Optimized ObjectEntityCache for Thread Safety and Memory Management #3392
Conversation
There was a problem hiding this comment.
Pull request overview
This PR enhances the ObjectEntityCache class with thread safety mechanisms and memory management capabilities. The implementation adds concurrent access support through read-write locks and prevents unbounded cache growth through an LRU eviction policy.
Key Changes:
- Introduced
ReadWriteLocksynchronization for thread-safe cache operations - Implemented LRU eviction with a 10,000 entry limit to prevent memory leaks
- Added reverse mapping cleanup to maintain consistency during evictions
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
| resourceCache.remove(oldestKey); | ||
|
|
||
| uuidReverseMap.entrySet().removeIf(entry -> getCacheKeyFromUUID(entry.getValue()).equals(oldestKey)); |
There was a problem hiding this comment.
The getCacheKeyFromUUID method is called for every entry in uuidReverseMap during eviction, resulting in O(n*m) complexity where n is the size of uuidReverseMap and m is the size of resourceCache. This is highly inefficient. Instead, retrieve the evicted object from resourceCache before removal and use it to directly remove from uuidReverseMap: Object evictedEntity = resourceCache.get(oldestKey); resourceCache.remove(oldestKey); uuidReverseMap.remove(evictedEntity);
| resourceCache.remove(oldestKey); | |
| uuidReverseMap.entrySet().removeIf(entry -> getCacheKeyFromUUID(entry.getValue()).equals(oldestKey)); | |
| Object evictedEntity = resourceCache.get(oldestKey); | |
| resourceCache.remove(oldestKey); | |
| uuidReverseMap.remove(evictedEntity); |
| private String getCacheKeyFromUUID(String uuidValue) { | ||
| for (String key : resourceCache.keySet()) { | ||
| if (key.endsWith("_" + uuidValue)) { | ||
| return key; | ||
| } | ||
| } | ||
| return null; | ||
| } |
There was a problem hiding this comment.
The getCacheKeyFromUUID method assumes cache keys end with _<uuid>, but the actual key format from getCacheKey(type, id) is not visible in this diff. This assumption may be incorrect and could cause the reverse mapping cleanup to fail silently. Additionally, iterating through all cache keys is inefficient. Consider storing the type-id pair in uuidReverseMap alongside the UUID, or maintaining a separate map from entity objects to cache keys.
| lock.readLock().lock(); | ||
| try { | ||
| return resourceCache.get(getCacheKey(type, id)); | ||
| } finally { | ||
| lock.readLock().unlock(); |
There was a problem hiding this comment.
Using a read lock with an access-ordered LinkedHashMap (created with accessOrder=true on line 27) is unsafe. The get() operation on an access-ordered map modifies the internal structure by moving the accessed entry to the end, which is a write operation. This will cause inconsistent state when multiple threads read concurrently. Either use a write lock for get() operations or switch to insertion-ordered LinkedHashMap (set third parameter to false).
| lock.readLock().lock(); | |
| try { | |
| return resourceCache.get(getCacheKey(type, id)); | |
| } finally { | |
| lock.readLock().unlock(); | |
| lock.writeLock().lock(); | |
| try { | |
| return resourceCache.get(getCacheKey(type, id)); | |
| } finally { | |
| lock.writeLock().unlock(); |
| uuidReverseMap.put(entity, id); | ||
|
|
||
| if (resourceCache.size() >= MAX_CACHE_SIZE) { | ||
| evictLRU(); | ||
| } | ||
|
|
||
| return resourceCache.put(getCacheKey(type, id), entity); |
There was a problem hiding this comment.
The reverse mapping is added before checking cache size and evicting. If the entity being added already exists in the cache at a different position, and eviction occurs, the old entry's reverse mapping may not be properly cleaned up. Additionally, if the same entity is re-added with a different ID, this will overwrite the previous UUID mapping. Consider handling updates and eviction order more carefully to maintain mapping consistency.
2 participants
Resolves # (if appropriate): (New enhancement)
Description
Enhanced the
ObjectEntityCacheclass with thread safety and memory management optimizations. The implementation now supports concurrent access patterns and prevents memory leaks through LRU eviction.ReadWriteLocksynchronization for thread-safe cache operationsMotivation and Context
The original
ObjectEntityCacheimplementation was unbounded cache growth with no eviction mechanism and wasn't optimized for concurrent access patterns. This enhancement attempts to address these issues while maintaining the existing public API making Elide more robust in high concurrency scenarios.How Has This Been Tested?
ReadWriteLockusage with separate read/write pathsEnvironment:
Java 17 compilation target
Maven build system
Existing Elide test suite compatibility verified
Screenshots (if appropriate): N/A - Code optimization without UI changes
License
I confirm that this contribution is made under an Apache 2.0 license and that I have the authority necessary to make this contribution on behalf of its copyright owner.