[JBMAR-225] Metaspace Memory Leak - SerializableClassRegistry Prevents ClassLoader Garbage Collection

[dmlloyd]

Joshua Swett opened JBMAR-225 and commented

jboss marshalling caches SerailizableClass instances using Java's ClassValue API. The intention here is to be able to weakly cache stuff using the java.lang.Class instance as the cache key while also referencing the java.lang.Class in the cache value (SerializableClass).

Unfortunately, the solution as it stands prevents ClassLoader garbage collection because SeriallzableClassRegistry keeps the ClassValue instance as a static variable which creates a hard reference chain between the ClassLoader and the ClassValue instance:

Some ClassLoader -> SerializableClassRegistry.class -> ClassValue instance

In this case, "Some ClassLoader" is a plugin/module class loader with an augmented class path compared to the System class loader.

The reason "Some ClassLoader" can't be GCed is because there is reference chain starting at the System Class loader that is preventing it that looks something like this:

System ClassLoader -> java.util.HashMap.class (this could be any Serializable class loaded by the System ClassLoader) ->
classValueMap -> SerilizableClass.class (entry in the map) -> "Some ClassLoader" (it loaded SerializableClass)

---

Affects: 2.0.7.Final

Attachments:

• TestCase.zip (384.55 kB)

[dmlloyd]

@dmlloyd commented

I haven't gotten to analyze the test case yet, but one thing I wonder: why are the ClassValue instances not being GC'd? The class value map itself should be keyed by an opaque Identity object (example https://github.com/unofficial-openjdk/openjdk/blob/jdk/jdk/src/java.base/share/classes/java/lang/ClassValue.java#L262) so I don't see how e.g. HashMap.class or any other class object could possibly hold a strong reference to it.

[dmlloyd]

Joshua Swett commented

Based on the heap dumps I analyzed, the ClassValue instances are not getting GC'ed because of the fact that it is a static field in the SeriallzableClassRegistry class. Because of this, the plugin/module ClassLoader has a hard reference to it since the ClassLoader has a list of classes it has loaded (one of which is SerializableClassRegistry.class), and that class has a hard reference to the ClassValue instance. This overall scenario is basically the same thing described on JDK-8136353, which was decided to not be a JDK bug, but an issue in the client code (although there is some disagreement about this on the ticket).