Tuesday, May 1, 2007

JavaOne / memory leaks revisited...

Memory leaks in print 

A few months ago, Gregg Sporar together with A. Sundararajan started an article on memory leaks in the magazine Software Test & Performance. While writing that, he stumbled upon my blog and decided to cover the "java.lang.OutOfMemoryError: PermGen space" exception too. I offered to collaborate on the article. The article eventually grew so much it was split in two. Part one was published a month ago. Yesterday, part two was published.

Memory leaks at JavaOne

Edward Chou submitted a proposal for a BOF at JavaOne 2007. He and I will be presenting a BOF on the "java.lang.OutOfMemoryError: PermGen space" exception. I'll try to record the session with my MP3 player and post it on my blog.

In preparation for our presentation, we've been looking at some real-life examples of permgen memory leaks. We took a few memory dumps that came from actual customers in actual production environments. We discovered a few more improvements we could make to jhat: it was already fairly simple to track the leaks with jhat; with these changes it becomes really simple. We were actually quite surprised how simple. More on that in a future entry, either on my blog or on Edward's.

More at JavaOne

Speaking about JavaOne... I have my hands full. Next to the memory leaks BOF, I'm also presenting a BOF on JBI ("How to develop JBI components") and I'll be co-presenting another BOF on "EE and JBI."


16 comments:

Markus Kohler said...

Hi,
You might be interested in the new SAP
Memory Analyzer tool.
For details check
https://www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/wlg/6433
We easily found classloader leaks with this tool.
There's also a session about this tool at JavaOne (TS-21935).
Regards,
Markus

Michael Bartmann said...

I attended a session about the SAP Memory Analyzer at the german JAX conference and was very impressed. The tool is on the same DVD as the SAP J2EE environment, but it can be installed using a separate installer binary.
Regards, Michael

Markus Kohler said...

Hi Michael,
I'm glad to hear that you liked the tool.
This is only the first version.
The next version, will be \*much\* better.
By the way,the same DVD will be available at JavaOne.
Regards,
Markus

Michael Bartmann said...

Hi Markus,
will I need to attend yet another conference to get the next version, or
is there an url to watch?
Regards, Michael.

Markus Kohler said...

Hi Michael,
Watch my weblog http://https//www.sdn.sap.com/irj/sdn/weblogs?blog=/pub/u/6389
The next release should come out in summer.
Regards,
Markus

Edward Chou said...

Hi Markus, I just tried SAP Memory Analyzer tool from the JavaOne DVD, and I have to say I am pretty impressed with it. Very nice UI, and performance is great also. It also works well to find Classloader leaks, although I did find a bug with tool. When I try to trace paths from GC roots to an object without weak and soft reference, it still returns results that include ThreadLocal$ThreadLocalMap$Entry as a node in the path, however this inner class extends from WeakReference, so it should not be included. Other than that, I have no complaints.

Markus Kohler said...

Hi Edward,
Thanks for your positive feedback :)
We will try to verify that bug.
In case you have a simple test case for it, please let my know.
e-mail to "my first name dot my last name at gmail dot com"
Regards,
Markus

Krum said...

Hi Edward,
Only paths through the referent field will be avoided. Is the one you see through this field or through another one (e.g. value)?
Regards,
Krum

Edward Chou said...

Hi Krum, the path is through a ThreadLocal variable. See the screenshot of the path.

Vedran Lerenc said...

Hi Edward, the ThreadLocal$ThreadLocalMap$Entry is itself a WeakReference, but this alone is not all that matters to the JVM and its reference handling during the GC. Only the referent field (within WeakReference) is handled specifically, therefore the MemoryAnalyzer sees the object hold through the field value as strong reference, just as the JVM/GC would see it as a strong reference. This might irritate you as the value will be expunged by the data structure (important: by the data structure, not the GC) when it detects the clearing of the referent, but it is correct behaviour and has already proven to be important as e.g. with WeakHashMaps which become memory leaks if they are not accessed after the referents got cleared. The problem is that the data structure itself takes care for its data hold through the field value, not the JVM/GC and it does so only on access within its own code. Regards, Vedran

Edward Chou said...

Hi Vedran, I went back to look at the JDK source code for ThreadLocal, and you are correct regarding the "referent" field. The constructor of ThreadLocal$ThreadLocalMap$Entry calls "super(k)", so only the ThreadLocal instance if weak-referenced, and not the target object. My mistake, and thanks for the clarification, :-)

Agampreet said...

Hi,

I am trying to open a 1.5 gb heap dump with Memory Analyzer tool, I found that the in MemoryAnalyzer.ini the following heap size is setup:

-vmargs

-Xmx1024m

I tried to increase the size to 2048, but upon running the exe file, its spitting the following error:

JVM Terminated. Exit Code=-1

Please let me know what should I try to have this dump opened. Thanks

Regards,

Agam

Frank Kieviet said...

Re Agampreet,

I've heard good things about the SAP Memory Analyzer Tool. If I'm not mistaken, the MAT is now part of Eclipse and you can download it from eclipse.org.

HTH,

Frank

agampreet said...

Thanks for replying back Frank.

Yes, I have downloaded MAT and use it all the time to analyze the heap dumps. Its a wonderful tool. It works good for heap dumps below 1 GB, but now I am in situation where the heap dump grows to 2 Gig and I am trying to open that heap dump, I have 4 Gig of memory on my windows machine, but when I am trying to open the 2 gig heap dump, it keep on analyzing but never returned with results, or sometime spit OOM. Please advise. Thanks much.

Regards,

Agam

agampreet said...

Thanks for replying back Frank.

Yes, I have downloaded MAT and use it all the time to analyze the heap dumps. Its a wonderful tool. It works good for heap dumps below 1 GB, but now I am in situation where the heap dump grows to 2 Gig and I am trying to open that heap dump, I have 4 Gig of memory on my windows machine, but when I am trying to open the 2 gig heap dump, it keep on analyzing but never returned with results, or sometime spit OOM. Please advise. Thanks much.

Regards,

Agam

Carl-Erik said...

Your articles (part 1 and 2) are gone. Maybe time to update the links?