Mantis - Resin
Viewing Issue Advanced Details
ID: Category: Severity: Reproducibility: Date Submitted: Last Update:
600 minor always 12-30-05 09:31 02-27-06 12:40
Reporter: ferg Platform:  
Assigned To: ferg OS:  
Priority: urgent OS Version:  
Status: closed Product Version: 3.0.17  
Product Build: Resolution: fixed  
Projection: none      
ETA: none Fixed in Version: 3.0.19  
Summary: 0000600: OOM
Description: (rep by uwe schaefer)

weŽre facing some OOMs in our production servers, just like other guys
did in the past and probably still do.
profiling our application did not shed any light to the issue, so i scaled
everything down and found something interesting:

to reproduce use this setting:

- using SuSE linux (9.3) (probably irrelevant)
- any (!) sun JVM (1.4.1_06, _07, 1.5.0_05) (probably irrelevant)
- running under JProfiler (reproducible without it)
- starting resin with -server -Xmx64m -Xms64m -verbose:gc (just to make
the error appear faster)

plain resin in standard config, just as it comes.
replace standard host with the javadocs from JDK (yes, all these .html pages)
and configure resin to use the JSP-Servlet (instead of the fileservlet) to
handle them.

start resin and throw 10 concurrent htdigs at it and youŽll see an OOM in
about 5-10 mins (probably depending on cpu and IO speed) talking about
the infamous Jar.getSafeJarEntry method.

exactly before OOM occured, the GC output showed a Full GC like this:
[Full GC 64426K->4232K(64640K), 0.1457610 secs]

which makes me think that this isnŽt really a heap related issue.

it looks like, the problem has to do with native stuff.
iŽve read something a few months ago about some memleak problem with
java+jni+some gzip lib. coincidence ?

this seems easily reproduceable without any custom java-code involved.
doing the same to resin3 crashed it within 3 mins without the slightest
hint from the profiler that resin REALLY ran out of heap-memory.

any suggestions highly welcome.

PS: what SEEMS to help is +UseParallelGC with jdk1.5, but i am not sure
yet. maybe this only defers the problem.
Steps To Reproduce:
Additional Information:
Relationships
Attached Files:

Notes
(0000625)
anonymous   
01-02-06 00:42   
when additionally using -XX:MaxPermSize=64m, i see the following output from one Full-GC run:

[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
[Unloading class _errorpages._404__html]
...

is it possible, that there are more classloaders involved, than i expected with one application ? or is this class loaded for more than one time within the very same classloader ?
(0000767)
ferg   
01-24-06 11:55   
The JSP caching can be improved. Currently, the loaded JSP pages are immune to garbage collection. If the application has a large number of JSPs, even if a large number are rarely used, most of them will be part of the heap.

It is possible to change the caching, but it will take a bit of refactoring and some juggling to make the jspDestroy work properly.
(0000907)
ferg   
02-27-06 12:40   
Added SoftReference to PageFilterChain. Also added page-cache-max to JspServlet (as an <init> parameter).