exactly one hour after the first request, the performance drops dramatically and garbage collection goes off the roof.
After one hour? That is interesting. These options in the run script sets the RMI cleanup thread to run a full GC every hour:
-Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000
Try adding a 0 (or 00) to the end of each to increase the delay time. Or add -XX:+DisableExplicitGC to turn it off completely.
(By the way, I know why the RMI GC interval is set to 1 hour - it was set to that for a spoecific test environment but in production that is the wrong time interval.)
The other thing you could do is turn on -verbose:gc and track the garbage collections. You will have to graph the output to see what is going on.
For suggestions on how to gather GC data and analyze it, see these white papers:
Java Garbage Collection Statistical Analysis 101
Java Garbage Collection Performance Analysis 201
at http://www.cmg.org/cgi-bin/search.cgi?q=java+peter+johnson&x=30&y=10