As Peter said it is a bit rare information.
But I support a application with about 400-500 transactions per second for one JBoss instance.
It run with 1500m heap (Xms and Xmx with same value) and 128m permSize.
The maximum of memory depends on the OS and JVM implementation, simple to test with the Xms option, the JVM did not start if the memory is not supported. (e.g. we use Solaris 64Bit with the original Sun 32Bit-JVM with 3600m for some cases)
The GC behaviour is extreme application dependend! A good practice is to have objects with short livecycle, in this case the GC is able to run faster.
In general you have to understand how the GC works. Good starting points are [1][2].
Also you might test the JVM tools JStat and VisualGC from Sun (sorry Oracle ;-)
search at oracle techwork for 'GC tuning' or the tools.
[1] JVM5 http://www.oracle.com/technetwork/java/gc-tuning-5-138395.html
[2] JVM6 http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html