1) As I said -Xmx does not change the threadstacksize
2) It's more difficult ...
You should not start threads by yourselve running JEE under AS control!
If you don't do this your code should be ok.
Different might be the MBean handling.
Do you check whether you have enough memory system wide? What OS you use?