No you do need the truststore on the client side to be able to verify the certificate although you should be able to use the following properties on the client without affecting the JVM installation: -
-Djavax.net.ssl.trustStore=client.truststore -Djavax.net.ssl.trustStorePassword=truststore_password
The issue is that if the client is inadvertantly forwarded to connect to a different address a man in the middle type attack with a fake private key is fairly simple with that intermediarry now having full access to the data being exchanged.