Just a small comment on things which are not obvious - Ray created his own custom autoscaler implementation. Both OpenShift and Kubernetes have support for so called Custom Metrics [1][2] which you can use for autoscaling. 

However, trying to scale based on the memory consumption is a tricky business [3][4]. I've been talking with Andrew about this and he says that settings `-Xms` to some low value and `-Xmx` reasonably high might do the trick. When you add a node to the cluster (assuming only DIST caches) the total amount of used memory in the Pods will (possibly) drop after some time. This however might not work for CMS GC. 

Anyway - those are the things which are on my plate for this and next week. Once I have some results I would be glad to share them with you. 

And as always - any help with this would be highly appreciated.

Thanks

Sebastian

[1] https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/custom-metrics.md

[2] https://github.com/openshift/origin-metrics

[3] https://developerblog.redhat.com/2014/07/15/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-1/

[4] https://developerblog.redhat.com/2014/07/22/dude-wheres-my-paas-memory-tuning-javas-footprint-in-openshift-part-2/