I tried changing jboss-service.xml to change the DefaultCacheTimeout to 0 from 1800, and also tried
String domain = "jmx-console";
ObjectName jaasMgr = new ObjectName("jboss.security:service=JaasSecurityManager");
Object[] params = {domain};
String[] signature = {"java.lang.String"};
MBeanServer server = (MBeanServer) MBeanServerFactory.findMBeanServer(null).get(0);
server.invoke(jaasMgr, "flushAuthenticationCache", params, signature);
But that didn't seem to work either. I'm stumped :(