What version of the JDK are you running Wildfly on?  A quick Googling of some of the issues in your stack trace point to this thread (https://github.com/notnoop/java-apns/issues/116) which claims a similar issue is a JVM bug.

It sounds like you might be hitting a (similar) race condition.

A few things that come to mind to check:

Do you notice if the server is using a lot of CPU or memory?  
Are you possibly out of disk space?
Do your system logs have anything interesting which might shed some light on the issue?
Is selinux blocking something?

Sorry you are having troubles,

Summers

On Thu, Sep 1, 2016 at 10:57 AM, Yoann Archambault <ya@sherpa-solutions.com> wrote:
Hello,

Our deployment of Aerogear in production has stopped responding to requests on the ag-push.war or auth.war applications (wildfly admin console works fine). I can only see one type of stack trace when i do a kill -3 (see bellow). So far i've tried restarting wildfly, undeploying/deploying both WARs, rebooting the server and nothing has had any effect. There has been no change in configuration or code for a few weeks now and i've had a couple of service restarts without a issue. I'm using UPS 1.1.3-final on a CentOS server.

Also i'm not sure if its relevant but since the start we've been using a slightly modified version of UPS which we built with maven. The only change is using the push_message_info.client_identifier for our own purpose. Its basically a one line code change.

"default task-62" #221 prio=5 os_prio=0 tid=0x00007fc7901a9000 nid=0x32c4 runnable [0x00007fc6eb7f5000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:170)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
        at sun.security.ssl.InputRecord.read(InputRecord.java:503)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
        - locked <0x00000006c5abff60> (a java.lang.Object)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
        at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
        - locked <0x00000006c5ac0020> (a sun.security.ssl.AppInputStream)
        at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
        at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
        at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
        at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
        at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
        at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:223)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
        at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
        at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
        at org.keycloak.adapters.AdapterDeploymentContext.resolveRealmKey(AdapterDeploymentContext.java:105)
        at org.keycloak.adapters.AdapterDeploymentContext.resolveDeployment(AdapterDeploymentContext.java:83)
        at org.keycloak.adapters.PreAuthActionsHandler.preflightCors(PreAuthActionsHandler.java:71)
        at org.keycloak.adapters.PreAuthActionsHandler.handleRequest(PreAuthActionsHandler.java:47)
        at org.keycloak.adapters.undertow.ServletPreAuthActionsHandler.handleRequest(ServletPreAuthActionsHandler.java:68)
        at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:282)
        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:261)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:80)
        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:172)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:199)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:774)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)


I've been trying to solve this for 2 days without success, so at this point any help would be greatly appreciated!

--
Yoann

_______________________________________________
Aerogear-users mailing list
Aerogear-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/aerogear-users