[Aerogear-users] Default heap size and OOM
Matthias Wessendorf
matzew at apache.org
Tue Jun 23 09:06:19 EDT 2015
Hi Mo,
the PR has been merged, and it should improve the situation a bit
PS: next will be removing SimplePush from the cartridge
On Thu, Jun 18, 2015 at 5:17 PM, Matthias Wessendorf <matzew at apache.org>
wrote:
> Hi ,
>
>
>
> [1]
> https://github.com/aerogear/openshift-origin-cartridge-aerogear-push/pull/18
>
> On Thu, Jun 18, 2015 at 4:57 PM, mo <michi.oshima at gmail.com> wrote:
>
>> Thanks, Matthias. I take your reply to mean "No, one shouldn't have to
>> increase JVM heap size."
>>
>
> yeah, if you really want to, you can try tweaking the jvm heap size (as
> long as you still use a function of the max memory available on the gear)
>
>
> But also please review the changes done in
>
> https://github.com/aerogear/openshift-origin-cartridge-aerogear-push/pull/18
>
> You could port them over to your instance. Makes sense ?
>
> More coming soon.
>
>
>
>>
>> Let me list some more parameters, in the hope that you might spot
>> something
>> obvious that we can correct on our side. It's been difficult to
>> troubleshoot this, because when the problem happens we can't even SSH into
>> the server. I believe whatever the problem is, it's exhausting resources
>> on
>> the server.
>>
>> Parameters:
>>
>> 1. Deployed on OpenShift using the cartridge found here:
>> https://github.com/aerogear/openshift-origin-cartridge-aerogear-push
>> <https://github.com/aerogear/openshift-origin-cartridge-aerogear-push>
>> 2. AeroGear UnifiedPush Server 1.0.3 and WildFly, MySQL 5.5, Medium gear.
>> 3. One application is defined with one variant (simple-push), 131
>> installations registered.
>> 4. We are using all default settings. We haven't customized the JVM max
>> heap size, for example. The server log tells me "-Xmx512m".
>>
>> Here are the reproduction steps. This happened reliably two days ago. We
>> couldn't reproduce this yesterday after a fresh restart of AeroGear.
>>
>> 1. Logged into AeroGear as admin
>> 2. Select "Send Push"
>> 3. Select an application
>> 4. Hit "Send Push Notification", multiple times one after another. About
>> 10
>> consecutive messages would cause OutOfMemoryError on our server.
>>
>> Here's a relevant snippet of the server log:
>>
>> [0m [0m17:03:29,511 INFO [PushNotificationSenderEndpoint] (default
>> task-28) Message submitted to PushNetworks for further processing
>> [0m [0m17:03:29,513 INFO [JPAPushMessageInformationDao] (EJB default -
>> 6)
>> Processing send request with '[alert=, criteria=[aliases=null,
>> deviceTypes=null, categories=null, variants=null], time-to-live=-1]'
>> payload
>> [0m [31m17:03:29,837 ERROR [org.jboss.as.ejb3.invocation] (EJB default -
>> 6)
>> JBAS014134: EJB Invocation failed on component SenderServiceImpl for
>> method
>> public abstract void
>>
>> org.jboss.aerogear.unifiedpush.message.SenderService.send(org.jboss.aerogear.unifiedpush.api.PushApplication,org.jboss.aerogear.unifiedpush.message.UnifiedPushMessage):
>> javax.ejb.EJBException: JBAS014580: Unexpected Error
>> at
>>
>> org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:187)
>> [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final]
>> at
>>
>> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275)
>> [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final]
>>
>> .
>> .
>> .
>>
>> [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final]
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> [rt.jar:1.7.0_79]
>> at
>>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> [rt.jar:1.7.0_79]
>> at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_79]
>> at org.jboss.threads.JBossThread.run(JBossThread.java:122)
>> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
>> at java.lang.Thread.start0(Native Method) [rt.jar:1.7.0_79]
>> at java.lang.Thread.start(Thread.java:714) [rt.jar:1.7.0_79]
>> at
>> sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1056)
>> [jsse.jar:1.7.0_79]
>> at
>>
>> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
>> [jsse.jar:1.7.0_79]
>> at
>> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359)
>> [jsse.jar:1.7.0_79]
>> .
>> .
>> .
>>
>>
>> org.jboss.aerogear.unifiedpush.message.sender.SimplePushNotificationSender.put(SimplePushNotificationSender.java:111)
>> [unifiedpush-push-1.0.3.jar:1.0.3]
>> at
>>
>> org.jboss.aerogear.unifiedpush.message.sender.SimplePushNotificationSender.sendPushMessage(SimplePushNotificationSender.java:65)
>> [unifiedpush-push-1.0.3.jar:1.0.3]
>> at
>>
>> org.jboss.aerogear.unifiedpush.message.SenderServiceImpl.send(SenderServiceImpl.java:110)
>> [unifiedpush-push-1.0.3.jar:1.0.3]
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> [rt.jar:1.7.0_79]
>> at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> [rt.jar:1.7.0_79]
>> at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> [rt.jar:1.7.0_79]
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> [rt.jar:1.7.0_79]
>> at
>>
>> org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52)
>>
>> .
>> .
>> .
>>
>> at
>>
>> org.jboss.as.ejb3.component.interceptors.NonPooledEJBComponentInstanceAssociatingInterceptor.processInvocation(NonPooledEJBComponentInstanceAssociatingInterceptor.java:59)
>> [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final]
>> at
>>
>> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:309)
>> at
>>
>> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:273)
>> [wildfly-ejb3-8.1.0.Final.jar:8.1.0.Final]
>> ... 38 more
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://aerogear-users.1116366.n5.nabble.com/Default-heap-size-and-OOM-tp56p58.html
>> Sent from the aerogear-users mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> Aerogear-users mailing list
>> Aerogear-users at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/aerogear-users
>>
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
>
--
Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/aerogear-users/attachments/20150623/cbbd75c6/attachment-0001.html
More information about the Aerogear-users
mailing list