[aerogear-dev] UPS Production worthiness
Matthias Wessendorf
matzew at apache.org
Fri Jul 18 09:41:28 EDT 2014
On Wed, Jul 2, 2014 at 4:40 PM, Vivek Pandey <vivek.pandey at pinelabs.com>
wrote:
> Hello Matthias,
>
>
>
> An ‘importer’ might help, but smaller changes in existing domain/service
> might help more.
>
> I ran a stress test on the installation endpoint yesterday, after merging
> Eric’s changes on source code of 0.10.x branch.
>
> I saw that even after using the merged code, the response worsened
> continuously.
>
> Looking closing, I saw that GenericVariantServiceImpl. addInstallation
> causes initialization of the now “lazy” collection
> AbstractVariant.installations.
>
>
>
> I changed the model a bit, by adding variantID to Installation, and made
> it the owner of the variant-installation relationship.
>
> That changed the code in InstallationRegistrationEndpoint. registerInstallation
> from
>
> // store the installation:
>
> entity = clientInstallationService.addInstallation(entity);
>
> // add installation to the matching variant
>
> genericVariantService.addInstallation(variant, entity);
>
>
>
> to
>
> // store the installation:
>
>
> entity.setVariantID(variant);
>
> entity =
> clientInstallationService.addInstallation(entity);
>
>
>
> This fixed the problem for me and now I am getting constant time
> installation registration with dramatically better performance
>
Hey Vivek!,
as a FYI - I ran some perf. tests as well, and noticed a huge improvement
on the regitration as well, after Erik included (parts of) your patch
>
>
> Please let me know if you want me to add this to AGPUSH-661
>
>
>
> Thanks
>
> Vivek
>
>
>
> *From:* mwessendorf at gmail.com [mailto:mwessendorf at gmail.com] *On Behalf
> Of *Matthias Wessendorf
> *Sent:* Wednesday, July 02, 2014 6:26 PM
> *To:* AeroGear Developer Mailing List; vivek.pandey at pinelabs.com
>
> *Subject:* Re: [aerogear-dev] UPS Production worthiness
>
>
>
> Hello Vivek,
>
>
>
> On Mon, May 26, 2014 at 1:23 PM, Matthias Wessendorf <matzew at apache.org>
> wrote:
>
>
>
>
>
> On Mon, May 26, 2014 at 12:59 PM, Erik Jan de Wit <edewit at redhat.com>
> wrote:
>
> Hi Vivek,
>
>
>
> We’ve fixed that
> https://github.com/aerogear/aerogear-unifiedpush-server/commit/1357c9e834286a9227a90e9ab618226aa54bbbf3 the
> collection is now lazy. But I’m sure other query optimisations can be made.
>
>
>
> +1
>
>
>
> btw. Vivek, I added your mail to an existing ticket:
>
> https://issues.jboss.org/browse/AGPUSH-661
>
>
>
>
>
> coming back from vacation and going over the JIRAs, I was wondering if an
> 'importer' helps?
>
>
>
> That would be a RESTful endpoint that accepts a JSON file, containing all
> the devices you want to register (for a given variant).
>
>
>
> That way it would be also a bit nicer on your side, to perform the import:
>
> * generate a "large" JSON file (instead of a ton of requests)
>
> * only one HTTP request
>
> * server accepts and will process it in the background
>
>
>
> I think that would be a nice migration tool.
>
>
>
> Let me know what you think!
>
>
>
> -Matthias
>
>
>
>
>
> -Matthias
>
>
>
>
>
> Cheers,
>
> Erik Jan
>
>
>
> On 26 May,2014, at 12:27 , Vivek Pandey <vivek.pandey at pinelabs.com> wrote:
>
>
>
> Hi Matthias,
>
>
>
> It is good to know that activity in java-apns is picking up and also that
> you are looking at pushy.
>
>
>
> I did a few tests which added installations to UPS with a concurrency of
> 4-8 threads. I was using Postgres 9.3 and UPS 0.10.3 war
>
>
>
> I noticed that response slowed down considerably after some time with high
> CPU usage and continued to get worse. After doing some profiling, I found
> that bulk of CPU cycles are being taken by
> org.postgresql.core.VisibleBufferedInputStream.readMore. The entire thread
> stack is attached. Also postgres continuously flagged
>
>
>
> select installati0_.variantID as variant10_0_0_, installati0_.id as
> id1_3_0_, installati0_.id as id1_3_1_, installati0_.alias as alias2_3_1_,
> installati0_.deviceToken as deviceTo3_3_1_, installati0_.deviceType as
> deviceTy4_3_1_, installati0_.enabled as enabled5_3_1_,
> installati0_.operatingSystem as operatin6_3_1_, installati0_.osVersion as
> osVersio7_3_1_, installati0_.platform as platform8_3_1_,
> installati0_.simplePushEndpoint as simplePu9_3_1_ from InstallationImpl
> installati0_ where installati0_.variantID=$1
>
>
>
> as the slow query. I am pretty sure that eager collection
> AbstractVariant.installations is the root cause of the problem.
>
>
>
> Please let me know if you need any more information.
>
>
>
> Thanks
>
> Vivek
>
>
>
> *From:* mwessendorf at gmail.com [mailto:mwessendorf at gmail.com
> <mwessendorf at gmail.com>] *On Behalf Of *Matthias Wessendorf
> *Sent:* Wednesday, May 21, 2014 5:55 PM
> *To:* vivek.pandey at pinelabs.com; AeroGear Developer Mailing List
> *Subject:* Re: [aerogear-dev] UPS Production worthiness
>
>
>
> Hello Vivek!
>
>
>
> On Wed, May 21, 2014 at 2:07 PM, Vivek Pandey <vivek.pandey at pinelabs.com>
> wrote:
>
> Hi Jay,
>
>
>
> Thanks for your reply.
>
>
>
> While we have not faced any issues in using UPS in our limited testing, I
> often see info stacktraces in ups logs
>
>
>
> 2014-05-16 10:19:20,032 INFO
> [com.notnoop.apns.internal.ApnsConnectionImpl] (Thread-118) Exception while
> waiting for error code: java.net.SocketException: Socket closed
>
> at java.net.SocketInputStream.socketRead0(Native Method)
> [rt.jar:1.7.0_51]
>
> at java.net.SocketInputStream.read(SocketInputStream.java:152)
> [rt.jar:1.7.0_51]
>
> at java.net.SocketInputStream.read(SocketInputStream.java:122)
> [rt.jar:1.7.0_51]
>
> ……………
>
> at
> com.notnoop.apns.internal.ApnsConnectionImpl$1MonitoringThread.run(ApnsConnectionImpl.java:114)
> [apns-0.2.3.jar:]
>
>
>
>
>
> These stacktraces coupled with low dev activity of noop/java-apns project
> are disconcerting to me.
>
>
>
> the stack-trace is no harm - it's only happening w/ doing a monitoring of
> the thread (that's what we currently do, when setting up ApnsService - I
> thought about explicitly disable that)
>
>
>
> The activity of the underlying java-apns is very low, yes! However @froh42
> is getting back:
>
> https://github.com/notnoop/java-apns/commits/master
>
>
>
> There will be a new release in the near future; @froh42 asked me if I
> could help with pushing the bits to maven central
>
>
>
>
>
> That said, I recently started looking at pushy:
>
> https://github.com/relayrides/pushy
>
>
>
> I also sent a PR that would allow us to feed pushy w/ our certificate from
> the database:
>
> https://github.com/relayrides/pushy/pull/87
>
>
>
> Hope that helps
>
> -Matthias
>
>
>
>
>
> I am currently using UPS 0.10.2 war.
>
>
>
> Thanks,
>
> Vivek
>
> *From:* Jay Balunas [mailto:jbalunas at redhat.com]
> *Sent:* Tuesday, May 20, 2014 11:36 PM
> *To:* vivek.pandey at pinelabs.com; AeroGear Developer Mailing List
> *Cc:* Jay Balunas
> *Subject:* Re: [aerogear-dev] UPS Production worthiness
>
>
>
> Hi Vivek,
>
>
>
> It's awesome to hear that you have integrated the UPS into your backend
> systems and some of your mobile apps!
>
>
>
> We have a lot of confidence around the UPS, its functionality, and
> performance. Our team has been working hard on improvements and stability
> including our QE team. Also, as you may have seen we're planning a 1.0
> release of the UPS this summer.
>
>
>
> However at this time we don't have specific references or success stories
> outside of what you can see in the community mailing lists - other users
> using it ;-) We're also about to kick off some performance and scale
> testing in the next couple of months.
>
>
>
> Have you run into any issues that drove these questions about production
> worthiness? If so please let us know and we'll certainly take a look.
>
>
>
> Hope this helps!
>
>
>
> Thanks,
>
> Jay Balunas
>
>
>
> On May 20, 2014, at 8:09 AM, Vivek Pandey <vivek.pandey at pinelabs.com>
> wrote:
>
>
>
> Hello Aerogear dev team,
>
>
>
> We integrated UPS into our backend server which is serving various mobile
> apps. While the development and testing phase went well, my manager is
> questioning me about production worthiness about Aerogear. It would be
> great help if you could point me to references/success stories where UPS is
> being used in production environments and scaling well in medium to high
> loads.
>
>
>
> Thanks,
>
> Vivek
>
>
>
>
>
>
> ------------------------------
>
> This message may contain privileged and confidential information and is
> solely for the use of intended recipient. The views expressed in this email
> are those of the sender and not of Pine Labs. The recipient should check
> this email and attachments for the presence of viruses / malwares etc. Pine
> Labs accepts no liability for any damage caused by any virus transmitted by
> this email. Pine Labs may monitor and record all emails.
> ------------------------------
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
>
>
> ------------------------------
>
> This message may contain privileged and confidential information and is
> solely for the use of intended recipient. The views expressed in this email
> are those of the sender and not of Pine Labs. The recipient should check
> this email and attachments for the presence of viruses / malwares etc. Pine
> Labs accepts no liability for any damage caused by any virus transmitted by
> this email. Pine Labs may monitor and record all emails.
> ------------------------------
>
>
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
>
>
> ------------------------------
>
> This message may contain privileged and confidential information and is
> solely for the use of intended recipient. The views expressed in this email
> are those of the sender and not of Pine Labs. The recipient should check
> this email and attachments for the presence of viruses / malwares etc. Pine
> Labs accepts no liability for any damage caused by any virus transmitted by
> this email. Pine Labs may monitor and record all emails.
> ------------------------------
>
>
>
> <threadstack.txt>_______________________________________________
>
>
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
>
>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>
>
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
>
>
>
>
>
> --
> Matthias Wessendorf
>
> blog: http://matthiaswessendorf.wordpress.com/
> sessions: http://www.slideshare.net/mwessendorf
> twitter: http://twitter.com/mwessendorf
>
> ------------------------------
> This message may contain privileged and confidential information and is
> solely for the use of intended recipient. The views expressed in this email
> are those of the sender and not of Pine Labs. The recipient should check
> this email and attachments for the presence of viruses / malwares etc. Pine
> Labs accepts no liability for any damage caused by any virus transmitted by
> this email. Pine Labs may monitor and record all emails.
> ------------------------------
>
>
--
Matthias Wessendorf
blog: http://matthiaswessendorf.wordpress.com/
sessions: http://www.slideshare.net/mwessendorf
twitter: http://twitter.com/mwessendorf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/aerogear-dev/attachments/20140718/248a7296/attachment-0001.html
More information about the aerogear-dev
mailing list