[aerogear-dev] [simplepush] Performance

[JR Conlin]

The concern with using Mongo is that it's very lossy and fairly
unreliable as a data store for a number of reasons. (It's fine for
simple, low demand systems, but has complications once you really start
to hammer on it.)

http://blog.schmichael.com/2011/11/05/failing-with-mongodb/
http://blog.engineering.kiip.me/post/20988881092/a-year-with-mongodb
etc.

I'd suggest sticking with other DBs unless you're ok with loss or don't
plan on heavily exercising it.

On 2013/9/25 9:17 AM, Karel Piwko wrote:
> Looks promising, however Redis has limitation - all data are basically stored in
> memory, requiring additional setup for enterprise application.
>
> I was wondering whether SimplePush could use for instance Hibernate OGM as
> abstraction, giving user possibility to switch DB storage, while Redis seems to
> be reasonable default [1]? On OpenShift, it would for instance make sense to use
> MongoDB, as this is already a cartridge available.
>
> Karel
>
> [1] This is an interesting comparison of NoSQL databases:
> http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis
>
> On Wed, 25 Sep 2013 14:16:30 +0200
> Daniel Bevenius <daniel.bevenius at gmail.com> wrote:
>
>> To get a feel for what would be involved to use a key/value datastore we've
>> done some experimenting with Redis[1]. There might be other non-relational
>> databases more suited or perhaps Redis is a good choice for this, I don't
>> know. But I think we should decide if this is worth pursuing and in that
>> case what database to use before spending more time on this.
>> Let us know what you think.
>>
>> [1] https://gist.github.com/danbev/6606289#using-redis-as-a-data-store
>>
>>
>> On 19 September 2013 18:08, JR Conlin <jrconlin at gmail.com> wrote:
>>
>>>  On 2013/9/19 5:18 AM, Lucas Holmquist wrote:
>>>
>>>
>>>  On Sep 19, 2013, at 12:34 AM, Daniel Bevenius <daniel.bevenius at gmail.com>
>>> wrote:
>>>
>>>  >I wonder what kind of numbers would we get by ditching JPA completely and
>>> using a non-relational DB like Redis
>>> Yeah, I think we will most likely need to if we want to come close to the
>>> other implementations performance wise. Others use Memcache and I've seen
>>> MongoDB in use as well.
>>>
>>>  Perhaps I should just add performance tests for the rest of the
>>> SimplePush operations so that we have them covered and then look into using
>>> a non-relational DB. Once that is done we can revisit this performance task.
>>> What do people thing about that?
>>>
>>>
>>>  +1,  relational DB's are dinosaours
>>>
>>> Hardly. It's just a question of what the right tool for a given job is.
>>> (I'll note that Google is spending quite a bit of time and effort improving
>>> Maria because they use a LOT of relational DBs for very large data.
>>>
>>> In this case, however, it's pretty easy to reduce things to simple
>>> key/value. I picked Memcache, partly because of the fact that it does
>>> record auto-expiration, which means that I don't have to do garbage
>>> collection on uncollected records. If you switched to an alternate schema
>>> (such as keeping a single record per UAID that contained all the CHID data
>>> as well as stuff like the proprietary info or other crap), you could even
>>> use simple flat files and skip the DB requirement altogether.
>>>
>>> We were kicking the idea around of only storing undeliverable data into
>>> the DB, and instead letting each websock connector deal with managing it's
>>> own data. For our implementation, I've already preferenced delivery over
>>> storage for connected clients and seen a fair bit of improvement on
>>> delivery. (Remember, SimplePush is not a 100% guaranteed delivery system,
>>> so please avoid using it for nuclear reactor management or pacemakers.)
>>>
>>> We'll probably hold off on doing further memory refinement until we get
>>> some actual use data, but I like having options available.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 19 September 2013 06:03, Bruno Oliveira <bruno at abstractj.org> wrote:
>>>
>>>> Hmmm tempting idea :)
>>>>
>>>>> On Sep 19, 2013, at 12:23 AM, Douglas Campos <qmx at qmx.me> wrote:
>>>>>
>>>>> That's a nice report!
>>>>>
>>>>> I wonder what kind of numbers would we get by ditching JPA completely
>>>>> and using a non-relational DB like Redis...
>>>>>
>>>>> --
>>>>> qmx
>>>>  > _______________________________________________
>>>>> aerogear-dev mailing list
>>>>> aerogear-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>> _______________________________________________
>>>> aerogear-dev mailing list
>>>> aerogear-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>>
>>>  _______________________________________________
>>> aerogear-dev mailing list
>>> aerogear-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> aerogear-dev mailing
>>> listaerogear-dev at lists.jboss.orghttps://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>
>>>
>>>
>>> _______________________________________________
>>> aerogear-dev mailing list
>>> aerogear-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/aerogear-dev
>>>
> _______________________________________________
> aerogear-dev mailing list
> aerogear-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/aerogear-dev