[hibernate-dev] [OGM] MongoDB time spikes

[Emmanuel Bernard]

Hi Alan and all,

I have been researching the spikes issue you encounter in the stress test from a theoretical point of view.
You were trying a different associations storage approach (splitting associations as one row per document rather than the whole association per document). Does that return better results?

I am skeptical for a few reasons. MongoDB has a global write lock per mongod process (they are working on a more fine grained solution). So if the spikes are due to lock contention, shuffling data won't help much. Also make sure you use MongoDB 2.0 instead of 1.8 as they yield lock on page fault which should solve a lot of these spikes problems.

I have found this blog entry to be quite insightful http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html

Generally speaking, if all data can stay in memory, MongoDB should behave wonderfully. 

Which leads to my demo and the time difference between Infinispan 5s and Mongodb 20s. I can see several reasons:

- we don't really batch operations in the mongodb dialect and we should.
  We should accumulate operations and apply them at the end of the flush operation in one batch. That will require some new infrastructure from OGM's engine though
  to tell the dialect when to "flush".
- my VM might swap memory on disk which would explain the difference
- or it could be that Infinispan is simply 4 times faster which would not be too surprising as Infinispan is in-process.

Emmanuel