[infinispan-dev] HSEARCH-1296

Fri Apr 12 08:41:24 EDT 2013

The index master is essentially a queue + the queue processor.
So if many people as the index master to index data (even as async).
the sync call will have to wait for all the other ones to be done before
carrying on.
But are the other put involving indexed entities? If not, then that's
not what is going on.

I can't explain one index operation taking 50s if nothing is happening
in parallel.
As Sanne said, use the blackhole backend on your index master node. We
will know if the index is really taking all that time.

Emmanuel

On Fri 2013-04-12 14:19, Ales Justin wrote:
> To our "default" cache - which is where we store Datastore entries,
> the test only does 1 put, 1 get, 1 delete, 1 query.
> 
> https://github.com/capedwarf/capedwarf-blue/blob/master/cluster-tests/src/test/java/org/jboss/test/capedwarf/cluster/test/QueryTest.java
> 
> There are other indexes that participate in this test as well, "logging" cache, etc.
> But only "default" is configured with new "sync" stuff,
> others all have this:
> 
>                     <indexing index="LOCAL">
>                         <property name="hibernate.search.default.worker.execution">async</property>
> 
> So I fail to see how this is "bunch of them".
> 
> -Ales
> 
> On Apr 12, 2013, at 2:12 PM, Emmanuel Bernard <emmanuel at hibernate.org> wrote:
> 
> > No, it is one RPC call for every put or delete in the grid. I'm sure the
> > test makes a bunch of them.
> > 
> > On Fri 2013-04-12 12:40, Bela Ban wrote:
> >> I might have misunderstood something: if this is *one* sync RPC, then 
> >> enabling or disabling of bundling won't have an impact; as 60ms added to 
> >> 50 secs doesn't make a diff.
> >> 
> >> On 4/12/13 12:20 PM, Sanne Grinovero wrote:
> >>> +1
> >>> I'm going to verify that. Some index tuning options won't hurt either,
> >>> but first I'll set the blackhole to confirm that indexing is the cause.
> >>> 
> >>> On Apr 12, 2013 11:12 AM, "Bela Ban" <bban at redhat.com
> >>> <mailto:bban at redhat.com>> wrote:
> >>> 
> >>>    OK, then it isn't JGroups related. Probably some indexing work done by
> >>>    HS is on the critical path now, and wasn't before. The test probably
> >>>    returned before that work was done.
> >>> 
> >>>    On 4/12/13 11:21 AM, Ales Justin wrote:
> >>>> Still the same -- after changing this to false.
> >>>> 
> >>>> 11:20:04,013 INFO
> >>>      [org.hibernate.search.indexes.impl.DirectoryBasedIndexManager]
> >>>    (http-/192.168.30.235:8080-2) HSEARCH000168: Serialization service
> >>>    Avro SerializationProvider v1.0 being used for index
> >>>    'default_capedwarf-test__com.google.appengine.api.datastore.Entity'
> >>>> 11:20:54,730 INFO  [org.jboss.web] (ServerService Thread Pool --
> >>>    59) JBAS018224: Unregister web context: /capedwarf-tests
> >>>> 
> >>>> 
> >>>> On Apr 12, 2013, at 9:43 AM, Bela Ban <bban at redhat.com
> >>>    <mailto:bban at redhat.com>> wrote:
> >>>> 
> >>>>> You need to set enable_bundling to *false*, not true !
> >>>>> 
> >>>>> On 4/11/13 9:13 PM, Ales Justin wrote:
> >>>>>> Although this change fixes query lookup,
> >>>>>> it adds horrible performance:
> >>>>>> 
> >>>>>> Running CapeDwarf cluster QueryTest:
> >>>>>> 
> >>>>>> with HSEARCH-1296
> >>>>>> 
> >>>>>> 21:00:27,188 INFO
> >>>      [org.hibernate.search.indexes.impl.DirectoryBasedIndexManager]
> >>>    (http-/192.168.1.102:8080-1) HSEARCH000168: Serialization service
> >>>    Avro SerializationProvider v1.0 being used for index
> >>>    'default_capedwarf-test__com.google.appengine.api.datastore.Entity'
> >>>>>> 21:01:17,911 INFO  [org.jboss.web] (ServerService Thread Pool
> >>>    -- 49) JBAS018224: Unregister web context: /capedwarf-tests
> >>>>>> 
> >>>>>> 50sec
> >>>>>> 
> >>>>>> old 4.2.0.Final HS
> >>>>>> 
> >>>>>> 21:08:19,988 INFO
> >>>      [org.hibernate.search.indexes.impl.DirectoryBasedIndexManager]
> >>>    (http-/192.168.1.102:8080-2) HSEARCH000168: Serialization service
> >>>    Avro SerializationProvider v1.0 being used for index
> >>>    'default_capedwarf-test__com.google.appengine.api.datastore.Entity'
> >>>>>> 21:08:20,829 INFO  [org.jboss.web] (ServerService Thread Pool
> >>>    -- 49) JBAS018224: Unregister web context: /capedwarf-tests
> >>>>>> 
> >>>>>> 841ms
> >>>>>> 
> >>>>>> ---
> >>>>>> 
> >>>>>> I added
> >>>>>> 
> >>>>>>                     <property
> >>>    name="enable_bundling">true</property>
> >>>>>> 
> >>>>>> to AS jgroups transport config, but no improvement.
> >>>>>> 
> >>>>>> Any (other) idea?
> >>>>>> 
> >>>>>> -Ales
> >>>>>> 
> >>>>>> 
> >>>>>> _______________________________________________
> >>>>>> infinispan-dev mailing list
> >>>>>> infinispan-dev at lists.jboss.org
> >>>    <mailto:infinispan-dev at lists.jboss.org>
> >>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>>>> 
> >>>>> 
> >>>>> --
> >>>>> Bela Ban, JGroups lead (http://www.jgroups.org)
> >>>>> _______________________________________________
> >>>>> infinispan-dev mailing list
> >>>>> infinispan-dev at lists.jboss.org
> >>>    <mailto:infinispan-dev at lists.jboss.org>
> >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>> 
> >>>> 
> >>>> _______________________________________________
> >>>> infinispan-dev mailing list
> >>>> infinispan-dev at lists.jboss.org
> >>>    <mailto:infinispan-dev at lists.jboss.org>
> >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>>> 
> >>> 
> >>>    --
> >>>    Bela Ban, JGroups lead (http://www.jgroups.org)
> >>>    _______________________________________________
> >>>    infinispan-dev mailing list
> >>>    infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
> >>>    https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>> 
> >>> 
> >>> 
> >>> _______________________________________________
> >>> infinispan-dev mailing list
> >>> infinispan-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >>> 
> >> 
> >> -- 
> >> Bela Ban, JGroups lead (http://www.jgroups.org)
> >> _______________________________________________
> >> infinispan-dev mailing list
> >> infinispan-dev at lists.jboss.org
> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> 

> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev