Re: [infinispan-dev] Is anyone else experiencing JGRP-1675

I've seen StateTransferLargeObjectTest hang on my machine with all OOB threads waiting in FlowControl$Credit.decrementIfEnoughCredits, but I thought that was because the JGroups internal thread pool wasn't enabled in the test configuration. Now that I've seen it's enabled by default, I think it could be a JGroups problem after all.

I'll enable TRACE logging for org.jgroups and hopefully I'll reproduce it again. For the record, the test suite is running with TCP, and the internal thread pool has queueing enabled by default, so I don't think any REPLENISH message could be lost.

Cheers Dan On Fri, Oct 11, 2013 at 4:12 PM, Radim Vansa <rvansa(a)redhat.com <mailto:rvansa@redhat.com>> wrote: I remember seeing that mostly on UDP, but I've had an issue with this over TCP as well on the cross-site link with multiple site masters (synchronous 1pc xs replication). Radim On 10/11/2013 02:51 PM, Ray Tsang wrote: > Udp or tcp? > > On Oct 11, 2013, at 8:41, Radim Vansa <rvansa(a)redhat.com <mailto:rvansa@redhat.com>> wrote: > >> Hi, >> >> since Infinispan moved to JGroups 3.4, we're experiencing occassional >> deadlocks in some tests - most of threads that send anything over >> JGroups are waiting in JGroups' FlowControl.decrementCredits. The >> problem sometimes goes away after several seconds, but it produces some >> ugly spikes in our througput/response time charts. Originally this >> affected just some RadarGun tests but this is appearing in some >> client-server tests as well (we've recently investigated an issue where >> this appeared in a regular soak test). >> I was looking into that [1] for some time but haven't really figured out >> the cause. The workaround is to set up MFC and UFC credits high enough >> (I use 10M) and stuff works then. I was trying to reproduce that on pure >> JGroups, but unsuccessfully. >> I am not asking anyone to dig into that, but I wanted to know whether QA >> is alone experiencing that or if there are more of us. >> >> Radim >> >> [1] https://issues.jboss.org/browse/JGRP-1675 >> >> -- >> Radim Vansa <rvansa(a)redhat.com <mailto:rvansa@redhat.com>> >> JBoss DataGrid QA >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Radim Vansa <rvansa(a)redhat.com <mailto:rvansa@redhat.com>> JBoss DataGrid QA _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org <mailto:infinispan-dev@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev