[infinispan-issues] [JBoss JIRA] Commented: (ISPN-691) ReplicationQueue has an out-of-order issue

[craig bomba (JIRA)]

    [ https://jira.jboss.org/browse/ISPN-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558559#action_12558559 ] 

craig bomba commented on ISPN-691:
----------------------------------

Both cases that this can occur.  One case is max size hit by a worker thread while the background timer thread expires and sends newer items ahead of the work on the worker thread.  The other case is 2 threads maxing the queue but they get sent out of order (likely to occur only during heavy load).  I have a test that was executed on Sept 3rd that I uploaded to: http://www.megaupload.com/?d=45E02OEC  To make things easier to search in those logs I have uploaded a diagnosis of the problem with 3 keys that were problematic and show the problem (you can search on those 3 keys in the megaupload logs).  The diagnosis file is at: http://pastebin.mozilla.org/823172

Finally, will create a JIRA for this and post another comment

> ReplicationQueue has an out-of-order issue
> ------------------------------------------
>
>                 Key: ISPN-691
>                 URL: https://jira.jboss.org/browse/ISPN-691
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Core API
>    Affects Versions: 4.1.0.Final, 4.2.0.ALPHA1, 4.2.0.ALPHA2
>            Reporter: craig bomba
>            Assignee: craig bomba
>
> The ReplicationQueue has an exposure to distributing items out of order.  The ReplicationQueue may flush items in either of 2 ways.  One way is the background thread provided by a ThreadPoolExecutor flushing on a timer.  The other is via the current thread that does a put (which calls ReplicationQueue.add).  In the case of the call to add if it hits the max size (set in the config by replQueueMaxElements) then items may get flushed to other nodes out of order.  This is not evident when a test case only includes puts (new items or updated items in a cache).  Your test must include removals to expose this concern.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira