[jbossts-issues] [JBoss JIRA] Commented: (JBTM-611) performance tune transaction reaper

[Jonathan Halliday (JIRA)]

    [ https://jira.jboss.org/jira/browse/JBTM-611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12482730#action_12482730 ] 

Jonathan Halliday commented on JBTM-611:
----------------------------------------

Although performance is now good for small number of threads, the proportional cost of the reaper rises steeply as the number of threads does, mainly due to increased lock contention. The cost of the reaper is essentially proportional to hte number of tx being created/terminated. Instead we would like the cost to be proportional to the number of tx timing out. We can get closer to this ideal by delaying the insert of new tx into the reaper queue if they would timeout after the earlier tx currently in the queue. That way we may never have to pay the cost of inserting them at all, if they get terminated and hence removed before that earlier timeout is reached. On the down side certain errors won't be caught at inert time, because the insert is happening asynchronously, but we can live with that.

Checking in prototype for this change, to be sanity checked by adinn.

> performance tune transaction reaper
> -----------------------------------
>
>                 Key: JBTM-611
>                 URL: https://jira.jboss.org/jira/browse/JBTM-611
>             Project: JBoss Transaction Manager
>          Issue Type: Task
>      Security Level: Public(Everyone can see) 
>          Components: Transaction Core
>    Affects Versions: 4.7.0
>            Reporter: Jonathan Halliday
>            Assignee: Andrew Dinn
>             Fix For: 4.8.0
>
>
> In situations where a large number of tx are being created and destroyed concurrently, the TransactionReaper represents a potential bottleneck due to the necessary synchronization. For most worklolads this is insignificant as it is dominated by the objectstore I/O. However, for tx scenarios that don't write the store (1PC, local resources, etc) it's a potential problem, so it's worth tuning a bit.
> There are some things we can do to reduce the workload inside the synchronized scope.
> 1- ReaperElements are kept in a sorted list, by expiry time. To impose the total ordering required by SortedSet, items with the same expire time (to the nearest ms) are distinguished by Uid. This is an expensive comparison. Introduce a sequence number or other cheap (i.e. int) element to the sort order so as to lessen the comparison (i.e. set insert/remove) cost.
> 2 - When a new tx is inserted into the reaper queue, we can avoid waking the reaper and having it recalculate the sleep interval, provided the new tx is not at the head of the queue i.e. does not have an expire time earlier than the existing wakeup time.
> 3 - Some debug logging is performed in synchronized blocks. Whilst this should not be removed, we can ensure it correctly checks the log threshold configuration, as generating log output statements unnecessarily is expensive
> 4 - The transactions and timeouts collections are currently synchronized Collections. Since almost all access to them is guarded by a lock on the TransactionReaper instance, they could be replaced by unsynchronized versions with a little work. This may or may not be helpful.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://jira.jboss.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira