[JBoss JIRA] (JBTM-1808) Investigate why QA tests take 5hrs to run per ORB
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1808?page=com.atlassian.jira.plugin.... ]
Paul Robinson updated JBTM-1808:
--------------------------------
Assignee: Tom Jenkinson (was: Amos Feng)
> Investigate why QA tests take 5hrs to run per ORB
> -------------------------------------------------
>
> Key: JBTM-1808
> URL: https://issues.jboss.org/browse/JBTM-1808
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: Build System, Testing
> Reporter: Paul Robinson
> Assignee: Tom Jenkinson
> Priority: Minor
> Fix For: 5.0.0.Final
>
>
> Our full build and test suite takes 12hrs to run. This is causing severe strain on our CI resources.
> Out of this 12hrs, 10hrs is consumed by the QA test suite (5hrs per orb). It would be good to understand where this time is spent so that we can hopefully bring the test time down.
> If there is no reasonable way to bring the timing down, we could chop the tests up into 5 groups (per orb) taking 1hr each. This would make better use of our CI cluster during quiet periods, but will not eleviate the load during busy periods. It is also likely to be a poor usage of resources as I suspect the CPU utilisation of each job will be low.
> I suspect the problem is that we have a lot of Thread.sleeps in the tests. Therefore we are likely to see long tests with low resource (CPU) utilisation. However, further investigation is required to understand where the time is spent.
> As an initial investigation, it would be good to profile the CPU utilisation on one of the CI nodes (take it offline) whilst running these tests. Also, it would be handy to find out how much time is spent in a Thread.sleep call (assuming that is where the time is spent).
> My understanding is that Thread.sleep is used to wait for certain background tasks to complete. It's likely that Byteman could be used to replace the sleeps. However, due to the number of tests this may not be feasible. Alternatively, we may discover that there is a common set of Byteman rules that could be applied to the majority of tests, which may prove "good enough".
> Also, we may be able to run these tests in parallel. This could drive up CPU utilisation, but would probably also interfere with the timings of the tests meaning that the current thread.sleeps are no longer adequate.
> To summarise, this is a general task to investigate why the tests take so long and to also come up with some candidate solutions to bring the total test duration down.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-1527) scripts/rebase-jbossas-branch.sh pushes empty 'A' and 'M' files when merging
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1527?page=com.atlassian.jira.plugin.... ]
Paul Robinson resolved JBTM-1527.
---------------------------------
Fix Version/s: 5.0.0.M5
(was: 5.0.0.Final)
Resolution: Out of Date
No longer applies as we don't attempt to automate conflict resolution anymore.
> scripts/rebase-jbossas-branch.sh pushes empty 'A' and 'M' files when merging
> ----------------------------------------------------------------------------
>
> Key: JBTM-1527
> URL: https://issues.jboss.org/browse/JBTM-1527
> Project: JBoss Transaction Manager
> Issue Type: Bug
> Security Level: Public(Everyone can see)
> Components: Build System
> Reporter: Paul Robinson
> Assignee: Paul Robinson
> Priority: Minor
> Fix For: 5.0.0.M5
>
> Original Estimate: 1 hour
> Remaining Estimate: 1 hour
>
> The portion of the scrip that resolves merge conflicts seems to be thinking the 'A' and 'M' attributes of the 'git status' output are files and manages to add them.
> I think we can only debug this issue when there's a merge conflict.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-1763) Consider running the QA tests in parrallel
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1763?page=com.atlassian.jira.plugin.... ]
Paul Robinson commented on JBTM-1763:
-------------------------------------
I don't think this will work as the times each test takes to run will be unpredictable and therefore will affect the timings required for the Thread.sleeps.
> Consider running the QA tests in parrallel
> ------------------------------------------
>
> Key: JBTM-1763
> URL: https://issues.jboss.org/browse/JBTM-1763
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: Testing
> Reporter: Paul Robinson
> Assignee: Tom Jenkinson
> Priority: Minor
> Fix For: 5.0.0.Final
>
>
> The QA tests currently take 5hrs to run per orb. This constitutes 83% of our CI run time.
> My understanding is that the time is mostly taken up in Thread.sleep. The test times range from 1 to 99seconds. So there are no major hotspots. The problem is that there are so many tests.
> I suspect we could run these tests in parallel to make better utilisation of resources.
> The tests don't run in an AS, but they do acquire ports. There may be other restrictions that make running them in parallel hard or impossible.
> It would be easier to simply carve up the tests into, say 5 parts per orb and have CI do the parallelism. However, this will consume more CI nodes than if we ran them in parallel on a single node.
> I think it would be worth investigating if we can run the tests in n batches (say n=10 initially, but should be configurable). Where each batch runs with a different port configuration. This may prove too disruptive to the tests,
> and also more bother than it is worth. Maybe we could do an initial investigation to see if this has legs?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-970) Allow a ServiceRequest to be annotated as read only
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-970?page=com.atlassian.jira.plugin.s... ]
Paul Robinson resolved JBTM-970.
--------------------------------
Assignee: Paul Robinson
Fix Version/s: 5.0.0.M5
(was: 5.0.0.Final)
Resolution: Out of Date
No longer applicable for the Compensations API
> Allow a ServiceRequest to be annotated as read only
> ---------------------------------------------------
>
> Key: JBTM-970
> URL: https://issues.jboss.org/browse/JBTM-970
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: TXFramework
> Reporter: Paul Robinson
> Assignee: Paul Robinson
> Labels: TXFramework
> Fix For: 5.0.0.M5
>
>
> The RequestType attribute on the ServiceRequest annotation identifies whether the service request is always read only as far as transactional modifications are concerned or, alternatively, that it may make changes to transactional data. in the latter case the service request method can indicate a read-only outcome by invoking the readOnly method of an injected control.
> {code}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.METHOD)
> public @interface ServiceRequest
> {
> public RequestType requestType() default RequestType.MODIFY;
> }
> {code}
> The framework does not normally register lifecycle callbacks for ServiceRequest methods whose request type is READ_ONLY (see the next paragraph for the rider which qualifies this statement). The framework does normally register callbacks for ServiceRequest methods whose request type is MODIFY, on the assumption that the method has performed transactional changes. The latter methods may inhibit registration of lifecycle methods by calling the readOnly() method of an injected transaction control.
> The reason why it is important to identify READ_ONLY methods is that they affect the way the framework performs PREPARE or COMPLETE processing (as appropriate to the protocol in question). If all ServiceRequest methods in the transaction are READ_ONLY the participant will trigger READ_ONLY or EXIT callbacks, respectively, for these lifecycle events, precluding further involvement in the transaction. If any of the ServiceRequest methods is not READ_ONLY then the framework will trigger PREPARE or COMPLETE callbacks, respectively. Clearly, it would be futile for a READ_ONLY declaration or notification to inhibit registration of callbacks which are triggered by READ_ONLY or EXIT events. So, the framework only registers READ_ONLY or EXIT lifecycle callbacks when they are triggered by READ_ONLY methods or by MODIFY methods which notify their READ_ONLY status using an injected control.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-1398) Review subsystem usage across Narayana
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1398?page=com.atlassian.jira.plugin.... ]
Paul Robinson updated JBTM-1398:
--------------------------------
Assignee: Tom Jenkinson
> Review subsystem usage across Narayana
> --------------------------------------
>
> Key: JBTM-1398
> URL: https://issues.jboss.org/browse/JBTM-1398
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: TXFramework
> Reporter: Paul Robinson
> Assignee: Tom Jenkinson
> Fix For: 5.0.0.Final
>
> Original Estimate: 2 days
> Time Spent: 1 day, 7 hours
> Remaining Estimate: 2 days, 6 hours
>
> h2. Components and their subsystem requirements:
> h5. Transactions
> * To Investigate
>
> h5. XTS
> * Bootstrap coordinator
> * Setup WS endpoints for coordinator
> * Setup WS endpoints for the participants
> * Respond to configuration from standalone-xts.xml
> h5. REST-TX
> * Boostrap the coordinator
> * Setup a REST endpoint for the coordinator
> * Setup a REST endpoint for the participants
> h5. Blacktie
> * Register MDB
> * Register MBean
> h5. STM
> * To Investigate
> h5. TXF
> * Modify the WS handler chain on application deployment
> * Registers a CDI extension
> h2. Notable Dependencies
> h5. TXF
> * Weld -> EJB3 -> Transactions (Could check that each link is valid. Check if subsys or regular dep)
> * JBossWS. Might be able to remove by putting the WS specific stuff into the XTS subsystem.
> h2. Subsystem Breakdown
> Providing we can have 'soft' dependencies it would seem favourable to have all the core features in the transactions subsystem and the optional 'transports' in additional subsystems.
> h5. Transactions
> * All current stuff
> * STM
> * TXF (Unlikely to work due to dependency on Weld for loading a portable extension)
> h5. XTS
> * All Current Stuff
> h5. RTS (new)
> h5. Blacktie (new)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-975) Allow ServiceRequest methods to have own set of lifecycle methods
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-975?page=com.atlassian.jira.plugin.s... ]
Paul Robinson resolved JBTM-975.
--------------------------------
Assignee: Paul Robinson
Fix Version/s: 5.0.0.M5
(was: 5.0.0.Final)
Resolution: Out of Date
> Allow ServiceRequest methods to have own set of lifecycle methods
> -----------------------------------------------------------------
>
> Key: JBTM-975
> URL: https://issues.jboss.org/browse/JBTM-975
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: TXFramework
> Reporter: Paul Robinson
> Assignee: Paul Robinson
> Labels: TXFramework
> Fix For: 5.0.0.M5
>
>
> By default if only one type of LifecycleClass is specified, or if the default of the current class is used, then the same LifecycleClass is registered for each @ServiceRequest method.
> Setting the attribute 'single' on @ParticipantService to 'false' allows a different instance of the LifeCycle class to be used for each @ServiceRequest method.
> {code}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.TYPE)
> public @interface ParticipantService
> {
> public boolean single() default true;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-974) Allow single lifecycle invocation for multiple ServiceRequest invocations
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-974?page=com.atlassian.jira.plugin.s... ]
Paul Robinson commented on JBTM-974:
------------------------------------
Still applies to Compensations API, but the description needs updating.
> Allow single lifecycle invocation for multiple ServiceRequest invocations
> -------------------------------------------------------------------------
>
> Key: JBTM-974
> URL: https://issues.jboss.org/browse/JBTM-974
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: TXFramework
> Reporter: Paul Robinson
> Labels: TXFramework
> Fix For: 5.0.0.Final
>
>
> By setting the 'single' attribute on a Lifecycle Handler Annotation (such as @Prepare or @Compensate) identifies whether more than one callback can be registered for the lifecycle method in response to multiple invocations of the ServiceRequest method. If it is true then the lifecycle method will be called once to the ServiceRequest method. If it is false then the lifecycle method will be called once in response to each (non-read only) call to the ServiceRequest method. The default is 'false'.
> Annotation:
> {code}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.METHOD)
> public @interface Compensate
> {
> public boolean single() default false;
> }
> {code}
> Example:
> {code}
> @ServiceRequest
> public void addToBasket()
> {
> //Add another item to the basket.
> }
> @Compensate(single=true)
> public void emptyBasket()
> {
> //Empty every item from the basket
> }
> {code}
> In this example the @ServiceRequest may be invoked many times to add many items to the basket. The @Compensate method is only needed to be invoked once as the basket can be emptied in one operation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-1398) Review subsystem usage across Narayana
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-1398?page=com.atlassian.jira.plugin.... ]
Paul Robinson commented on JBTM-1398:
-------------------------------------
I think this is resolved for now. Do you agree Tom?
> Review subsystem usage across Narayana
> --------------------------------------
>
> Key: JBTM-1398
> URL: https://issues.jboss.org/browse/JBTM-1398
> Project: JBoss Transaction Manager
> Issue Type: Task
> Security Level: Public(Everyone can see)
> Components: TXFramework
> Reporter: Paul Robinson
> Fix For: 5.0.0.Final
>
> Original Estimate: 2 days
> Time Spent: 1 day, 7 hours
> Remaining Estimate: 2 days, 6 hours
>
> h2. Components and their subsystem requirements:
> h5. Transactions
> * To Investigate
>
> h5. XTS
> * Bootstrap coordinator
> * Setup WS endpoints for coordinator
> * Setup WS endpoints for the participants
> * Respond to configuration from standalone-xts.xml
> h5. REST-TX
> * Boostrap the coordinator
> * Setup a REST endpoint for the coordinator
> * Setup a REST endpoint for the participants
> h5. Blacktie
> * Register MDB
> * Register MBean
> h5. STM
> * To Investigate
> h5. TXF
> * Modify the WS handler chain on application deployment
> * Registers a CDI extension
> h2. Notable Dependencies
> h5. TXF
> * Weld -> EJB3 -> Transactions (Could check that each link is valid. Check if subsys or regular dep)
> * JBossWS. Might be able to remove by putting the WS specific stuff into the XTS subsystem.
> h2. Subsystem Breakdown
> Providing we can have 'soft' dependencies it would seem favourable to have all the core features in the transactions subsystem and the optional 'transports' in additional subsystems.
> h5. Transactions
> * All current stuff
> * STM
> * TXF (Unlikely to work due to dependency on Weld for loading a portable extension)
> h5. XTS
> * All Current Stuff
> h5. RTS (new)
> h5. Blacktie (new)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months
[JBoss JIRA] (JBTM-969) Specify external participant lifecycle class
by Paul Robinson (JIRA)
[ https://issues.jboss.org/browse/JBTM-969?page=com.atlassian.jira.plugin.s... ]
Paul Robinson resolved JBTM-969.
--------------------------------
Assignee: Paul Robinson
Fix Version/s: 5.0.0.M5
Resolution: Out of Date
Something similar is now offered by the Compensations API
> Specify external participant lifecycle class
> --------------------------------------------
>
> Key: JBTM-969
> URL: https://issues.jboss.org/browse/JBTM-969
> Project: JBoss Transaction Manager
> Issue Type: Feature Request
> Security Level: Public(Everyone can see)
> Components: TXFramework
> Reporter: Paul Robinson
> Assignee: Paul Robinson
> Priority: Minor
> Labels: TXFramework
> Fix For: 5.0.0.M5
>
>
> This class level annotation allows the class which is the target of lifecycle handler callbacks to be specified.
> {code}
> @Retention(RetentionPolicy.RUNTIME)
> @Target(ElementType.TYPE)
> public @interface ParticipantService
> {
> public Class lifecycleClass() default Default.class;
> }
> {code}
> This field may be overridden in the ServiceRequest annotation itself. If the value of this field is Default.class then the ParticipantService class is used as the default class to resolve POJO lifecycle methods.
> {code}
> @Retention(RetentionPolicy.RUNTIME)
> @Target({ElementType.METHOD, ElementType.TYPE})
> public @interface ServiceRequest
> {
> public Class lifecycleClass() default Default.class;
> }
> {code}
> If the lifecycleClass attribute is not specified in either place, the class containing the @ServiceInvocation is assumed to also contain the associated Lifecycle Handler methods.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 7 months