[ https://issues.jboss.org/browse/JBIDE-10974?page=com.atlassian.jira.plugi... ] Denis Golovin commented on JBIDE-10974: --------------------------------------- It is not really major problem to spend time for :) Slowness of our builds is related to: 1. Metadata downloading and analyzing (for example common component needs only TP and tests, but it is anyway use nightly composit repository with metadata for all components, another example is using remote and local version of target platform at the same time and etc) 2. Building parent pom in every single project even if it is not required anymore (this is minor) 3. Checkout from SVN is extremely slow and it happens pretty often because of slaves cleanup 4. Using very generic ant pattern for searching files like **/TEXT*.xml, force hudson to go over all project tree which is huge and takes time, I prefer use something more determined like sources/tests/*/target/surefire-reports/TEST*.xml 5. Two steps build per component (first build everything without tests, publish, then build tests and run them) Could be good to have one test plugin that aggregates all TestSuites from component test plugins to run them at once. 6. Extra component bits publishing. When component build triggered by component build it depends on (like common triggers jst) bits get published even there were no changes 7. All tests are UI tests even for *.core plugins. It happens because most of our *.core plugins actually references eclipse.ui packages where workbench instance is required 8. Mixing of shell, ant and maven instead of using maven only with ant calls form it where it is really needed (minor one, but really annoying when you need run it on windows and have to install cygwin) 9. Every change in build component triggers full rebuild for all components 10. Aggregation builds happen to often and it reduces number of slaves available for component builds. That's what I remember from top of my head -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JBIDE-10974?page=com.atlassian.jira.plugi... ] Nick Boldt edited comment on JBIDE-10974 at 3/4/12 11:33 AM: ------------------------------------------------------------- 10. Aggregation happens less than it used to because it's now set to "block when upstream is building"; as a result, most aggregations happen as a result of several upstream jobs asking it to be run. The opposite problem now occurs: people complain that aggregation doesn't happen often enough. :) 9. This could be fixed by instead of watching the /build folder in SVN for changes, we simply fetch it using `svn co` or `svn up` as a prelim step in the job. Perhaps taking this a step further, we could fetch the whole SVN tree and store it outside the workspaces so that jobs would simply re-use local shared space instead of clean-fetching everything every time they run on a new slave. See JBIDE-11155. 8. Shell is currently only used for bootstrapping Jenkins and publishing -- does not affect offline builds. AFAIK, none of the Ant scripts still depend on any <exec> calls to something platform-specific except: build/aggregate/build.xml -- see JBIDE-11156. Even in that case, you can skip the step that needs wget with *-Dcollect.zips.skip=true*; therefore you don't need cygwin anymore to run builds offline. 7. This should definitely be changed so that core tests don't need UI. *Denis, Can you open jobs for each component that's doing this wrong?* 6. This is necessary so that upstream API changes will trickle down correctly. Otherwise Maven fails because of changes in seam, vpe, jst, common, etc. 5. Building all the tests separately in another job is done in the .tests job, which can take >6hrs. But each component needs faster results than that, so they run their tests personally too. 4. This should be fixed. See JBIDE-11158 3. See JBIDE-11155. 2. Takes about 30 seconds - not significant. But really, we should be using Nexus here anyway. See JBIDE-11157 1. *Denis, what do you propose we do differently here? Jenkins jobs should ONLY use local profiles in place of remote ones?* 0. Randomly crashing slaves IS a problem, but it's a hard one to diagnose and fix. So, yes, there are more "low hanging fruit" options available. :) was (Author: nickboldt): 10. Aggregation happens less than it used to because it's now set to "block when upstream is building"; as a result, most aggregations happen as a result of several upstream jobs asking it to be run. The opposite problem now occurs: people complain that aggregation doesn't happen often enough. :) 9. This could be fixed by instead of watching the /build folder in SVN for changes, we simply fetch it using `svn co` or `svn up` as a prelim step in the job. Perhaps taking this a step further, we could fetch the whole SVN tree and store it outside the workspaces so that jobs would simply re-use local shared space instead of clean-fetching everything every time they run on a new slave. See JBIDE-11155. 8. Shell is currently only used for bootstrapping Jenkins and publishing -- does not affect offline builds. AFAIK, none of the Ant scripts still depend on any <exec> calls to something platform-specific except: build/aggregate/build.xml -- see JBIDE-11156. Even in that case, you can skip the step that needs wget with *-Dcollect.zips.skip=true*; therefore you don't need cygwin anymore to run builds offline. 7. This should definitely be changed so that core tests don't need UI. *Denis, Can you open jobs for each component that's doing this wrong?* 6. This is necessary so that upstream API changes will trickle down correctly. Otherwise Maven fails because of changes in seam, vpe, jst, common, etc. 5. Building all the tests separately in another job is done in the .tests job, which can take >6hrs. But each component needs faster results than that, so they run their tests personally too. 4. This should be fixed. See JBIDE-11158 3. See JBIDE-11155. 2. Takes about 30 seconds - not significant. But really, we should be using Nexus here anyway. See JBIDE-11157 1. Denis, what do you propose we do differently here? Jenkins jobs should ONLY use local.* profiles in place of remote ones? 0. Randomly crashing slaves IS a problem, but it's a hard one to diagnose and fix. So, yes, there are more "low hanging fruit" options available. :) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JBIDE-10974?page=com.atlassian.jira.plugi... ] Mickael Istria updated JBIDE-10974: ----------------------------------- Fix Version/s: 3.3.0.Beta3 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JBIDE-10974?page=com.atlassian.jira.plugi... ] Mickael Istria commented on JBIDE-10974: ---------------------------------------- Xvfb is working fine. However, it is difficult to get conclusions on how much time to we gain/loose since builds do not happen on same slaves: https://hudson.qa.jboss.com/hudson/view/DevStudio/view/DevStudio_Trunk/jo... https://hudson.qa.jboss.com/hudson/view/DevStudio/view/DevStudio_Trunk/jo... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JBIDE-10974?page=com.atlassian.jira.plugi... ] Mickael Istria closed JBIDE-10974. ---------------------------------- Resolution: Done Applied on Trunk and atable_branch jobs. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira

[ https://issues.jboss.org/browse/JBIDE-10974?page=com.atlassian.jira.plugi... ] Mickael Istria commented on JBIDE-10974: ---------------------------------------- Indeed, switching back from Xvfb to Xvnc when necessary is possible. But we also make husge efforts to get build behave locally almost the same way they behave on CI, so I hope thet in case a build is blocked on CI, it is always possible to reproduce the issue locally, which is more convenient for debugging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jboss.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira