Hi guys, Given all the compiling problems we had since we've split in multiple github repos (server, stores and embedded) makes me think that the split wasn't such a great idea after all( and that I was hmm, wrong). Shall we move everything back into a single repo? We can still keep different CI runs for cache stores, server etc, but at least all this builds will compile everything. wdyt? Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org) _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On Nov 15, 2013, at 2:02 PM, Sanne Grinovero <sanne(a)infinispan.org&gt; wrote: > +1 > (as I've always been, damn my "pessimistic" opinions) > > However there is an alternative. If I recall correctly, one of your > motivations for the split was to not keep maintaining all the > non-essential components. > I really like that idea, I just think it was implemented the wrong way. > > If you identify which components are non-essential (and you don't need > them on the release day), you should keep them out but unlock them to > a different version numbering, and especially never ever depend on > snapshot versions. > > Let's try define the requirements: > - any project should compile successfully out of the box > - last minute "surprises" before a release needs to be done is not manageable > > You could get this easily if you allow the projects which are in a > different repository to lag behind: you can't require to release those > other projects each time you release an Infinispan "core" version. > Which inherently implies that if you have something which is essential > for a core release, it needs to be moved back to catch failures early > on and keep them in perfect sync. > > Other module maintainers would then catch up on compatibility breaking > changes of Infinispan core as (and when) they can. We can of course > have a goal of keeping aligned quickly, but it should be optional > (i.e. non-blocking for innovation and progress on the core module). + 1 to all said above and Mircea's suggestion. The thing is that Infinispan Server, and a subset of the cache stores really need to be released at the same time as Infinispan core. All those modules would benefit from living in same repo, making the compilation, CI and release much easier. Some examples: Level DB store needs to be brought back, whereas Cassandra cache store can live on its own. > It's perfectly doable, all our other projects depending on Infinispan > have lived well so far, with the occasional pain to fix some backwards > compatibility issue but that's part of the dance. For example Search > depends on Hibernate ORM, JGroups and Infinispan.. we try to catch > backwards compatibility with ad-hoc CI jobs but it's not going to > catch all problems, nor it's our intention to "block" other projects > from innovating: it's rather for us to be aware of possible problems > asap.. but none of these problems are allowed to prevent us to release > early and often. > > [BTW the reason the Search case isn't perfect is because we target > specific platforms like AS/Wildfly, not necessarily the latest > Infinispan] > > Cheers, > Sanne > > > On 15 November 2013 12:43, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >> Hi guys, >> >> Given all the compiling problems we had since we've split in multiple github repos (server, stores and embedded) makes me think that the split wasn't such a great idea after all( and that I was hmm, wrong). Shall we move everything back into a single repo? We can still keep different CI runs for cache stores, server etc, but at least all this builds will compile everything. >> >> wdyt? >> >> Cheers, >> -- >> Mircea Markus >> Infinispan lead (www.infinispan.org) >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño galder(a)redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I've created ISPN-3728 to track this. On Nov 18, 2013, at 2:16 PM, Tristan Tarrant <ttarrant(a)redhat.com&gt; wrote: > +1, also only the gui-demo should be in the main repo. All other > examples/demos should be moved to quickstarts or their own repos. > > Tristan > > On 11/18/2013 09:15 AM, Galder Zamarreño wrote: >> On Nov 15, 2013, at 2:02 PM, Sanne Grinovero <sanne(a)infinispan.org&gt; wrote: >> >>> +1 >>> (as I've always been, damn my "pessimistic" opinions) >>> >>> However there is an alternative. If I recall correctly, one of your >>> motivations for the split was to not keep maintaining all the >>> non-essential components. >>> I really like that idea, I just think it was implemented the wrong way. >>> >>> If you identify which components are non-essential (and you don't need >>> them on the release day), you should keep them out but unlock them to >>> a different version numbering, and especially never ever depend on >>> snapshot versions. >>> >>> Let's try define the requirements: >>> - any project should compile successfully out of the box >>> - last minute "surprises" before a release needs to be done is not manageable >>> >>> You could get this easily if you allow the projects which are in a >>> different repository to lag behind: you can't require to release those >>> other projects each time you release an Infinispan "core" version. >>> Which inherently implies that if you have something which is essential >>> for a core release, it needs to be moved back to catch failures early >>> on and keep them in perfect sync. >>> >>> Other module maintainers would then catch up on compatibility breaking >>> changes of Infinispan core as (and when) they can. We can of course >>> have a goal of keeping aligned quickly, but it should be optional >>> (i.e. non-blocking for innovation and progress on the core module). >> + 1 to all said above and Mircea's suggestion. >> >> The thing is that Infinispan Server, and a subset of the cache stores really need to be released at the same time as Infinispan core. All those modules would benefit from living in same repo, making the compilation, CI and release much easier. Some examples: Level DB store needs to be brought back, whereas Cassandra cache store can live on its own. >> >>> It's perfectly doable, all our other projects depending on Infinispan >>> have lived well so far, with the occasional pain to fix some backwards >>> compatibility issue but that's part of the dance. For example Search >>> depends on Hibernate ORM, JGroups and Infinispan.. we try to catch >>> backwards compatibility with ad-hoc CI jobs but it's not going to >>> catch all problems, nor it's our intention to "block" other projects >>> from innovating: it's rather for us to be aware of possible problems >>> asap.. but none of these problems are allowed to prevent us to release >>> early and often. >>> >>> [BTW the reason the Search case isn't perfect is because we target >>> specific platforms like AS/Wildfly, not necessarily the latest >>> Infinispan] >>> >>> Cheers, >>> Sanne >>> >>> >>> On 15 November 2013 12:43, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >>>> Hi guys, >>>> >>>> Given all the compiling problems we had since we've split in multiple github repos (server, stores and embedded) makes me think that the split wasn't such a great idea after all( and that I was hmm, wrong). Shall we move everything back into a single repo? We can still keep different CI runs for cache stores, server etc, but at least all this builds will compile everything. >>>> >>>> wdyt? >>>> >>>> Cheers, >>>> -- >>>> Mircea Markus >>>> Infinispan lead (www.infinispan.org) >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Galder Zamarreño >> galder(a)redhat.com >> twitter.com/galderz >> >> Project Lead, Escalante >> http://escalante.io >> >> Engineer, Infinispan >> http://infinispan.org >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev Cheers, -- Mircea Markus Infinispan lead (www.infinispan.org) _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On 19.11.2013 09:59, Dan Berindei wrote: > BTW, Maven rebuilds every module in the reactor every time you run a > build. On my machine, it takes 6 minutes to build everything in the > Infinispan reactor. Would you really want to wait 6 minutes every time you > want to test something in the server? > Actually, we often end up doing that cos we first need to build ISPN core and then the server which is dependent on ISPN core. It happens quite often that ISPN server works only with latest version of ISPN core. So if I understand it correctly, the ISPN server would be integrated in the main ISPN repository. I guess it would be a specific module which you could build on its own (and also run its tests). This would cover the case when you don't need to build ISPN core in order to run the server. Or did I miss something?

On Nov 19, 2013, at 10:23 AM, Dan Berindei <dan.berindei(a)gmail.com&gt; wrote: > > On Tue, Nov 19, 2013 at 11:21 AM, Martin Gencur <mgencur(a)redhat.com&gt; wrote: > On 19.11.2013 09:59, Dan Berindei wrote: > BTW, Maven rebuilds every module in the reactor every time you run a build. On my machine, it takes 6 minutes to build everything in the Infinispan reactor. Would you really want to wait 6 minutes every time you want to test something in the server? > > Actually, we often end up doing that cos we first need to build ISPN core and then the server which is dependent on ISPN core. It happens quite often that ISPN server works only with latest version of ISPN core. > > So if I understand it correctly, the ISPN server would be integrated in the main ISPN repository. I guess it would be a specific module which you could build on its own (and also run its tests). This would cover the case when you don't need to build ISPN core in order to run the server. Or did I miss something? > > > I'm pretty sure it wouldn't be a single module… The parent, X, can be a single module (or maven project), and then it can list the modules to build, e.g. the current list of Infinispan Server submodules: <module>versions</module> <module>jgroups</module> <module>infinispan</module> <module>integration</module> <module>build</module> <module>management/server-rhq-plugin</module> Given that Infinispan Server already uses <groupId>org.infinispan.server</groupId>, it might make sense for X to be `server`. Thoughts?

> _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño galder(a)redhat.com twitter.com/galderz Project Lead, Escalante http://escalante.io Engineer, Infinispan http://infinispan.org

Hi guys, Given all the compiling problems we had since we've split in multiple github repos (server, stores and embedded) makes me think that the split wasn't such a great idea after all( and that I was hmm, wrong). Shall we move everything back into a single repo? We can still keep different CI runs for cache stores, server etc, but at least all this builds will compile everything. wdyt? Cheers,

On 19 Nov 2013, at 22:44, Manik Surtani <manik(a)infinispan.org&gt; wrote: Can't this be achieved by checking out and building all relevant repos? This could be scripted. > On 15 November 2013 04:43, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: > Hi guys, > > Given all the compiling problems we had since we've split in multiple github repos (server, stores and embedded) makes me think that the split wasn't such a great idea after all( and that I was hmm, wrong). Shall we move everything back into a single repo? We can still keep different CI runs for cache stores, server etc, but at least all this builds will compile everything. > > wdyt? > > Cheers, > -- > Mircea Markus > Infinispan lead (www.infinispan.org) > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

As in, for easy refactoring? True that helps. But it doesn't help you with modular releases. On 19 November 2013 15:00, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: > > That only half-solves the problem: having everything in the same IDE at > the same time is more preentive. > > On 19 Nov 2013, at 22:44, Manik Surtani <manik(a)infinispan.org&gt; wrote: > > Can't this be achieved by checking out and building all relevant repos? > This could be scripted. > > > On 15 November 2013 04:43, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >> >> Hi guys, >> >> Given all the compiling problems we had since we've split in multiple >> github repos (server, stores and embedded) makes me think that the split >> wasn't such a great idea after all( and that I was hmm, wrong). Shall we >> move everything back into a single repo? We can still keep different CI runs >> for cache stores, server etc, but at least all this builds will compile >> everything. >> >> wdyt? >> >> Cheers, >> -- >> Mircea Markus >> Infinispan lead (www.infinispan.org) >> >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Hi Manik, the problem the team has been facing past week is that the other modules are depending on a -SNAPSHOT version, and after having done some final touches to the modules in the core repo, we then find out that the other modules don't build anymore. This happened several times, and while it's normal for dependant projects to occasionally break, it's not so nice that the current structure doesn't allow for it to be spotted, or managed time-wise: it is important to be able to release at any point in time, but having potential "surprises" in different modules makes it hard to predict how long it will take to get to a stable tag. My suggestion is not to necessarily return all external modules in the core repository, but rather to make sure the modules which are separately also have the benefit of a different release cycle, so that you can always release any module at any time. This could mean that they have different version numbers but not necessarily: we could make it a convention to release compatible optional modules using the same version. For example - the optional CacheStore implementation for Infinispan 6.0.0.Final are released also as 6.0.0.Final but not _necessarily_ on the same day of the core module. Probably works best to have a same major,minor number, and be flexible with micro releases. You'd never want to depend on a snapshot version unless it's a module in your same repo. As mentioned, if a new Infinispan "improvement" breaks the Hibernate Search build, I have the option to decide to not upgrade; you don't have this flexibility if you're depending on snapshots. Cheers, Sanne On 19 November 2013 23:17, Manik Surtani <manik(a)infinispan.org&gt; wrote: > As in, for easy refactoring? True that helps. But it doesn't help you with > modular releases. > > > On 19 November 2013 15:00, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >> >> That only half-solves the problem: having everything in the same IDE at >> the same time is more preentive. >> >> On 19 Nov 2013, at 22:44, Manik Surtani <manik(a)infinispan.org&gt; wrote: >> >> Can't this be achieved by checking out and building all relevant repos? >> This could be scripted. >> >> >> On 15 November 2013 04:43, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >>> >>> Hi guys, >>> >>> Given all the compiling problems we had since we've split in multiple >>> github repos (server, stores and embedded) makes me think that the split >>> wasn't such a great idea after all( and that I was hmm, wrong). Shall we >>> move everything back into a single repo? We can still keep different CI runs >>> for cache stores, server etc, but at least all this builds will compile >>> everything. >>> >>> wdyt? >>> >>> Cheers, >>> -- >>> Mircea Markus >>> Infinispan lead (www.infinispan.org) >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

On Nov 20, 2013, at 11:03 AM, Dan Berindei <dan.berindei(a)gmail.com&gt; wrote: > When it comes to IDEs, I'd rather remove some more modules from the main project in order to make it more responsive and easier to use. Launching a test from IntelliJ takes minutes to build (or parse, or analyze, or whatever) everything, when it works. Extra dependencies on old versions in the compatibility modules broke the "Analyze stacktrace" feature, which used to be very nice for analyzing logs. ^ Isnt't that a problem with the IDE which is messing up the code that's really in use?

Hi Manik, the problem the team has been facing past week is that the other modules are depending on a -SNAPSHOT version, and after having done some final touches to the modules in the core repo, we then find out that the other modules don't build anymore. This happened several times, and while it's normal for dependant projects to occasionally break, it's not so nice that the current structure doesn't allow for it to be spotted, or managed time-wise: it is important to be able to release at any point in time, but having potential "surprises" in different modules makes it hard to predict how long it will take to get to a stable tag. My suggestion is not to necessarily return all external modules in the core repository, but rather to make sure the modules which are separately also have the benefit of a different release cycle, so that you can always release any module at any time.

This could mean that they have different version numbers but not necessarily: we could make it a convention to release compatible optional modules using the same version. For example - the optional CacheStore implementation for Infinispan 6.0.0.Final are released also as 6.0.0.Final but not _necessarily_ on the same day of the core module. Probably works best to have a same major,minor number, and be flexible with micro releases. You'd never want to depend on a snapshot version unless it's a module in your same repo. As mentioned, if a new Infinispan "improvement" breaks the Hibernate Search build, I have the option to decide to not upgrade; you don't have this flexibility if you're depending on snapshots. Cheers, Sanne On 19 November 2013 23:17, Manik Surtani <manik(a)infinispan.org&gt; wrote: > As in, for easy refactoring? True that helps. But it doesn't help you with > modular releases. > > > On 19 November 2013 15:00, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >> >> That only half-solves the problem: having everything in the same IDE at >> the same time is more preentive. >> >> On 19 Nov 2013, at 22:44, Manik Surtani <manik(a)infinispan.org&gt; wrote: >> >> Can't this be achieved by checking out and building all relevant repos? >> This could be scripted. >> >> >> On 15 November 2013 04:43, Mircea Markus <mmarkus(a)redhat.com&gt; wrote: >>> >>> Hi guys, >>> >>> Given all the compiling problems we had since we've split in multiple >>> github repos (server, stores and embedded) makes me think that the split >>> wasn't such a great idea after all( and that I was hmm, wrong). Shall we >>> move everything back into a single repo? We can still keep different CI runs >>> for cache stores, server etc, but at least all this builds will compile >>> everything. >>> >>> wdyt? >>> >>> Cheers, >>> -- >>> Mircea Markus >>> Infinispan lead (www.infinispan.org) >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev