Hello JBoss community, This year I want to participate in an OS project and GSoC gives me great opportunity to do so. Here is my idea as GSoC 2013 proposal: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/adamw/1 I'd be very happy to hear your opinions (I hope it's not too late). _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev

AFAIK, gradle team tells tooling that want to understand gradle build files is to use and *run* gradle-tooling-api to parse it. Don't try parse it. I believe eclipse plugin uses that. See http://www.gradle.org/docs/current/userguide/embedding.html (it's one of the reasons I'm not really happy about gradle since its dog start/slow to run on large set of projects - they keep saying that will improve though...) /max On Mon, May 06, 2013 at 07:27:20PM -0400, Lincoln Baxter, III wrote: >Hey Adam, > >Thank you so much for your proposal! We have reviewed your submission and >so far all the mentors have given it a high rating :) > >I took a look at your prototype groovy-manager project, which looks very >interesting. You are doing source parsing for groovy, which is pretty >challenging. > >While writing a parser from scratch can be a thrilling challenge, I have to >say that it can be quite a burden! We started writing our own Java-parser >for Forge, but quickly came to find that even parsing a type-safe language >is incredibly difficult. > >We ended up using the Eclipse JDT as a base implementation for our >JavaParser. > >Have you looked for any Groovy parsers that may already exist today? This >would give you a quick boost when working on this project. It may be the >case that the parser would require some usability improvements to make >integration with Forge simpler, and simpler for plugin devs, but I think it >is safe to say that a search for existing parsers is warranted. > >If a suitable parser can be found, I would suggest that we revise your >proposal to allow for several weeks to become familiar with that parser, >and then allocate the rest of the time to actually working with it to >implement the gradle support. > >I'd suggest starting here, with the Groovy Eclipse Plugin - they almost >certainly already support groovy syntax: >http://groovy.codehaus.org/Eclipse+Plugin#EclipsePlugin-KeyFeatures > >The problem with writing your own parser is that you have to maintain it :) >Better to delegate. > >~Lincoln > > > > >On Sun, May 5, 2013 at 5:05 PM, Adam Wyłuda <adamwyl92(a)gmail.com&gt; wrote: > >> Hello JBoss community, >> This year I want to participate in an OS project and GSoC gives me great >> opportunity to do so. >> Here is my idea as GSoC 2013 proposal: >> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/adamw/1 >> I'd be very happy to hear your opinions (I hope it's not too late). >> >> _______________________________________________ >> forge-dev mailing list >> forge-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/forge-dev >> > > > >-- >Lincoln Baxter, III >http://ocpsoft.org >"Simpler is better." >_______________________________________________ >forge-dev mailing list &gt;forge-dev(a)lists.jboss.org >https://lists.jboss.org/mailman/listinfo/forge-dev _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev

Hey Adam, Thank you so much for your proposal! We have reviewed your submission and so far all the mentors have given it a high rating :) I took a look at your prototype groovy-manager project, which looks very interesting. You are doing source parsing for groovy, which is pretty challenging. While writing a parser from scratch can be a thrilling challenge, I have to say that it can be quite a burden! We started writing our own Java-parser for Forge, but quickly came to find that even parsing a type-safe language is incredibly difficult. We ended up using the Eclipse JDT as a base implementation for our JavaParser. Have you looked for any Groovy parsers that may already exist today? This would give you a quick boost when working on this project. It may be the case that the parser would require some usability improvements to make integration with Forge simpler, and simpler for plugin devs, but I think it is safe to say that a search for existing parsers is warranted. If a suitable parser can be found, I would suggest that we revise your proposal to allow for several weeks to become familiar with that parser, and then allocate the rest of the time to actually working with it to implement the gradle support. I'd suggest starting here, with the Groovy Eclipse Plugin - they almost certainly already support groovy syntax: http://groovy.codehaus.org/Eclipse+Plugin#EclipsePlugin-KeyFeatures The problem with writing your own parser is that you have to maintain it :) Better to delegate. ~Lincoln On Sun, May 5, 2013 at 5:05 PM, Adam Wyłuda <adamwyl92(a)gmail.com&gt; wrote: > Hello JBoss community, > This year I want to participate in an OS project and GSoC gives me great > opportunity to do so. > Here is my idea as GSoC 2013 proposal: > http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/adamw/1 > I'd be very happy to hear your opinions (I hope it's not too late). > > _______________________________________________ > forge-dev mailing list > forge-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/forge-dev > -- Lincoln Baxter, III http://ocpsoft.org "Simpler is better." _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev

Just a warning about using groovy eclipse plugin - it historically have had a dependency on a patched Eclipse JDT meaning it's not installable on just any eclipse version. I would hope if you just need a groovy parser that you don't have a depenency on Eclipse JDT for that - but if you do then that might become tricky (might also be osgi dependencies to deal with, but that is another concern ;) If the dependency will be included into eclipse as a forge loaded lib we might avoid it all - but since forge itself also have a dependency on JDT it is something to keep an eye on for possible version mismatches. /max On Sun, May 12, 2013 at 06:00:20PM +0200, Adam Wyłuda wrote: >Hi Lincoln! >Thank you for your response, it's nice to hear that my proposal is rated >positively. > >I first thought that creating our own Groovy parser which works directly on >Groovy AST would be a good idea because that would give us more >possibilities than existing solutions, but you are right - there is a risk >that it might take more time than planned (although basic support could be >provided which could be sufficient to work on Gradle project files). > >I have found this paper which gave me some insights on problems related to >Groovy script parsing and modification: >http://groovy.ifs.hsr.ch/groovy-refactoring.pdf > >It seems that the biggest issue is how to rewrite code without changing its >formatting and how to keep comments in code. Groovy AST doesn't contain >comment nodes so if we want to use pure Groovy AST to parse it, we need to >include all whitespace and comments in our model, what is possible because >Groovy AST nodes include position in source information, so we could just >substring our source between statements, but it's hard because we need to >check thoroughly every expression as comments can appear between any nodes, >like this: >x = y + /* comment */ z > >So as you said, it's better to use existing solution so we won't reinvent >the wheel. I think that Groovy Eclipse plugin is the best candidate because >it's API is similar to JDT plugin used in JavaParser (actually Groovy >plugin uses JDT and it even seems to be a part of JDT), so it will be more >convenient for other developers with JavaParser experience to maintain it. >The are also alternatives like NetBeans or IntelliJ IDEA plugin, but I >think Eclipse plugin is sufficient enough to work effectively with Groovy >sources. > >Like you suggested I updated my proposal to give myself some time to >understand Groovy Eclipse plugin API and start working on Gradle project >support earlier. > >2013/5/7 Lincoln Baxter, III <lincolnbaxter(a)gmail.com&gt; > >> Hey Adam, >> >> Thank you so much for your proposal! We have reviewed your submission and >> so far all the mentors have given it a high rating :) >> >> I took a look at your prototype groovy-manager project, which looks very >> interesting. You are doing source parsing for groovy, which is pretty >> challenging. >> >> While writing a parser from scratch can be a thrilling challenge, I have >> to say that it can be quite a burden! We started writing our own >> Java-parser for Forge, but quickly came to find that even parsing a >> type-safe language is incredibly difficult. >> >> We ended up using the Eclipse JDT as a base implementation for our >> JavaParser. >> >> Have you looked for any Groovy parsers that may already exist today? This >> would give you a quick boost when working on this project. It may be the >> case that the parser would require some usability improvements to make >> integration with Forge simpler, and simpler for plugin devs, but I think it >> is safe to say that a search for existing parsers is warranted. >> >> If a suitable parser can be found, I would suggest that we revise your >> proposal to allow for several weeks to become familiar with that parser, >> and then allocate the rest of the time to actually working with it to >> implement the gradle support. >> >> I'd suggest starting here, with the Groovy Eclipse Plugin - they almost >> certainly already support groovy syntax: >> http://groovy.codehaus.org/Eclipse+Plugin#EclipsePlugin-KeyFeatures >> >> The problem with writing your own parser is that you have to maintain it >> :) Better to delegate. >> >> ~Lincoln >> >> >> >> >> On Sun, May 5, 2013 at 5:05 PM, Adam Wyłuda <adamwyl92(a)gmail.com&gt; wrote: >> >>> Hello JBoss community, >>> This year I want to participate in an OS project and GSoC gives me great >>> opportunity to do so. >>> Here is my idea as GSoC 2013 proposal: >>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/adamw/1 >>> I'd be very happy to hear your opinions (I hope it's not too late). >>> >>> _______________________________________________ >>> forge-dev mailing list >>> forge-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/forge-dev >>> >> >> >> >> -- >> Lincoln Baxter, III >> http://ocpsoft.org >> "Simpler is better." >> >> _______________________________________________ >> forge-dev mailing list >> forge-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/forge-dev >> >_______________________________________________ >forge-dev mailing list &gt;forge-dev(a)lists.jboss.org >https://lists.jboss.org/mailman/listinfo/forge-dev _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev

On Mon, May 13, 2013 at 02:38:21PM -0400, Lincoln Baxter, III wrote: >Do you have a suggestion for an alternate Groovy parser to use if we cannot >use groovy eclipse? Nope - its one of the reasons I consider Gradle as evil as Ant in many places :) >Also, even if we do, this should not (unless the Eclipse JDT does some >*really bad* things), be a problem because it will be loaded in an isolated >classloader. Just please consider memory usage - these are not exactly "small" on memory usage. /max > > >On Mon, May 13, 2013 at 12:55 PM, Max Rydahl Andersen &gt;&lt;manderse(a)redhat.com&gt;wrote: > >> Just a warning about using groovy eclipse plugin - it historically have >> had a dependency >> on a patched Eclipse JDT meaning it's not installable on just any eclipse >> version. >> >> I would hope if you just need a groovy parser that you don't have a >> depenency on Eclipse JDT for that >> - but if you do then that might become tricky (might also be osgi >> dependencies to deal with, but >> that is another concern ;) >> >> If the dependency will be included into eclipse as a forge loaded lib we >> might avoid it all - but >> since forge itself also have a dependency on JDT it is something to keep >> an eye on for possible >> version mismatches. >> >> /max >> >> On Sun, May 12, 2013 at 06:00:20PM +0200, Adam Wyłuda wrote: >> >Hi Lincoln! >> >Thank you for your response, it's nice to hear that my proposal is rated >> >positively. >> > >> >I first thought that creating our own Groovy parser which works directly >> on >> >Groovy AST would be a good idea because that would give us more >> >possibilities than existing solutions, but you are right - there is a risk >> >that it might take more time than planned (although basic support could be >> >provided which could be sufficient to work on Gradle project files). >> > >> >I have found this paper which gave me some insights on problems related to >> >Groovy script parsing and modification: >> >http://groovy.ifs.hsr.ch/groovy-refactoring.pdf >> > >> >It seems that the biggest issue is how to rewrite code without changing >> its >> >formatting and how to keep comments in code. Groovy AST doesn't contain >> >comment nodes so if we want to use pure Groovy AST to parse it, we need to >> >include all whitespace and comments in our model, what is possible because >> >Groovy AST nodes include position in source information, so we could just >> >substring our source between statements, but it's hard because we need to >> >check thoroughly every expression as comments can appear between any >> nodes, >> >like this: >> >x = y + /* comment */ z >> > >> >So as you said, it's better to use existing solution so we won't reinvent >> >the wheel. I think that Groovy Eclipse plugin is the best candidate >> because >> >it's API is similar to JDT plugin used in JavaParser (actually Groovy >> >plugin uses JDT and it even seems to be a part of JDT), so it will be more >> >convenient for other developers with JavaParser experience to maintain it. >> >The are also alternatives like NetBeans or IntelliJ IDEA plugin, but I >> >think Eclipse plugin is sufficient enough to work effectively with Groovy >> >sources. >> > >> >Like you suggested I updated my proposal to give myself some time to >> >understand Groovy Eclipse plugin API and start working on Gradle project >> >support earlier. >> > >> >2013/5/7 Lincoln Baxter, III <lincolnbaxter(a)gmail.com&gt; >> > >> >> Hey Adam, >> >> >> >> Thank you so much for your proposal! We have reviewed your submission >> and >> >> so far all the mentors have given it a high rating :) >> >> >> >> I took a look at your prototype groovy-manager project, which looks very >> >> interesting. You are doing source parsing for groovy, which is pretty >> >> challenging. >> >> >> >> While writing a parser from scratch can be a thrilling challenge, I have >> >> to say that it can be quite a burden! We started writing our own >> >> Java-parser for Forge, but quickly came to find that even parsing a >> >> type-safe language is incredibly difficult. >> >> >> >> We ended up using the Eclipse JDT as a base implementation for our >> >> JavaParser. >> >> >> >> Have you looked for any Groovy parsers that may already exist today? >> This >> >> would give you a quick boost when working on this project. It may be the >> >> case that the parser would require some usability improvements to make >> >> integration with Forge simpler, and simpler for plugin devs, but I >> think it >> >> is safe to say that a search for existing parsers is warranted. >> >> >> >> If a suitable parser can be found, I would suggest that we revise your >> >> proposal to allow for several weeks to become familiar with that parser, >> >> and then allocate the rest of the time to actually working with it to >> >> implement the gradle support. >> >> >> >> I'd suggest starting here, with the Groovy Eclipse Plugin - they almost >> >> certainly already support groovy syntax: >> >> http://groovy.codehaus.org/Eclipse+Plugin#EclipsePlugin-KeyFeatures >> >> >> >> The problem with writing your own parser is that you have to maintain it >> >> :) Better to delegate. >> >> >> >> ~Lincoln >> >> >> >> >> >> >> >> >> >> On Sun, May 5, 2013 at 5:05 PM, Adam Wyłuda <adamwyl92(a)gmail.com&gt; >> wrote: >> >> >> >>> Hello JBoss community, >> >>> This year I want to participate in an OS project and GSoC gives me >> great >> >>> opportunity to do so. >> >>> Here is my idea as GSoC 2013 proposal: >> >>> >> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/adamw/1 >> >>> I'd be very happy to hear your opinions (I hope it's not too late). >> >>> >> >>> _______________________________________________ >> >>> forge-dev mailing list >> >>> forge-dev(a)lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/forge-dev >> >>> >> >> >> >> >> >> >> >> -- >> >> Lincoln Baxter, III >> >> http://ocpsoft.org >> >> "Simpler is better." >> >> >> >> _______________________________________________ >> >> forge-dev mailing list >> >> forge-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/forge-dev >> >> >> >> >_______________________________________________ >> >forge-dev mailing list >> &gt;forge-dev(a)lists.jboss.org >> >https://lists.jboss.org/mailman/listinfo/forge-dev >> >> _______________________________________________ >> forge-dev mailing list >> forge-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/forge-dev >> > > > >-- >Lincoln Baxter, III >http://ocpsoft.org >"Simpler is better." >_______________________________________________ >forge-dev mailing list &gt;forge-dev(a)lists.jboss.org >https://lists.jboss.org/mailman/listinfo/forge-dev _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev

So, after studying JavaParser API, I've come to conclusion that creating exact clone of this API for Groovy would be completely irrelevant for this project. JavaParser only works on Java class/interface/enum/annotation structures, there is no way to modify method logic other than manually editing its source code (from Method.getBody() method, which returns plain String). What we really need, is a util which works on imperative expressions, not data structures, so it should be exactly opposite of JavaParser (in Gradle build script there will be rarely any class declarations). I think this issue can be dealt in two ways: 1) Create a full GroovyParser with all capabilities to synthesize any piece of code (including class structures and method bodies) - that's what I was thinking initially, surely nice to have, but it's very hard to even design an API (with statements and expressions its API size will be about twice of JavaParser), and implementing it. To do that we would need to either use one of the existing parsers which are not easy to use (mostly IDE plugins, which are not even available as a Maven dependencies, and they take quite a lot of memory space) or write our own Groovy AST rewriter which may be easy or hard depending on what output quality we expect from it - if we want to keep comments and formatting it certainly won't be easy to do. 2) Use my simple parser ( https://github.com/adamwy/simple-gradle-manager ) or rewrite it, and start working on Gradle support from now, it's fast and light solution (no dependencies other than Groovy library), and it's sufficient to make all necessary modifications to the Gradle build script. I could work on GroovyParser (which I promised to do) after Gradle support in Forge is done, and we could think about creating a common API for parsing Java, Groovy and maybe other languages with full support for expressions inside method bodies. Having this abstraction layer addons which need to synthesize code could work with any supported language without caring in which language the class they modify is written. What do you think is a better way? I planned to go with the first, but the many issue related to this made me think that maybe we should take more time to think and discuss how the Groovy (and probably Java) parser should look and work like, and it will be better to do that later (it's not necessary for Gradle addon, but a nice feature which we will probably need in the future anyway). 2013/5/5 Adam Wy?uda <adamwyl92(a)gmail.com <mailto:adamwyl92@gmail.com>> Hello JBoss community, This year I want to participate in an OS project and GSoC gives me great opportunity to do so. Here is my idea as GSoC 2013 proposal: http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/adamw/1 I'd be very happy to hear your opinions (I hope it's not too late). _______________________________________________ forge-dev mailing list forge-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/forge-dev