[windup-dev] Let's replace the GraphVisitor interface concept with something generic
Lincoln Baxter, III
lincolnbaxter at gmail.com
Mon May 5 10:34:28 EDT 2014
Yeah, I actually have to agree with Jess here. I think that the "add more
and more information as you go" concept is probably the simplest, and also
makes sense from the point-of-view of "improving" what we know the more
rules are run.
On Sun, May 4, 2014 at 7:30 PM, Jess Sightler <jsightle at redhat.com> wrote:
>
> On May 2, 2014 5:31 PM, Ondrej Zizka <ozizka at redhat.com> wrote:
> >
> >
> > On 2.5.2014 21:17, Lincoln Baxter wrote:
> >>
> >> Hey Ondra,
> >>
> >> I'm moving this discussion to windup-dev at lists.jboss.org - please use
> windup-dev for all non-confidential development discussion.
> >>
> >> In general I support your idea to replace the GraphVisitor interface -
> I don't think any of us is suggesting that we continue with that approach.
> But I am concerned that it is more complicated than necessary, and I
> do have a few concerns about what you've mocked up below:
> >>
> >> #1) Your concept of replacing nodes is interesting, but what is
> stopping a rule from replacing a node, then subsequently overwriting or
> being overwritten by another replacement? I see potential for multiple
> rules to interfere with each other's types in this way.
> >
> > That's the purpose. The rules would interfere. Or rather, infer.
> > They would have to be written in a way that they would not do that. You
> can write an EJB which kills JVM. You can write HQL query which will delete
> all your entities. You can call wait() in the DSL. Let's assume that user
> will not write silly rules. Let's give them some freedom and see the rules
> flourish.
>
> I prefer to think of this in terms of RDF-style classes that operate on
> the open world assumption:
> http://en.m.wikipedia.org/wiki/Open_world_assumption
>
> In this model, you don't replace types, you add additional type
> information, and inferred properties as you discover more things about a
> particular entity. If you'd like, I can try to put together a little mini
> presentation on rdf and owl, as I believe we may be re-solving some
> problems that those groups have already solved pretty well. :-)
>
> From my research on Friday, I believe that this OWA type model may
> actually be supported pretty well by Tinkerpop frames.
>
> We can discuss further next week, as I am sure the above is insufficient.
> :-)
>
> >
> > Further - the replacement would simply happen for certain cases. E.g.
> Maven POM file will never be anything else, so a XmlFileNode can be changed
> to MavenPomFileNode. In other cases, rules would create new nodes and
> connect them. Also - these possible collisions can be easily detected - if
> some migrator asks for a XmlFileNode, and then asks to change it to some
> subtype while it already is another subtype, -> warning or error.
> >
> >> #2) I'm also not sure that the graph would allow you to dynamically
> replace nodes of one type with nodes of another. Can you verify that?
> >
> > I didn't find anything related in
> https://github.com/thinkaurelius/titan/wiki/Titan-Limitations (those type
> limitations seem to apply on other things). I'll try practically.
> >>
> >>
> >> #3) You asked: "Forge UI could be mapped to XML elements. I.e. <aaa
> foo="bar"/> could invoke command "aaa" with given params.
> >> I believe Forge already has this way of input, right?"
> >>
> >> I'm not completely following your example here, but in theory you could
> map XML elements and their attributes to forge commands and their options
> in this way; nonetheless, I don't think that this is a good use of the
> Forge command model. You're better off just mapping to Java objects of some
> type.
> >
> > Ok, I thought forge UI could be the way to do the mapping, since it has
> the type conversion already done, but we can duplicate that outside Forge,
> too.
> >>
> >>
> >> #4) This seems more complicated than the example Visitor you linked.
> Now instead of one Java file containing the rule, and two java interfaces
> to encapsulate the data storage in the graph, you have two very convoluted
> XML files. I actually think that the JAXB bit is fairly nice, but the code
> required to do that in your example would work in the current approach
> anyway because it would still need to be implemented somewhere.
> >
> > Yes and no.
> >
> > 1) The Visitor is custom Java code and of course, with access to all
> Java libraries in the world, you can make it matter of few lines. But we
> are heading towards rules which allow non-trivial operations while being
> limited to just several concepts. In this proposal, free-form Java code
> would be replaced with those few mentioned: Service calls, Graph queries,
> JAXB, EL, iteration, rules dependency,
> > IMO, if we don't create something such, we will end up with rules which
> will be just a thin wrapper around services.
> >
> > 2) This XML example doesn't map to the visitor 1:1, it's better. The
> approach is different. In that java code, there are two tasks mixed into
> one: Discover Maven POM files, and load the dependencies.
> > What if the pom files will be added by a custom migrator? E.g. when they
> have different doctype. Then this visitor would miss them and not scan
> their dependencies.
> > In this approach, the second rule would pick up the MavenPomFileNode no
> matter where it comes from.
> > I could have written the example 1:1 but wanted to show this task
> separation advantage.
> >
> >
> >>
> >> This is similar to what I am prototyping in the config addon, but in
> XML not in Java. If you want to continue with this idea of reducing the
> operations to operate more closely with the graph, I support that, but
> let's please try to find a way to mock it up using the config DSL instead.
> >
> > Okay, let's see what we have.
> >>
> >>
> >> As far as implementing this goes - the syntax you've described below
> would probably work by mapping to our Java DSL using Reflection, but that's
> to be done once the java config API is established.
> >>
> >> Java first. Then XML (or whatever).
> >
> > Hmm. About 4 months ago, it was exactly opposite: XML rules, no Java.
> > It was also one of the main reasons to abandon WindRide the XML rules
> were implemented.
> > The argument was that rules authors will not create Java projects and
> study some framework (e.g. Forge) to be able to write a trivial rule like
> "if com.foo.Bar is found, report a warning with a comment and a link to
> docs XY".
> > When did this change, and what's the guarantee that it won't change
> again?
> >
> > Ondra
> >
> >>
> >> ~Lincoln
> >>
> >> ________________________________
> >> From: "Ondrej Zizka" <ozizka at redhat.com>
> >> To: jboss-migration at redhat.com
> >> Sent: Thursday, May 1, 2014 12:05:23 AM
> >> Subject: Re: Let's replace the GraphVisitor interface concept with
> something generic
> >>
> >> I've put it to this doc so we can edit/comment there.
> >>
> >>
> https://docs.google.com/document/d/1UOihPv_zryFilCb7T0k8992wPUt3YNP4goh9rxTC7ng/edit#
> >>
> >> Ondra
> >>
> >>
> >>
> >> On 1.5.2014 04:35, Ondrej Zizka wrote:
> >>>
> >>> Hi,
> >>>
> >>> as we discussed before, I'd like to replace the GraphVisitor interface
> with something generic.
> >>> Seriously, having hard-coded interface with methods specific for e.g.
> MessageDrivenBean, EjbEntity, SpringConfiguration, etc. is IMO not the way
> to go. It is hard to extend. A rule system created around this would be
> cumbersome.
> >>>
> >>> public void visitEjbEntity(EjbEntityFacet entry);
> >>> public void visitEjbService(EjbSessionBeanFacet entry);
> >>> public void visitMessageDrivenBean(MessageDrivenBeanFacet entry);
> >>> public void visitEjbEntity(SpringBeanFacet entry);
> >>> ...
> >>>
> >>>
> >>>
> >>> Instead, we should focus on the graph, and have just very few node
> types in the core - FileNode, and the rest would be subtypes defined in
> addons. Addons would, through rules:
> >>> * replace a node with a subclass, e.g.
> >>> FileNode => XmlFileNode => MavenPomNode,
> >>> FileNode => JavaFileNode => AnnotationNode
> >>>
> >>> * add properties, e.g.
> >>> XmlFileNode's "doctype",
> >>> ClassNode's "blacklisted"
> >>> * connect nodes to them, e.g.
> >>> MavenPom ---contains---> MavenDependencyNode
> >>> JavaFile --- imports --> [ ClassNode, ClassNode, ... ]
> >>>
> >>> This approach would:
> >>> * Leverage of Forge modularity (e.g. Maven addon depending on
> XmlFile addon)
> >>> * Improve extendability (no need to squeeze everything into the
> GraphVisitor interface's methods or extend it)
> >>> * Lead to much more straightforward rules implementation - all
> rules would reduce to:
> >>> * matching graph information (Gremlin?)
> >>> * using DAO's / Services (for mining data from the files/...,
> and for writing them during active migration)
> >>> 1) Bundled - XPath, AST query, properties, Maven
> remote fetch, ...
> >>> 2) User's: .class packed within the addon or a
> Groovy class
> >>> * writing back to the graph
> >>> * rendering pieces of HTML for the report.
> >>>
> >>> Who's in? I need some scenarios where this wouldn't work. But from
> what I can tell, this would be more generic, but still simpler, than
> current "God-object" suffering GraphVisitor.
> >>>
> >>> As an example, take e.g. MavenRemoteFetchVisitor.
> https://github.com/windup/windup/blob/master/engine/rules/impl/src/main/java/org/jboss/windup/engine/visitor/inspector/MavenRemoteFetchVisitor.java
> >>>
> >>> All that is doable using few simple building blocks, directed by few
> lines of a rule like this:
> >>>
> >>> ------------------------------------------------------------------
> >>>
> >>> <var name="pomNS" val="http://maven.apache.org/POM/4.0.0">
> >>>
> >>> Rule 1) which would process all POM files and load the info into the
> graph.
> >>>
> >>> <rule id="maven.pomFiles" desc=" Analyze Maven POM files "
> >>> phase="StaticConfigAnalysis">
> >>> <graph match="XmlFileNode[doctype='
> http://maven.apache.org/POM/4.0.0']" toVar="xmls">
> >>> <for var="pom" in="xmls">
> >>> <graph:replaceSingle ref="pom" with="MavenPomFileNode">
> >>> <properties>
> >>> <!-- <jaxb> would invoce a call to a Service.
> >>> <properties> hander would take returned
> object's bean props.
> >>> toClass would load the class from CL or
> compile from .groovy coming with the migrator (addon).
> >>> -->
> >>> <jaxb toClass="PomJaxb.groovy"
> fromFile="${pom.path}">
> >>> <ns name="pom" uri="${pomNS}"/>
> >>> </jaxb>
> >>> </properties>
> >>> </graph>
> >>> </for>
> >>> </rule>
> >>>
> >>> @XmlRoot
> >>> class PomJaxb {
> >>> @XmlXPath("/pom:project/pom:modelVersion") String
> modelVersion;
> >>> @XmlXPath("/pom:project/pom:name") String
> name;
> >>> @XmlXPath("/pom:project/pom:description" String
> description;
> >>> @XmlXPath("/pom:project/pom:url"
> String url;
> >>> }
> >>>
> >>>
> >>> Rule 2) which would load the dependencies and describe them into Nodes
> and Edges.
> >>>
> >>> <rule id="maven.pomDependencies" desc=" Analyze Maven POM file
> dependencies "
> >>> phase="StaticConfigAnalysis">
> >>> <after rule="maven.pomFiles">
> >>>
> >>> <graph match="MavenPomFileNode" toVar="pomVertexes">
> >>> <for var="pomV" in="pomVertexes">
> >>>
> >>> <xpath toVar="deps" fromFile="${pomV.path}"
> match="/pom:project/pom:dependencies/pom:dependency"/>
> >>> <for var="depElement" in="deps">
> >>> <jaxb toVar="dep" toClass="MavenDepencencyJaxb.groovy"
> fromElement="depElement" ns="pom ${pomNS}"/>
> >>> <graph:query toVar="isBlacklisted"
> >>> q=" /* I don't know Gremlin so far, imagine an
> equiv of this XPath: */
> >>> MavenDependencyNode[
> >>> g=${dep.groupId} and a=${dep.artifactId}
> and v=${dep.version}
> >>> ]@blacklisted
> >>> " />
> >>> <continue if="isBlacklisted /* var, or Groovy (or EL)
> expression */" />
> >>> <!-- This would be useful for blacklists and filters in
> general, which appear often in real life rules. -->
> >>>
> >>> <graph:insertIfNotExists type="MavenDependencyNode"
> toVar="depVertex>
> >>> <properties from="dep"/>
> >>> </graph>
> >>> <graph:edge type="dependsOn" from="pomV" to="depVertex"/>
> >>> <!-- Maybe Gremlin could replace this? -->
> >>> </for>
> >>> </for>
> >>> </rule>
> >>>
> >>> @XmlRoot
> >>> class MavenDepencencyJaxb {
> >>> // GraphKeyProperty identifies those which are compared for
> insertIfNotExists.
> >>> @GraphKeyProperty @XmlXPath("./pom:groupId") String groupId;
> >>> @GraphKeyProperty @XmlXPath("./pom:artifactId") String
> artifactId;
> >>> @GraphKeyProperty @XmlXPath("./pom:version") String version;
> >>> }
> >>> ------------------------------------------------------------------
> >>>
> >>> As you can see, It creates independent rules which only communicate
> indirectly through the graph.
> >>> You can also see how nicely Java classes fit into this, and how Groovy
> could make this easier.
> >>>
> >>> SERVICES INVOCATION
> >>> Forge UI could be mapped to XML elements. I.e. <aaa foo="bar"/> could
> invoke command "aaa" with given params.
> >>> I believe Forge already has this way of input, right?
> >>>
> >>> GRAPH OPERATION
> >>> There would be several <graph:...> operations - CRUD plus some special.
> >>>
> >>> EXECUTION FLOW
> >>> The flow would be simple, from top to bottom, creating variables along
> the way, containing objects or iterable collections of objects. Those
> iterable could be used in <for>.
> >>>
> >>> Does Lincoln's executor fit this? I haven't still looked how it works.
> This tree would likely be executed classically with a stack and using tree
> reduction for operation arguments.
> >>>
> >>> For more complex logic, users would break the task into multiple
> rules, storing data into the graph intermediately.
> >>>
> >>> I'll check few more visitors to see if this is powerful enough to
> satisfy all the baneeds.
> >>>
> >>>
> >>> .............................
> >>>
> >>> Also, I'd like to eradicate any mention of an archive from most of the
> code - archives should be totally transparent for the migrators. There
> should be just FileNodes, connected with ArchiveNodes, and whoever needs an
> information that a file came from an archive, may look that up. See
> https://docs.google.com/drawings/d/1IMnds3Qu8Wwcf7_mr7NJ9a3YgtcGJ7dejl09EhWl7Vc
> for illustration.
> >>>
> >>> Ondra
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >
>
>
> _______________________________________________
> windup-dev mailing list
> windup-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/windup-dev
>
--
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/windup-dev/attachments/20140505/834182a5/attachment-0001.html
More information about the windup-dev
mailing list