3:20 p.m.
On 2.5.2014 21:17, Lincoln Baxter wrote:
Hey Ondra,
I'm moving this discussion to windup-dev(a)lists.jboss.org
<mailto:windup-dev@lists.jboss.org> - please use windup-dev for all
non-confidential development discussion.
In general I support your idea to replace the GraphVisitor interface -
I don't think any of us is suggesting that we continue with that
approach. But I am concerned that it is more complicated than
necessary, and I do have a few concerns about what you've mocked up below:
*#1)* Your concept of replacing nodes is interesting, but what is
stopping a rule from replacing a node, then subsequently overwriting
or being overwritten by another replacement? I see potential for
multiple rules to interfere with each other's types in this way.
That's the
purpose. The rules would interfere. Or rather, infer.
They would have to be written in a way that they would not do that. You
can write an EJB which kills JVM. You can write HQL query which will
delete all your entities. You can call wait() in the DSL. Let's assume
that user will not write silly rules. Let's give them some freedom and
see the rules flourish.
Further - the replacement would simply happen for certain cases. E.g.
Maven POM file will never be anything else, so a XmlFileNode can be
changed to MavenPomFileNode. In other cases, rules would create new
nodes and connect them. Also - these possible collisions can be easily
detected - if some migrator asks for a XmlFileNode, and then asks to
change it to some subtype while it already is another subtype, ->
warning or error.
*#2)* I'm also not sure that the graph would allow you to
dynamically
replace nodes of one type with nodes of another. Can you verify that?
I didn't
find anything related in
https://github.com/thinkaurelius/titan/wiki/Titan-Limitations (those
type limitations seem to apply on other things). I'll try practically.
*#3)* You asked: "Forge UI could be mapped to XML elements. I.e. <aaa
foo="bar"/> could invoke command "aaa" with given params.
I believe Forge already has this way of input, right?"
I'm not completely following your example here, but in theory you
could map XML elements and their attributes to forge commands and
their options in this way; nonetheless, I don't think that this is a
good use of the Forge command model. You're better off just mapping to
Java objects of some type.
Ok, I thought forge UI could be the way to do the
mapping, since it has
the type conversion already done, but we can duplicate that outside
Forge, too.
*#4)* This seems more complicated than the example Visitor you linked.
Now instead of one Java file containing the rule, and two java
interfaces to encapsulate the data storage in the graph, you have two
very convoluted XML files. I actually think that the JAXB bit is
fairly nice, but the code required to do that in your example would
work in the current approach anyway because it would still need to be
implemented somewhere.
Yes and no.
1) The Visitor is custom Java code and of course, with access to all
Java libraries in the world, you can make it matter of few lines. But we
are heading towards rules which allow non-trivial operations while being
limited to just several concepts. In this proposal, free-form Java code
would be replaced with those few mentioned: Service calls, Graph
queries, JAXB, EL, iteration, rules dependency,
IMO, if we don't create something such, we will end up with rules which
will be just a thin wrapper around services.
2) This XML example doesn't map to the visitor 1:1, it's better. The
approach is different. In that java code, there are two tasks mixed into
one: Discover Maven POM files, and load the dependencies.
What if the pom files will be added by a custom migrator? E.g. when they
have different doctype. Then this visitor would miss them and not scan
their dependencies.
In this approach, the second rule would pick up the MavenPomFileNode no
matter where it comes from.
I could have written the example 1:1 but wanted to show this task
separation advantage.
This is similar to what I am prototyping in the config addon, but in
XML not in Java. If you want to continue with this idea of reducing
the operations to operate more closely with the graph, I support that,
but let's please try to find a way to mock it up using the config DSL
instead.
Okay, let's see what we have.
As far as implementing this goes - the syntax you've described below
would probably work by mapping to our Java DSL using Reflection, but
that's to be done once the java config API is established.
Java first. Then XML (or whatever).
Hmm. About 4 months ago, it was exactly
opposite: XML rules, no Java.
It was also one of the main reasons to abandon WindRide the XML rules
were implemented.
The argument was that rules authors will not create Java projects and
study some framework (e.g. Forge) to be able to write a trivial rule
like "if com.foo.Bar is found, report a warning with a comment and a
link to docs XY".
When did this change, and what's the guarantee that it won't change again?
Ondra
~Lincoln
------------------------------------------------------------------------
*From: *"Ondrej Zizka" <ozizka(a)redhat.com>
*To: *jboss-migration(a)redhat.com
*Sent: *Thursday, May 1, 2014 12:05:23 AM
*Subject: *Re: Let's replace the GraphVisitor interface concept with
something generic
I've put it to this doc so we can edit/comment there.
https://docs.google.com/document/d/1UOihPv_zryFilCb7T0k8992wPUt3YNP4goh9r...
Ondra
On 1.5.2014 04:35, Ondrej Zizka wrote:
Hi,
as we discussed before, I'd like to replace the GraphVisitor
interface with something generic.
Seriously, having hard-coded interface with methods specific for
e.g. MessageDrivenBean, EjbEntity, SpringConfiguration, etc. is
IMO not the way to go. It is hard to extend. A rule system created
around this would be cumbersome.
public void visitEjbEntity(EjbEntityFacet entry);
public void visitEjbService(EjbSessionBeanFacet entry);
public void visitMessageDrivenBean(MessageDrivenBeanFacet entry);
public void visitEjbEntity(SpringBeanFacet entry);
...
<http://www.google.hu/imgres?imgurl=http%3A%2F%2Fimages.firstcovers.com%2F...
Instead, we should focus on the graph, and have just very few node
types in the core - FileNode, and the rest would be subtypes
defined in addons. Addons would, through rules:
* replace a node with a subclass, e.g.
FileNode => XmlFileNode => MavenPomNode,
FileNode => JavaFileNode => AnnotationNode
* add properties, e.g.
XmlFileNode's "doctype",
ClassNode's "blacklisted"
* connect nodes to them, e.g.
MavenPom ---contains---> MavenDependencyNode
JavaFile --- imports --> [ ClassNode, ClassNode, ... ]
This approach would:
* Leverage of Forge modularity (e.g. Maven addon depending on
XmlFile addon)
* Improve extendability (no need to squeeze everything into
the GraphVisitor interface's methods or extend it)
* Lead to much more straightforward rules implementation - all
rules would reduce to:
* matching graph information (Gremlin?)
* using DAO's / Services (for mining data from the
files/..., and for writing them during active migration)
1) Bundled - XPath, AST query, properties, Maven
remote fetch, ...
2) User's: .class packed within the addon or a
Groovy class
* writing back to the graph
* rendering pieces of HTML for the report.
Who's in? I need some scenarios where this wouldn't work. But from
what I can tell, this would be more generic, but still simpler,
than current "God-object" suffering GraphVisitor.
As an example, take e.g. MavenRemoteFetchVisitor.
https://github.com/windup/windup/blob/master/engine/rules/impl/src/main/j...
All that is doable using few simple building blocks, directed by
few lines of a rule like this:
------------------------------------------------------------------
<var name="pomNS" val="http://maven.apache.org/POM/4.0.0">
Rule 1) which would process all POM files and load the info into
the graph.
<rule id="maven.pomFiles" desc=" Analyze Maven POM files "
phase="StaticConfigAnalysis">
<graph
match="XmlFileNode[doctype='http://maven.apache.org/POM/4.0.0...
toVar="xmls">
<for var="pom" in="xmls">
<graph:replaceSingle ref="pom"
with="MavenPomFileNode">
<properties>
<!-- <jaxb> would invoce a call to a Service.
<properties> hander would take returned
object's bean props.
toClass would load the class from CL or
compile from .groovy coming with the migrator (addon).
-->
<jaxb toClass="PomJaxb.groovy"
fromFile="${pom.path}">
<ns name="pom" uri="${pomNS}"/>
</jaxb>
</properties>
</graph>
</for>
</rule>
@XmlRoot
class PomJaxb {
@XmlXPath("/pom:project/pom:modelVersion") String
modelVersion;
@XmlXPath("/pom:project/pom:name") String name;
@XmlXPath("/pom:project/pom:description"String description;
@XmlXPath("/pom:project/pom:url"
String url;
}
Rule 2) which would load the dependencies and describe them into
Nodes and Edges.
<rule id="maven.pomDependencies" desc=" Analyze Maven POM file
dependencies "
phase="StaticConfigAnalysis">
<after rule="maven.pomFiles">
<graph match="MavenPomFileNode" toVar="pomVertexes">
<for var="pomV" in="pomVertexes">
<xpath toVar="deps" fromFile="${pomV.path}"
match="/pom:project/pom:dependencies/pom:dependency"/>
<for var="depElement" in="deps">
<jaxb toVar="dep"
toClass="MavenDepencencyJaxb.groovy" fromElement="depElement"
ns="pom ${pomNS}"/>
<graph:query toVar="isBlacklisted"
q=" /* I don't know Gremlin so far, imagine an
equiv of this XPath: */
MavenDependencyNode[
g=${dep.groupId} and
a=${dep.artifactId} and v=${dep.version}
]@blacklisted
" />
<continue if="isBlacklisted /* var, or Groovy (or
EL) expression */" />
<!-- This would be useful for blacklists and filters
in general, which appear often in real life rules. -->
<graph:insertIfNotExists type="MavenDependencyNode"
toVar="depVertex>
<properties from="dep"/>
</graph>
<graph:edge type="dependsOn" from="pomV"
to="depVertex"/>
<!-- Maybe Gremlin could replace this? -->
</for>
</for>
</rule>
@XmlRoot
class MavenDepencencyJaxb {
// GraphKeyProperty identifies those which are compared for
insertIfNotExists.
@GraphKeyProperty @XmlXPath("./pom:groupId") String groupId;
@GraphKeyProperty @XmlXPath("./pom:artifactId") String
artifactId;
@GraphKeyProperty @XmlXPath("./pom:version") String version;
}
------------------------------------------------------------------
As you can see, It creates independent rules which only
communicate indirectly through the graph.
You can also see how nicely Java classes fit into this, and how
Groovy could make this easier.
SERVICES INVOCATION
Forge UI could be mapped to XML elements. I.e. <aaa foo="bar"/>
could invoke command "aaa" with given params.
I believe Forge already has this way of input, right?
GRAPH OPERATION
There would be several <graph:...> operations - CRUD plus some
special.
EXECUTION FLOW
The flow would be simple, from top to bottom, creating variables
along the way, containing objects or iterable collections of
objects. Those iterable could be used in <for>.
Does Lincoln's executor fit this? I haven't still looked how it
works. This tree would likely be executed classically with a stack
and using tree reduction for operation arguments.
For more complex logic, users would break the task into multiple
rules, storing data into the graph intermediately.
I'll check few more visitors to see if this is powerful enough to
satisfy all the baneeds.
.............................
Also, I'd like to eradicate any mention of an archive from most of
the code - archives should be totally transparent for the
migrators. There should be just FileNodes, connected with
ArchiveNodes, and whoever needs an information that a file came
from an archive, may look that up. See
https://docs.google.com/drawings/d/1IMnds3Qu8Wwcf7_mr7NJ9a3YgtcGJ7dejl09E...
for illustration.
Ondra
