<html><body><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><div><span style="font-size: 12pt;">To address your points:</span></div><div><br></div><div><strong><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">#1) </span>"<span style="font-family: Helvetica, Arial, sans-serif; font-size: 12pt;" data-mce-style="font-family: Helvetica, Arial, sans-serif; font-size: 12pt;">Let's assume that user will not write silly rules. Let's give them some freedom and see the rules flourish."</span></strong></div><div><span style="font-family: Helvetica, Arial, sans-serif; font-size: 12pt;"><br></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">We can't assume that users won't do silly things. Users always do silly things and always find ways to break things. The Java-based rules DSL that I've proposed both encourages the user to be a "good citizen" and allows them the freedom to do whatever they like. As I said before, I'd like it if you could think about how to implement your approach using this as a starting point. I agree with you that the data can be represented in the graph in any way imaginable. I'm a little worried that you're still looking at the GraphVisitor stuff that's there now and thinking that's the direction we are headed in - The GraphVisitor is all going to be replaced with the config java DSL (or something java if that doesn't work out.) </span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></div><div><strong><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">#3) <span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">I thought forge UI could be the way to do the mapping, since it has the type conversion already done</span></span></strong></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">If you're looking for dynamic/automatic type conversion, just use the Forge conversion addon - https://github.com/forge/core/tree/master/convert#features - no need to bring in the UI for that, as far as I know.</span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></div><div><strong><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">#4) <span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">About 4 months ago, it was exactly opposite: XML rules, no Java. <span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">When did this change, and what's the guarantee that it won't change again?</span></span></span></strong></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">This hasn't changed. We won't not require users to write Java and compile projects to write rules, I'm saying that we should design the Java APIs first or at *least* in parallel to designing any XML or non-native rule format. If you <span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">take a look at the config addon - it should look very similar to the XML you've written below: <a href="https://github.com/windup/windup/tree/master/engine/config/api/src/main/java/org/jboss/windup/addon/config">https://github.com/windup/windup/tree/master/engine/config/api/src/main/java/org/jboss/windup/addon/config</a></span></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">Is it possible that the Java DSL approach won't work out? Yes, but I think, at this moment, it provides us a great deal of flexibility, and at the same time does not impede your goal of representing the graph data with lower-level, more abstract constructs. Think of it as an implementation of what you have proposed in this thread - the java that makes your XML work.</span></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;">~Lincoln</span></div><div><span style="font-family: Helvetica, Arial, sans-serif;" data-mce-style="font-family: Helvetica, Arial, sans-serif;"><br></span></div><div><br></div><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Ondrej Zizka" <ozizka@redhat.com><br><b>To: </b>"Lincoln Baxter" <lbaxter@redhat.com><br><b>Cc: </b>windup-dev@lists.jboss.org<br><b>Sent: </b>Friday, May 2, 2014 4:20:46 PM<br><b>Subject: </b>Re: Let's replace the GraphVisitor interface concept with something generic<br><div><br></div>
<br>
<div class="moz-cite-prefix">On 2.5.2014 21:17, Lincoln Baxter
wrote:<br>
</div>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div>Hey Ondra,</div>
<div><br>
</div>
<div>I'm moving this discussion to <a href="mailto:windup-dev@lists.jboss.org" target="_blank">windup-dev@lists.jboss.org</a> -
please use windup-dev for all non-confidential development
discussion.</div>
<div><br>
</div>
<div>In general I support your idea to replace the GraphVisitor
interface - I don't think any of us is suggesting that we
continue with that approach. But I am concerned that it is
more complicated than necessary, and I do<span style="font-size: 12pt;"> have a few concerns about what
you've mocked up below:</span></div>
<div><br>
</div>
<div><strong>#1)</strong> Your concept of replacing nodes is
interesting, but what is stopping a rule from replacing a
node, then subsequently overwriting or being overwritten by
another replacement? I see potential for multiple rules to
interfere with each other's types in this way.</div>
</div>
</blockquote>
That's the purpose. The rules would interfere. Or rather, infer.<br>
They would have to be written in a way that they would not do that.
You can write an EJB which kills JVM. You can write HQL query which
will delete all your entities. You can call wait() in the DSL. Let's
assume that user will not write silly rules. Let's give them some
freedom and see the rules flourish.<br>
<br>
Further - the replacement would simply happen for certain cases.
E.g. Maven POM file will never be anything else, so a XmlFileNode
can be changed to MavenPomFileNode. In other cases, rules would
create new nodes and connect them. Also - these possible collisions
can be easily detected - if some migrator asks for a XmlFileNode,
and then asks to change it to some subtype while it already is
another subtype, -> warning or error.<br>
<br>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div><strong>#2)</strong> I'm also not sure that the graph would
allow you to dynamically replace nodes of one type with nodes
of another. Can you verify that?</div>
</div>
</blockquote>
I didn't find anything related in
<a class="moz-txt-link-freetext" href="https://github.com/thinkaurelius/titan/wiki/Titan-Limitations" target="_blank">https://github.com/thinkaurelius/titan/wiki/Titan-Limitations</a> (those
type limitations seem to apply on other things). I'll try
practically.<br>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div><br>
</div>
<div><strong>#3)</strong> You asked: "<span style="font-family:
'times new roman', 'new york', times, serif;">Forge UI could
be mapped to XML elements. I.e. <aaa foo="bar"/>
could invoke command "aaa" with given params.</span><br style="font-family: Helvetica, Arial, sans-serif;">
<span style="font-family: 'times new roman', 'new york',
times, serif;">I believe Forge already has this way of
input, right?"</span></div>
<div><br>
</div>
<div>I'm not completely following your example here, but in
theory you could map XML elements and their attributes to
forge commands and their options in this way; nonetheless, I
don't think that this is a good use of the Forge command
model. You're better off just mapping to Java objects of some
type.</div>
</div>
</blockquote>
Ok, I thought forge UI could be the way to do the mapping, since it
has the type conversion already done, but we can duplicate that
outside Forge, too.<br>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div><br>
</div>
<div><strong>#4)</strong> This seems more complicated than the
example Visitor you linked. Now instead of one Java file
containing the rule, and two java interfaces to encapsulate
the data storage in the graph, you have two very convoluted
XML files. I actually think that the JAXB bit is fairly nice,
but the code required to do that in your example would work in
the current approach anyway because it would still need to be
implemented somewhere.</div>
</div>
</blockquote>
Yes and no. <br>
<br>
1) The Visitor is custom Java code and of course, with access to all
Java libraries in the world, you can make it matter of few lines.
But we are heading towards rules which allow non-trivial operations
while being limited to just several concepts. In this proposal,
free-form Java code would be replaced with those few mentioned:
Service calls, Graph queries, JAXB, EL, iteration, rules dependency,
<br>
IMO, if we don't create something such, we will end up with rules
which will be just a thin wrapper around services.<br>
<br>
2) This XML example doesn't map to the visitor 1:1, it's better. The
approach is different. In that java code, there are two tasks mixed
into one: Discover Maven POM files, and load the dependencies.<br>
What if the pom files will be added by a custom migrator? E.g. when
they have different doctype. Then this visitor would miss them and
not scan their dependencies.<br>
In this approach, the second rule would pick up the MavenPomFileNode
no matter where it comes from.<br>
I could have written the example 1:1 but wanted to show this task
separation advantage.<br>
<br>
<br>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div><br>
</div>
<div>This is similar to what I am prototyping in the config
addon, but in XML not in Java. If you want to continue with
this idea of reducing the operations to operate more closely
with the graph, I support that, but let's please try to find a
way to mock it up using the config DSL instead. </div>
</div>
</blockquote>
Okay, let's see what we have.<br>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div><br>
</div>
<div>As far as implementing this goes - the syntax you've
described below would probably work by mapping to our Java DSL
using Reflection, but that's to be done once the java config
API is established.</div>
<div><br>
</div>
<div>Java first. Then XML (or whatever).</div>
</div>
</blockquote>
Hmm. About 4 months ago, it was exactly opposite: XML rules, no
Java.<br>
It was also one of the main reasons to abandon WindRide the XML
rules were implemented.<br>
The argument was that rules authors will not create Java projects
and study some framework (e.g. Forge) to be able to write a trivial
rule like "if com.foo.Bar is found, report a warning with a comment
and a link to docs XY".<br>
When did this change, and what's the guarantee that it won't change
again?<br>
<br>
Ondra<br>
<br>
<blockquote cite="mid:455050578.12453669.1399058265626.JavaMail.zimbra@redhat.com">
<div style="font-family: times new roman, new york, times, serif;
font-size: 12pt; color: #000000">
<div><br>
</div>
<div>~Lincoln</div>
<div><br>
</div>
<hr id="zwchr">
<div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From:
</b>"Ondrej Zizka" <a class="moz-txt-link-rfc2396E" href="mailto:ozizka@redhat.com" target="_blank"><ozizka@redhat.com></a><br>
<b>To: </b><a class="moz-txt-link-abbreviated" href="mailto:jboss-migration@redhat.com" target="_blank">jboss-migration@redhat.com</a><br>
<b>Sent: </b>Thursday, May 1, 2014 12:05:23 AM<br>
<b>Subject: </b>Re: Let's replace the GraphVisitor interface
concept with something generic<br>
<div><br>
</div>
I've put it to this doc so we can edit/comment there.<br>
<br>
<a class="moz-txt-link-freetext" href="https://docs.google.com/document/d/1UOihPv_zryFilCb7T0k8992wPUt3YNP4goh9rxTC7ng/edit#" target="_blank">https://docs.google.com/document/d/1UOihPv_zryFilCb7T0k8992wPUt3YNP4goh9rxTC7ng/edit#</a><br>
<br>
Ondra<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 1.5.2014 04:35, Ondrej Zizka
wrote:<br>
</div>
<blockquote cite="mid:5361B2DE.60101@redhat.com"> Hi,<br>
<br>
as we discussed before, I'd like to replace the GraphVisitor
interface with something generic.<br>
Seriously, having hard-coded interface with methods specific
for e.g. MessageDrivenBean, EjbEntity, SpringConfiguration,
etc. is IMO not the way to go. It is hard to extend. A rule
system created around this would be cumbersome.<br>
<br>
public void visitEjbEntity(EjbEntityFacet entry);<br>
public void visitEjbService(EjbSessionBeanFacet entry);<br>
public void
visitMessageDrivenBean(MessageDrivenBeanFacet entry);<br>
public void visitEjbEntity(SpringBeanFacet entry);<br>
...<br>
<br>
<a style="width:352px;height:130px;left:0px;right:" href="http://www.google.hu/imgres?imgurl=http%3A%2F%2Fimages.firstcovers.com%2Fcovers%2Fuserquotes%2Fn%2Fno_matter_how_far-133551.jpg&imgrefurl=http%3A%2F%2Fwww.firstcovers.com%2Fuserquotes%2F133551%2Fno%2Bmatter%2Bhow%2Bfar.html&h=315&w=850&tbnid=KPt-4-d8G4ur7M%3A&zoom=1&docid=70YydZev__q0ZM&ei=35phU6_WJ8LcOevqgPgP&tbm=isch&client=ubuntu&ved=0CF0QMygIMAg&iact=rc&uact=3&dur=651&page=1&start=0&ndsp=31" class="rg_l" target="_blank"><img src="cid:part3.00050005.00060307@redhat.com" style="width:352px;height:130px;margin-left:0px;margin-right:0px;margin-top:0px" class="rg_i"></a><br>
<br>
Instead, we should focus on the graph, and have just very
few node types in the core - FileNode, and the rest would
be subtypes defined in addons. Addons would, through rules:<br>
* replace a node with a subclass, e.g.<br>
FileNode => XmlFileNode => MavenPomNode,<br>
FileNode => JavaFileNode =>
AnnotationNode<br>
<br>
* add properties, e.g.<br>
XmlFileNode's "doctype",<br>
ClassNode's "blacklisted"<br>
* connect nodes to them, e.g.<br>
MavenPom ---contains--->
MavenDependencyNode<br>
JavaFile --- imports --> [ ClassNode,
ClassNode, ... ]<br>
<br>
This approach would:<br>
* Leverage of Forge modularity (e.g. Maven addon
depending on XmlFile addon)<br>
* Improve extendability (no need to squeeze everything
into the GraphVisitor interface's methods or extend it)<br>
* Lead to much more straightforward rules implementation
- all rules would reduce to:<br>
* matching graph information (Gremlin?)<br>
* using DAO's / Services (for mining data from the
files/..., and for writing them during active migration)<br>
1) Bundled - XPath, AST query, properties,
Maven remote fetch, ...<br>
2) User's: .class packed within the addon
or a Groovy class<br>
* writing back to the graph<br>
* rendering pieces of HTML for the report.<br>
<br>
Who's in? I need some scenarios where this wouldn't work.
But from what I can tell, this would be more generic, but
still simpler, than current "God-object" suffering
GraphVisitor.<br>
<br>
As an example, take e.g. MavenRemoteFetchVisitor. <a class="moz-txt-link-freetext" href="https://github.com/windup/windup/blob/master/engine/rules/impl/src/main/java/org/jboss/windup/engine/visitor/inspector/MavenRemoteFetchVisitor.java" target="_blank">https://github.com/windup/windup/blob/master/engine/rules/impl/src/main/java/org/jboss/windup/engine/visitor/inspector/MavenRemoteFetchVisitor.java</a><br>
<br>
All that is doable using few simple building blocks,
directed by few lines of a rule like this:<br>
<br>
------------------------------------------------------------------<br>
<br>
<var name="pomNS" val=<a class="moz-txt-link-rfc2396E" href="http://maven.apache.org/POM/4.0.0" target="_blank">"http://maven.apache.org/POM/4.0.0"</a>><br>
<br>
Rule 1) which would process all POM files and load the info
into the graph.<br>
<br>
<rule id="maven.pomFiles" desc=" Analyze Maven POM
files "<br>
phase="StaticConfigAnalysis"><br>
<graph match="XmlFileNode[doctype='<a class="moz-txt-link-freetext" href="http://maven.apache.org/POM/4.0.0" target="_blank">http://maven.apache.org/POM/4.0.0</a>']"
toVar="xmls"><br>
<for var="pom" in="xmls"><br>
<graph:replaceSingle ref="pom"
with="MavenPomFileNode"><br>
<properties><br>
<!-- <jaxb> would invoce a call
to a Service. <br>
<properties> hander would
take returned object's bean props. <br>
toClass would load the class from
CL or compile from .groovy coming with the migrator (addon).<br>
--><br>
<jaxb toClass="PomJaxb.groovy"
fromFile="${pom.path}"><br>
<ns name="pom"
uri="${pomNS}"/><br>
</jaxb><br>
</properties><br>
</graph><br>
</for><br>
</rule><br>
<br>
@XmlRoot<br>
class PomJaxb {<br>
@XmlXPath(<span class="s">"/pom:project/pom:modelVersion"</span><span class="o"></span><span class="n"></span><span class="o"></span>)
<span class="n">String</span> <span class="n">modelVersion</span><span class="o"></span>;
<div class="line" id="LC77"> @XmlXPath(<span class="s"></span><span class="s">"/pom:project/pom:name"</span><span class="o"></span>)
<span class="n">String</span> <span class="n">name</span>;<span class="o"></span></div>
<div class="line" id="LC78"> @XmlXPath(<span class="s"></span><span class="s"></span><span class="n"><span class="s">"/pom:project/pom:description"</span><span class="o"> </span>String</span> <span class="n">description</span><span class="s">;</span><span class="o"></span> <span class="o"></span></div>
<div class="line" id="LC79"> @XmlXPath(<span class="s">"/pom:project/pom:url"</span><span class="o"></span><span class="s"></span><span class="s"></span><span class="n"> String</span> <span class="n">url;</span><br>
}<br>
</div>
<br>
<br>
Rule 2) which would load the dependencies and describe them
into Nodes and Edges.<br>
<br>
<rule id="maven.pomDependencies" desc=" Analyze Maven
POM file dependencies "<br>
phase="StaticConfigAnalysis"><br>
<after rule="maven.pomFiles"><br>
<br>
<graph match="MavenPomFileNode"
toVar="pomVertexes"><br>
<for var="pomV" in="pomVertexes"><br>
<br>
<xpath toVar="deps" fromFile="${pomV.path}"
match="/pom:project/pom:dependencies/pom:dependency"/><br>
<for var="depElement" in="deps"><br>
<jaxb toVar="dep"
toClass="MavenDepencencyJaxb.groovy"
fromElement="depElement" ns="pom ${pomNS}"/><br>
<graph:query toVar="isBlacklisted" <br>
q=" /* I don't know Gremlin so far,
imagine an equiv of this XPath: */<br>
MavenDependencyNode[<br>
g=${dep.groupId} and
a=${dep.artifactId} and v=${dep.version}<br>
]@blacklisted<br>
" /><br>
<continue if="isBlacklisted /* var, or
Groovy (or EL) expression */" /><br>
<!-- This would be useful for blacklists
and filters in general, which appear often in real life
rules. --><br>
<br>
<graph:insertIfNotExists
type="MavenDependencyNode" toVar="depVertex><br>
<properties from="dep"/><br>
</graph><br>
<graph:edge type="dependsOn" from="pomV"
to="depVertex"/><br>
<!-- Maybe Gremlin could
replace this? --><br>
</for><br>
</for><br>
</rule><br>
<br>
@XmlRoot<br>
class MavenDepencencyJaxb {<br>
// GraphKeyProperty identifies those which are
compared for insertIfNotExists.<br>
@GraphKeyProperty @XmlXPath("./pom:groupId") String
groupId;<br>
@GraphKeyProperty @XmlXPath("./pom:artifactId")
String artifactId;<br>
@GraphKeyProperty @XmlXPath("./pom:version") String
version;<br>
}<br>
------------------------------------------------------------------<br>
<br>
As you can see, It creates independent rules which only
communicate indirectly through the graph.<br>
You can also see how nicely Java classes fit into this, and
how Groovy could make this easier.<br>
<br>
SERVICES INVOCATION<br>
Forge UI could be mapped to XML elements. I.e. <aaa
foo="bar"/> could invoke command "aaa" with given params.<br>
I believe Forge already has this way of input, right?<br>
<br>
GRAPH OPERATION<br>
There would be several <graph:...> operations - CRUD
plus some special.<br>
<br>
EXECUTION FLOW<br>
The flow would be simple, from top to bottom, creating
variables along the way, containing objects or iterable
collections of objects. Those iterable could be used in
<for>.<br>
<br>
Does Lincoln's executor fit this? I haven't still looked how
it works. This tree would likely be executed classically
with a stack and using tree reduction for operation
arguments.<br>
<br>
For more complex logic, users would break the task into
multiple rules, storing data into the graph intermediately.<br>
<br>
I'll check few more visitors to see if this is powerful
enough to satisfy all the baneeds.<br>
<br>
<br>
.............................<br>
<br>
Also, I'd like to eradicate any mention of an archive from
most of the code - archives should be totally transparent
for the migrators. There should be just FileNodes, connected
with ArchiveNodes, and whoever needs an information that a
file came from an archive, may look that up. See <a class="moz-txt-link-freetext" href="https://docs.google.com/drawings/d/1IMnds3Qu8Wwcf7_mr7NJ9a3YgtcGJ7dejl09EhWl7Vc" target="_blank">https://docs.google.com/drawings/d/1IMnds3Qu8Wwcf7_mr7NJ9a3YgtcGJ7dejl09EhWl7Vc</a>
for illustration.<br>
<br>
Ondra<br>
<br>
<br>
<br>
<br>
</blockquote>
<br>
</div>
<div><br>
</div>
</div>
</blockquote>
<br>
</div><div><br></div></div></body></html>