[hibernate-dev] [HV/HSEARCH] Free form

[Gunnar Morling]

Emmanuel,

In your PoC, how would a complete tree-like structure be traversed?
It's not clear to me, who is driving StructureTraverser, i.e. which
component will call processSubstructureInContainer() et al. when
traversing an entire tree.

@Yoann, maybe you can add a usage example similar to Emmanuel's? You
have a lot of framework code, but I'm not sure about how it'd be used.

For Hibernate Search, the traversal pattern I implemented for the
ScenicView PoC may be of interest. Its general idea is to represent a
tree traversal as a sequence of events which a traverser
implementation receives and can act on, e.g. to create a corresponding
de-normalized structure, Lucene document etc. The retrieval of values
and associated objects happens lazily as the traverser
("TreeTraversalEventConsumer" in my lingo) pulls events from the
sequence, similar to what some XML parsers do.

The main contract can be found at [1]. There are two event sequence
implements, one based on Hibernate's meta-model [2] and one for
java.util.Map [3]. An example event consumer implementation which
creates MongoDB documents can be found at [4].

As said I think it'd nicely fit for Hibernate Search, for HV I'm not
so sure. The reason being that the order of traversal may very,
depending on the defined validation groups and sequences. Sometimes we
need to go "depth first". I've been contemplating to employ an
event-like approach as described above for HV, but it may look
different than the one used for HSEARCH.

--Gunnar

[1] https://github.com/gunnarmorling/scenicview-mvp/blob/master/core/src/main/java/org/hibernate/scenicview/spi/backend/model/TreeTraversalSequence.java.
[2] https://github.com/gunnarmorling/scenicview-mvp/blob/master/core/src/main/java/org/hibernate/scenicview/internal/model/EntityStateBasedTreeTraversalSequence.java
[3] https://github.com/gunnarmorling/scenicview-mvp/blob/master/core/src/test/java/org/hibernate/scenicview/test/traversal/MapTreeTraversalSequence.java
[4] https://github.com/gunnarmorling/scenicview-mvp/blob/master/mongodb/src/main/java/org/hibernate/scenicview/mongodb/internal/MongoDbDenormalizationBackend.java#L91..L128



2017-02-06 16:49 GMT+01:00 Emmanuel Bernard <emmanuel at hibernate.org>:
> Your prototype is very Hibernate Search tainted. I wonder how or whether we want it reusable across Hibernate Validator, Search and possibly more.
>
> Have you captured somewhere the discussion about the new document builder so I could get a better grip of what’s at bay?
> Would this reverse of logic also be embraced by Hibernate Validator? There are runtime decisions done in HV during traversal that made me doubt that it would be as pertinent.
>
>
>
>> On 30 Jan 2017, at 11:21, Yoann Rodiere <yrodiere at redhat.com> wrote:
>>
>> Hi,
>>
>> Did the same this week-end, and adapted your work to match the bigger picture of what we discussed on Friday.
>> Basically the "StructureTraverser" is now called "ValueProcessor", because it's not responsible for exposing the internals of a structure anymore, but only to process a structure according to previously defined metadata, passing the output to the "DocumentContext". I think it's the second option you suggested. It makes sense in my opinion, since metadata will be defined differently for different source types (POJO, JSON, ...).
>> This design allows in particular what Sanne suggested: when bootstrapping, we can build some kind of "walker" (a composition of "ValueProcessors") from the metadata, and avoid metadata lookup at runtime.
>>
>> The snippet is there: https://gist.github.com/yrodiere/9ff8fe8a8c7f59c1a051b36db20fbd4d <https://gist.github.com/yrodiere/9ff8fe8a8c7f59c1a051b36db20fbd4d>
>>
>> I'm sure it'll have to be refined to address additional constraints, but in its current state it seems to address all of our requirements.
>>
>> Yoann Rodière <yrodiere at redhat.com <mailto:yrodiere at redhat.com>>
>> Software Engineer
>> Red Hat / Hibernate NoORM Team
>>
>> On 27 January 2017 at 18:23, Emmanuel Bernard <emmanuel at hibernate.org <mailto:emmanuel at hibernate.org>> wrote:
>> I took the flight home to play with free form and specifically how we would retrieve data from the free form structure.
>> By free-form I mean non POJO but they will have schema (not expressed here).
>>
>> https://github.com/emmanuelbernard/hibernate-search/commit/0bd3fbab137bdad81bfa5b9934063792a050f537 <https://github.com/emmanuelbernard/hibernate-search/commit/0bd3fbab137bdad81bfa5b9934063792a050f537>
>>
>> And in particular
>> https://github.com/emmanuelbernard/hibernate-search/blob/freeform/freeform/src/main/java/org/hibernate/freeform/StructureTraverser.java <https://github.com/emmanuelbernard/hibernate-search/blob/freeform/freeform/src/main/java/org/hibernate/freeform/StructureTraverser.java>
>> https://github.com/emmanuelbernard/hibernate-search/blob/freeform/freeform/src/main/java/org/hibernate/freeform/pojo/impl/PojoStructureTraverser.java <https://github.com/emmanuelbernard/hibernate-search/blob/freeform/freeform/src/main/java/org/hibernate/freeform/pojo/impl/PojoStructureTraverser.java>
>>
>> It probably does not compile, I could not make the build work.
>>
>> I figured it was important to dump this raw thinking because it will influence and will be influenced by the redesign of the DocumentBuilder of Hibernate Search.
>>
>> There are several options for traversing a free form structure
>> - expose  the traversing API as a holder to  navigate all properties per structure and sub structure. This is what the prototype shows. Caching needs to be accessed via a hashmap get or other lookup. Metadata and the traversing structure will be navigated in parallel
>> - expose a structure that is specialized to a single property or container unwrapping aspect. The structures will be spread across and embedded in the Metadata
>>
>>
>> Another angle:
>> - create a traversable object per payload to carry it (sharing metadata info per type)
>> - have a stateless traversable object that is provided the payload for each access
>>
>> The former seems better as it does not create a traversable object per object navigated.
>> The latter is better for payloads that need parsing or are better at sequential access since state could be cached.
>>
>> We need to discuss that and know where DocumentBuilder is going to properly design this API.
>>
>> Emmanuel
>> _______________________________________________
>> hibernate-dev mailing list
>> hibernate-dev at lists.jboss.org <mailto:hibernate-dev at lists.jboss.org>
>> https://lists.jboss.org/mailman/listinfo/hibernate-dev <https://lists.jboss.org/mailman/listinfo/hibernate-dev>
>>
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev