[HSEARCH][V6-POC] FieldBridge 2.0
by Yoann Rodiere
Hello,
In Search 6, we are changing the field bridge APIs to move away from Lucene
dependencies. Similarly to the query DSL, this breaks backward
compatibility whatever we do, so we had a look to improving the field
bridge API a bit with some other changes.
The result is we have three types of bridges:
- *ValueBridges* can be applied to values, i.e. either a property value
directly, or container values extracted from the property (list elements,
map keys, ...). They cannot be parameterized and cannot have dedicated
annotations (at least for now), and they are applied to a mapping using
@Field(valueBridge = ...). They are supposed to be applied to simple types
(LocalDate, some Enum, some immutable user type, ...) and perform a 1-1
mapping between the value and a *single* index field. Implementations
are supposed to work as bidirectional functions: one method transforms the
POJO-side type into the Index-side type, and one method does the opposite.
- *TypeBridges* can be applied to types, can be parameterized, they each
have a dedicated annotation (@MyCustomTypeBridge for instance). Their
implementations are supposed to use specific APIs to access both the
property values and the index fields, so that Hibernate Search is aware of
the necessary input data (useful for dirty checking) and of the index
fields created by the bridge, along with their characteristics (field type,
stored or not, analyzer, ...).
- *PropertyBridges* can be applied to properties, can be
parameterized, they each have a dedicated annotation
(@MyCustomPropertyBridge for instance). Their implementations are supposed
to use specific APIs to access both the property values and the index
fields, so that Hibernate Search is aware of the necessary input data
(useful for dirty checking) and of the index fields created by the bridge,
along with their characteristics (field type, stored or not, analyzer, ...).
The expectation is that most users will only need to implement ValueBridge,
which is the simpler one. Users with more advanced needs will be able to
implement TypeBridge/PropertyBridge, which are more complex but are also
much more powerful.
Some aspects of the bridge API are still in the works, so don't be alarmed
to not see them:
- Access to the type/property data in TypeBridge and PropertyBridge is
still a bit rough, missing in particular support for container types (List,
Map, ...).
- During projections, transforming the projected data using bridges is
not supported yet.
The showcase contains a few simple bridge implementations.
- org.hibernate.search.v6poc.integrationtest.showcase.library.bridge.
*ISBNBridge* is a ValueBridge. Hibernate Search is able to detect from
its class metadata that it can only be applied to ISBN values, and
generates a String field.
The bridge is applied using a @Field annotation
on org.hibernate.search.v6poc.integrationtest.showcase.library.model.Book#isbn.
You can find an equivalent use of the bridge in the programmatic mapping
in org.hibernate.search.v6poc.integrationtest.showcase.library.model.ProgrammaticMappingContributor.
- Another value bridge is applied
to org.hibernate.search.v6poc.integrationtest.showcase.library.model.Library#services
and is implicitly applied to every value of the list.
- org.hibernate.search.v6poc.integrationtest.showcase.library.bridge.
*MultiKeywordStringBridge* is a PropertyBridge which turns a
comma-separated string into multiple field values. It is admittedly a very
simple PropertyBridge, but it has the advantage of showing pretty clearly
how a custom bridge is declared and implemented.
The bridge is applied using its custom "@MultiKeywordStringBridge"
annotation
on org.hibernate.search.v6poc.integrationtest.showcase.library.model.Document#tags.
You can find an equivalent use of the bridge in the programmatic mapping
in
org.hibernate.search.v6poc.integrationtest.showcase.library.model.ProgrammaticMappingContributor:
look for the line where "MultiKeywordStringBridge.Builder" is used.
- org.hibernate.search.v6poc.entity.pojo.bridge.builtin.spatial.
*GeoPointBridge* is a more representative example of what an advanced
bridge could be. It's a built-in bridge, but it could definitely be
implemented by a third-party: no internal APIs are used in its
implementation. Note that one characteristic of this field is that it can
be used both as a TypeBridge or as a PropertyBridge, just as the @Spatial
annotation in Search 5.
The bridge is applied using its dedicated "@GeoPointBridge" annotation
to
type org.hibernate.search.v6poc.integrationtest.showcase.library.model.Library,
with "@Latitude" and "@Longitude" annotations on the relevant properties of
this type.
You can find an equivalent use of the bridge in the programmatic mapping
in
org.hibernate.search.v6poc.integrationtest.showcase.library.model.ProgrammaticMappingContributor:
look for the line where "GeoPointBridge.Builder" is used, and the lines
where "LatitudeMarker.Builder" and "LongitudeMarker.Builder" are used.
So, with those few examples of bridges... WDYT?
--
Yoann Rodiere
yoann(a)hibernate.org / yrodiere(a)redhat.com
Software Engineer
Hibernate NoORM team
6 years, 1 month
Cache testing & JCache
by Steve Ebersole
The initial hibernate-jcache code would create JCache Cache instance "on
the fly" as they were needed if not already defined. As part of a
discussion about JCache support, it was agreed that this should not happen.
To be clear... the integration code
calls javax.cache.CacheManager#getCache(java.lang.String). If we are
returned nothing, its a choice between throwing an exception or
calling javax.cache.CacheManager#createCache. The consensus among the
cache providers was that an exception should be thrown as the standard
outcome.
However, in testing I wonder if we want to allow that "on the fly" creation
to happen. Opinions? And if not, thoughts on how to best pre-define all
the needed Cache instances?
6 years, 1 month
[HSEARCH][V6-POC] Syntax for query building
by Yoann Rodiere
Hello,
In Search 6, we are changing the Query DSL APIs to move away from Lucene
dependencies. Since this breaks backward compatibility whatever we do, we
had a look to improve the the syntax a little bit with some other changes.
In particular, for query predicate building, we experimented with three
syntaxes when it comes to nesting:
- a fully-fluid syntax, allowing nested predicate definition through
chained calls (and potentially an ".end()" call where necessary to get back
to the parent predicate).
- a lambda-based syntax, allowing nested predicate definition by passing
lambdas that create the nested predicate.
- an "object-based" syntax, allowing to create an object representing
your predicate, and then pass it as an argument to another predicate to
nest it. This is pretty similar to the QueryBuilder syntax in Search 5.
In the showcase module of the proof-of-concept (
https://github.com/hibernate/hibernate-search-6-poc/tree/master/integrati...),
I included multiple implementations of the DAOs to demonstrate various
syntaxes. Since the fully-fluid syntax does not allow to handle conditions
or loops within a single chained call, some of the implementations actually
mix multiple syntaxes:
- Package
org.hibernate.search.v6poc.integrationtest.showcase.library.dao.syntax.object
demonstrates pure "object-based" syntax
- Package org.hibernate.search.v6poc.integrationtest.showcase.library.dao.syntax.lambda
demonstrates pure "lambda-based" syntax
- Package org.hibernate.search.v6poc.integrationtest.showcase.library.dao.syntax.fluidandlambda
demonstrates fluid syntax resorting to lambda-based syntax when conditions
or loops are necessary.
- Package org.hibernate.search.v6poc.integrationtest.showcase.library.dao.syntax.fluidandobject
demonstrates fluid syntax resorting to object-based syntax when conditions
or loops are necessary.
My questions:
- Do all of these query building syntaxes make sense to you?
- Do you think we need all of them?
- Is one better than the others?
Some elements of thought:
- There may not be a "one-size-fits-all" solution; I personally like the
"fluid" syntax best for "shallow" queries with little to no nesting, and
the "fluid+lambda" syntax best for deeply nested queries. To make your own
opinion, you can have a look at implementations of
"DocumentDao#searchAroundMe" for a query with complex nesting, and at
implementations of "LibraryDao#search" for a "shallow" query.
- Purely lambda or purely object syntax, in my opinion, are more
verbose. The syntax feels more "to the point" where we can use chained
calls (fluid syntax).
- The fluid syntax is not very IDE-friendly when nesting predicates, as
code formatting will ignore the implicit structure of chained calls (it
will remove indents that previously made nesting obvious).
- With lambdas being regularly advised against for various reasons, we
might want to offer at least one other way of doing things.
--
Yoann Rodiere
yoann(a)hibernate.org / yrodiere(a)redhat.com
Software Engineer
Hibernate NoORM team
6 years, 1 month
[HSEARCH][V6-POC] RFC on the Hibernate Search 6 APIs
by Yoann Rodiere
Hello,
It's been some time we have been working on a proof-of-concept for Search
6, and I think it is mature enough to receive some early feedback. So to
anyone interested: I would love to read your comments! Note that the APIs
are not set in stone yet, everything could still change depending on the
feedback.
To help you form an opinion, I added a "showcase" module with a relatively
realistic domain model (that of Libraries and their Books), a Hibernate
Search mapping and some queries. The showcase uses the Elasticsearch
backend, which is (for now) the only one available; Guillaume is working on
the Lucene one.
The proof-of-concept is available here:
https://github.com/hibernate/hibernate-search-6-poc and the showcase is
located in the "integrationtest/showcase/library" directory.
You can find:
- the domain model, mapped using annotations,
in org.hibernate.search.v6poc.integrationtest.showcase.library.model
- the data access objects implementing CRUD operations
in org.hibernate.search.v6poc.integrationtest.showcase.library.dao; there
are various implementations of the same DAO contracts (more on this in
another email thread).
- the custom bridge implementations
in org.hibernate.search.v6poc.integrationtest.showcase.library.bridge (more
on this in another email thread)
A test allowing you to see it all in action is available in
src/test/java/org/hibernate/search/v6poc/integrationtest/showcase/OrmElasticsearchLibraryShowcaseIT.java
. You can launch it:
- from the command line with "mvn clean install -DskipTests; mvn clean
verify -pl integrationtest/showcase/library" (an Elasticsearch instance
will be spawned automatically)
- or more interstingly from your IDE, with breakpoints. You will need an
Elasticsearch cluster running and accessible from localhost:9200.
I will be starting separate email threads for some topics I would
especially like feedback on. If there are some other topics you would like
discussed, feel free to answer here or start another thread.
As a reminder, and as an explanation of why so many things changed, here
are the main goals of Hibernate Search 6:
- Overhaul the APIs to remove the dependency to Lucene, so that other
technologies can be integrated without hacks (Elasticsearch in particular,
but also maybe in the future Solr).
- Overhaul the bridge APIs to allow:
- Better integration with other technologies (such as Elasticsearch),
in particular by abstracting access to indexed documents (HSEARCH-2186
<https://hibernate.atlassian.net/browse/HSEARCH-2186>)
- Detection by Hibernate Search of the properties being used, for
better dirty checking [not yet implemented, but the APIs should
allow that]
(HSEARCH-2496 <https://hibernate.atlassian.net/browse/HSEARCH-2496>)
- Overhaul the Query API to:
- offer a less verbose syntax for simpler queries
- return typed results: if you query the Book index, you get a
List<Book>, not a raw List (HSEARCH-2225
<https://hibernate.atlassian.net/browse/HSEARCH-2225>)
- be generic, yet extensible depending on the integration (e.g. allow
to nest Elasticsearch JSON inside a query predicate) (HSEARCH-2498
<https://hibernate.atlassian.net/browse/HSEARCH-2498>)
- Generally ensure the architecture:
- will allow detection by Hibernate Search of the load graph to use
when fetching entities from Hibernate ORM for mass indexing.
- will make it easier to implement alternative mappers in the future (
HSEARCH-1800 <https://hibernate.atlassian.net/browse/HSEARCH-1800>,
HSEARCH-1401 <https://hibernate.atlassian.net/browse/HSEARCH-1401>).
While we were at it, I gave a shot to some additional features I felt could
be useful:
- User-defined bridge annotations for user-defined bridges (HSEARCH-175
<https://hibernate.atlassian.net/browse/HSEARCH-175>)
- Give access to more application context in the bridges (HSEARCH-1839
<https://hibernate.atlassian.net/browse/HSEARCH-1839>)
- Comply with the JPA access type when accessing entity properties (
HSEARCH-383 <https://hibernate.atlassian.net/browse/HSEARCH-383>)
--
Yoann Rodiere
yoann(a)hibernate.org / yrodiere(a)redhat.com
Software Engineer
Hibernate NoORM team
6 years, 1 month
5.3 - hibernate-ehcache
by Steve Ebersole
Given the changes to the second-level cache SPI, I wonder if we want to
take that as an opportunity to consider dropping the `hibernate-ehcache`
module?
There are a few reasons I am considering this...
1. We have the JCache integration in place and users can use Ehcache
via that support already.
2. This is really the same discussion we had wrt Spring Cache[1]. IMO
the answer should be consistent. We moved hibernate-infinispan for many of
these same reasons...
3. A large part of the argument for keeping hibernate-ehcache was that
it is already in place and therefore did not require much effort to
support/maintain. However, given the new SPI there is actually a pretty
big effort to adapt hibernate-ehcache to those new SPIs. So far I am the
one doing that - which in and of itself is fine since I am the one who
changed the SPIs ;) But it makes me think it is far less of a maintenance
effort just to drop it and support just hibernate-ehcache.
WDYT?
[1] https://github.com/hibernate/hibernate-orm/pull/1639
6 years, 1 month
Re: [hibernate-dev] 5.3 - hibernate-ehcache
by Sanne Grinovero
On 15 March 2018 at 13:58, Steve Ebersole <steve(a)hibernate.org> wrote:
> On Thu, Mar 15, 2018 at 8:47 AM Sanne Grinovero <sanne(a)hibernate.org> wrote:
>>
>>
>> BTW what about all of Alex's sophisticated integration tests? I assume
>> they are more thorough in the older Ehcache module than in the JCache
>> module; would be a shame to lose those verifications.
>
>
> We'd pull those over to hibernate-jcache. I think that is actually a good
> start on a "caching TCK" like Radim has asked about a few times.
Sounds great, I just hope that it won't make the amount of work for
you even worse than just adjusting the SPIs. Probably still worth it
for the better maintainable outcome.
6 years, 1 month
Joins over type variable defined assocations NPE problems
by Christian Beikov
Hey,
so the problems related to the type variable handling I did aren't fully
solved. There is one other case that was reported today:
https://hibernate.atlassian.net/browse/HHH-12375
I'm not sure how to handle this. Let me explain you the model and the
problem.
@Entity class BaseEntity { @Id Integer id; }
@Entity class EntityA extends BaseEntity { @ManyToOne EntityC relation; }
@Entity class EntityB extends BaseEntity { @ManyToOne EntityD relation; }
@Entity class EntityC {...}
@Entity class EntityD {...}
The essence is, that EntityA and EntityB define properties with the same
names but have unrelated types. It's actually the same problem for
collections.
Since EntityA and EntityB extend BaseEntity and because BaseEntity is an
@Entity, the Hibernate code tries to add "EntityA.relation" and
"EntityB.relation" to the PropertyMapping of BaseEntity. Before my fix,
the property which is added first won and subsequent adds are ignored.
This is presumably done to support implicit downcasting. My fix was
about making use of a common super type if possible and disallowing
ambiguous access.
The problem the user now reported is, when using FetchType.EAGER there
is an exception which states that property "relation" could not be found
on type "BaseEntity". This is because I register "null" as type to
signal that it's not really possible to use that property with that
persister(BaseEntity in this case). During booting, the load plan for
BaseEntity is built, for which the SQL is pre-generated. At this point,
the exception happens, because "null" is registered as type for the
property "relation". This will cause the property to be considered as
non-existant wrt getPropertyType. The code before my fix was essentially
broken, because it doesn't handle eager fetching properly due to the
"first property wins" strategy. A property might only be loaded through
a secondary query due to lazy loading rather than by the defined fetch
style. An implicit downcasted join using the property might lead to
non-deterministic results.
One of the reasons for doing the fix, was to prevent the possibility of
being hit by the non-determinism. I had a mapping that was using the
type variable approach and I had to find out the non-determinism the
hard way. I'd like to prevent access to such conflicting attributes from
the top level type through implicit downcasting, so IMO the only real
solution is to exclude the conflicting attributes in the load plan.
The "best" solution would be to introduce a synthetic type that
represents the union of the unrelated property types, but to me this
seems to be quite a lot of work.
I have implemented the proposed solution here:
https://github.com/hibernate/hibernate-orm/pull/2190
We need to release this fix soon as the current state is broken for the
EAGER configuration. What do you think about the solution or the matter
in general?
--
Mit freundlichen Grüßen,
------------------------------------------------------------------------
*Christian Beikov*
6 years, 1 month
Hibernate support for JDBC drivers targeting multiple JVM versions
by Bregler, Jonathan
Hi,
in a recent version of the HANA JDBC driver the new JDBC 4.3 features that came with Java 9 have been implemented. The driver itself is still compiled for Java 7 (javac -target 1.7). So the driver should also be usable with a JVM 7 or 8. This works as expected until Java reflection is used to determine, for example, the existence of a method on the connection class. Hibernate uses this approach in org.hibernate.engine.jdbc.env.internal.DefaultSchemaNameResolver#determineAppropriateResolverDelegate to determine if the connection class implements the #getSchema method. In this case the JVM tries to load the entire connection class including the non-existing new interface java.sql.ShardingKey. The result is a java.lang.NoClassDefFoundError being thrown which isn't caught anywhere causing the Hibernate bootstrapping process to fail. I've attached a sample stack trace to this mail.
Situations like this can also occur in other places, for example, when getting a connection from a Hikari connection pool.
My question is now how you think Hibernate should handle situations like this. Do you see it as a JDBC driver issue? Should Hibernate ignore the error and continue with a conservative guess if possible (e.g. assume that the connection class doesn't implement #getSchema)?
Thanks,
Jonathan
6 years, 1 month
