[Search] Deprecating @Boost and related options
by Sanne Grinovero
Index time boosting is deprecated in Apache Lucene 6, and will be
removed in Lucene 7.
While we currently are using Lucene 5, since we'll likely jump
directly to Lucene 7 with the next major version of Hibernate Search,
I think we should prepare our users already.
Let's deprecate @Boost and any option to control index-time boosting,
so that people can start already to move to query-time boosting.
- https://hibernate.atlassian.net/browse/HSEARCH-2735
6 years, 11 months
Re: [hibernate-dev] "rebranding" the Hibernate Search "ram" directory
by Gunnar Morling
Can't you just move it to the test JAR if it's for testing only?
Am 18.05.2017 6:29 nachm. schrieb "Sanne Grinovero" <sanne(a)hibernate.org>:
Right technically it's not a unit test. But I'd like to focus on the
testing aspect, as "local-ram" might still convey concepts as "fast",
maybe even expect it to engage Infinispan's off-heap capabilities, or
just being an option to consider for other reasons.
"testing" ?
On 18 May 2017 at 17:20, Adrian Nistor <anistor(a)redhat.com> wrote:
> I agree, but probably "unit-testing" is not such a good name either.
> Technically, that's a functional test.
> I think I like "local-ram" better, implying that it is not
> shared/distributed and it is also volatile.
>
>
> On 05/18/2017 06:07 PM, Sanne Grinovero wrote:
>>
>> As anyone who's bothered to read the manual knows, the "ram" directory
>> should really only be used for unit tests. The other implementations,
>> while typically disk based, are also faster (memory mapped files) and
>> more efficient (better locking design) so there's really no reason to
>> use it, not even performance except for trivial, small, non important
>> data sets.
>>
>> For example the Elasticsearch team is making sure of this by having
>> totally removed the option of using the RAMDirectory - something I
>> actually don't appreciate as our unit tests could benefit from it,
>> having slow storage on our test environments.
>>
>> Tristan is reporting that the "ram" terminology is confusing people,
>> not least in the Infinispan community as "RAM" might be ambiguous
>> since everything is in memory, and people get surprised it's not
>> replicated in the "in memory cluster".
>>
>> I wouldn't want to go to the extremes of the Elasticsearch team as I
>> believe having this option is very useful, especially for testing.
>>
>> Should we rename (rebrand) its short name "ram" into "unit-testing" ?
>>
>> I suspect that would make people think a bit more before pushing it
>> into production...
>>
>>
>> Thanks,
>> Sanne
>
>
>
_______________________________________________
hibernate-dev mailing list
hibernate-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/hibernate-dev
6 years, 11 months
"rebranding" the Hibernate Search "ram" directory
by Sanne Grinovero
As anyone who's bothered to read the manual knows, the "ram" directory
should really only be used for unit tests. The other implementations,
while typically disk based, are also faster (memory mapped files) and
more efficient (better locking design) so there's really no reason to
use it, not even performance except for trivial, small, non important
data sets.
For example the Elasticsearch team is making sure of this by having
totally removed the option of using the RAMDirectory - something I
actually don't appreciate as our unit tests could benefit from it,
having slow storage on our test environments.
Tristan is reporting that the "ram" terminology is confusing people,
not least in the Infinispan community as "RAM" might be ambiguous
since everything is in memory, and people get surprised it's not
replicated in the "in memory cluster".
I wouldn't want to go to the extremes of the Elasticsearch team as I
believe having this option is very useful, especially for testing.
Should we rename (rebrand) its short name "ram" into "unit-testing" ?
I suspect that would make people think a bit more before pushing it
into production...
Thanks,
Sanne
6 years, 11 months
org.hibernate.event.spi Listener changes for 6 ?
by Sanne Grinovero
Since 6 is coming I'd like to suggest two trivial changes for the listeners:
# Serializable
I assume we can drop the "extend Serializable" from all these interfaces ?
# PostInsertEventListener.requiresPostCommitHanding(EntityPersister)
I suspect there is a typo in this method?
N.B. there are some more copies of `requiresPostCommitHanding` in
other listeners, beyond the PostInsertEventListener.
Thanks,
Sanne
6 years, 11 months
[ORM] Event Listeners: what's the consequence of not implementing Serializable correctly?
by Sanne Grinovero
As you might know Hibernate Search is implemented mainly as a complex
Event Listener to events
from Hibernate ORM.
The Hibernate ORM event listener contracts "suggest" that such event
listeners need to be Serializable, but it seems our listeners is not
actually serializable since some time.
It appears to be serializable and there's quite some complexity to
make it happen, but I'm realizing that the tests are incomplete and
it's unlikely to work in practice.
Yet everything seems to be fine: no tests failing, no people
complaining, nor bugs have been raised.
Does someone know if there's a realistic use case being broken? How
would a reasonable test look like?
Thanks,
Sanne
6 years, 11 months
Test case for HHH-11740
by Gail Badner
HHH-11740 is a bug that shows up on DB2 because Hibernate is using the
wrong DDL to create a global temporary table that can be shared by multiple
sessions; instead it is creating a temporary table that is visible only to
the connection that creates it.
On Friday I pushed a test case to master that was intended to check that
bulk operations can be executed using two different connections. My
intention was to expose any other dialects that had the same issue as DB.
Unfortunately, the test I pushed caused pessimistic lock exceptions on
multiple dialects. I see that Vlad corrected the test to deal with the
pessimistic lock exceptions. I see now that I should have sent email about
the possible failures and kept a lookout for the test results to avoid the
extra effort on Vlad's part.
I have since pushed an update to that test that avoids the pessimistic lock
failure, and should expose similar problems with other dialects. I'll keep
an eye on it and deal with any failures that show up.
Regards,
Gail
6 years, 11 months
[SEARCH] Master nodes without class definitions
by Yoann Rodiere
Hello,
In our roadmap for 5.8, we have this:
Removal of class definitions from master nodes
> The integration of a "master node" needs to be simplified, both in terms
> of user configuration and implementation. We plan to not need user classes
> (entities) on such a node, so that in future it could be a separate
> download which works out of the box with minimal configuration.
I guess what you are doing, Sanne, about replacing Class<?> references with
an internal binding representation, is part of that, but there doesn't seem
to be any trace of those efforts in JIRA.
Is there a ticket about this? I don't think there is one with a 5.8 "Fix
version", but maybe somewhere else... ?
On a side note, there may be hidden complexity in having
mapping-independent master nodes: we can probably use a generic
representation of documents for document additions/updates, but with
Elasticsearch, and probably very soon also with Lucene 7, we need to
initialize indexes with a schema. So at least one node needs both write
access to the indexes *and* knowledge of the mapping... Which
mapping-independent master nodes would prevent.
Yoann Rodière
Hibernate NoORM Team
yoann(a)hibernate.org
6 years, 11 months
Preventing duplicate ForeignKey generation
by Milo van der Zee
Hello all,
During development of applications I'm used to set the schema creation
to 'update' (hbm2ddl.auto = update). This makes life a bit easier.
Issue with the newer version of Hibernate is that the name of the
generated keys changed and so all keys are regenerated. For the large
databases I use this takes hours and has to be done every time a fresh
copy from production is taken to the development environment.
I do use meaningful names for the indexes where possible. But when using
abstract classes used by the entities that is not possible because the
same fields from the abstract are used by many entity classes and so
would end up having the same index names.
I checked the code that decides if the index needs to be created and
found that it only checks the name of the index. Not what the index
actually does. This is why I changed that piece of code to be a bit
smarter. It is desinged for simple constraints from one column to
another column. Not for multi to multi column indexes and constraints.
I created a Jira issue for it but nobody notices it and there are no
comments or anything else. So now I try it here :)
Jira HHH-10934 (https://hibernate.atlassian.net/browse/HHH-10934)
Code fragment I put in SchemaMigratorImpl.java:
private ForeignKeyInformation findMatchingForeignKey(ForeignKey foreignKey, TableInformation tableInformation) {
if (foreignKey.getName() ==null) {
return null;
}
/*
* Find existing keys based on referencing column and referencedTable
*/
String referencingColumn = foreignKey.getColumn(0).getName();
String referencedTableName = foreignKey.getReferencedTable().getName();
Iterable<ForeignKeyInformation> existingForeignKeys = tableInformation.getForeignKeys();
for (ForeignKeyInformation existingKey : existingForeignKeys) {
Iterable<ColumnReferenceMapping> columnReferenceMappings = existingKey.getColumnReferenceMappings();
for (ColumnReferenceMapping mapping : columnReferenceMappings) {
String existingReferencingColumn = mapping.getReferencingColumnMetadata().getColumnIdentifier().getText();
String existingReferencedTableName = mapping.getReferencedColumnMetadata().getContainingTableInformation().getName().getTableName().getCanonicalName();
if (referencingColumn.equals(existingReferencingColumn) && referencedTableName.equals(existingReferencedTableName)) {
return existingKey;
}
}
}
// If not yet found check based on key name return tableInformation.getForeignKey(Identifier.toIdentifier(foreignKey.getName()));
}
Or if you prever the Java 8 way:
private ForeignKeyInformation findMatchingForeignKey(ForeignKey foreignKey, TableInformation tableInformation) {
log.debug("findMatchingForeignKey");
if (foreignKey.getName() ==null)return null;
/*
* Find existing keys based on referencing column and referencedTable
*/
String referencingColumn = foreignKey.getColumn(0).getName();
String referencedTableName = foreignKey.getReferencedTable().getName();
Predicate<ColumnReferenceMapping> mappingPredicate = m -> referencingColumn.equals(m.getReferencingColumnMetadata().getColumnIdentifier().getText())
&& referencedTableName.equals(m.getReferencedColumnMetadata().getContainingTableInformation().getName().getTableName().getCanonicalName());
for (ForeignKeyInformation existingKey : tableInformation.getForeignKeys()) {
boolean found = StreamSupport.stream(existingKey.getColumnReferenceMappings().spliterator(),false).anyMatch(mappingPredicate);
if (found)return existingKey;
}
// If not yet found check based on key name return tableInformation.getForeignKey(Identifier.toIdentifier(foreignKey.getName()));
}
The calling method does not use the returned value. It only checks if
the returned value is null or not. So this could also be cleaned by
changing the method to return a boolean and then remove the for loop in
java-8 and use flatmap. But first let us agree on the validity of the
idea to change this piece of code.
I hope anybody would like to have a look at it and if there is any
change that the idea (not this actual very quick/dirty implementation)
goes into the system I'll clean it up and do some actual tests for more
complex database structures. I did not even check the junit tests yet.
At the moment it is good enough for me but I think it could be something
more people would benefit from.
Thanks,
Milo van der Zee
6 years, 11 months
Questions on HS + ES
by Gunnar Morling
Hi,
I recently got some good questions on Hibernate Search + Elastisearch:
* Can we have multiple indexes for the same entity, with different
fields, geared towards different use cases?
* How could we deal with authorization for seeing specific ES query
results? I reckon that's something to mostly (solely?) handle on the
ES side?
* Are dynamic analyzers supported for the ES backend?
* And a wish for the docs: make clear which kind of changes towards
mapped entities require a re-index run?
Any replies would be welcome.
Thanks,
--Gunnar
6 years, 11 months
