October 2014 - hibernate-dev - Jboss List Archives

Improving performance of index-writing for Hibernate Search

by Sanne Grinovero

Hi all, after some performance tests on Infinispan we concluded that the way Hibernate Search writes to the index is not as efficient as we'd like to. While these results came from extensive Infinispan testing, it's actually highlighting a performance issue which isn't strictly related to Infinispan storage, and could affect any kind of index storage which we support. Surprisingly this wasn't too bad for normal disk based FSDirectory, especially as we expect users to got for the NRT backend, but as you might know an NRT backend is not suited for clustered deployments (be it using a master/slave approach or the Infinispan index, or other more experimental shared index strategies like NFS based sharing). So, while NRT performs great, the "normal" backend actually doesn't perform very well because of the high frequency of commits. Surprisingly even an ASYNC backend does the same amount of commits, so enabling the async workers of Hibernate Search would decouple the latency of the main thread from the speed of the backend, but if you have a sustained amount of writes the overall throughput that the backend is able to deliver would not change. A much better design is to allow the ASYNC backends to _not_ flush at each commit, but to flush periodically. The overall throughput increase is estimated to be 2 to 3 orders of magnitude, details depending on the kind of storage you're having. This idea is being proposed as: - https://hibernate.atlassian.net/browse/HSEARCH-1693 Gustavo sent a first pull request for this already, some preliminary results can be found on the pull request description: https://github.com/hibernate/hibernate-search/pull/681 Note that his tests focus on Infinispan storage but the code is unrelated to Infinispan and will benefit all users independently from the storage technology. This patch will expose a new configuration property which will allow users to specify how often they need the indexes to be committed; in other words, you'll be able to specify that Async is acceptable for your application, but as long as the "staleness" of your index doesn't exceed a set threshold of time. Now the best part of this, is that we can also propose a solution for the SYNC backends which require an illusion of transactional behaviour. Essentially we're aiming at implementing a synchronous backend which has performance close to the capabilities of NRT, but without the drawbacks so that it can safely be used on shared indexes. This ideas is a bit more complex to implement, I've attempted to describe it on: https://hibernate.atlassian.net/browse/HSEARCH-1699 Sanne

11 years, 2 months

1
0
0 / 0

Re: [hibernate-dev] Analysis of hibernate-release-4.3.6 dependency on JDK-Internal APIs

by Emmanuel Bernard

Hi, Three of these errors are related to Infinispan which you have already contacted. Two are related to dom4j which we use for XML parsing. While I think our next gen version will no longer use dom4j, we don’t plan on moving away from dom4j for the older and current releases. So I am not sure why you guys are removing org.relaxng form the JDK but if you do then we won’t be able to have Hibernate ORM running on JDK 9. Emmanuel On 10 Oct 2014, at 12:00, Rory O'Donnell Oracle, Dublin Ireland <rory.odonnell(a)oracle.com> wrote: > Hi Sanne, > > Did you have time to review the report, any feedback ? > > Rgds,Rory > > On 11/09/2014 15:14, Rory O'Donnell Oracle, Dublin Ireland wrote: >> Hi Sanne, >> >> As part of the preparations for JDK 9, Oracle’s engineers have been analyzing open source projects like yours to understand usage. >> One area of concern involves identifying compatibility problems, such as reliance on JDK-internal APIs. >> >> Our engineers have already prepared guidance on migrating some of the more common usage patterns of JDK-internal APIs to supported public interfaces. The list is on the OpenJDK wiki [0], along with instructions on how to run the jdeps analysis tool yourself . >> >> As part of the ongoing development of JDK 9, I would like to encourage migration from JDK-internal APIs towards the supported Java APIs. I have prepared a report for your project release hibernate-release-4.3.6 based on the jdeps output. >> >> The report is attached to this e-mail. >> >> For anything where your migration path is unclear, I would appreciate comments on the JDK-internal API usage patterns in the attached jdeps report, (if any) - in particular comments elaborating on the rationale for them - either to me or on the adoption-discuss [1] mailing list at OpenJDK. >> >> Finding suitable replacements for unsupported interfaces is not always straightforward, which is why I am reaching out to you early in the JDK 9 development cycle so you can give feedback about new APIs that may be needed to facilitate this exercise. >> >> Thank you in advance for any efforts and feedback helping us make JDK 9 better, >> Rgds,Rory >> >> [0] https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool >> [1] http://mail.openjdk.java.net/mailman/listinfo/adoption-discuss >> > > -- > Rgds,Rory O'Donnell > Quality Engineering Manager > Oracle EMEA , Dublin, Ireland > > > >

11 years, 2 months

2
1
0 / 0

IRC

by Steve Ebersole

just a heads up that I have not been able to connect to freenode for quite a few weeks now. Still no idea what is up there.

11 years, 2 months

5
7
0 / 0

WildFly deployments: how to access the (user app) deployment classloader from Hibernate code?

by Sanne Grinovero

It looks like that when Hibernate Search is being bootstrapped in a JPA deployment on WidFly the current context classloader was overriden. That's expected I guess, but it's of a type which surprised me: - org.hibernate.boot.registry.classloading.internal.ClassLoaderServiceImpl$AggregatedClassLoader while I really need to access the deployment classloader; on WildFly this would be a ModularClassloader having a toString of the form: ModuleClassLoader for Module "deployment.DeploymentName:main" from Service Module Loader The ClassLoaderService itself is of no use in this case as it only exposes the same aggregate ClassLoader which is loading Hibernate Search, while I need to get fine-grained access to the ClassLoader which is loading the user deployment. There is no such problem when I start the Hibernate SessionFactory directly (not in a container): in that case the Context Classloader matches the system one so I don't think is being set by Hibernate ORM. If such a semantics is needed in WildFly, could we then expose the original TCCL via an accessor on ClassLoaderService ? Thanks, Sanne

11 years, 2 months

1
0
0 / 0

Publishing documentation as rendered by asciidoc

by Sanne Grinovero

Hi all, I would love to publish the documentation the way it looks like from the asciidoc rendering (before the transformations via docbook), as it looks like much more readable. But I'd like to keep our style and branding rather than the default docbook template; did someone already experiment with that? Or could anyone volunteer please as my design skills are better avoided :-) Cheers, Sanne

11 years, 2 months

4
10
0 / 0

dehydrate knows acout the Hibernate mapping, disassemble does not.

by Eric J. Van der Velden

Hello, I have to correct my code because I work with a second level cache in the following case (1-N bidirectional). Suppose we have a Container class type, with a Set<Item>, and an Item class type. In the Hibernate mapping we make the Set non-inverse. For persistence of the Item instances we only have to add them to the Container instance's Set, and I do not have to set the Container reference in the Item instance. When an Item is inserted in the database table, the Item-persister.dehydrate method skips the Container reference in the Item instance, and takes the Long which is there because of the BackrefPropertyAccessor$BackrefGetter.getForInsert() method. This will be the foreign key in the row. The Item-persister.getPropertyInsertability()=[true,true,false,true] for example, and the method Item-persister.dehydrate knows about it: The false means: skip the Container reference in the Item instance, and the llast true means: take this number (instead). My question is: Why does the Item-persister.disassemble not take into account the Hibernate mapping? Because it does not, I have to make a correction in my code to set the Container reference in the Item instances, only for the second level cache. Thanks.

11 years, 2 months

2
1
0 / 0

Signing up for JIRA

by Gunnar Morling

Hi, In JIRA I'm seeing this permanent message: "Need to file a bug report and don't have an account? Email emmanuel at hibernate dot org with JIRA Account creation as subject. You'll receive an invite by email." So have we abandoned the self sign-up? Why is this, due to spamming issues? Not sure whether it's really a problem, but this may raise the barrier for getting a JIRA account and thus stop some new users from reporting bugs. --Gunnar

11 years, 2 months

4
10
0 / 0

[Search] Internship next year

by Guillaume Smet

Hi, Just a heads up: I'm going to propose an internship this year (scholar year more precisely) on Hibernate Search contribution. I'm not sure I'll find the good intern for this but that's the plan. The internship season usually starts in Feb/March in France so it's for early next year. The internship is located in Lyon, France and if you happen to meet a good student interested in it, I'll be glad to meet him. I usually am interested in final year student of French "Grandes Ecoles" as we name it here but brilliant persons in general are welcome :). I have a couple of subjects in mind (like finishing the work I started on plain text search) but I'm sure we'll be able to find good ideas to keep a person busy during several months! -- Guillaume

11 years, 2 months

2
1
0 / 0

MongoDB object id support and restructured public API - Hibernate OGM 4.1.0.Beta7 is out

by Gunnar Morling

Hi, It’s my great pleasure to announce the release of Hibernate OGM 4.1.0.Beta7! The release brings support for MongoDB’s object id, a clarification of what are our public API/SPI packages as well as several bugfixes and many other improvements. Check out the announcement post [1] for all the details. --Gunnar [1] http://in.relation.to/Bloggers/MongoDBObjectIdSupportAndRestructuredPubli...

11 years, 2 months

1
0
0 / 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

hibernate-dev October 2014