[hibernate-dev] [Hibernate Search] Database back end worker

[Flemming Harms]

Hi Martin

For this version the AbstractDatabaseHibernateSearchController is not able
to process Lucene workers simultaneously. When we build it our initial
requirement was only one node should process the workers at a time, but the
“master” was floating. We use Quartz to get this type of functionality and
it will synchronizing the execution between the nodes. But you could also
use an HA-singleton to dedicate a specific node to process the workers.

We had been playing with an idea where we stamp the LuceneDatabaseWork with
the known cluster nodes, and then the last node will remove it from the
database or a scheduled job can take care of it. The advance of this
solution is it will make Infinispan optional, and it can store the indexes
on each node instead in a shared cache.

Your idea and work look very nice. Pretty awesome feature to support
different JPA providers.

-- 
cheers
Flemming


2015-08-05 11:57 GMT+02:00 Martin Braun <martinbraun123 at aol.com>:

> Hi,
>
>
> Note: I am no core developer of Hibernate Search, but I am currently
> working on something
> that looks quite similar to what you are doing :). One part of it is an
> updating mechanism based on triggers
> that uses the database as a event-storage as well. It's not the exact same
> thing, but related.
>
>
> https://github.com/Hotware/Hibernate-Search-JPA
>
>
>
> The idea is quite nice, but after looking at the sourcecode I am wondering
> how the different nodes are able to work together, because in
> AbstractDatabaseHibernateSearchController you remove the entity
> from the persistence context and I wasn't able to find code that would
> make up for that.
>
>
> Doesn't that mean that the other workers will not be able to read that
> entity?
> Or will users of this need to implement their own synchronization
> mechanism between
> the different nodes?
>
>
> Martin Braun
> martinbraun123 at aol.com
> www.github.com/s4ke
>
>
>
>
> -----Original Message-----
> From: Flemming Harms <flemming.harms at gmail.com>
> To: Hibernate.org <hibernate-dev at lists.jboss.org>
> Sent: Tue, Aug 4, 2015 6:40 pm
> Subject: [hibernate-dev] [Hibernate Search] Database back end worker
>
>
> Hey guys
>
> I want to introduce myself and a new database back-end worker, me
> and
> another guy have build for hibernate search. I already had some initial
> talk
> with Sanne regarding if this could be interested to the hibernate
> search
> project.
>
> I have been working with Hibernate Search from some time and actually
> done
> various small custom modification to search since 3.x, especial
> around
> running in a cluster and indexing. To make a long story short when
> we
> upgraded Hibernate search we thought it would be ideal to use a SQL
> database
> as storage for lucene workers for 3 main reasons.
>
> - The database was shared
> between the nodes
> - The workers was persistent in case of a node crash.
> - No
> master/slave
>
>
> *In some way it’s very similar to the JMS back-end worker, where
> the user
> also have to implement a MDB that process the workers. In our case
> they
> will have to implement a job using something like quartz or a
> timer
> service. *
>
> *We are using JPA as persistence layer for the database, even
> it’s a fairly
> simple entity we persistent, but it make sense for supporting
> various
> databases and schema update out of the box. We have tried to make it’s
> as
> easy as possible to set-up by minimizing the number of properties, and
> it’s
> all configurable from the persistence.xml*
>
> *The actually work can* be
> *find
> here
> https://github.com/umbrew/org.umbrew.hibernate.database.worker.backend
> <https://github.com/umbrew/org.umbrew.hibernate.database.worker.backend>*
>
>
>
> *So
> based on this introduction and the code, is this something you could
> use? (of
> course with the modification it requires to follow the design,
> style, docs etc
> for the search)*--
>
> Kind regards
> Flemming
> Harms
> _______________________________________________
> hibernate-dev mailing
> list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev
>
>
> _______________________________________________
> hibernate-dev mailing list
> hibernate-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/hibernate-dev




-- 

Kind regards / Med Venlig Hilsen
Flemming Harms

   -

https://twitter.com/fnharms
https://dk.linkedin.com/in/fharms