[Hibernate-JIRA] Created: (HSEARCH-880) Discussion on how to support backward / forward compatible serialization layer
by Emmanuel Bernard (JIRA)
Discussion on how to support backward / forward compatible serialization layer
------------------------------------------------------------------------------
Key: HSEARCH-880
URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-880
Project: Hibernate Search
Issue Type: New Feature
Components: serialization
Reporter: Emmanuel Bernard
h1. General principles
The serialized message needs the following elements:
* index name: to redirect the flux to the appropriate backend
* serialization provider id: if not present, a cluster must make sure to use the same SerializationProvider for a given IndexManager
* protocol version: today the version is major.minor where the major increase means incompatibility at the stream level, whereas minor means compatibility but with missing features
* stream: this is the SerializationProvider specific byte[]
bq. Do we need a serialization provider id? In other words, do we need to be able to hot-upgrade the SerializationProvider in a cluster?
h1. Exchanging messages in an heterogeneous cluster
h2. Cluster with one way communication (JMS)
In this case the master receives a message and must try and process it.
Receives an index name + serial provider id.
Use the serial provider id to deserialize the message.
If message_major > node_major, the serialization provider fails
If message_minor > node_minor, the serialization provider proceeds but some features might not be supported and the deserialization might fail.
In the minor bump case:
* some feature might not be deserialized and simply ignored. A user is aware of the list of features differences between each node.
* the stream might not be readable by an old version after all due to the use of some new features => Exception
If message_major or message_minor < node_major or node_minor, we use the older protocol deserializer.
bq. could there ever be a problem where a new HSearch Engine cannot deal with an old HSearch engine's message?
h2. Cluster with two way communication (JMS)
Each time a node A needs to send a message to a node B for the first time. It sends the list of supported SerializationProvider id and for each the list of Versions supported. The first SerializationProvider id is preferred and the latest versions are preferred.
A version is more recent if majorA > majorB and with majorA = majorB if minorA > minorB.
Node B receives the handshake message and returns the appropriate serialization provider id and version. Subsequent messages are exchanged with this accepted version between A and B
bq. Is the JGroups clustering using multicast to send change messages ie does it know which node it sends the message to to do the handshake?
bq. What happens if B goes down and back up? Does it have a "new" name that uniquely identify it?
h1. API changes
SerializationProvider will need the following adjustments:
* a getSupportedVersions()
* a getSerializer(Version)
* a getDeserializer(Version)
bq. could it be that Serializer / Deserializer / LuceneWorksBuilder lead to the inability to support a version n-1 (by adding of new methods or stuff like that?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[Hibernate-JIRA] Created: (HSEARCH-841) after deleting an entity lucene index updating fails due to NullPointerException
by Bogdan Butnaru (JIRA)
after deleting an entity lucene index updating fails due to NullPointerException
--------------------------------------------------------------------------------
Key: HSEARCH-841
URL: http://opensource.atlassian.com/projects/hibernate/browse/HSEARCH-841
Project: Hibernate Search
Issue Type: Bug
Components: engine
Affects Versions: 3.4.0.Final
Environment: Hibernate 3.4.0 (on MySQL 5.1.54, but any database is affected AFAIK)
Reporter: Bogdan Butnaru
When entities are deleted, I noticed an exception with a rather large stack trace (below), which appears to be caused by the fact that Lucene is told to remove from the index a document with a null id.
After (quite) a bit of digging, I believe the source is the top of this stack trace:
DocumentBuilderIndexedEntity<T>.getId(Object) line: 662
WorkPlan$PerClassWork<T>.extractProperId(Work<T>) line: 233
WorkPlan$PerClassWork<T>.addWork(Work<T>) line: 205
WorkPlan.addWork(Work<T>) line: 79
WorkQueue.add(Work) line: 73
BatchedQueueingProcessor.add(Work, WorkQueue) line: 115
PostTransactionWorkQueueSynchronization.add(Work) line: 70
TransactionalWorker.performWork(Work<?>, TransactionContext) line: 81
FullTextIndexEventListener.processWork(T, Serializable, WorkType, AbstractEvent, boolean) line: 215
FullTextIndexEventListener.processCollectionEvent(AbstractCollectionEvent) line: 262
FullTextIndexEventListener.onPostRemoveCollection(PostCollectionRemoveEvent) line: 227
CollectionRemoveAction.postRemove() line: 146
CollectionRemoveAction.execute() line: 119
ActionQueue.execute(Executable) line: 273
ActionQueue.executeActions(List) line: 265
ActionQueue.executeActions() line: 186
EJB3FlushEventListener(AbstractFlushingEventListener).performExecutions(EventSource) line: 321
EJB3FlushEventListener(DefaultFlushEventListener).onFlush(FlushEvent) line: 51
SessionImpl.flush() line: 1216
SessionImpl.managedFlush() line: 383
JDBCTransaction.commit() line: 133
TransactionImpl.commit() line: 76
[only application-specific lines below here, omitted]
Specifically, org.hibernate.search.engine.DocumentBuilderIndexedEntity.getId(Object) does:
Object unproxiedEntity = HibernateHelper.unproxy( entity );
return (Serializable) ReflectionHelper.getMemberValue( unproxiedEntity, idGetter );
As far as I can tell, unproxy(entity) will return an entity with a null id if called during postRemove; the returned result is null, although the debugger says that entity.id actually contains the correct (non-null, pre-deletion) value.
org.hibernate.search.engine.WorkPlan.PerClassWork.extractProperId(Work<T>), right below the top of the stack, does:
if ( entity == null || documentBuilder.requiresProvidedId() ||
( work.isIdentifierWasRolledBack() && documentBuilder.isIdMatchingJpaId() )) {
return work.getId();
} else {
return documentBuilder.getId( entity );
}
This condition fails and goes into the "else" block, which is unfortunate, because work.getId() actually contains the correct value.
The error doesn't occur right away: org.hibernate.search.engine.WorkPlan.PerClassWork.addWork(Work<T>) (a bit below the top of the stack) calls "entityById.put( id, entityWork );", with id=null due to the above.
A while later the following happens (the bottom two items are the same as the stack trace above):
DocumentBuilderIndexedEntity<T>.addWorkToQueue(Class<T>, T, Serializable, boolean, boolean, boolean, List<LuceneWork>) line: 355
WorkPlan$PerEntityWork<T>.enqueueLuceneWork(Class<T>, Serializable, AbstractDocumentBuilder<T>, List<LuceneWork>) line: 449
WorkPlan$PerClassWork<T>.enqueueLuceneWork(List<LuceneWork>) line: 250
WorkPlan.getPlannedLuceneWork() line: 150
WorkQueue.prepareWorkPlan() line: 134
BatchedQueueingProcessor.prepareWorks(WorkQueue) line: 124
PostTransactionWorkQueueSynchronization.beforeCompletion() line: 89
EventSourceTransactionContext$DelegateToSynchronizationOnBeforeTx.doBeforeTransactionCompletion(SessionImplementor) line: 172
ActionQueue$BeforeTransactionCompletionProcessQueue.beforeTransactionCompletion() line: 543
ActionQueue.beforeTransactionCompletion() line: 216
SessionImpl.beforeTransactionCompletion(Transaction) line: 571
JDBCContext.beforeTransactionCompletion(Transaction) line: 250
JDBCTransaction.commit() line: 138
TransactionImpl.commit() line: 76
The entry at the top belongs to
public void addWorkToQueue(Class<T> entityClass, T entity, Serializable id, boolean delete, boolean add, boolean batch, List<LuceneWork> queue) {
String idInString = objectToString( idBridge, idKeywordName, id );
if ( delete ) {
queue.add( new DeleteLuceneWork( id, idInString, entityClass ) );
}
if ( add ) {
queue.add( createAddWork( entityClass, entity, id, idInString, HibernateStatelessInitializer.INSTANCE, batch ) );
}
}
The argument delete is true and id is null (it's the value of WorkPlan$PerClassWork<T>.enqueueLuceneWork(List<LuceneWork>) gets it from the documentBuilder.getId(entity) call mentioned above). A little while later the actual exception is triggered:
java.lang.NullPointerException
at org.apache.lucene.index.Term.compareTo(Term.java:115)
at org.apache.lucene.index.TermInfosReader.getIndexOffset(TermInfosReader.java:189)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:268)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:208)
at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:57)
at org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.java:312)
at org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.java:289)
at org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.java:191)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3358)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3296)
at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1836)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1800)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1764)
at org.hibernate.search.backend.Workspace.closeIndexWriter(Workspace.java:244)
at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:113)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
ERROR o.h.s.e.i.LogErrorHandler - Exception occurred java.lang.NullPointerException
Primary Failure:
Entity com.dragg.server.model.Walk Id null Work Type org.hibernate.search.backend.DeleteLuceneWork
Subsequent failures:
Entity com.dragg.server.model.Walk Id null Work Type org.hibernate.search.backend.AddLuceneWork
Entity com.dragg.server.model.Walk Id 5552 Work Type org.hibernate.search.backend.DeleteLuceneWork
java.lang.NullPointerException
at org.apache.lucene.index.Term.compareTo(Term.java:115)
at org.apache.lucene.index.TermInfosReader.getIndexOffset(TermInfosReader.java:189)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:268)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:208)
at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:57)
at org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.java:312)
at org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.java:289)
at org.apache.lucene.index.BufferedDeletes.applyDeletes(BufferedDeletes.java:191)
at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3358)
at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3296)
at org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1836)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1800)
at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1764)
at org.hibernate.search.backend.Workspace.closeIndexWriter(Workspace.java:244)
at org.hibernate.search.backend.impl.lucene.PerDPQueueProcessor.run(PerDPQueueProcessor.java:113)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
org.apache.lucene.index.Term.compareTo fails because this.id is null, having received its value a bit earlier from the null document id above. (Technically, I think this also a bug in lucene; surrounding code, for example Term.equals, does test for null, so either Term.compareTo should do it, or the Term should have been checked for null quite a while earlier. Anyway, the results would have been the same, except for easier debugging.)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months
[Hibernate-JIRA] Created: (HHH-6802) NullPointerException in addValueToMiddleTable with @MapKeyJoinColumn
by Marcin Kobylarz (JIRA)
NullPointerException in addValueToMiddleTable with @MapKeyJoinColumn
--------------------------------------------------------------------
Key: HHH-6802
URL: http://opensource.atlassian.com/projects/hibernate/browse/HHH-6802
Project: Hibernate Core
Issue Type: Bug
Components: envers
Affects Versions: 3.6.8
Environment: Hibernate + Envers 3.6.8
Reporter: Marcin Kobylarz
The following mapping:
@Audited
@OneToMany(mappedBy = "formData", cascade = CascadeType.ALL, orphanRemoval = true)
@MapKeyJoinColumn(name = "SPR_FORMULARZ_MODEL_POLE_ID", nullable = false)
private Map<FieldModel, FormFieldValue> values = new HashMap<FieldModel, FormFieldValue();
causes the exception while bringing hibernate up:
Caused by: java.lang.NullPointerException
at org.hibernate.envers.configuration.metadata.CollectionMetadataGenerator.addValueToMiddleTable(CollectionMetadataGenerator.java:458)
at org.hibernate.envers.configuration.metadata.CollectionMetadataGenerator.addIndex(CollectionMetadataGenerator.java:398)
at org.hibernate.envers.configuration.metadata.CollectionMetadataGenerator.addOneToManyAttached(CollectionMetadataGenerator.java:196)
at org.hibernate.envers.configuration.metadata.CollectionMetadataGenerator.addCollection(CollectionMetadataGenerator.java:157)
at org.hibernate.envers.configuration.metadata.AuditMetadataGenerator.addValue(AuditMetadataGenerator.java:204)
at org.hibernate.envers.configuration.metadata.AuditMetadataGenerator.addProperties(AuditMetadataGenerator.java:223)
at org.hibernate.envers.configuration.metadata.AuditMetadataGenerator.generateSecondPass(AuditMetadataGenerator.java:484)
at org.hibernate.envers.configuration.EntitiesConfigurator.configure(EntitiesConfigurator.java:115)
at org.hibernate.envers.configuration.AuditConfiguration.<init>(AuditConfiguration.java:97)
at org.hibernate.envers.configuration.AuditConfiguration.getFor(AuditConfiguration.java:129)
at org.hibernate.envers.event.AuditEventListener.initialize(AuditEventListener.java:337)
at org.hibernate.event.EventListeners$1.processListener(EventListeners.java:198)
at org.hibernate.event.EventListeners.processListeners(EventListeners.java:181)
at org.hibernate.event.EventListeners.initializeListeners(EventListeners.java:194)
This is caused because queryGeneratorBuilder is null in the addValueToMiddleTable:458 method.
We suppose that method AuditedPropertiesReader:289:
private void addPropertyMapKey(XProperty property, PropertyAuditingData propertyData) {
MapKey mapKey = property.getAnnotation(MapKey.class);
if (mapKey != null) {
propertyData.setMapKey(mapKey.name());
}
}
needs improvement to accept also @MapKeyJoinColumn annotation not only @MapKey.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
12 years, 5 months