[JBoss JIRA] (ISPN-7779) State transfer does not work with protobuf encoded entities
by AHMAD ELABIARY (JIRA)
[ https://issues.jboss.org/browse/ISPN-7779?page=com.atlassian.jira.plugin.... ]
AHMAD ELABIARY updated ISPN-7779:
---------------------------------
Description:
I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
{code:java}
15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4)
DGISPN0001: Started ___protobuf_metadata cache from clustered container
{code}
It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
*State Transfer Exception*
{code:java}
2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
... 56 more
{code}
was:
I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
{code:java}
15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started ___protobuf_metadata cache from clustered container
{code}
It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
*State Transfer Exception*
{code:java}
2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
... 56 more
{code}
> State transfer does not work with protobuf encoded entities
> -----------------------------------------------------------
>
> Key: ISPN-7779
> URL: https://issues.jboss.org/browse/ISPN-7779
> Project: Infinispan
> Issue Type: Feature Request
> Components: State Transfer
> Affects Versions: 9.0.0.Final
> Reporter: AHMAD ELABIARY
> Priority: Critical
>
> I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
> Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
> If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
> {code:java}
> 15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4)
> DGISPN0001: Started ___protobuf_metadata cache from clustered container
> {code}
> It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
> *State Transfer Exception*
> {code:java}
> 2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
> entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
> field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
> at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
> at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
> at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
> at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
> at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
> at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
> at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
> at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
> at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
> at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
> at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
> at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
> at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
> at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
> at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
> at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
> at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
> at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
> at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
> at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
> at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
> at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
> at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
> at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
> at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
> at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
> ... 56 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (ISPN-7779) State transfer does not work with protobuf encoded entities
by AHMAD ELABIARY (JIRA)
AHMAD ELABIARY created ISPN-7779:
------------------------------------
Summary: State transfer does not work with protobuf encoded entities
Key: ISPN-7779
URL: https://issues.jboss.org/browse/ISPN-7779
Project: Infinispan
Issue Type: Feature Request
Components: State Transfer
Affects Versions: 9.0.0.Final
Reporter: AHMAD ELABIARY
Priority: Critical
I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started ___protobuf_metadata cache from clustered container
It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
*State Transfer Exception*
{code:java}
2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
... 56 more
{code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (ISPN-7779) State transfer does not work with protobuf encoded entities
by AHMAD ELABIARY (JIRA)
[ https://issues.jboss.org/browse/ISPN-7779?page=com.atlassian.jira.plugin.... ]
AHMAD ELABIARY updated ISPN-7779:
---------------------------------
Description:
I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
{code:java}
15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started ___protobuf_metadata cache from clustered container
{code}
It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
*State Transfer Exception*
{code:java}
2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
... 56 more
{code}
was:
I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started ___protobuf_metadata cache from clustered container
It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
*State Transfer Exception*
{code:java}
2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
... 56 more
{code}
> State transfer does not work with protobuf encoded entities
> -----------------------------------------------------------
>
> Key: ISPN-7779
> URL: https://issues.jboss.org/browse/ISPN-7779
> Project: Infinispan
> Issue Type: Feature Request
> Components: State Transfer
> Affects Versions: 9.0.0.Final
> Reporter: AHMAD ELABIARY
> Priority: Critical
>
> I am currently testing remote deployment of JBoss Data Grid 7.1 with a 3-node replicated cluster.
> Items stored in the replicated cache are protobuf encoded and indexed. I have enabled global-state persistence on each of the cluster nodes to ensure that the protobuf schema cache is not purged on startup. However, when state-transfer occurs to a node joining the cluster, the transfer triggers an array of exceptions as shown below.
> If i then kill the nodes transferring state to this erroring new joiner, the node continues to start-up and only at that point is this log message show
> {code:java}
> 15:54:19,991 INFO [org.jboss.as.clustering.infinispan] (MSC service thread 1-4) DGISPN0001: Started ___protobuf_metadata cache from clustered container
> {code}
> It seems as through the protobuf metadata cache starts after the configured caches which means that during their startup they do not have access to the persisted schema
> *State Transfer Exception*
> {code:java}
> 2017-04-28 15:03:08,036 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (stateTransferExecutor-thread--p7-t14) ISPN000136: Error executing command PutKeyValueCommand, writing keys [[B0x4A0E4C45494C4757..[16]]: org.hibernate.search.bridge.BridgeException: Exception while calling bridge#set
> entity class: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapper
> field bridge: org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge@44f36069
> at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper.buildBridgeException(ContextualExceptionBridgeHelper.java:103)
> at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:137)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFieldForClassBridges(DocumentBuilderIndexedEntity.java:890)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.buildDocumentFields(DocumentBuilderIndexedEntity.java:454)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.getDocument(DocumentBuilderIndexedEntity.java:391)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.createUpdateWork(DocumentBuilderIndexedEntity.java:301)
> at org.hibernate.search.engine.spi.DocumentBuilderIndexedEntity.addWorkToQueue(DocumentBuilderIndexedEntity.java:243)
> at org.hibernate.search.engine.impl.WorkPlan$PerEntityWork.enqueueLuceneWork(WorkPlan.java:520)
> at org.hibernate.search.engine.impl.WorkPlan$PerClassWork.enqueueLuceneWork(WorkPlan.java:282)
> at org.hibernate.search.engine.impl.WorkPlan.getPlannedLuceneWork(WorkPlan.java:154)
> at org.hibernate.search.backend.impl.WorkQueue.prepareWorkPlan(WorkQueue.java:114)
> at org.hibernate.search.backend.impl.BatchedQueueingProcessor.prepareWorks(BatchedQueueingProcessor.java:57)
> at org.hibernate.search.backend.impl.PerTransactionWorker.performWork(PerTransactionWorker.java:86)
> at org.infinispan.query.backend.QueryInterceptor.performSearchWorks(QueryInterceptor.java:241)
> at org.infinispan.query.backend.QueryInterceptor.performSearchWork(QueryInterceptor.java:235)
> at org.infinispan.query.backend.QueryInterceptor.updateIndexes(QueryInterceptor.java:228)
> at org.infinispan.query.backend.QueryInterceptor.processPutKeyValueCommand(QueryInterceptor.java:457)
> at org.infinispan.query.backend.QueryInterceptor.lambda$visitPutKeyValueCommand$0(QueryInterceptor.java:154)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenAccept(BaseAsyncInterceptor.java:108)
> at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:154)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:107)
> at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:39)
> at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:77)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:397)
> at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:340)
> at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:123)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:164)
> at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:153)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:57)
> at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutKeyValueCommand(IsMarshallableInterceptor.java:49)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextThenApply(BaseAsyncInterceptor.java:77)
> at org.infinispan.interceptors.compat.BaseTypeConverterInterceptor.visitPutKeyValueCommand(BaseTypeConverterInterceptor.java:82)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:126)
> at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:92)
> at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:59)
> at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:53)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitPutKeyValueCommand(DDAsyncInterceptor.java:59)
> at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:79)
> at org.infinispan.interceptors.DDAsyncInterceptor.visitCommand(DDAsyncInterceptor.java:49)
> at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:282)
> at org.infinispan.statetransfer.StateConsumerImpl.doApplyState(StateConsumerImpl.java:621)
> at org.infinispan.statetransfer.StateConsumerImpl.applyChunk(StateConsumerImpl.java:589)
> at org.infinispan.statetransfer.StateConsumerImpl.access$000(StateConsumerImpl.java:93)
> at org.infinispan.statetransfer.StateConsumerImpl$1.run(StateConsumerImpl.java:552)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.IllegalArgumentException: Message descriptor not found : flightpricestore.SectorDayCacheEntry
> at org.infinispan.protostream.impl.SerializationContextImpl.getMessageDescriptor(SerializationContextImpl.java:166)
> at org.infinispan.query.remote.impl.indexing.WrappedMessageTagHandler.onTag(WrappedMessageTagHandler.java:65)
> at org.infinispan.protostream.ProtobufParser.parseMessage(ProtobufParser.java:75)
> at org.infinispan.protostream.ProtobufParser.parseInternal(ProtobufParser.java:57)
> at org.infinispan.protostream.ProtobufParser.parse(ProtobufParser.java:44)
> at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.decodeAndIndex(ProtobufValueWrapperFieldBridge.java:58)
> at org.infinispan.query.remote.impl.indexing.ProtobufValueWrapperFieldBridge.set(ProtobufValueWrapperFieldBridge.java:46)
> at org.hibernate.search.bridge.util.impl.ContextualExceptionBridgeHelper$OneWayConversionContextImpl.set(ContextualExceptionBridgeHelper.java:134)
> ... 56 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
6 years, 12 months
[JBoss JIRA] (ISPN-7704) AbstractFileLookup#lookupFileStrict should also support absolute paths
by Sebastian Łaskawiec (JIRA)
[ https://issues.jboss.org/browse/ISPN-7704?page=com.atlassian.jira.plugin.... ]
Sebastian Łaskawiec commented on ISPN-7704:
-------------------------------------------
Awesome! Thanks for checking [~danielbarbarian]!
> AbstractFileLookup#lookupFileStrict should also support absolute paths
> ----------------------------------------------------------------------
>
> Key: ISPN-7704
> URL: https://issues.jboss.org/browse/ISPN-7704
> Project: Infinispan
> Issue Type: Bug
> Components: JCache, Spring Integration
> Affects Versions: 9.0.0.Final
> Reporter: Sebastian Łaskawiec
> Assignee: Sebastian Łaskawiec
> Priority: Critical
> Fix For: 9.1.0.Final, 9.0.1.Final, 9.1.0.Alpha1
>
>
> When specifying {{spring.cache.jcache.config}} value in Spring's application.properties, Spring passes us a scheme with absolute path (e.g. {{file:/home/slaskawi/work/infinispan/CacheTest/target/classes/infinispan.xml}}). This causes problems with {{AbstractFileLookup#lookupFileStrict(java.net.URI, java.lang.ClassLoader)}} since it should construct a new {{File}} rather than load this resource from the classloader.
> Related Pull Request: https://github.com/infinispan/infinispan/pull/4474
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years
[JBoss JIRA] (ISPN-5021) Nodes that finish the rebalance later can see outdated values
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-5021?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-5021:
-------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.1.0.Alpha1
Resolution: Done
> Nodes that finish the rebalance later can see outdated values
> -------------------------------------------------------------
>
> Key: ISPN-5021
> URL: https://issues.jboss.org/browse/ISPN-5021
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.2.Final
> Reporter: Dan Berindei
> Assignee: Pedro Ruivo
> Priority: Critical
> Labels: consistency, testsuite_stability
> Fix For: 9.1.0.Final, 9.1.0.Alpha1
>
>
> Copied from [ISPN-4444|https://issues.jboss.org/browse/ISPN-4444?focusedCommentId=1302...]
> If the CH_UPDATE command is delayed on the old owner, the new owners might update the key without the old owner knowing, and a locality check on the old owner won't help.
> I remember one thing that struck me when reading the Raft algorithm was that they install configuration changes symmetrically, in 3 phases. We might need to do the same for our rebalance:
> 1. T0: read_ch=old, write_ch=old
> 2. start a rebalance
> 3. T1: read_ch=old, write_ch=old+new
> 4. new owners have all the data
> 5. T2: read_ch=new, write_ch=old+new
> 6. remove old cache entries and ignore further writes
> 7. T3: read_ch=new, write_ch=new
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years
[JBoss JIRA] (ISPN-5021) Nodes that finish the rebalance later can see outdated values
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-5021?page=com.atlassian.jira.plugin.... ]
Dan Berindei reassigned ISPN-5021:
----------------------------------
Assignee: Radim Vansa (was: Pedro Ruivo)
> Nodes that finish the rebalance later can see outdated values
> -------------------------------------------------------------
>
> Key: ISPN-5021
> URL: https://issues.jboss.org/browse/ISPN-5021
> Project: Infinispan
> Issue Type: Bug
> Components: Core, State Transfer
> Affects Versions: 7.0.2.Final
> Reporter: Dan Berindei
> Assignee: Radim Vansa
> Priority: Critical
> Labels: consistency, testsuite_stability
> Fix For: 9.1.0.Final, 9.1.0.Alpha1
>
>
> Copied from [ISPN-4444|https://issues.jboss.org/browse/ISPN-4444?focusedCommentId=1302...]
> If the CH_UPDATE command is delayed on the old owner, the new owners might update the key without the old owner knowing, and a locality check on the old owner won't help.
> I remember one thing that struck me when reading the Raft algorithm was that they install configuration changes symmetrically, in 3 phases. We might need to do the same for our rebalance:
> 1. T0: read_ch=old, write_ch=old
> 2. start a rebalance
> 3. T1: read_ch=old, write_ch=old+new
> 4. new owners have all the data
> 5. T2: read_ch=new, write_ch=old+new
> 6. remove old cache entries and ignore further writes
> 7. T3: read_ch=new, write_ch=new
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years
[JBoss JIRA] (ISPN-7604) StateTransferLockImpl.stop() never runs
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-7604?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-7604:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.1.0.Alpha1
Resolution: Done
> StateTransferLockImpl.stop() never runs
> ---------------------------------------
>
> Key: ISPN-7604
> URL: https://issues.jboss.org/browse/ISPN-7604
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 9.0.0.CR2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Fix For: 9.1.0.Final, 9.1.0.Alpha1
>
>
> {{StateTransferLockImpl.stop()}} is supposed to signal the installation of topology {{Integer.MAX_VALUE}}, in order to unblock any commands waiting for a new topology. However, it doesn't have the {{@Stop}} annotation, so it's never called, and threads waiting on a new topology will block forever:
> {noformat}
> "jgroups-4,test-NodeB-41665@3770" prio=5 tid=0x1f nid=NA waiting
> java.lang.Thread.State: WAITING
> at sun.misc.Unsafe.park(Unsafe.java:-1)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
> at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
> at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
> at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.runSync(BaseBlockingRunnable.java:48)
> at org.infinispan.remoting.inboundhandler.BaseBlockingRunnable.run(BaseBlockingRunnable.java:38)
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handleStateRequestCommand(TrianglePerCacheInboundInvocationHandler.java:171)
> at org.infinispan.remoting.inboundhandler.TrianglePerCacheInboundInvocationHandler.handle(TrianglePerCacheInboundInvocationHandler.java:109)
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleCacheRpcCommand(GlobalInboundInvocationHandler.java:120)
> at org.infinispan.remoting.inboundhandler.GlobalInboundInvocationHandler.handleFromCluster(GlobalInboundInvocationHandler.java:79)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.executeCommandFromLocalCluster(CommandAwareRpcDispatcher.java:175)
> at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:149)
> at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:383)
> at org.jgroups.blocks.RequestCorrelator.dispatch(RequestCorrelator.java:356)
> at org.jgroups.blocks.RequestCorrelator.receiveMessageBatch(RequestCorrelator.java:326)
> at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:586)
> at org.jgroups.JChannel.up(JChannel.java:813)
> at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:896)
> at org.jgroups.protocols.RSVP.up(RSVP.java:233)
> at org.jgroups.protocols.FlowControl.up(FlowControl.java:418)
> at org.jgroups.stack.Protocol.up(Protocol.java:344)
> at org.jgroups.stack.Protocol.up(Protocol.java:344)
> at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:293)
> at org.jgroups.protocols.UNICAST3.deliverBatch(UNICAST3.java:1083)
> at org.jgroups.protocols.UNICAST3.removeAndDeliver(UNICAST3.java:892)
> at org.jgroups.protocols.UNICAST3.handleBatchReceived(UNICAST3.java:858)
> at org.jgroups.protocols.UNICAST3.up(UNICAST3.java:529)
> at org.jgroups.protocols.pbcast.NAKACK2.up(NAKACK2.java:695)
> at org.jgroups.stack.Protocol.up(Protocol.java:344)
> at org.jgroups.protocols.TP.passBatchUp(TP.java:1229)
> at org.jgroups.util.MaxOneThreadPerSender$BatchHandlerLoop.passBatchUp(MaxOneThreadPerSender.java:284)
> at org.jgroups.util.SubmitToThreadPool$BatchHandler.run(SubmitToThreadPool.java:136)
> at org.jgroups.util.MaxOneThreadPerSender$BatchHandlerLoop.run(MaxOneThreadPerSender.java:273)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years
[JBoss JIRA] (ISPN-7598) Core test suite leaks threads
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-7598?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-7598:
----------------------------------
Status: Resolved (was: Pull Request Sent)
Fix Version/s: 9.1.0.Alpha1
Resolution: Done
> Core test suite leaks threads
> -----------------------------
>
> Key: ISPN-7598
> URL: https://issues.jboss.org/browse/ISPN-7598
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 9.0.0.CR2
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: testsuite_stability
> Fix For: 9.1.0.Final, 9.1.0.Alpha1
>
>
> Each {{LocalDistributedExecutorTest}} test method creates a new {{DefaultExecutorService}} instance, and each instance uses a new local executor created with {{Executors.newCachedThreadPool(...)}}. But the {{DefaultExecutorService}} is created with {{takeExecutorOwnership = false}}, and so the local executor is not stopped on shutdown.
--
This message was sent by Atlassian JIRA
(v7.2.3#72005)
7 years