[JBoss JIRA] (ISPN-4027) TransactionTable.start() initialize the TxServiceCleanup thread pool even when the cache is NON_TRANSACTIONAL
by Takayoshi Kimura (JIRA)
[ https://issues.jboss.org/browse/ISPN-4027?page=com.atlassian.jira.plugin.... ]
Takayoshi Kimura updated ISPN-4027:
-----------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2955
> TransactionTable.start() initialize the TxServiceCleanup thread pool even when the cache is NON_TRANSACTIONAL
> -------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-4027
> URL: https://issues.jboss.org/browse/ISPN-4027
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.1.Final
> Reporter: Guillermo GARCIA OCHOA
> Assignee: Mircea Markus
> Labels: 630
>
> In the {{TransactionTable.start()}} each cache creates a thread pool and a job is scheduled to clean up completed transactions.
> {code:java}
> private void start() {
> ...
> totalOrder = configuration.transaction().transactionProtocol().isTotalOrder();
> if (!totalOrder) {
> // Periodically run a task to cleanup the transaction table from completed transactions.
> ThreadFactory tf = new ThreadFactory() {
> @Override
> public Thread newThread(Runnable r) {
> String address = rpcManager != null ? rpcManager.getTransport().getAddress().toString() : "local";
> Thread th = new Thread(r, "TxCleanupService," + cacheName + "," + address);
> th.setDaemon(true);
> return th;
> }
> };
> executorService = Executors.newSingleThreadScheduledExecutor(tf);
> long interval = configuration.transaction().reaperWakeUpInterval();
> executorService.scheduleAtFixedRate(new Runnable() {
> @Override
> public void run() {
> cleanupCompletedTransactions();
> }
> }, interval, interval, TimeUnit.MILLISECONDS);
> }
> }
> {code}
> As you can see in the code, even is the cache is {{NON_TRANSACTIONAL}} the job is scheduled, consuming resources to do nothing (the {{completedTransactions}} map is always empty)
> Maybe I'm missing something, but our application profiling is showing us that these threads do nothing but they are consuming precious resources because we have more than 1000 {{NON_TRANSACTIONAL}} caches.
> (i) This can be considered when solving ISPN-3702 too.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4027) TransactionTable.start() initialize the TxServiceCleanup thread pool even when the cache is NON_TRANSACTIONAL
by Takayoshi Kimura (JIRA)
[ https://issues.jboss.org/browse/ISPN-4027?page=com.atlassian.jira.plugin.... ]
Takayoshi Kimura reassigned ISPN-4027:
--------------------------------------
Assignee: Takayoshi Kimura (was: Mircea Markus)
> TransactionTable.start() initialize the TxServiceCleanup thread pool even when the cache is NON_TRANSACTIONAL
> -------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-4027
> URL: https://issues.jboss.org/browse/ISPN-4027
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.1.Final
> Reporter: Guillermo GARCIA OCHOA
> Assignee: Takayoshi Kimura
> Labels: 630
>
> In the {{TransactionTable.start()}} each cache creates a thread pool and a job is scheduled to clean up completed transactions.
> {code:java}
> private void start() {
> ...
> totalOrder = configuration.transaction().transactionProtocol().isTotalOrder();
> if (!totalOrder) {
> // Periodically run a task to cleanup the transaction table from completed transactions.
> ThreadFactory tf = new ThreadFactory() {
> @Override
> public Thread newThread(Runnable r) {
> String address = rpcManager != null ? rpcManager.getTransport().getAddress().toString() : "local";
> Thread th = new Thread(r, "TxCleanupService," + cacheName + "," + address);
> th.setDaemon(true);
> return th;
> }
> };
> executorService = Executors.newSingleThreadScheduledExecutor(tf);
> long interval = configuration.transaction().reaperWakeUpInterval();
> executorService.scheduleAtFixedRate(new Runnable() {
> @Override
> public void run() {
> cleanupCompletedTransactions();
> }
> }, interval, interval, TimeUnit.MILLISECONDS);
> }
> }
> {code}
> As you can see in the code, even is the cache is {{NON_TRANSACTIONAL}} the job is scheduled, consuming resources to do nothing (the {{completedTransactions}} map is always empty)
> Maybe I'm missing something, but our application profiling is showing us that these threads do nothing but they are consuming precious resources because we have more than 1000 {{NON_TRANSACTIONAL}} caches.
> (i) This can be considered when solving ISPN-3702 too.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4840) LockControlCommand timeouts can cause orphaned locks.
by Erik Salter (JIRA)
Erik Salter created ISPN-4840:
---------------------------------
Summary: LockControlCommand timeouts can cause orphaned locks.
Key: ISPN-4840
URL: https://issues.jboss.org/browse/ISPN-4840
Project: Infinispan
Issue Type: Bug
Components: Transactions
Affects Versions: 7.0.0.CR1
Reporter: Erik Salter
Assignee: Dan Berindei
If an originator times out on sending a pessimistic LockControlCommand, the receiver may still get the message. In this case, the originator will send a TxCompletionCommand. Because of this, the receiver will remove the registered remote transaction from its transaction table. If, however, another thread is processing the remote LCC command, it could acquire the lock(s) after the referencing remoteTx is removed. Thus, the affected keys will remain locked indefinitely.
A simple solution would be to add a check to see if there's a remote transaction when the LCC thread verifies the remote transaction. This would be in addition to checking if the transaction is completed.
See the following TRACE messages:
-- Local TX created
2014-10-10 11:27:28,899 TRACE [org.infinispan.transaction.TransactionTable] (OOB-1353,session-resource-cluster,240-east-dht2.comcast.net-46326(CMC-Denver-CO)) Created a new local transaction: LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, isFromStateTransfer=false} globalTx=GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:local, topologyId=64, age(ms)=0
-- Remote TX created
2014-10-10 11:27:28,525 TRACE [org.infinispan.transaction.TransactionTable] (OOB-2850,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) Created and registered remote transaction RemoteTransaction{modifications=[], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=null, missingLookedUpEntries=false, isMarkedForRollback=false} globalTx=GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:remote, topologyId=63, age(ms)=0
-- Originator times out on LCC
2014-10-10 11:27:36,902 WARN [org.infinispan.remoting.rpc.RpcManagerImpl] (OOB-1353,session-resource-cluster,240-east-dht2.comcast.net-46326(CMC-Denver-CO)) ISPN000071: Caught exception when handling command LockControlCommand{cache=eigAllocation, keys=[EdgeResourceCacheKey[edgeDeviceId=2878,resourceId=11130]], flags=null, unlock=false}
org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to 240-west-dht2.comcast.net-30190(CH2-Chicago-IL)
at org.infinispan.util.Util.rewrapAsCacheException(Util.java:542)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:188)
...
at org.infinispan.interceptors.distribution.TxDistributionInterceptor.visitLockControlCommand(TxDistributionInterceptor.java:204)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:177)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:235)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:177)
at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:116)
...
at org.infinispan.CacheImpl.lock(CacheImpl.java:565)
at org.infinispan.CacheImpl.lock(CacheImpl.java:548)
...
Caused by: org.jgroups.TimeoutException: timeout sending message to 240-west-dht2.comcast.net-30190(CH2-Chicago-IL)
at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:390)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:303)
at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:181)
... 91 more
2014-10-10 11:27:36,936 TRACE [org.infinispan.interceptors.TxInterceptor] (OOB-1353,session-resource-cluster,240-east-dht2.comcast.net-46326(CMC-Denver-CO)) invokeNextInterceptorAndVerifyTransaction :: originatorMissing=false, alreadyCompleted=false
2014-10-10 11:27:36,937 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (OOB-1353,session-resource-cluster,240-east-dht2.comcast.net-46326(CMC-Denver-CO)) ISPN000136: Execution error
org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to 240-west-dht2.comcast.net-30190(CH2-Chicago-IL)
at org.infinispan.util.Util.rewrapAsCacheException(Util.java:542)
...
-- Same stack trace
-- Remote TX removed -- must be from TxCompletionMessage since isMarkedForRollback == false
2014-10-10 11:27:36,530 TRACE [org.infinispan.transaction.TransactionTable] (OOB-2963,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) Removed remote transaction GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:local ? RemoteTransaction{modifications=[], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=null, missingLookedUpEntries=false, isMarkedForRollback=false} globalTx=GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:remote, topologyId=63, age(ms)=8004
2014-10-10 11:27:36,530 TRACE [org.infinispan.transaction.TransactionTable] (OOB-2963,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) Removed RemoteTransaction{modifications=[], lookedUpEntries={}, lockedKeys=null, backupKeyLocks=null, missingLookedUpEntries=false, isMarkedForRollback=false} globalTx=GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:remote, topologyId=63, age(ms)=8004 from transaction table.
-- Local TX removed
2014-10-10 11:27:36,937 TRACE [org.infinispan.transaction.TransactionTable] (OOB-1353,session-resource-cluster,240-east-dht2.comcast.net-46326(CMC-Denver-CO)) Removed LocalTransaction{remoteLockedNodes=[240-east-dht2.comcast.net-46326(CMC-Denver-CO), 240-west-dht2.comcast.net-30190(CH2-Chicago-IL)], isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, isFromStateTransfer=false} globalTx=GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:local, topologyId=64, age(ms)=8038 from transaction table.
-- Lock acquisition completes!
2014-10-10 11:27:39,195 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (OOB-2850,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) Attempting to lock EdgeResourceCacheKey[edgeDeviceId=2878,resourceId=11130] with acquisition timeout of 5000 millis
2014-10-10 11:27:39,198 TRACE [org.infinispan.util.concurrent.locks.LockManagerImpl] (OOB-2850,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) Successfully acquired lock EdgeResourceCacheKey[edgeDeviceId=2878,resourceId=11130]!
2014-10-10 11:27:39,202 TRACE [org.infinispan.transaction.TransactionTable] (OOB-2850,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) Transaction=GlobalTransaction:<240-east-dht2.comcast.net-46326(CMC-Denver-CO)>:859804:remote, nodeMaxPrunedTxId=858439
-- Not in completion map since transaction was removed without commit or rollback. Lock never releases
2014-10-10 11:27:39,203 TRACE [org.infinispan.interceptors.TxInterceptor] (OOB-2850,session-resource-cluster,240-west-dht2.comcast.net-30190(CH2-Chicago-IL)) invokeNextInterceptorAndVerifyTransaction :: originatorMissing=false, alreadyCompleted=false
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4071) Switch to automated generation of RHQ plugin for server
by William Burns (JIRA)
[ https://issues.jboss.org/browse/ISPN-4071?page=com.atlassian.jira.plugin.... ]
William Burns commented on ISPN-4071:
-------------------------------------
Just to keep this around
{quote}
ttarrant: maybe it would make sense to dump the DMR of our three subsystems and somehow transform that to XML
{quote}
> Switch to automated generation of RHQ plugin for server
> -------------------------------------------------------
>
> Key: ISPN-4071
> URL: https://issues.jboss.org/browse/ISPN-4071
> Project: Infinispan
> Issue Type: Enhancement
> Affects Versions: 7.0.0.Alpha1
> Reporter: Tomas Sykora
> Assignee: William Burns
> Priority: Minor
>
> InVM plugin for RHQ is generated automatically. However, the plugin for server is not and therfore, it is easy to miss some of the possible management operations.
> The principal intention is to automate server plugin generation and avoid the case when we are missing any of possible operations/metrics/traits due to out of synchronization of code vs rhq-plugin.xml file.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4826) X-Site State transfer values not propagated correctly
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-4826?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-4826:
------------------------------
Status: Pull Request Sent (was: Coding In Progress)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2953
The problem is the x-site control command was forward in the remote site cluster using the cache name from the sender site.
> X-Site State transfer values not propagated correctly
> ------------------------------------------------------
>
> Key: ISPN-4826
> URL: https://issues.jboss.org/browse/ISPN-4826
> Project: Infinispan
> Issue Type: Bug
> Components: Cross-Site Replication
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
> Assignee: Pedro Ruivo
> Fix For: 7.0.0.CR2
>
>
> Used configuration:
> a) SITE1: 2 nodes, cache testCacheSite1
> <backups>
> <backup site="SITE2"/>
> </backups>
> b) SITE2: 3 nodes, cache testCacheSite1_backup – backup cache for testCacheSite1
> <backup-for remote-cache="testCacheSite1" remote-site="SITE1"/>
> When using backup cache with name (testCacheSite1_backup) different from the name of the main cache in SITE1 (testCacheSite1), the data is not propagated to the backup cache completely. The issue seems to be fixed by using the same name for the backup cache (testCacheSite1).
> Scenario
> 1. Start site1 and write data into it (1000 entries)
> 2. Start site2 and invoke XsiteAdminOperations.pushState(“SITE2”)
> 3. Wait 2 minutes
> 4. Check whether the state was transferred to site2 (tested on dist & repl backup cache configs)
> a) distributed mode (numOwners=2) - expected 2000 entries in total, was 648 on site2 master & 0 on other nodes
> b) replicated mode – expected 3000 entries in total, was 1000 on site2 master & 0 on other nodes
>
> Trace log:
> 04:14:39,116 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (OOB-10,edg-perf13-23152) Silently ignoring that testCacheSite1 cache is not defined
> 04:14:39,375 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-10,edg-perf13-23152) Attempting to execute command: SingleRpcCommand{cacheName='testCacheSite1_backup', command=PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}} [sender=edg-perf14-31850]
> 04:14:39,376 TRACE [org.infinispan.statetransfer.StateTransferLockImpl] (OOB-10,edg-perf13-23152) Checking if transaction data was received for topology 4, current topology is 4
> 04:14:39,376 TRACE [org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl] (OOB-10,edg-perf13-23152) Added a new task: 0 task(s) are waiting
> 04:14:39,376 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread--p3-t2) Calling perform() on SingleRpcCommand{cacheName='testCacheSite1_backup', command=PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}}
> 04:14:39,378 TRACE [org.infinispan.commands.remote.BaseRpcInvokingCommand] (remote-thread--p3-t2) Invoking command PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}, with originLocal flag set to false
> 04:14:39,378 TRACE [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p3-t2) Invoked with command PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@266883cb]
> 04:14:39,379 TRACE [org.infinispan.statetransfer.StateTransferInterceptor] (remote-thread--p3-t2) handleNonTxWriteCommand for command PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}
> 04:14:39,380 TRACE [org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor] (remote-thread--p3-t2) Are (edg-perf13-23152) we the lock owners for key 'key_0000000000000001'? false
> 04:14:39,380 TRACE [org.infinispan.interceptors.EntryWrappingInterceptor] (remote-thread--p3-t2) Wrapping entry 'key_0000000000000001'? true
> 04:14:39,380 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Exists in context? null
> 04:14:39,382 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Retrieved from container null (isL1Enabled=false, isLocal=true)
> 04:14:39,382 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Creating new entry.
> 04:14:39,388 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Wrap key_0000000000000001 for put. Entry=ReadCommittedEntry(197b92bc){key=key_0000000000000001, value=null, oldValue=null, isCreated=true, isChanged=false, isRemoved=false, isValid=true, skipRemoteGet=false, metadata=EmbeddedMetadata{version=null}}
> 04:14:39,390 TRACE [org.infinispan.interceptors.CallInterceptor] (remote-thread--p3-t2) Executing command: PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}.
> 04:14:39,391 TRACE [org.infinispan.interceptors.EntryWrappingInterceptor] (remote-thread--p3-t2) About to commit entry ReadCommittedEntry(197b92bc){key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, oldValue=null, isCreated=true, isChanged=true, isRemoved=false, isValid=true, skipRemoteGet=false, metadata=EmbeddedMetadata{version=null}}
> 04:14:39,392 TRACE [org.infinispan.statetransfer.CommitManager] (remote-thread--p3-t2) Trying to commit. Key=key_0000000000000001. Operation Flag=PUT_FOR_X_SITE_STATE_TRANSFER, L1 invalidation=false
> 04:14:39,392 TRACE [org.infinispan.statetransfer.CommitManager] (remote-thread--p3-t2) Not committing key=key_0000000000000001. It is a state transfer key but no track is enabled!
> 04:14:39,392 TRACE [org.infinispan.interceptors.EntryWrappingInterceptor] (remote-thread--p3-t2) The return value is null
> Suspicious lines:
> 04:14:39,116 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (OOB-10,edg-perf13-23152) Silently ignoring that testCacheSite1 cache is not defined
> 04:14:39,392 TRACE [org.infinispan.statetransfer.CommitManager] (remote-thread--p3-t2) Not committing key=key_0000000000000001. It is a state transfer key but no track is enabled!
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4826) X-Site State transfer values not propagated correctly
by Pedro Ruivo (JIRA)
[ https://issues.jboss.org/browse/ISPN-4826?page=com.atlassian.jira.plugin.... ]
Pedro Ruivo updated ISPN-4826:
------------------------------
Fix Version/s: 7.0.0.CR2
> X-Site State transfer values not propagated correctly
> ------------------------------------------------------
>
> Key: ISPN-4826
> URL: https://issues.jboss.org/browse/ISPN-4826
> Project: Infinispan
> Issue Type: Bug
> Components: Cross-Site Replication
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
> Assignee: Pedro Ruivo
> Fix For: 7.0.0.CR2
>
>
> Used configuration:
> a) SITE1: 2 nodes, cache testCacheSite1
> <backups>
> <backup site="SITE2"/>
> </backups>
> b) SITE2: 3 nodes, cache testCacheSite1_backup – backup cache for testCacheSite1
> <backup-for remote-cache="testCacheSite1" remote-site="SITE1"/>
> When using backup cache with name (testCacheSite1_backup) different from the name of the main cache in SITE1 (testCacheSite1), the data is not propagated to the backup cache completely. The issue seems to be fixed by using the same name for the backup cache (testCacheSite1).
> Scenario
> 1. Start site1 and write data into it (1000 entries)
> 2. Start site2 and invoke XsiteAdminOperations.pushState(“SITE2”)
> 3. Wait 2 minutes
> 4. Check whether the state was transferred to site2 (tested on dist & repl backup cache configs)
> a) distributed mode (numOwners=2) - expected 2000 entries in total, was 648 on site2 master & 0 on other nodes
> b) replicated mode – expected 3000 entries in total, was 1000 on site2 master & 0 on other nodes
>
> Trace log:
> 04:14:39,116 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (OOB-10,edg-perf13-23152) Silently ignoring that testCacheSite1 cache is not defined
> 04:14:39,375 TRACE [org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher] (OOB-10,edg-perf13-23152) Attempting to execute command: SingleRpcCommand{cacheName='testCacheSite1_backup', command=PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}} [sender=edg-perf14-31850]
> 04:14:39,376 TRACE [org.infinispan.statetransfer.StateTransferLockImpl] (OOB-10,edg-perf13-23152) Checking if transaction data was received for topology 4, current topology is 4
> 04:14:39,376 TRACE [org.infinispan.util.concurrent.BlockingTaskAwareExecutorServiceImpl] (OOB-10,edg-perf13-23152) Added a new task: 0 task(s) are waiting
> 04:14:39,376 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (remote-thread--p3-t2) Calling perform() on SingleRpcCommand{cacheName='testCacheSite1_backup', command=PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}}
> 04:14:39,378 TRACE [org.infinispan.commands.remote.BaseRpcInvokingCommand] (remote-thread--p3-t2) Invoking command PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}, with originLocal flag set to false
> 04:14:39,378 TRACE [org.infinispan.interceptors.InvocationContextInterceptor] (remote-thread--p3-t2) Invoked with command PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@266883cb]
> 04:14:39,379 TRACE [org.infinispan.statetransfer.StateTransferInterceptor] (remote-thread--p3-t2) handleNonTxWriteCommand for command PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}
> 04:14:39,380 TRACE [org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor] (remote-thread--p3-t2) Are (edg-perf13-23152) we the lock owners for key 'key_0000000000000001'? false
> 04:14:39,380 TRACE [org.infinispan.interceptors.EntryWrappingInterceptor] (remote-thread--p3-t2) Wrapping entry 'key_0000000000000001'? true
> 04:14:39,380 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Exists in context? null
> 04:14:39,382 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Retrieved from container null (isL1Enabled=false, isLocal=true)
> 04:14:39,382 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Creating new entry.
> 04:14:39,388 TRACE [org.infinispan.container.EntryFactoryImpl] (remote-thread--p3-t2) Wrap key_0000000000000001 for put. Entry=ReadCommittedEntry(197b92bc){key=key_0000000000000001, value=null, oldValue=null, isCreated=true, isChanged=false, isRemoved=false, isValid=true, skipRemoteGet=false, metadata=EmbeddedMetadata{version=null}}
> 04:14:39,390 TRACE [org.infinispan.interceptors.CallInterceptor] (remote-thread--p3-t2) Executing command: PutKeyValueCommand{key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, flags=[SKIP_REMOTE_LOOKUP, PUT_FOR_X_SITE_STATE_TRANSFER, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=false, valueMatcher=MATCH_ALWAYS, metadata=EmbeddedMetadata{version=null}, successful=true}.
> 04:14:39,391 TRACE [org.infinispan.interceptors.EntryWrappingInterceptor] (remote-thread--p3-t2) About to commit entry ReadCommittedEntry(197b92bc){key=key_0000000000000001, value=value_key_0000000000000001_SITE1_ORIGINAL@testCacheSite1, oldValue=null, isCreated=true, isChanged=true, isRemoved=false, isValid=true, skipRemoteGet=false, metadata=EmbeddedMetadata{version=null}}
> 04:14:39,392 TRACE [org.infinispan.statetransfer.CommitManager] (remote-thread--p3-t2) Trying to commit. Key=key_0000000000000001. Operation Flag=PUT_FOR_X_SITE_STATE_TRANSFER, L1 invalidation=false
> 04:14:39,392 TRACE [org.infinispan.statetransfer.CommitManager] (remote-thread--p3-t2) Not committing key=key_0000000000000001. It is a state transfer key but no track is enabled!
> 04:14:39,392 TRACE [org.infinispan.interceptors.EntryWrappingInterceptor] (remote-thread--p3-t2) The return value is null
> Suspicious lines:
> 04:14:39,116 TRACE [org.infinispan.remoting.InboundInvocationHandlerImpl] (OOB-10,edg-perf13-23152) Silently ignoring that testCacheSite1 cache is not defined
> 04:14:39,392 TRACE [org.infinispan.statetransfer.CommitManager] (remote-thread--p3-t2) Not committing key=key_0000000000000001. It is a state transfer key but no track is enabled!
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months