[JBoss JIRA] Created: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster
9:24 a.m.
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in
cluster
--------------------------------------------------------------------------------------------------
Key: JBCACHE-1592
URL: https://jira.jboss.org/browse/JBCACHE-1592
Project: JBoss Cache
Issue Type: Bug
Security Level: Public (Everyone can see)
Components: Locking
Affects Versions: 3.2.5.GA
Environment: windows os,cluster node in one machine.
Reporter: l jw
Assignee: Manik Surtani
Priority: Blocker
When used in cluster,the following exception is throw frequently:
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
[/courseClass/360/male] after [5000] milliseconds for requestor
[GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by
[GlobalTransaction:<172.18.73.162:3428>:6416]
at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
at
org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
at
org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
at org.jgroups.JChannel.up(JChannel.java:1336)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
at
org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
and following is my config:
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.2">
<locking isolationLevel="REPEATABLE_READ"
lockParentForChildInsertRemove="false"
lockAcquisitionTimeout="5000"
nodeLockingScheme="mvcc"
writeSkewCheck="true"
useLockStriping="true"
concurrencyLevel="10000"/>
<clustering mode="replication"
clusterName="JBossCache-cluster">
<stateRetrieval timeout="5000" fetchInMemoryState="false"
nonBlocking="false"/>
<sync replTimeout="5000"/>
<jgroupsConfig>
<UDP discard_incompatible_packets="true"
enable_bundling="false" enable_diagnostics="false"
ip_ttl="2"
loopback="false" max_bundle_size="64000"
max_bundle_timeout="30" mcast_addr="224.0.0.1"
mcast_port="45588" mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
oob_thread_pool.enabled="true"
oob_thread_pool.keep_alive_time="10000"
oob_thread_pool.max_threads="4"
oob_thread_pool.min_threads="1"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="10"
oob_thread_pool.rejection_policy="Run"
thread_naming_pattern="pl" thread_pool.enabled="true"
thread_pool.keep_alive_time="30000"
thread_pool.max_threads="25" thread_pool.min_threads="1"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
tos="8" ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000" use_concurrent_stack="true"
use_incoming_packet_handler="true"/>
<PING num_initial_members="3" timeout="2000"/>
<MERGE2 max_interval="30000" min_interval="10000"/>
<FD_SOCK/>
<FD max_tries="5" shun="true"
timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK discard_delivered_msgs="true" gc_lag="0"
retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000" print_local_addr="true"
shun="false" view_ack_collection_timeout="5000"
view_bundling="true"/>
<FRAG2 frag_size="60000"/>
<pbcast.STREAMING_STATE_TRANSFER/>
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
</jbosscache>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:16 p.m.
New subject: [JBoss JIRA] Commented: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster
[
https://jira.jboss.org/browse/JBCACHE-1592?page=com.atlassian.jira.plugin...
]
l jw commented on JBCACHE-1592:
-------------------------------
Any Jboss pepole pay attention this bug trace system?
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
is very frequently in cluster
--------------------------------------------------------------------------------------------------
Key: JBCACHE-1592
URL: https://jira.jboss.org/browse/JBCACHE-1592
Project: JBoss Cache
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Locking
Affects Versions: 3.2.5.GA
Environment: windows os,cluster node in one machine.
Reporter: l jw
Assignee: Manik Surtani
Priority: Blocker
When used in cluster,the following exception is throw frequently:
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
[/courseClass/360/male] after [5000] milliseconds for requestor
[GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by
[GlobalTransaction:<172.18.73.162:3428>:6416]
at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
at
org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
at
org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
at org.jgroups.JChannel.up(JChannel.java:1336)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
at
org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
and following is my config:
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.2">
<locking isolationLevel="REPEATABLE_READ"
lockParentForChildInsertRemove="false"
lockAcquisitionTimeout="5000"
nodeLockingScheme="mvcc"
writeSkewCheck="true"
useLockStriping="true"
concurrencyLevel="10000"/>
<clustering mode="replication"
clusterName="JBossCache-cluster">
<stateRetrieval timeout="5000" fetchInMemoryState="false"
nonBlocking="false"/>
<sync replTimeout="5000"/>
<jgroupsConfig>
<UDP discard_incompatible_packets="true"
enable_bundling="false" enable_diagnostics="false"
ip_ttl="2"
loopback="false" max_bundle_size="64000"
max_bundle_timeout="30" mcast_addr="224.0.0.1"
mcast_port="45588" mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
oob_thread_pool.enabled="true"
oob_thread_pool.keep_alive_time="10000"
oob_thread_pool.max_threads="4"
oob_thread_pool.min_threads="1"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="10"
oob_thread_pool.rejection_policy="Run"
thread_naming_pattern="pl" thread_pool.enabled="true"
thread_pool.keep_alive_time="30000"
thread_pool.max_threads="25" thread_pool.min_threads="1"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
tos="8" ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000" use_concurrent_stack="true"
use_incoming_packet_handler="true"/>
<PING num_initial_members="3" timeout="2000"/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD max_tries="5" shun="true"
timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK discard_delivered_msgs="true"
gc_lag="0" retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000"
print_local_addr="true" shun="false"
view_ack_collection_timeout="5000"
view_bundling="true"/>
<FRAG2 frag_size="60000"/>
<pbcast.STREAMING_STATE_TRANSFER/>
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
</jbosscache>
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
https://jira.jboss.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:21 a.m.
New subject: [JBoss JIRA] Commented: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster
[
https://jira.jboss.org/browse/JBCACHE-1592?page=com.atlassian.jira.plugin...
]
Sebastian Gruber commented on JBCACHE-1592:
-------------------------------------------
We were experiencing the same problem (even in an unclustered environment) under heavy
load. An in-depth analysis of the problem yielded the following two approaches to solve or
work around the issue:
* increase the concurrency level (see
http://jbosscache.blogspot.com/2008/07/mvcc-has-landed.html)
* disable lock striping (see
http://community.jboss.org/wiki/LockingandConcurrency#Lock_striping)
For us, increasing the concurrency level (even up to 1000000) did not resolve the problem.
We needed to switch to disabling lock striping.
Details: See class {{org.jboss.cache.lock.MVCCLockManager}}. If lock striping is enabled
(default), an {{OwnableReentrantSharedLockContainer}} used, otherwise a
{{PerElementReentrantLockContainer}}.
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
is very frequently in cluster
--------------------------------------------------------------------------------------------------
Key: JBCACHE-1592
URL: https://jira.jboss.org/browse/JBCACHE-1592
Project: JBoss Cache
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Locking
Affects Versions: 3.2.5.GA
Environment: windows os,cluster node in one machine.
Reporter: l jw
Assignee: Manik Surtani
Priority: Blocker
When used in cluster,the following exception is throw frequently:
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
[/courseClass/360/male] after [5000] milliseconds for requestor
[GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by
[GlobalTransaction:<172.18.73.162:3428>:6416]
at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
at
org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
at
org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
at org.jgroups.JChannel.up(JChannel.java:1336)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
at
org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
and following is my config:
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.2">
<locking isolationLevel="REPEATABLE_READ"
lockParentForChildInsertRemove="false"
lockAcquisitionTimeout="5000"
nodeLockingScheme="mvcc"
writeSkewCheck="true"
useLockStriping="true"
concurrencyLevel="10000"/>
<clustering mode="replication"
clusterName="JBossCache-cluster">
<stateRetrieval timeout="5000" fetchInMemoryState="false"
nonBlocking="false"/>
<sync replTimeout="5000"/>
<jgroupsConfig>
<UDP discard_incompatible_packets="true"
enable_bundling="false" enable_diagnostics="false"
ip_ttl="2"
loopback="false" max_bundle_size="64000"
max_bundle_timeout="30" mcast_addr="224.0.0.1"
mcast_port="45588" mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
oob_thread_pool.enabled="true"
oob_thread_pool.keep_alive_time="10000"
oob_thread_pool.max_threads="4"
oob_thread_pool.min_threads="1"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="10"
oob_thread_pool.rejection_policy="Run"
thread_naming_pattern="pl" thread_pool.enabled="true"
thread_pool.keep_alive_time="30000"
thread_pool.max_threads="25" thread_pool.min_threads="1"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
tos="8" ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000" use_concurrent_stack="true"
use_incoming_packet_handler="true"/>
<PING num_initial_members="3" timeout="2000"/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD max_tries="5" shun="true"
timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK discard_delivered_msgs="true"
gc_lag="0" retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000"
print_local_addr="true" shun="false"
view_ack_collection_timeout="5000"
view_bundling="true"/>
<FRAG2 frag_size="60000"/>
<pbcast.STREAMING_STATE_TRANSFER/>
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
</jbosscache>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:23 a.m.
New subject: [JBoss JIRA] Issue Comment Edited: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster
[
https://jira.jboss.org/browse/JBCACHE-1592?page=com.atlassian.jira.plugin...
]
Sebastian Gruber edited comment on JBCACHE-1592 at 12/3/10 9:22 AM:
--------------------------------------------------------------------
We were experiencing the same problem (even in an unclustered environment) under heavy
load. An in-depth analysis of the problem yielded the following two approaches to solve or
work around the issue:
- increase the concurrency level (see
http://jbosscache.blogspot.com/2008/07/mvcc-has-landed.html )
- disable lock striping (see
http://community.jboss.org/wiki/LockingandConcurrency#Lock_striping )
For us, increasing the concurrency level (even up to 1000000) did not resolve the problem.
We needed to switch to disabling lock striping.
Details: See class {{org.jboss.cache.lock.MVCCLockManager}}. If lock striping is enabled
(default), an {{OwnableReentrantSharedLockContainer}} used, otherwise a
{{PerElementReentrantLockContainer}}.
was (Author: grubers):
We were experiencing the same problem (even in an unclustered environment) under heavy
load. An in-depth analysis of the problem yielded the following two approaches to solve or
work around the issue:
* increase the concurrency level (see
http://jbosscache.blogspot.com/2008/07/mvcc-has-landed.html)
* disable lock striping (see
http://community.jboss.org/wiki/LockingandConcurrency#Lock_striping)
For us, increasing the concurrency level (even up to 1000000) did not resolve the problem.
We needed to switch to disabling lock striping.
Details: See class {{org.jboss.cache.lock.MVCCLockManager}}. If lock striping is enabled
(default), an {{OwnableReentrantSharedLockContainer}} used, otherwise a
{{PerElementReentrantLockContainer}}.
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
is very frequently in cluster
--------------------------------------------------------------------------------------------------
Key: JBCACHE-1592
URL: https://jira.jboss.org/browse/JBCACHE-1592
Project: JBoss Cache
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Locking
Affects Versions: 3.2.5.GA
Environment: windows os,cluster node in one machine.
Reporter: l jw
Assignee: Manik Surtani
Priority: Blocker
When used in cluster,the following exception is throw frequently:
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
[/courseClass/360/male] after [5000] milliseconds for requestor
[GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by
[GlobalTransaction:<172.18.73.162:3428>:6416]
at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
at
org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
at
org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
at org.jgroups.JChannel.up(JChannel.java:1336)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
at
org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
and following is my config:
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.2">
<locking isolationLevel="REPEATABLE_READ"
lockParentForChildInsertRemove="false"
lockAcquisitionTimeout="5000"
nodeLockingScheme="mvcc"
writeSkewCheck="true"
useLockStriping="true"
concurrencyLevel="10000"/>
<clustering mode="replication"
clusterName="JBossCache-cluster">
<stateRetrieval timeout="5000" fetchInMemoryState="false"
nonBlocking="false"/>
<sync replTimeout="5000"/>
<jgroupsConfig>
<UDP discard_incompatible_packets="true"
enable_bundling="false" enable_diagnostics="false"
ip_ttl="2"
loopback="false" max_bundle_size="64000"
max_bundle_timeout="30" mcast_addr="224.0.0.1"
mcast_port="45588" mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
oob_thread_pool.enabled="true"
oob_thread_pool.keep_alive_time="10000"
oob_thread_pool.max_threads="4"
oob_thread_pool.min_threads="1"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="10"
oob_thread_pool.rejection_policy="Run"
thread_naming_pattern="pl" thread_pool.enabled="true"
thread_pool.keep_alive_time="30000"
thread_pool.max_threads="25" thread_pool.min_threads="1"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
tos="8" ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000" use_concurrent_stack="true"
use_incoming_packet_handler="true"/>
<PING num_initial_members="3" timeout="2000"/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD max_tries="5" shun="true"
timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK discard_delivered_msgs="true"
gc_lag="0" retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000"
print_local_addr="true" shun="false"
view_ack_collection_timeout="5000"
view_bundling="true"/>
<FRAG2 frag_size="60000"/>
<pbcast.STREAMING_STATE_TRANSFER/>
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
</jbosscache>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
8:23 a.m.
New subject: [JBoss JIRA] Issue Comment Edited: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster
[
https://jira.jboss.org/browse/JBCACHE-1592?page=com.atlassian.jira.plugin...
]
Sebastian Gruber edited comment on JBCACHE-1592 at 12/3/10 9:23 AM:
--------------------------------------------------------------------
We were experiencing the same problem (even in an unclustered environment) under heavy
load. An in-depth analysis of the problem yielded the following two approaches to solve or
work around the issue:
- increase the concurrency level (see
http://jbosscache.blogspot.com/2008/07/mvcc-has-landed.html )
- disable lock striping (see
http://community.jboss.org/wiki/LockingandConcurrency#Lock_striping )
For us, increasing the concurrency level (even up to 1000000) did not resolve the problem.
We needed to switch to disabling lock striping.
Details: See class org.jboss.cache.lock.MVCCLockManager. If lock striping is enabled
(default), an OwnableReentrantSharedLockContainer used, otherwise a
PerElementReentrantLockContainer.
was (Author: grubers):
We were experiencing the same problem (even in an unclustered environment) under heavy
load. An in-depth analysis of the problem yielded the following two approaches to solve or
work around the issue:
- increase the concurrency level (see
http://jbosscache.blogspot.com/2008/07/mvcc-has-landed.html )
- disable lock striping (see
http://community.jboss.org/wiki/LockingandConcurrency#Lock_striping )
For us, increasing the concurrency level (even up to 1000000) did not resolve the problem.
We needed to switch to disabling lock striping.
Details: See class {{org.jboss.cache.lock.MVCCLockManager}}. If lock striping is enabled
(default), an {{OwnableReentrantSharedLockContainer}} used, otherwise a
{{PerElementReentrantLockContainer}}.
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
is very frequently in cluster
--------------------------------------------------------------------------------------------------
Key: JBCACHE-1592
URL: https://jira.jboss.org/browse/JBCACHE-1592
Project: JBoss Cache
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Locking
Affects Versions: 3.2.5.GA
Environment: windows os,cluster node in one machine.
Reporter: l jw
Assignee: Manik Surtani
Priority: Blocker
When used in cluster,the following exception is throw frequently:
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
[/courseClass/360/male] after [5000] milliseconds for requestor
[GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by
[GlobalTransaction:<172.18.73.162:3428>:6416]
at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
at
org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
at
org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
at org.jgroups.JChannel.up(JChannel.java:1336)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
at
org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
and following is my config:
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.2">
<locking isolationLevel="REPEATABLE_READ"
lockParentForChildInsertRemove="false"
lockAcquisitionTimeout="5000"
nodeLockingScheme="mvcc"
writeSkewCheck="true"
useLockStriping="true"
concurrencyLevel="10000"/>
<clustering mode="replication"
clusterName="JBossCache-cluster">
<stateRetrieval timeout="5000" fetchInMemoryState="false"
nonBlocking="false"/>
<sync replTimeout="5000"/>
<jgroupsConfig>
<UDP discard_incompatible_packets="true"
enable_bundling="false" enable_diagnostics="false"
ip_ttl="2"
loopback="false" max_bundle_size="64000"
max_bundle_timeout="30" mcast_addr="224.0.0.1"
mcast_port="45588" mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
oob_thread_pool.enabled="true"
oob_thread_pool.keep_alive_time="10000"
oob_thread_pool.max_threads="4"
oob_thread_pool.min_threads="1"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="10"
oob_thread_pool.rejection_policy="Run"
thread_naming_pattern="pl" thread_pool.enabled="true"
thread_pool.keep_alive_time="30000"
thread_pool.max_threads="25" thread_pool.min_threads="1"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
tos="8" ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000" use_concurrent_stack="true"
use_incoming_packet_handler="true"/>
<PING num_initial_members="3" timeout="2000"/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD max_tries="5" shun="true"
timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK discard_delivered_msgs="true"
gc_lag="0" retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000"
print_local_addr="true" shun="false"
view_ack_collection_timeout="5000"
view_bundling="true"/>
<FRAG2 frag_size="60000"/>
<pbcast.STREAMING_STATE_TRANSFER/>
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
</jbosscache>
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
9:44 a.m.
New subject: [JBoss JIRA] Commented: (JBCACHE-1592) org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn is very frequently in cluster
[
https://issues.jboss.org/browse/JBCACHE-1592?page=com.atlassian.jira.plug...
]
Manik Surtani commented on JBCACHE-1592:
----------------------------------------
Yes, disabling lock striping will help
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
is very frequently in cluster
--------------------------------------------------------------------------------------------------
Key: JBCACHE-1592
URL: https://issues.jboss.org/browse/JBCACHE-1592
Project: JBoss Cache
Issue Type: Bug
Security Level: Public(Everyone can see)
Components: Locking
Affects Versions: 3.2.5.GA
Environment: windows os,cluster node in one machine.
Reporter: l jw
Assignee: Manik Surtani
Priority: Blocker
When used in cluster,the following exception is throw frequently:
org.jboss.cache.lock.TimeoutException: Unable to acquire lock on Fqn
[/courseClass/360/male] after [5000] milliseconds for requestor
[GlobalTransaction:<172.18.73.162:3430>:1843]! Lock held by
[GlobalTransaction:<172.18.73.162:3428>:6416]
at org.jboss.cache.mvcc.MVCCNodeHelper.acquireLock(MVCCNodeHelper.java:159)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:217)
at org.jboss.cache.mvcc.MVCCNodeHelper.wrapNodeForWriting(MVCCNodeHelper.java:186)
at
org.jboss.cache.interceptors.MVCCLockingInterceptor.handlePutKeyValueCommand(MVCCLockingInterceptor.java:101)
at
org.jboss.cache.interceptors.base.PrePostProcessingCommandInterceptor.visitPutKeyValueCommand(PrePostProcessingCommandInterceptor.java:88)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.ReplicationInterceptor.handleCrudMethod(ReplicationInterceptor.java:150)
at
org.jboss.cache.interceptors.ReplicationInterceptor.visitPutKeyValueCommand(ReplicationInterceptor.java:107)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:65)
at
org.jboss.cache.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:100)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.TxInterceptor.replayModifications(TxInterceptor.java:501)
at
org.jboss.cache.interceptors.TxInterceptor.handleRemotePrepare(TxInterceptor.java:388)
at
org.jboss.cache.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:134)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:131)
at
org.jboss.cache.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:140)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:116)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:178)
at
org.jboss.cache.interceptors.InvocationContextInterceptor.visitPrepareCommand(InvocationContextInterceptor.java:106)
at org.jboss.cache.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:68)
at
org.jboss.cache.interceptors.InterceptorChain.invokeRemote(InterceptorChain.java:316)
at
org.jboss.cache.commands.remote.ReplicateCommand.processSingleCommand(ReplicateCommand.java:139)
at org.jboss.cache.commands.remote.ReplicateCommand.perform(ReplicateCommand.java:115)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.executeCommand(CommandAwareRpcDispatcher.java:319)
at
org.jboss.cache.marshall.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:246)
at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
at org.jgroups.JChannel.up(JChannel.java:1336)
at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:454)
at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:486)
at
org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:319)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:820)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:328)
at org.jgroups.protocols.pbcast.NAKACK.handleMessage(NAKACK.java:895)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:708)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
at org.jgroups.protocols.FD.up(FD.java:284)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:307)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
at org.jgroups.protocols.Discovery.up(Discovery.java:264)
at org.jgroups.protocols.PING.up(PING.java:273)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1273)
at org.jgroups.protocols.TP.access$100(TP.java:49)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1826)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1805)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
and following is my config:
<jbosscache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="urn:jboss:jbosscache-core:config:3.2">
<locking isolationLevel="REPEATABLE_READ"
lockParentForChildInsertRemove="false"
lockAcquisitionTimeout="5000"
nodeLockingScheme="mvcc"
writeSkewCheck="true"
useLockStriping="true"
concurrencyLevel="10000"/>
<clustering mode="replication"
clusterName="JBossCache-cluster">
<stateRetrieval timeout="5000" fetchInMemoryState="false"
nonBlocking="false"/>
<sync replTimeout="5000"/>
<jgroupsConfig>
<UDP discard_incompatible_packets="true"
enable_bundling="false" enable_diagnostics="false"
ip_ttl="2"
loopback="false" max_bundle_size="64000"
max_bundle_timeout="30" mcast_addr="224.0.0.1"
mcast_port="45588" mcast_recv_buf_size="25000000"
mcast_send_buf_size="640000"
oob_thread_pool.enabled="true"
oob_thread_pool.keep_alive_time="10000"
oob_thread_pool.max_threads="4"
oob_thread_pool.min_threads="1"
oob_thread_pool.queue_enabled="true"
oob_thread_pool.queue_max_size="10"
oob_thread_pool.rejection_policy="Run"
thread_naming_pattern="pl" thread_pool.enabled="true"
thread_pool.keep_alive_time="30000"
thread_pool.max_threads="25" thread_pool.min_threads="1"
thread_pool.queue_enabled="true"
thread_pool.queue_max_size="10" thread_pool.rejection_policy="Run"
tos="8" ucast_recv_buf_size="20000000"
ucast_send_buf_size="640000" use_concurrent_stack="true"
use_incoming_packet_handler="true"/>
<PING num_initial_members="3" timeout="2000"/>
<MERGE2 max_interval="30000"
min_interval="10000"/>
<FD_SOCK/>
<FD max_tries="5" shun="true"
timeout="10000"/>
<VERIFY_SUSPECT timeout="1500"/>
<pbcast.NAKACK discard_delivered_msgs="true"
gc_lag="0" retransmit_timeout="300,600,1200,2400,4800"
use_mcast_xmit="false"/>
<UNICAST timeout="300,600,1200,2400,3600"/>
<pbcast.STABLE desired_avg_gossip="50000"
max_bytes="400000" stability_delay="1000"/>
<pbcast.GMS join_timeout="5000"
print_local_addr="true" shun="false"
view_ack_collection_timeout="5000"
view_bundling="true"/>
<FRAG2 frag_size="60000"/>
<pbcast.STREAMING_STATE_TRANSFER/>
<pbcast.FLUSH timeout="0"/>
</jgroupsConfig>
</clustering>
</jbosscache>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
4918
days inactive
5220
days old
jbosscache-issues@lists.jboss.org
5 comments
3 participants
participants (3)
-
l jw (JIRA) -
Manik Surtani (JIRA)
-
Sebastian Gruber (JIRA)