11:43 a.m.
Hi all
We are having some kind of strange locks that comes from our cache that we have in
production. This problem have come up from two different situations:
The first one is during a scheduler execution that just evicts a node at certain time.
The second one is during normal applications execution.
This second situation is quite confused to us. Stack trace below:
"http-170.32.21.12-8080-250" daemon prio=10 tid=6000000007078c50 nid=43815
lwp_id=9087433 in Object.wait() [9fffffffd84c0000..9ffff
| fffd84c0c40]
| at java.lang.Object.wait(Native Method)
| at java.lang.Object.wait(Object.java:474)
| at
org.jboss.cache.interceptors.BaseCacheLoaderInterceptor.obtainLoaderLock(BaseCacheLoaderInterceptor.java:49)
| - locked <9ffffffee2c31308> (a
org.jboss.cache.interceptors.CacheStoreInterceptor)
| at
org.jboss.cache.interceptors.BaseCacheLoaderInterceptor.obtainLoaderLocks(BaseCacheLoaderInterceptor.java:91)
| at
org.jboss.cache.interceptors.CacheStoreInterceptor.invoke(CacheStoreInterceptor.java:92)
| at org.jboss.cache.interceptors.Interceptor.invoke(Interceptor.java:68)
| at
org.jboss.cache.interceptors.TxInterceptor.handleCommitRollback(TxInterceptor.java:719)
| at
org.jboss.cache.interceptors.TxInterceptor.runCommitPhase(TxInterceptor.java:761)
| at
org.jboss.cache.interceptors.TxInterceptor$RemoteSynchronizationHandler.afterCompletion(TxInterceptor.java:1076)
| at
org.jboss.cache.interceptors.TxInterceptor$LocalSynchronizationHandler.afterCompletion(TxInterceptor.java:1175)
| at
org.jboss.cache.interceptors.OrderedSynchronizationHandler.afterCompletion(OrderedSynchronizationHandler.java:83)
| at
com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:136)
| at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:342)
| at
com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:95)
| at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
| at
com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1256)
| at
com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
| at
com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
| at
org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
|
This cache lock leads to a 8080 port blocking, that means no service availability.
Our cache configuration is as follows:
| <server>
|
| <classpath codebase="./lib" archives="jboss-cache.jar,
jgroups.jar"/>
|
|
| <!-- ====================================================================
-->
| <!-- Defines TreeCache configuration
-->
| <!-- ====================================================================
-->
|
|
| <mbean code="org.jboss.cache.TreeCache"
| name="jboss.cache:service=FooCache">
|
| <depends>jboss:service=Naming</depends>
| <depends>jboss:service=TransactionManager</depends>
|
| <attribute
name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>
|
| <!--
| Isolation level : SERIALIZABLE
| REPEATABLE_READ (default)
| READ_COMMITTED
| READ_UNCOMMITTED
| NONE
| -->
| <attribute name="IsolationLevel">NONE</attribute>
|
| <!--
| Valid modes are LOCAL
| REPL_ASYNC
| REPL_SYNC
| INVALIDATION_ASYNC
| INVALIDATION_SYNC
| -->
| <attribute name="CacheMode">REPL_SYNC</attribute>
|
| <!--
| Just used for async repl: use a replication queue
| -->
| <attribute name="UseReplQueue">false</attribute>
|
| <!--
| Replication interval for replication queue (in ms)
| -->
| <attribute name="ReplQueueInterval">0</attribute>
|
| <!--
| Max number of elements which trigger replication
| -->
| <attribute name="ReplQueueMaxElements">0</attribute>
|
| <!-- Name of cluster. Needs to be the same for all clusters, in order
| to find each other
| -->
| <attribute
name="ClusterName">FooCache-Cluster</attribute>
|
| <!-- JGroups protocol stack properties. Can also be a URL,
| e.g. file:/home/bela/default.xml
| <attribute name="ClusterProperties"></attribute>
| -->
|
| <attribute name="ClusterConfig">
| <config>
| <!-- UDP: if you have a multihomed machine,
| set the bind_addr attribute to the appropriate NIC IP address, e.g
bind_addr="192.168.0.2"
| -->
| <!-- UDP: On Windows machines, because of the media sense feature
| being broken with multicast (even after disabling media sense)
| set the loopback attribute to true -->
| <UDP mcast_addr="${jboss.cache.FooCache.addr:228.1.2.3}"
| mcast_port="${jboss.cache.FooCache.port:48866}"
| ip_ttl="64" ip_mcast="true"
| mcast_send_buf_size="150000"
mcast_recv_buf_size="80000"
| ucast_send_buf_size="150000"
ucast_recv_buf_size="80000"
| loopback="false"/>
| <PING timeout="2000" num_initial_members="3"
| up_thread="false" down_thread="false"/>
| <MERGE2 min_interval="10000"
max_interval="20000"/>
| <!-- <FD shun="true" up_thread="true"
down_thread="true" />-->
| <FD_SOCK/>
| <VERIFY_SUSPECT timeout="1500"
| up_thread="false" down_thread="false"/>
| <pbcast.NAKACK gc_lag="50"
retransmit_timeout="600,1200,2400,4800"
| max_xmit_size="8192" up_thread="false"
down_thread="false"/>
| <UNICAST timeout="600,1200,2400"
window_size="100" min_threshold="10"
| down_thread="false"/>
| <pbcast.STABLE desired_avg_gossip="20000"
| up_thread="false" down_thread="false"/>
| <FRAG frag_size="8192"
| down_thread="false" up_thread="false"/>
| <pbcast.GMS join_timeout="5000"
join_retry_timeout="2000"
| shun="true" print_local_addr="true"/>
| <pbcast.STATE_TRANSFER up_thread="true"
down_thread="true"/>
| </config>
| </attribute>
|
|
| <!--
| Whether or not to fetch state on joining a cluster
| NOTE this used to be called FetchStateOnStartup and has been renamed to be
more descriptive.
| -->
| <attribute name="FetchInMemoryState">true</attribute>
|
| <!--
| The max amount of time (in milliseconds) we wait until the
| initial state (ie. the contents of the cache) are retrieved from
| existing members in a clustered environment
| -->
| <attribute
name="InitialStateRetrievalTimeout">15000</attribute>
|
| <!--
| Number of milliseconds to wait until all responses for a
| synchronous call have been received.
| -->
| <attribute name="SyncReplTimeout">15000</attribute>
|
| <!-- Max number of milliseconds to wait for a lock acquisition -->
| <attribute
name="LockAcquisitionTimeout">10000</attribute>
|
| <!-- Name of the eviction policy class. -->
| <attribute
name="EvictionPolicyClass">org.jboss.cache.eviction.LRUPolicy</attribute>
| <!-- Specific eviction policy configurations. This is LRU -->
| <attribute name="EvictionPolicyConfig">
| <config>
| <!-- This attribute will be share by all eviction policies -->
| <attribute
name="wakeUpIntervalSeconds">5</attribute>
| <!-- Cache wide default 86400-->
| <region name="/_default_">
| <attribute
name="maxNodes">500000</attribute>
| <attribute
name="timeToLiveSeconds">86400</attribute>
| </region>
| </config>
| </attribute>
|
|
| <!--
| Indicate whether to use region based marshalling or not. Set this to true if
you are running under a scoped
| class loader, e.g., inside an application server. Default is
"false".
| -->
|
| <attribute
name="UseRegionBasedMarshalling">true</attribute>
| <attribute name="InactiveOnStartup">true</attribute>
|
| <attribute name="CacheLoaderConfiguration">
| <config>
| <passivation>false</passivation>
| <preload>/</preload>
| <shared>true</shared>
| <!-- Own implementation of a ClusteredCacheLoader that
allows act/deact of regions -->
| <cacheloader>
| <class>
| com.foo.bar.loader.ExtendedClusteredCacheLoader
| </class>
| <properties>timeout=1000</properties>
| <async>true</async>
| <fetchPersistentState>false</fetchPersistentState>
| <ignoreModifications>false</ignoreModifications>
| </cacheloader>
|
| </config>
| </attribute>
|
| </mbean>
|
|
| <!-- Uncomment to get a graphical view of the TreeCache MBean above -->
| <!-- <mbean code="org.jboss.cache.TreeCacheView"
name="jboss.cache:service=TreeCacheView">-->
| <!-- <depends>jboss.cache:service=TreeCache</depends>-->
| <!-- <attribute
name="CacheService">jboss.cache:service=TreeCache</attribute>-->
| <!-- </mbean>-->
|
|
| </server>
|
Hope someone could help us.
Thanks in advance!
View the original post :
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4107831#...
Reply to the post :
http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...