[JBoss Cache Users] - Re: MVCC leading to deadlocks in synchronous replication mod
by veklov
As I understand your scenario is not primary use case for JBossCache. People from Hibernate brought up this issue some time ago and as the result the method putForExternalRead was added:
/**
| * Under special operating behavior, associates the value with the specified key for a node identified by the Fqn passed in.
| * <ul>
| * <li> Only goes through if the node specified does not exist; no-op otherwise.</i>
| * <li> Force asynchronous mode for replication to prevent any blocking.</li>
| * <li> invalidation does not take place. </li>
| * <li> 0ms lock timeout to prevent any blocking here either. If the lock is not acquired, this method is a no-op, and swallows the timeout exception.</li>
| * <li> Ongoing transactions are suspended before this call, so failures here will not affect any ongoing transactions.</li>
| * <li> Errors and exceptions are 'silent' - logged at a much lower level than normal, and this method does not throw exceptions</li>
| * </ul>
| * This method is for caching data that has an external representation in storage, where, concurrent modification and
| * transactions are not a consideration, and failure to put the data in the cache should be treated as a 'suboptimal outcome'
| * rather than a 'failing outcome'.
| * <p/>
| * An example of when this method is useful is when data is read from, for example, a legacy datastore, and is cached before
| * returning the data to the caller. Subsequent calls would prefer to get the data from the cache and if the data doesn't exist
| * in the cache, fetch again from the legacy datastore.
| * <p/>
| * See <a href="http://jira.jboss.com/jira/browse/JBCACHE-848">JBCACHE-848</a> for details around this feature.
| * <p/>
| *
| * @param fqn <b><i>absolute</i></b> {@link Fqn} to the {@link Node} to be accessed.
| * @param key key with which the specified value is to be associated.
| * @param value value to be associated with the specified key.
| * @throws IllegalStateException if {@link #getCacheStatus()} would not return {@link org.jboss.cache.CacheStatus#STARTED}.
| */
| void putForExternalRead(Fqn fqn, K key, V value);
|
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4268556#4268556
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4268556
14 years, 9 months
[JBoss Cache Users] - Re: How to prevent TimeoutException?
by galder.zamarreno＠jboss.com
As indicated in the documentation, JBoss Cache uses lock stripping by default and you can control the shared pool of locks by tweaking the concurrency level, as you already did. However, lock striping, under certain circumstances, can lead to deadlocks as indicated in https://jira.jboss.org/jira/browse/JBCACHE-1494. So, something good to try is to run your benchmark with:
<locking .... useLockStriping="false" />
More generally, debugging TimeoutExceptions involves generally producing some thread dumps when these are reported. The idea here is finding out which other thread might be holding that particular lock and see whether it's hanged or waiting for something else and see if that other resource can be tweaked accordingly.
Tools like JVisualVM (part of the jdk) can both generate and analyse thread dumps, otherwise you can use the rudimentary kill -3 or equivalent method and use standalone tools like to inspect them, i.e.:
- TDA (https://tda.dev.java.net/)
- or Samurai (http://yusuke.homeip.net/samurai/en/index.html)
Other tips to get around TimeoutExceptions include reducing the transactional scope of the operation that is holding the lock. That way, the time the lock is held is reduced and hence it's released earlier. Reducing the transactional scope can be achieved by either moving some operations out of the transaction or finding more efficient ways to do what is being done within the transaction.
Sometimes these TimeoutException can hide behind memory issues that produce long garbage collection periods. Turning on garbage collection logging (http://www.jboss.org/community/docs/DOC-12451) can help visualise whether there are noticeable stoppages. Turning on VM parameters can sometimes help reduce the stoppages which in turn increases speed of operation and consequently reduces TimeoutException occurrences.
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4268551#4268551
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4268551
14 years, 9 months
[jBPM Users] - Re: Variables keep old values
by aroeder
I wrote a JUnit test for showing my problem:
| package de.firstdata.test;
|
| import java.io.IOException;
| import java.util.HashMap;
| import java.util.Map;
| import java.util.Set;
|
| import org.jbpm.graph.def.ProcessDefinition;
| import org.jbpm.graph.exe.ProcessInstance;
| import org.jbpm.taskmgmt.exe.TaskInstance;
|
| import de.firstdata.config.TicketServiceConfigService;
| import de.firstdata.dm.JBPMContextWrapper;
| import de.firstdata.dm.TicketSystemDomainModelFacade;
|
| public class JbpmVariablesTest extends ATicketServiceTestA {
|
| private TicketSystemDomainModelFacade facade = null;
|
| @SuppressWarnings("unchecked")
| public void testVariables() throws IOException {
|
| JBPMContextWrapper ctx = getContextWrapper();
|
| try {
| ProcessDefinition processDefinition = ProcessDefinition
| .parseXmlString("<process-definition xmlns='' name='ABR'>" + "<start-state name='start-state1'>"
| + "<transition to='edit'></transition>" + "</start-state>" + "<task-node name='edit'>"
| + "<task name='edit'>" + " <controller>"
| + " <variable access='read,write,required' name='inbox'></variable>" + " </controller>"
| + "</task>" + "<transition to='change' name='tochange'></transition>"
| + "<transition to='end-state1' name='done'></transition>" + "</task-node>"
| + "<task-node name='change'>" + "<task name='indexDataChange'>" + " <controller>"
| + " <variable access='read,write,required' name='inbox'></variable>" + " </controller>"
| + "</task>" + "<transition to='edit' name='back'></transition>" + "</task-node>"
| + "<end-state name='end-state1'></end-state>" + "</process-definition>");
|
| Map<String, String> myMap = new HashMap<String, String>();
| myMap.put("inbox", "COMPLAINTS_MANAGEMENT");
|
| ctx.deployProcessDefinition(processDefinition);
| ProcessInstance processInstance = processDefinition.createProcessInstance(myMap);
| ctx.save(processInstance);
| processInstance.signal();
|
| TaskInstance taskEdit = getTaskInstance(processInstance, "edit");
| taskEdit.end("tochange");
|
| ctx.save(taskEdit);
|
| TaskInstance taskChange = getTaskInstance(processInstance, "indexDataChange");
| Map<String, String> variables = taskChange.getContextInstance().getVariables();
| variables.put("inbox", "CS_ISO");
| taskChange.getProcessInstance().getContextInstance().addVariables(variables);
| ctx.save(taskChange);
| ctx.close();
| ctx = getContextWrapper();
|
| taskChange.end("back");
|
| taskEdit = getTaskInstance(processInstance, "edit");
| Map<String, String> changedVariables = taskEdit.getProcessInstance().getContextInstance().getVariables();
|
| String inbox = (String) changedVariables.get("inbox");
|
| taskEdit.end("done");
| processInstance.end();
| ctx.deleteProcessDefinition(processDefinition);
|
| assertEquals("CS_ISO", inbox);
| } finally {
| ctx.close();
| }
| }
|
| private JBPMContextWrapper getContextWrapper() {
| if (facade == null) {
| facade = (TicketSystemDomainModelFacade) TicketServiceConfigService.getConfig().getFacade();
| }
| return JBPMContextWrapper.getInstance("jbpm.cfg.xml", facade);
| }
|
| @SuppressWarnings("unchecked")
| private TaskInstance getTaskInstance(final ProcessInstance processInstance, final String taskInstanceName) {
| TaskInstance returnInstance = null;
| Set<TaskInstance> tasks = (Set<TaskInstance>) processInstance.getTaskMgmtInstance().getTaskInstances();
|
| for (TaskInstance ti : tasks) {
| if (ti.getName().equals(taskInstanceName) && !ti.hasEnded()) {
| returnInstance = ti;
| }
| }
|
| return returnInstance;
| }
| }
|
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4268545#4268545
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4268545
14 years, 9 months
[JBoss Cache Users] - Current transation can not see modifications done by committ
by veklov
Current transation can not see modifications done by committed transactions while isolation level is READ_COMMITTED.
Cache version is 3.2.1.GA. Can be replicated when cache mode is REPL_SYNC or INVALIDATION_SYNC (the other are not tested). Please see below test cases for details.
Note: Test cases use DummyTransactionManager and behavior of second test case for REPL_SYNC mode depends on whether calls are done in context of JTA transaction or not, so in this mode the issue can be caused by a bug in DummyTransactionManager.
INVALIDATION_SYNC:
import java.util.ArrayList;
| import java.util.Collections;
| import java.util.List;
| import java.util.concurrent.CyclicBarrier;
| import java.util.concurrent.TimeUnit;
| import javax.transaction.TransactionManager;
|
| import junit.framework.TestCase;
| import org.jboss.cache.Cache;
| import org.jboss.cache.CacheFactory;
| import org.jboss.cache.CacheSPI;
| import org.jboss.cache.DefaultCacheFactory;
| import org.jboss.cache.config.Configuration;
| import org.jboss.cache.lock.IsolationLevel;
|
| public class TestInvalidationSyncReadCommited extends TestCase {
|
| private static final String NODE_FQN = "/node";
| private static final String KEY = "key";
| private static final String VALUE1 = "value1";
| private static final String VALUE2 = "value2";
|
| private static Cache createCache() {
| final CacheFactory cf = new DefaultCacheFactory();
| final Configuration configuration = new Configuration();
| configuration.setCacheMode(Configuration.CacheMode.INVALIDATION_SYNC);
| configuration.setTransactionManagerLookupClass("org.jboss.cache.transaction.DummyTransactionManagerLookup");
| configuration.setIsolationLevel(IsolationLevel.READ_COMMITTED);
| configuration.setLockAcquisitionTimeout(10000L);
| return cf.createCache(configuration, true);
| }
|
| public void testPut() throws Exception {
| final Cache cache1 = createCache();
| final Cache cache2 = createCache();
| assertEquals("Members count", 2, cache1.getMembers().size());
| assertEquals("Members count", 2, cache2.getMembers().size());
|
| final CyclicBarrier barrier = new CyclicBarrier(2);
| final List exceptions = Collections.synchronizedList(new ArrayList());
|
| cache1.put(NODE_FQN, KEY, VALUE1);
|
| final Thread thread1 = new Thread() {
| public void run() {
| try {
| getTransactionManager(cache1).begin();
| assertEquals("Must be in cache", VALUE1, cache1.get(NODE_FQN, KEY));
| await(barrier);
| await(barrier);
| // assertEquals("For some reason it is in cache", VALUE1, cache1.get(NODE_FQN, KEY));
| assertNull("Must be invalidated before commit", cache1.get(NODE_FQN, KEY));
| getTransactionManager(cache1).commit();
| assertNull("Must be invalidated", cache1.get(NODE_FQN, KEY));
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
| final Thread thread2 = new Thread() {
| public void run() {
| try {
| await(barrier);
| getTransactionManager(cache2).begin();
| cache2.put(NODE_FQN, KEY, VALUE2);
| getTransactionManager(cache2).commit();
| await(barrier);
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
|
| thread1.start();
| thread2.start();
| thread1.join();
| thread2.join();
|
| cache1.stop();
| cache2.stop();
|
| if (!exceptions.isEmpty()) {
| fail(exceptions.toString());
| }
| }
|
| public void testRemoveNode() throws Exception {
| final Cache cache1 = createCache();
| final Cache cache2 = createCache();
| assertEquals("Members count", 2, cache1.getMembers().size());
| assertEquals("Members count", 2, cache2.getMembers().size());
|
| final CyclicBarrier barrier = new CyclicBarrier(2);
| final List exceptions = Collections.synchronizedList(new ArrayList());
|
| cache1.put(NODE_FQN, KEY, VALUE1);
|
| final Thread thread1 = new Thread() {
| public void run() {
| try {
| getTransactionManager(cache1).begin();
| assertEquals("Must be in cache", VALUE1, cache1.get(NODE_FQN, KEY));
| await(barrier);
| await(barrier);
| // assertEquals("For some reason it is in cache", VALUE1, cache1.get(NODE_FQN, KEY));
| assertNull("Must be invalidated before commit", cache1.get(NODE_FQN, KEY));
| getTransactionManager(cache1).commit();
| assertNull("Must be invalidated", cache1.get(NODE_FQN, KEY));
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
| final Thread thread2 = new Thread() {
| public void run() {
| try {
| await(barrier);
| getTransactionManager(cache2).begin();
| cache2.removeNode(NODE_FQN);
| getTransactionManager(cache2).commit();
| await(barrier);
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
|
| thread1.start();
| thread2.start();
| thread1.join();
| thread2.join();
|
| cache1.stop();
| cache2.stop();
|
| if (!exceptions.isEmpty()) {
| fail(exceptions.toString());
| }
| }
|
| private static TransactionManager getTransactionManager(final Cache cache) {
| return ((CacheSPI) cache).getTransactionManager();
| }
|
| private static void await(final CyclicBarrier barrier) throws Exception {
| barrier.await(20, TimeUnit.SECONDS);
| }
| }
|
REPL_SYNC (This case can be fixed by commenting out begin/commit calls in thread2, e.g. behavior depends on whether calls are done in context of JTA transaction or not. Is that an issue with DummyTransactionManager or with cache itself?):
import java.util.ArrayList;
| import java.util.Collections;
| import java.util.List;
| import java.util.concurrent.BrokenBarrierException;
| import java.util.concurrent.CyclicBarrier;
| import java.util.concurrent.TimeUnit;
| import java.util.concurrent.TimeoutException;
| import javax.transaction.TransactionManager;
|
| import junit.framework.TestCase;
| import org.jboss.cache.Cache;
| import org.jboss.cache.CacheFactory;
| import org.jboss.cache.CacheSPI;
| import org.jboss.cache.DefaultCacheFactory;
| import org.jboss.cache.config.Configuration;
| import org.jboss.cache.lock.IsolationLevel;
|
| public class TestReplSyncReadCommitted2 extends TestCase {
|
| private static final String NODE_FQN = "/node";
| private static final String KEY = "key";
| private static final String VALUE1 = "value1";
| private static final String VALUE2 = "value2";
|
| private static Cache createCache() {
| final CacheFactory cf = new DefaultCacheFactory();
| final Configuration configuration = new Configuration();
| configuration.setCacheMode(Configuration.CacheMode.REPL_SYNC);
| configuration.setTransactionManagerLookupClass("org.jboss.cache.transaction.DummyTransactionManagerLookup");
| configuration.setIsolationLevel(IsolationLevel.READ_COMMITTED);
| configuration.setLockAcquisitionTimeout(10000L);
| return cf.createCache(configuration, true);
| }
|
| public void testRemoveNodeTwoCaches() throws InterruptedException {
| final Cache cache1 = createCache();
| final Cache cache2 = createCache();
| assertEquals("Members count", 2, cache1.getMembers().size());
| assertEquals("Members count", 2, cache2.getMembers().size());
|
| final CyclicBarrier barrier = new CyclicBarrier(2);
| final List exceptions = Collections.synchronizedList(new ArrayList());
|
| final Thread thread1 = new Thread() {
| public void run() {
| try {
| await(barrier);
| await(barrier);
| getTransactionManager(cache1).begin();
| assertEquals("Must be replicated", VALUE1, cache1.get(NODE_FQN, KEY));
| await(barrier);
| await(barrier);
| assertNull("Must be removed", cache1.get(NODE_FQN, KEY));
| getTransactionManager(cache1).commit();
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
| final Thread thread2 = new Thread() {
| public void run() {
| try {
| await(barrier);
| getTransactionManager(cache2).begin();
| cache2.put(NODE_FQN, KEY, VALUE1);
| getTransactionManager(cache2).commit();
| await(barrier);
| await(barrier);
| getTransactionManager(cache2).begin();
| cache2.removeNode(NODE_FQN);
| getTransactionManager(cache2).commit();
| await(barrier);
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
|
| thread1.start();
| thread2.start();
| thread1.join();
| thread2.join();
|
| cache1.stop();
| cache2.stop();
|
| if (!exceptions.isEmpty()) {
| fail(exceptions.toString());
| }
| }
|
| public void testPutTwoCaches() throws InterruptedException {
| final Cache cache1 = createCache();
| final Cache cache2 = createCache();
| assertEquals("Members count", 2, cache1.getMembers().size());
| assertEquals("Members count", 2, cache2.getMembers().size());
|
| final CyclicBarrier barrier = new CyclicBarrier(2);
| final List exceptions = Collections.synchronizedList(new ArrayList());
|
| final Thread thread1 = new Thread() {
| public void run() {
| try {
| await(barrier);
| await(barrier);
| getTransactionManager(cache1).begin();
| assertEquals("Must be replicated", VALUE1, cache1.get(NODE_FQN, KEY));
| await(barrier);
| await(barrier);
| assertEquals("Must be replicated", VALUE2, cache1.get(NODE_FQN, KEY));
| getTransactionManager(cache1).commit();
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
| final Thread thread2 = new Thread() {
| public void run() {
| try {
| await(barrier);
| getTransactionManager(cache2).begin();
| cache2.put(NODE_FQN, KEY, VALUE1);
| getTransactionManager(cache2).commit();
| await(barrier);
| await(barrier);
| getTransactionManager(cache2).begin();
| cache2.put(NODE_FQN, KEY, VALUE2);
| getTransactionManager(cache2).commit();
| await(barrier);
| } catch (Throwable e) {
| exceptions.add(e);
| }
| }
| };
|
| thread1.start();
| thread2.start();
| thread1.join();
| thread2.join();
|
| cache1.stop();
| cache2.stop();
|
| if (!exceptions.isEmpty()) {
| fail(exceptions.toString());
| }
| }
|
| private static TransactionManager getTransactionManager(final Cache cache) {
| return ((CacheSPI) cache).getTransactionManager();
| }
|
| private static void await(final CyclicBarrier barrier)
| throws InterruptedException, BrokenBarrierException, TimeoutException {
| barrier.await(20, TimeUnit.SECONDS);
| }
| }
|
View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4268540#4268540
Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=4268540
14 years, 9 months