[infinispan-dev] Remote command smarter dispatcher (merge ISPN-2808 and ISPN-2849)

[Erik Salter]

Hi Pedro,

May I steal this implementation for improving xsite replication? =)

Erik

-----Original Message-----
From: infinispan-dev-bounces at lists.jboss.org
[mailto:infinispan-dev-bounces at lists.jboss.org] On Behalf Of Pedro Ruivo
Sent: Monday, March 18, 2013 12:10 PM
To: ispn-dev
Subject: [infinispan-dev] Remote command smarter dispatcher (merge ISPN-2808
and ISPN-2849)

Hi all,

To solve ISPN-2808 (avoid blocking JGroups threads in order to allow to
deliver the request responses), I've created another thread pool to move the
possible blocking commands (i.e. the commands that may block until some
state is achieved).

Problem description:

With this solution, the new thread pool should be large in order to be able
to handle the remote commands without deadlocks. The problem is that all the
threads can be block to process the command that may unblock other commands.

Example: a bunch of commands are blocked waiting for a new topology ID and
the command that will increment the topology ID is in the thread pool queue.

Solution:

Use a smart command dispatcher, i.e., keep the command in the queue until we
are sure that it will not wait for other commands. I've already implemented
some kind of executor service (ConditionalExecutorService, in ISPN-2635 and
ISPN-2636 branches, Total Order stuff) that only put the Runnable (more
precisely a new interface called ConditionalRunnable) in the thread pool
when it is ready to be processed. Creative guys, it may need a better name
:)

The ConditionalRunnable has a new method (boolean isReady()) that should
return true when the runnable should not block.

Example how to apply this to ISPN-2808:

Most of the commands awaits for a particular topology ID and/or for lock
acquisition. In this way, the isReady() implementation can be something
like:

isReady()
  return commandTopologyId <= currentTopologyId && (for all keys; do if
!lock(key).tryLock(); return false; done)

With this, I believe we can keep the number of thread low and avoid the
thread deadlocks.

Now, I have two possible implementations:

1) put a reference for StateTransferManager and/or LockManager in the
commands, and invoke the methods directly (a little dirty)

2) added new method in the CommandInterceptor like: boolean
preProcess<command>(Command, InvocationContext). each interceptor will check
if the command will block on it (returning false) or not (invoke the next
interceptor). For example, the StateTransferInterceptor returns immediately
false if the commandToplogyId is higher than the currentTopologyId and the
*LockingIntercerptor will return false if it cannot acquire some lock.

Any other suggestions? If I was not clear let me know.

Thanks.

Cheers,
Pedro
_______________________________________________
infinispan-dev mailing list
infinispan-dev at lists.jboss.org
https://lists.jboss.org/mailman/listinfo/infinispan-dev