[Design of JBossCache] - Re: Listener Notifications to happen asyncronously?
by jason.greene@jboss.com
"bstansberry(a)jboss.com" wrote :
| 1) Notifications are dispatched in a separate thread, and the thread that caused the notification does not block waiting for the notifications to finish. Thus, by the time the notification is received the cache could have processed numerous other changes. I could also conceivably get a notification for a node from change 1 after I get a notification for change 2.
|
Yes order is an issue, there is another thread I brought up which talks about the need for further changes to solve ordering problems. We should also be able to maintain TX order through use of a blocking queue. At the moment change order within a TX is guaranteed, which is a must, but the interleaving problem I described in the thread could still occur.
http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4054779
anonymous wrote :
| 2) All notifications for events that occur within a transactional context are dispatched as part of tx commit. And again, the calling thread doesn't block. Also, the transactional context is not associated with the notification thread, so if my app wanted to do transactional work as part of the notification, that won't work. Even if the tx context were associated with the notification thread, the tx is in the commit phase so it can't be used.
|
This was a change that was made earlier (before introducing the thread pool). The problem is that optimistic locking and pessimistic locking had completely different behaviors. Optimistic locking sent notifications on transaction commit. Pessimistic locking would send notifications even if a transaction was rolled back, and in that case would also send undo notifications.
Also, the cache listener code was not designed to handle reentrancy. So if you tried to make changes when receiving a notification, it could lead to all kinds of problems. Specifically, there are issues with the fact that the TX has already been committed.
anonymous wrote :
| 3) I believe the notifications are issued after locks are released. So, even if the main thread blocked while the notifications were done, the listener would have no guarantee that some other thread hasn't altered the cache state.
|
Right, this is why the cache listener is passed the state of the node at the time of the change. See the thread I mentioned about, I had suggested merging pre and post to make accessing this easier.
This still poses a problem for me though, because for pojo notifications, I need to hit the cache when processing a notification, since I don't have all of the state available (object references to other nodes).
anonymous wrote :
| I'm not sure if a notification is much use to me if I can't rely on the timing of the notification with respect to the state of the cache system. Getting tx notifications at commit could possibly be OK if #3 were solved, but #1 is a big problem.
|
| Note that a major use of notifications is to make app code aware of remotely originating modifications to the cached data. There's no other convenient mechanism for this code of cluster-wide signalling.
The ordering issues can be solved, the ability to intercept and modify cache state during any modification is not possible with the current design. For that to work we would essentially need them to be more like a database trigger than a notification. So in both the pessimistic and optimistic locking scenarios, the callbacks would have to occur after each change regardless of the success of the transaction. or this to occur we would need to make the cache interceptor stack reentrant. Also, This would mean that app listeners would have to be able to handle changes that never committed.
To be honest this type of design would be easier for me to map for POJO cache, however it conflicts with the goal for notifications to be sent only when the tx is committed.
-Jason
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4057204#4057204
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4057204
18 years, 9 months
[Design of Messaging on JBoss (Messaging/JBoss)] - Re: DB performances for large number of messages
by timfox
I think the way this should be done is to enable the user to configure a specific destination to use a specific persistence manager.
This would involve an extra (optional) attribute "PersistenceManager" on the destination.
When doing any database operations associated with a particular destination, we just need to make sure the correct persistence manager is used, then (hopefully) pretty much all the rest of the code doesn't need to change.
There is the following complication.
When doing transactional operations (sending/acking) to more than one persistence manager in the same transaction, we need that to be transactional.
So we can either
1) Implement our own XAResource to handle this (not recommended)
2) Leverage any XA capabilities of the underlying database.
If we make sure we use an XADatasource for each database, we should be able to leverage the JCA adapter to handle transaction enlistment.
Of course, if the underlying database does not support XA we cannot do this.
Going ahead I think this will less of a problem since we will support local file based persistence stores so each node in the cluster can have its own storage.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4057189#4057189
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4057189
18 years, 9 months
[Design of JBossCache] - Re: Listener Notifications to happen asyncronously?
by bstansberry@jboss.com
We're not just forcing the use of a different thread; we are changing the timing of the notifications.
For readers, these are some differences in 2.0.0.CR2 vs. the 1.x releases:
1) Notifications are dispatched in a separate thread, and the thread that caused the notification does not block waiting for the notifications to finish. Thus, by the time the notification is received the cache could have processed numerous other changes. I could also conceivably get a notification for a node from change 1 after I get a notification for change 2.
2) All notifications for events that occur within a transactional context are dispatched as part of tx commit. And again, the calling thread doesn't block. Also, the transactional context is not associated with the notification thread, so if my app wanted to do transactional work as part of the notification, that won't work. Even if the tx context were associated with the notification thread, the tx is in the commit phase so it can't be used.
3) I believe the notifications are issued after locks are released. So, even if the main thread blocked while the notifications were done, the listener would have no guarantee that some other thread hasn't altered the cache state.
I'm not sure if a notification is much use to me if I can't rely on the timing of the notification with respect to the state of the cache system. Getting tx notifications at commit could possibly be OK if #3 were solved, but #1 is a big problem.
Note that a major use of notifications is to make app code aware of remotely originating modifications to the cached data. There's no other convenient mechanism for this code of cluster-wide signalling.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4057137#4057137
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4057137
18 years, 9 months
[Design of Messaging on JBoss (Messaging/JBoss)] - JBossMessaging with Remoting http transport
by ron.sigal@jboss.com
I've introduced a new blocking mode for pull callback that should help with Remoting http transport performance. Rather than polling periodically for callbacks, org.jboss.remoting.callback.CallbackPoller can be configured to block on the server side and wait for a callback to be created, at which point it returns immediately. There is also a timeout period, after which an empty list will be returned. For details, see the Remoting developers forum thread "New blocking mode for pul callbacks" (http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4057082#4057082).
To use blocking mode in JBM, replace the remoting-service.xml line
| <attribute name="callbackPollPeriod" isParam="true">100</attribute>
|
with
| <attribute name="blockingMode" isParam="true">blocking</attribute>
|
I've run the JBM http stress tests in blocking mode, and they pass. (This is in JBossMessaging 1.2.0.SP.) The only problem I was having was that org.jboss.remoting.ConnectionValidator was occasionally timing out. There was a hardcoded timeout value of 1000 ms in ConnectionValidator, and, on my year old Windows laptop (Dell 9400 dual core), RelayStressTest seemed to slow things down enough to cause a timeout (though the test passed). I made the ConnectionValidator timeout value configurable, and, when I set the timeout to 4000, the ConnectionValidator timeouts went away. At 3000 I got a timeout on one run and no timeout on another run. I tried increasing the tomcat server threads, but it didn't seem to help. Note that on my new T60 RHEL5 machine, the tests passed without timeouts with the default timeout value of 1000 ms. By the way, the http stress tests take about 90 minutes on the Windows machine and about 22 minutes on the linux machine. (!)
Being concerned about timeouts, I ran an http version of the clustering tests
| <antcall target="clustering-tests">
| <param name="test.remoting" value="http"/>
| </antcall>
|
and that passed as well.
This change currently lives on the remoting_2_x branch. I'll try to keep track of the releases of the various JBossAS versions so I can get it into the next releases that will accept new features.
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4057084#4057084
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4057084
18 years, 9 months
[Design of JBoss Remoting, Unified Invokers] - New blocking mode for pul callbacks
by ron.sigal@jboss.com
Remoting has a new callback polling mode, in response to the forum thread http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3990104#3990104.
In the original mode, now known as "nonblocking" mode, org.jboss.remoting.Client.getCallbacks() makes an invocation on the server and returns all waiting callbacks. If there are no callbacks to be retrieved, it returns an empty list. This call may be made directly from the application, in the manner of pull callbacks, or it can be made from an instance of org.jboss.remoting.callback.CallbackPoller, which simulates push callbacks by pulling callbacks from the server and then pushing them to an org.jboss.remoting.callback.InvokerCallbackHandler. See Section 5.6 "Callbacks" in the Remoting Guide (http://labs.jboss.com/jbossremoting/docs/guide/ch05.html#section-callbacks) for more information.
The problem with nonblocking mode is latency. For example, if CallbackPoller is used, the default behavior is to poll for callbacks every five seconds. In the new "blocking" mode, on the other hand, Client.getCallbacks() will either
1. return all waiting callbacks, if there are any, or
2. if there are no waiting callbacks, block on the server side and wait until a callback is created or until the timeout period elapses.
In other words, in blocking mode, a callback will be returned soon after it is created
The default behavior is to use nonblocking mode, to avoid behavioral changes without explicit action. To switch to blocking mode, the key org.jboss.remoting.ServerInvoker.BLOCKING_MODE (actual value "blockingMode") needs to be added to the appropriate configuration map, as explained below, with the value ServerInvoker.BLOCKING (actual value "blocking").
To accomodate pull callbacks, a new method
| public List getCallbacks(InvokerCallbackHandler callbackHandler, Map metadata) throws Throwable;
|
has been added to the Client class, and the configuration information can be passed in the metadata map. If CallbackPoller is used, then one of the methods from the Client class
| public void addListener(InvokerCallbackHandler callbackhandler, Map metadata) throws Throwable;
|
| public void addListener(InvokerCallbackHandler callbackhandler, Map metadata,
| Object callbackHandlerObject) throws Throwable;
|
| public void addListener(InvokerCallbackHandler callbackhandler, Map metadata,
| Object callbackHandlerObject, boolean serverToClient) throws Throwable;
|
may be used, with the configuration information passed in the metadata map.
A couple of additional configuration parameters are available, which are passed the same way as the blocking mode.
1. ServerInvoker.BLOCKING_TIMEOUT (actual value "blockingTimeout") determines how long the Client.getCallbacks() will block on the server side, waiting for a callback to be created, before timing out and returning an empty list. The default value is 5000 ms.
2. CallbackPoller.MAX_ERROR_COUNT (actual value "maxErrorCount") determines how many errors CallbackPoller will sustain before it stops polling and shuts itself down. This parameter applies to both blocking and nonblocking polling. The default value is 5
View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4057082#4057082
Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4057082
18 years, 9 months