"galder.zamarreno(a)jboss.com" wrote : A month ago, Bela, Vladimir, Jimmy and myself were discussing my suggestions proposed in this forum thread. Apologies if I haven't been able to post the notes earlier. | I was wondering where this went! :-) "galder.zamarreno(a)jboss.com" wrote : | Interceptor vs listener vs HA-Singleton: | - based on a configuration option, an interceptor (preferred) or cache listener could be created that encapsulates solution; this is preferred to a standalone solution based on HA-Singleton. | - interceptor allows for greater flexibility and simplicity in terms of relaying data between dcs (data centres), filtering, transactional work and state transfer propagation than cache listener solution. | - this interceptor is only active if it's the coordinator of the local datacentre; use similar technique to what Singleton Store Cache Loader uses. | Why is the interceptor approach better than the cache listener one? The reason why I say this is from an integration perspective, a cache listener is far less tightly coupled to JBC internals than an interceptor approach. In terms of achieving goals, I don't see why this is hard: 1) A CL should be registered on every cache instance. 2) Querying viewChange events will tell the CL if it is the intra-ds coord (and hence whether or not to relay stuff to the inter-ds group 3) Registers a channel for the inter-ds group, listens for method invocations which it applies to the cache (if it is coord) or temporarily caches modifications in a Collection and removes them on commit (if it is the 2nd in line) "galder.zamarreno(a)jboss.com" wrote : | Relaying: | - in the context of this use case, relaying refers to inter dc communication. | - relaying could be done periodically, or per transaction/operation (put, remove, clear...etc). | - relaying would be asynchronous as data correctness is not paramount and it gives better performance. | Could use a replication queue. "galder.zamarreno(a)jboss.com" wrote : | - inter dc relay ping pong effect should be avoided so that inter dc changes that are applied locally do not bounce back to other dcs. requires further thought on how to avoid it. | The Cache Listener would only relay stuff to other DCs if it is marked as being in the "active" dc, given than the dc switchover would be manual. This would prevent the "dc ping pong" you describe. "galder.zamarreno(a)jboss.com" wrote : | State Transfer: | - if a new node starts and it's first in local data centre, inter data centre interceptor is active and joins the mux channel, potentially requesting a state transfer from the coordinator. The coordinator of the inter dc cluster does not necessarily have to be the primary relayer, but inter dc cluster members should have the same data. | - if a new node starts and it's not first in local data centre, standard local state transfer rules apply. | - streaming state transfer at inter data centre level would require further thought as potential intermediate firewalls would come into play. This needs careful thought, especially where blocking, WAN links and large states are concerned. An entire new ds coming up could block the inter-ds group for quite a while, and this could throttle the inter-ds proxy on the active ds. Even if this is async, if we're talking of several GBs of data over a WAN link, this could mean the inter ds group being blocked for hours, which could easily lead to the inter-ds proxy throwing OOMEs on queued async calls. I think we need something better WRT state transfer before we can think of applying this to a WAN scenario. Perhaps something where state is chunked and delivered in several bursts of, for example, under 50MB at a time, so that the group isn't blocked for too long. I don't have a solution here (yet), just thinking aloud - I know it's not an easy problem to solve. View the original post : http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4102571#... Reply to the post : http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&a...