5:55 a.m.
Adding Infinispan dev list.
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
----- galder(a)redhat.com wrote:
As a FYI:
I'm currently testing this and it's interesting to see that the order
of 1) and 2) as below matters:
1) disp1=new MyRpcDispatcher((short)100, c1, null, null, new
Server("foo1()"));
disp2=new MyRpcDispatcher((short)0, c1, null, null, new
Server("foo2()"));
disp3=new MyRpcDispatcher((short)150, c1, null, null, new
Server("foo3()"));
2) MyUpHandler handler=new MyUpHandler();
c1.setUpHandler(handler);
The MyRpcDispatcher constructor relies on MessageDispatcher ctror that
does:
prot_adapter=new ProtocolAdapter();
...
channel.setUpHandler(prot_adapter);
And after all those ctros have executed, we override the handler in
2). If you execute 2) then 1), all messages end up in the last
uphandler set, which means that all calls go to foo3(). The reason I'm
looking into this is because from a consumer perspective, I'm trying
to understand what Hot Rod server would need to give to Infinispan or
whoever managers the channel.
Note that after 2) comes:
3) handler.put((short)100, disp1.getProtocolAdapter());
handler.put((short)0, disp2.getProtocolAdapter());
handler.put((short)150, disp3.getProtocolAdapter());
Bearing all this mind, I see Hot Rod server providing its own
Rpcdispatcher in 1) and then 3). 3) might not need to rely on Hot Rod
server entire if out MyRpcDispatcher, whoever executes 3) can figure
out the scope number. In other words 3) could be make to look like
this assuming that disp1, disp2 and disp3 are defined as
MyRpcDispatcher rather than RpcDispatcher
handler.put(disp1.scope, disp1.getProtocolAdapter());
handler.put(disp2.scope, disp2.getProtocolAdapter());
handler.put(disp3.scope, disp3.getProtocolAdapter());
My investigation continues...
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
----- "Bela Ban" <bban(a)redhat.com> wrote:
> OK, why don't you take a look ? JGRP-1177 has sample code attached
> showing how to use this.
>
> I'll respond in more detail tomorrow
>
> galder(a)redhat.com wrote:
> > Indeed, I think I need this as well for the Hot Rod use case. Hot
> Rod servers need to know which other Hot Rod servers are running in
> the cluster so that they can forward the corresponding information
> back to clients for load balancing, failover...etc.
> >
> > I don't want to polute the core cache channel code having to deal
> with Hot Rod rpcs.
> >
> > Also, a separate channel over a shared transport does not make
sense
> since the Hot Rod is tightly linked to the core cache channel that
> provides data replication capabilities amongst other things. IOW,
if
> Hot Rod used a separate channel for this, you could have scenarios
> where a Hot Rod servers thinks that HotRodServer1, HotRodServer2,
and
> HotRodServer3 are the hot rod servers available in the cluster, but
> underneath the core cache channel would only be formed of
CoreCache1
> and CoreCache2, with CoreCache3 isolated. This would be bad since a
> client could send requests to HotRodServer3 expecting it to
replicate
> to 1 or 2 but that would not happen. If the same view was used,
> clients would realise that HotRodServer3 is not part of the cluster
> any more.
> >
> > I'll check the current proposed solution for
> https://jira.jboss.org/jira/browse/JGRP-1177 tomorrow and comment
> further.
> >
> > Bela, when do u expect to have something that I can test?
> >
> > Cheers,
> >
> > p.s. I'm adding cluster-dev list to discussion since this is a
true
> inter cluster project concern.
> > --
> > Galder Zamarreño
> > Sr. Software Engineer
> > Infinispan, JBoss Cache
> >
> > ----- "Brian Stansberry" <brian.stansberry(a)redhat.com> wrote:
> >
> >
> >> Paul,
> >>
> >> Was just chatting with Galder and it seems Infinispan internally
> could
> >>
> >> also have a use for this. (Their HotRodServers would want to
make
> RPCs
> >>
> >> but he doesn't want to corrupt the core infinispan usage with
> that.)
> >>
> >> Can you pull together your thoughts on this, respond to Bela,
bring
> in
> >>
> >> Galder and get this pushed through? Galder's also the most
> >> knowledgable
> >> guy on the Hibernate/ISPN integration, which is the other big
use
> case
> >>
> >> for this.
> >>
> >> Thanks,
> >>
> >> Brian
> >>
> >> On 04/06/2010 03:48 PM, Brian Stansberry wrote:
> >>
> >>> Below is the transcript from our chat.
> >>>
> >>> Only other thoughts I had on this were:
> >>>
> >>> 1) If MyUpHandler exposed a setter for defaultHandler instead
of
> >>>
> >> using
> >>
> >>> scope 0 that would be good.
> >>>
> >>> 2) If JGroups shipped an UpHandler impl that sent a Message
with
> a
> >>> marker object like NoHandlerForRpc, whoever's responsible for
> >>>
> >> setting
> >>
> >>> MyUpHandler on the channel could also set that default handler.
> As
> >>>
> >> a
> >>
> >>> convenience an overloaded constructor on MyUpHandler could do
> this
> >>>
> >> for
> >>
> >>> you based on a boolean param.
> >>>
> >>> 3) If JGroups shipped a RspFilter impl that returned false to
> >>> isAcceptable(Object response, Address sender) if response
> >>>
> >> instanceof
> >>
> >>> NoHandlerForRpc, then code that was concerned about getting
> >>> NoHandlerForRpc could pass that to RpcDispatcher call remote
> >>>
> >> methods.
> >>
> >>> 4) The impl of 3) above should also accept an optional delegate
> >>>
> >> whose
> >>
> >>> methods would be invoked if the response isn't an instance of
> >>> NoHandlerForRpc. That could be used in case whatever was making
> the
> >>>
> >> RPC
> >>
> >>> also wanted to filter responses some other way.
> >>>
> >>> The above plus what we discussed below I think pretty nicely
> >>> encapsulates the guts of this inside JGroups code. The users of
> >>>
> >> this
> >>
> >>> stuff would then have to:
> >>>
> >>> A) instantiate the channel (if not already done)
> >>>
> >>> B) instantiate and set the MyUpHandler and set up the default
> >>>
> >> handler
> >>
> >>> (if not already done)
> >>>
> >>> C) Register their RPC dispatcher
> >>>
> >>> D) Pass in the channel to Infinispan, which can be clueless
about
> >>>
> >> all of
> >>
> >>> this if they so choose.
> >>>
> >>> E) When making any RPCs, pass in the RspFilter impl described
> >>>
> >> above.
> >>
> >>> If the Channel already existed (e.g. a second service sharing a
> >>> CacheManager), then the user service would need to figure out a
> way
> >>>
> >> to
> >>
> >>> get ahold of it (does Infinispan expose it)? and then do C and
E
> >>>
> >> above.
> >>
> >>> I think the latter scenario should discouraged though.
Different
> >>> services sharing the same channel is always problematic. Bela
> >>>
> >> convinced
> >>
> >>> me that by default the different AS services should use
different
> >>> CacheManager's/Channels, so all of the above would just be
needed
> if
> >>>
> >> we
> >>
> >>> want to support CacheManager sharing between
> >>>
> >> HTTP/SFSB/Hib/HAPartition.
> >>
> >>> (02:07:31 PM) besYIM: this class extends ReceiverAdapter, but
> AFAICT
> >>>
> >> it
> >>
> >>> doesn't need to; it doesn't do anything w/ that API
> >>> (02:08:21 PM) paul_m_ferraro: I hadn't even noticed that
> >>> (02:09:11 PM) besYIM: good; so that means I probably didn't
miss
> >>> anything :)
> >>> (02:10:15 PM) besYIM: the part I don't see is where MyUpHandler
> >>> put/remove get called
> >>> (02:10:30 PM) besYIM: ah, in start
> >>> (02:10:37 PM) paul_m_ferraro: yep
> >>> (02:12:28 PM) besYIM: this would need to be used from 1) AS 2)
> >>>
> >> Hibernate
> >>
> >>> 2LC provider 3) Infinispan
> >>> (02:13:14 PM) paul_m_ferraro: why not put in jgroups directly?
> >>> (02:13:56 PM) besYIM: yeah, i expect so
> >>> (02:13:58 PM) paul_m_ferraro: in org.jgroups.blocks?
> >>> (02:14:02 PM) besYIM: yep
> >>> (02:14:17 PM) besYIM: i was thinking about what clients would
> need
> >>>
> >> to be
> >>
> >>> adapted to use it
> >>> (02:15:33 PM) besYIM: i.e. what ISPN would have to change to
deal
> >>>
> >> with
> >>
> >>> the fact this *may* be there, but typically isn't
> >>> (02:17:15 PM) paul_m_ferraro: oh - I see you point
> >>> (02:17:19 PM) paul_m_ferraro: your
> >>> (02:17:22 PM) besYIM: i'm trying to figure where that
> >>> MyRpcDispatcher.createRequestCorrelator() method gets invoked
> >>> (02:17:40 PM) besYIM: since that method smells like a nice hook
> to
> >>>
> >> hide
> >>
> >>> all this :)
> >>> (02:18:21 PM) paul_m_ferraro: infinispan uses their own
> >>>
> >> RpcDispatcher
> >>
> >>> extension, right?
> >>> (02:18:33 PM) besYIM: yeah, i believe so
> >>> (02:19:07 PM) paul_m_ferraro: so the overridden UpHandler would
> >>>
> >> just
> >>
> >>> fall through to the default behavior if no scope header exists
> >>> (02:20:23 PM) paul_m_ferraro: the question is, how to override
> >>> infinispan so that it too can be multiplexed by scope
> >>> (02:20:31 PM) paul_m_ferraro: no?
> >>> (02:20:43 PM) besYIM: yep
> >>> (02:21:33 PM) besYIM: the createRequestCorrelatorMethod seems
to
> be
> >>>
> >> the way
> >>
> >>> (02:21:42 PM) besYIM: well, maybe
> >>> (02:22:20 PM) besYIM: if it had knowledge as whether the
> UpHandler
> >>>
> >> was
> >>
> >>> MyUpHandler it could change the type of RequestCorrelator it
> >>>
> >> creates
> >>
> >>> (02:22:37 PM) besYIM: std UpHandler, just create the std one
> >>> (02:22:47 PM) besYIM: special one, create MyRequestCorrelator
> >>> (02:25:49 PM) paul_m_ferraro: so where would Infinispan get
it's
> >>>
> >> scope
> >>
> >>> from?
> >>> (02:26:14 PM) besYIM: "well known scopes" wiki page?
> >>> (02:26:45 PM) besYIM: i suppose adding it to the ISPN config
> would
> >>>
> >> be
> >>
> >>> better
> >>> (02:28:28 PM) besYIM: ah, i found the call to
> >>>
> >> createRequestCorrelator --
> >>
> >>> in MessageDispatcher.start()
> >>> (02:31:16 PM) besYIM: none of the params passed to that method
> >>>
> >> provide
> >>
> >>> any info on the type of the UpHandler, but
> MessageDispatcher.channel
> >>>
> >> is
> >>
> >>> protected, so the method impl could just look at the Channel
> >>> (02:32:52 PM) paul_m_ferraro: I should think the
MyRpcDispatcher
> >>>
> >> should
> >>
> >>> also register itself w/the MyUpHandler, if its channel used it
> >>> (02:33:02 PM) besYIM: yep
> >>> (02:33:44 PM) besYIM: this could all be part of the std
> >>>
> >> RpcDispatcher
> >>
> >>> (02:34:43 PM) paul_m_ferraro: yep - and can use a default
scope,
> if
> >>>
> >> none
> >>
> >>> specified
> >>> (02:35:40 PM) besYIM: yeah, so infinispan could deal with
> >>>
> >> configuring
> >>
> >>> scope or not
> >>> (02:40:38 PM) besYIM: the other concern i had was lifecycle
> issues
> >>>
> >> --
> >>
> >>> i.e. transient issues with a channel not disconnected but a
> handler
> >>> removed e.g. during service undeployment
> >>> (02:41:34 PM) besYIM: or deployment
> >>> (02:42:52 PM) paul_m_ferraro: so, for example, a message
arrives
> >>>
> >> w/scope
> >>
> >>> X but scope X is no longer registered w/the UpHandler?
> >>> (02:43:16 PM) besYIM: or hasn't yet been registered
> >>> (02:43:52 PM) besYIM: like lets assume to make it hard that
HTTP
> >>> session, SFSB, Hib 2LC will all use the same CacheManager and
> thus
> >>>
> >> same
> >>
> >>> Channel
> >>> (02:44:44 PM) besYIM: nodes A and B are fully started; node C
is
> >>>
> >> coming
> >>
> >>> on line. It deploys a distributable webapp, so cache is
started,
> >>>
> >> Channel
> >>
> >>> connected, etc
> >>> (02:45:10 PM) besYIM: but there is no handler for the SFSB and
> Hib
> >>>
> >> 2LC
> >>
> >>> stuff yet because those aren't deployed yet
> >>> (02:46:11 PM) besYIM: HAPartition deals with this via
> >>>
> >> NoHandlerForRPC
> >>
> >>> (02:51:02 PM) paul_m_ferraro: Well, this is the responsibility
of
> >>> whomever created the scoped UpHandler
> >>> (02:51:57 PM) paul_m_ferraro: bela's impl throws a
> >>> IllegalArgumentException - but it should be something more
> specific
> >>> (02:52:01 PM) besYIM: yeah
> >>> (02:52:31 PM) besYIM: but if it throws an exception I believe
> that
> >>>
> >> will
> >>
> >>> propagate back to the MyRequestCorrelator who sent the RPC and
> then
> >>>
> >> get
> >>
> >>> thrown
> >>> (02:52:32 PM) paul_m_ferraro: and perhaps infinispan should
allow
> a
> >>> pluggable handler for this?
> >>> (02:53:02 PM) paul_m_ferraro: so those cache users that want to
> >>>
> >> ignore
> >>
> >>> it can, or retry it, etc.
> >>> (02:53:13 PM) besYIM: I don't think Infinispan will have a
> problem
> >>> (02:53:24 PM) paul_m_ferraro: retry the operation, if possible
> >>> (02:53:49 PM) besYIM: Infinispan is the one who actually calls
> >>> Channel.connect, so they are always registered first
> >>> (02:54:24 PM) besYIM: it's the specialized handlers we'd add
for
> >>>
> >> SFSB
> >>
> >>> ownership, or the Hibernate eviction thing that would be
missing
> >>> (02:59:17 PM) besYIM: i was looking at the org.jgroups.Request
> >>>
> >> class
> >>
> >>> (02:59:58 PM) besYIM: it's receiveResponse method is a
> centralized
> >>>
> >> place
> >>
> >>> that could deal with something like a NoHandlerForRPC
> >>> (03:00:10 PM) besYIM: i.e. discard it but mark the response
> >>>
> >> received
> >>
> >>> (03:02:41 PM) besYIM: a simple impl of
> org.jgroups.blocks.RspFilter
> >>> could also work -- add such a thing to JGroups and then AS,
EJB3,
> >>> Hibernate could use it for the few RPCs they make
> >>> (03:06:45 PM) paul_m_ferraro: I see
> >>> (03:06:57 PM) paul_m_ferraro: damn - its 4
> >>> (03:07:07 PM) paul_m_ferraro: today's my day to pick up marcus
> from
> >>> preschool
> >>> (03:07:19 PM) besYIM: that's always fun :). enjoy!
> >>>
> >> --
> >> Brian Stansberry
> >> Lead, AS Clustering
> >> JBoss by Red Hat
> >>
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss