[infinispan-dev] MyRpcDispatcher
Galder Zamarreno
galder at jboss.org
Fri Apr 9 06:55:35 EDT 2010
Adding Infinispan dev list.
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
----- galder at redhat.com wrote:
> As a FYI:
>
> I'm currently testing this and it's interesting to see that the order
> of 1) and 2) as below matters:
>
> 1) disp1=new MyRpcDispatcher((short)100, c1, null, null, new
> Server("foo1()"));
> disp2=new MyRpcDispatcher((short)0, c1, null, null, new
> Server("foo2()"));
> disp3=new MyRpcDispatcher((short)150, c1, null, null, new
> Server("foo3()"));
>
> 2) MyUpHandler handler=new MyUpHandler();
> c1.setUpHandler(handler);
>
> The MyRpcDispatcher constructor relies on MessageDispatcher ctror that
> does:
>
> prot_adapter=new ProtocolAdapter();
> ...
> channel.setUpHandler(prot_adapter);
>
> And after all those ctros have executed, we override the handler in
> 2). If you execute 2) then 1), all messages end up in the last
> uphandler set, which means that all calls go to foo3(). The reason I'm
> looking into this is because from a consumer perspective, I'm trying
> to understand what Hot Rod server would need to give to Infinispan or
> whoever managers the channel.
>
> Note that after 2) comes:
>
> 3) handler.put((short)100, disp1.getProtocolAdapter());
> handler.put((short)0, disp2.getProtocolAdapter());
> handler.put((short)150, disp3.getProtocolAdapter());
>
> Bearing all this mind, I see Hot Rod server providing its own
> Rpcdispatcher in 1) and then 3). 3) might not need to rely on Hot Rod
> server entire if out MyRpcDispatcher, whoever executes 3) can figure
> out the scope number. In other words 3) could be make to look like
> this assuming that disp1, disp2 and disp3 are defined as
> MyRpcDispatcher rather than RpcDispatcher
>
> handler.put(disp1.scope, disp1.getProtocolAdapter());
> handler.put(disp2.scope, disp2.getProtocolAdapter());
> handler.put(disp3.scope, disp3.getProtocolAdapter());
>
> My investigation continues...
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
>
> ----- "Bela Ban" <bban at redhat.com> wrote:
>
> > OK, why don't you take a look ? JGRP-1177 has sample code attached
> > showing how to use this.
> >
> > I'll respond in more detail tomorrow
> >
> > galder at redhat.com wrote:
> > > Indeed, I think I need this as well for the Hot Rod use case. Hot
> > Rod servers need to know which other Hot Rod servers are running in
> > the cluster so that they can forward the corresponding information
> > back to clients for load balancing, failover...etc.
> > >
> > > I don't want to polute the core cache channel code having to deal
> > with Hot Rod rpcs.
> > >
> > > Also, a separate channel over a shared transport does not make
> sense
> > since the Hot Rod is tightly linked to the core cache channel that
> > provides data replication capabilities amongst other things. IOW,
> if
> > Hot Rod used a separate channel for this, you could have scenarios
> > where a Hot Rod servers thinks that HotRodServer1, HotRodServer2,
> and
> > HotRodServer3 are the hot rod servers available in the cluster, but
> > underneath the core cache channel would only be formed of
> CoreCache1
> > and CoreCache2, with CoreCache3 isolated. This would be bad since a
> > client could send requests to HotRodServer3 expecting it to
> replicate
> > to 1 or 2 but that would not happen. If the same view was used,
> > clients would realise that HotRodServer3 is not part of the cluster
> > any more.
> > >
> > > I'll check the current proposed solution for
> > https://jira.jboss.org/jira/browse/JGRP-1177 tomorrow and comment
> > further.
> > >
> > > Bela, when do u expect to have something that I can test?
> > >
> > > Cheers,
> > >
> > > p.s. I'm adding cluster-dev list to discussion since this is a
> true
> > inter cluster project concern.
> > > --
> > > Galder Zamarreño
> > > Sr. Software Engineer
> > > Infinispan, JBoss Cache
> > >
> > > ----- "Brian Stansberry" <brian.stansberry at redhat.com> wrote:
> > >
> > >
> > >> Paul,
> > >>
> > >> Was just chatting with Galder and it seems Infinispan internally
> > could
> > >>
> > >> also have a use for this. (Their HotRodServers would want to
> make
> > RPCs
> > >>
> > >> but he doesn't want to corrupt the core infinispan usage with
> > that.)
> > >>
> > >> Can you pull together your thoughts on this, respond to Bela,
> bring
> > in
> > >>
> > >> Galder and get this pushed through? Galder's also the most
> > >> knowledgable
> > >> guy on the Hibernate/ISPN integration, which is the other big
> use
> > case
> > >>
> > >> for this.
> > >>
> > >> Thanks,
> > >>
> > >> Brian
> > >>
> > >> On 04/06/2010 03:48 PM, Brian Stansberry wrote:
> > >>
> > >>> Below is the transcript from our chat.
> > >>>
> > >>> Only other thoughts I had on this were:
> > >>>
> > >>> 1) If MyUpHandler exposed a setter for defaultHandler instead
> of
> > >>>
> > >> using
> > >>
> > >>> scope 0 that would be good.
> > >>>
> > >>> 2) If JGroups shipped an UpHandler impl that sent a Message
> with
> > a
> > >>> marker object like NoHandlerForRpc, whoever's responsible for
> > >>>
> > >> setting
> > >>
> > >>> MyUpHandler on the channel could also set that default handler.
> > As
> > >>>
> > >> a
> > >>
> > >>> convenience an overloaded constructor on MyUpHandler could do
> > this
> > >>>
> > >> for
> > >>
> > >>> you based on a boolean param.
> > >>>
> > >>> 3) If JGroups shipped a RspFilter impl that returned false to
> > >>> isAcceptable(Object response, Address sender) if response
> > >>>
> > >> instanceof
> > >>
> > >>> NoHandlerForRpc, then code that was concerned about getting
> > >>> NoHandlerForRpc could pass that to RpcDispatcher call remote
> > >>>
> > >> methods.
> > >>
> > >>> 4) The impl of 3) above should also accept an optional delegate
> > >>>
> > >> whose
> > >>
> > >>> methods would be invoked if the response isn't an instance of
> > >>> NoHandlerForRpc. That could be used in case whatever was making
> > the
> > >>>
> > >> RPC
> > >>
> > >>> also wanted to filter responses some other way.
> > >>>
> > >>> The above plus what we discussed below I think pretty nicely
> > >>> encapsulates the guts of this inside JGroups code. The users of
> > >>>
> > >> this
> > >>
> > >>> stuff would then have to:
> > >>>
> > >>> A) instantiate the channel (if not already done)
> > >>>
> > >>> B) instantiate and set the MyUpHandler and set up the default
> > >>>
> > >> handler
> > >>
> > >>> (if not already done)
> > >>>
> > >>> C) Register their RPC dispatcher
> > >>>
> > >>> D) Pass in the channel to Infinispan, which can be clueless
> about
> > >>>
> > >> all of
> > >>
> > >>> this if they so choose.
> > >>>
> > >>> E) When making any RPCs, pass in the RspFilter impl described
> > >>>
> > >> above.
> > >>
> > >>> If the Channel already existed (e.g. a second service sharing a
> > >>> CacheManager), then the user service would need to figure out a
> > way
> > >>>
> > >> to
> > >>
> > >>> get ahold of it (does Infinispan expose it)? and then do C and
> E
> > >>>
> > >> above.
> > >>
> > >>> I think the latter scenario should discouraged though.
> Different
> > >>> services sharing the same channel is always problematic. Bela
> > >>>
> > >> convinced
> > >>
> > >>> me that by default the different AS services should use
> different
> > >>> CacheManager's/Channels, so all of the above would just be
> needed
> > if
> > >>>
> > >> we
> > >>
> > >>> want to support CacheManager sharing between
> > >>>
> > >> HTTP/SFSB/Hib/HAPartition.
> > >>
> > >>> (02:07:31 PM) besYIM: this class extends ReceiverAdapter, but
> > AFAICT
> > >>>
> > >> it
> > >>
> > >>> doesn't need to; it doesn't do anything w/ that API
> > >>> (02:08:21 PM) paul_m_ferraro: I hadn't even noticed that
> > >>> (02:09:11 PM) besYIM: good; so that means I probably didn't
> miss
> > >>> anything :)
> > >>> (02:10:15 PM) besYIM: the part I don't see is where MyUpHandler
> > >>> put/remove get called
> > >>> (02:10:30 PM) besYIM: ah, in start
> > >>> (02:10:37 PM) paul_m_ferraro: yep
> > >>> (02:12:28 PM) besYIM: this would need to be used from 1) AS 2)
> > >>>
> > >> Hibernate
> > >>
> > >>> 2LC provider 3) Infinispan
> > >>> (02:13:14 PM) paul_m_ferraro: why not put in jgroups directly?
> > >>> (02:13:56 PM) besYIM: yeah, i expect so
> > >>> (02:13:58 PM) paul_m_ferraro: in org.jgroups.blocks?
> > >>> (02:14:02 PM) besYIM: yep
> > >>> (02:14:17 PM) besYIM: i was thinking about what clients would
> > need
> > >>>
> > >> to be
> > >>
> > >>> adapted to use it
> > >>> (02:15:33 PM) besYIM: i.e. what ISPN would have to change to
> deal
> > >>>
> > >> with
> > >>
> > >>> the fact this *may* be there, but typically isn't
> > >>> (02:17:15 PM) paul_m_ferraro: oh - I see you point
> > >>> (02:17:19 PM) paul_m_ferraro: your
> > >>> (02:17:22 PM) besYIM: i'm trying to figure where that
> > >>> MyRpcDispatcher.createRequestCorrelator() method gets invoked
> > >>> (02:17:40 PM) besYIM: since that method smells like a nice hook
> > to
> > >>>
> > >> hide
> > >>
> > >>> all this :)
> > >>> (02:18:21 PM) paul_m_ferraro: infinispan uses their own
> > >>>
> > >> RpcDispatcher
> > >>
> > >>> extension, right?
> > >>> (02:18:33 PM) besYIM: yeah, i believe so
> > >>> (02:19:07 PM) paul_m_ferraro: so the overridden UpHandler would
> > >>>
> > >> just
> > >>
> > >>> fall through to the default behavior if no scope header exists
> > >>> (02:20:23 PM) paul_m_ferraro: the question is, how to override
> > >>> infinispan so that it too can be multiplexed by scope
> > >>> (02:20:31 PM) paul_m_ferraro: no?
> > >>> (02:20:43 PM) besYIM: yep
> > >>> (02:21:33 PM) besYIM: the createRequestCorrelatorMethod seems
> to
> > be
> > >>>
> > >> the way
> > >>
> > >>> (02:21:42 PM) besYIM: well, maybe
> > >>> (02:22:20 PM) besYIM: if it had knowledge as whether the
> > UpHandler
> > >>>
> > >> was
> > >>
> > >>> MyUpHandler it could change the type of RequestCorrelator it
> > >>>
> > >> creates
> > >>
> > >>> (02:22:37 PM) besYIM: std UpHandler, just create the std one
> > >>> (02:22:47 PM) besYIM: special one, create MyRequestCorrelator
> > >>> (02:25:49 PM) paul_m_ferraro: so where would Infinispan get
> it's
> > >>>
> > >> scope
> > >>
> > >>> from?
> > >>> (02:26:14 PM) besYIM: "well known scopes" wiki page?
> > >>> (02:26:45 PM) besYIM: i suppose adding it to the ISPN config
> > would
> > >>>
> > >> be
> > >>
> > >>> better
> > >>> (02:28:28 PM) besYIM: ah, i found the call to
> > >>>
> > >> createRequestCorrelator --
> > >>
> > >>> in MessageDispatcher.start()
> > >>> (02:31:16 PM) besYIM: none of the params passed to that method
> > >>>
> > >> provide
> > >>
> > >>> any info on the type of the UpHandler, but
> > MessageDispatcher.channel
> > >>>
> > >> is
> > >>
> > >>> protected, so the method impl could just look at the Channel
> > >>> (02:32:52 PM) paul_m_ferraro: I should think the
> MyRpcDispatcher
> > >>>
> > >> should
> > >>
> > >>> also register itself w/the MyUpHandler, if its channel used it
> > >>> (02:33:02 PM) besYIM: yep
> > >>> (02:33:44 PM) besYIM: this could all be part of the std
> > >>>
> > >> RpcDispatcher
> > >>
> > >>> (02:34:43 PM) paul_m_ferraro: yep - and can use a default
> scope,
> > if
> > >>>
> > >> none
> > >>
> > >>> specified
> > >>> (02:35:40 PM) besYIM: yeah, so infinispan could deal with
> > >>>
> > >> configuring
> > >>
> > >>> scope or not
> > >>> (02:40:38 PM) besYIM: the other concern i had was lifecycle
> > issues
> > >>>
> > >> --
> > >>
> > >>> i.e. transient issues with a channel not disconnected but a
> > handler
> > >>> removed e.g. during service undeployment
> > >>> (02:41:34 PM) besYIM: or deployment
> > >>> (02:42:52 PM) paul_m_ferraro: so, for example, a message
> arrives
> > >>>
> > >> w/scope
> > >>
> > >>> X but scope X is no longer registered w/the UpHandler?
> > >>> (02:43:16 PM) besYIM: or hasn't yet been registered
> > >>> (02:43:52 PM) besYIM: like lets assume to make it hard that
> HTTP
> > >>> session, SFSB, Hib 2LC will all use the same CacheManager and
> > thus
> > >>>
> > >> same
> > >>
> > >>> Channel
> > >>> (02:44:44 PM) besYIM: nodes A and B are fully started; node C
> is
> > >>>
> > >> coming
> > >>
> > >>> on line. It deploys a distributable webapp, so cache is
> started,
> > >>>
> > >> Channel
> > >>
> > >>> connected, etc
> > >>> (02:45:10 PM) besYIM: but there is no handler for the SFSB and
> > Hib
> > >>>
> > >> 2LC
> > >>
> > >>> stuff yet because those aren't deployed yet
> > >>> (02:46:11 PM) besYIM: HAPartition deals with this via
> > >>>
> > >> NoHandlerForRPC
> > >>
> > >>> (02:51:02 PM) paul_m_ferraro: Well, this is the responsibility
> of
> > >>> whomever created the scoped UpHandler
> > >>> (02:51:57 PM) paul_m_ferraro: bela's impl throws a
> > >>> IllegalArgumentException - but it should be something more
> > specific
> > >>> (02:52:01 PM) besYIM: yeah
> > >>> (02:52:31 PM) besYIM: but if it throws an exception I believe
> > that
> > >>>
> > >> will
> > >>
> > >>> propagate back to the MyRequestCorrelator who sent the RPC and
> > then
> > >>>
> > >> get
> > >>
> > >>> thrown
> > >>> (02:52:32 PM) paul_m_ferraro: and perhaps infinispan should
> allow
> > a
> > >>> pluggable handler for this?
> > >>> (02:53:02 PM) paul_m_ferraro: so those cache users that want to
> > >>>
> > >> ignore
> > >>
> > >>> it can, or retry it, etc.
> > >>> (02:53:13 PM) besYIM: I don't think Infinispan will have a
> > problem
> > >>> (02:53:24 PM) paul_m_ferraro: retry the operation, if possible
> > >>> (02:53:49 PM) besYIM: Infinispan is the one who actually calls
> > >>> Channel.connect, so they are always registered first
> > >>> (02:54:24 PM) besYIM: it's the specialized handlers we'd add
> for
> > >>>
> > >> SFSB
> > >>
> > >>> ownership, or the Hibernate eviction thing that would be
> missing
> > >>> (02:59:17 PM) besYIM: i was looking at the org.jgroups.Request
> > >>>
> > >> class
> > >>
> > >>> (02:59:58 PM) besYIM: it's receiveResponse method is a
> > centralized
> > >>>
> > >> place
> > >>
> > >>> that could deal with something like a NoHandlerForRPC
> > >>> (03:00:10 PM) besYIM: i.e. discard it but mark the response
> > >>>
> > >> received
> > >>
> > >>> (03:02:41 PM) besYIM: a simple impl of
> > org.jgroups.blocks.RspFilter
> > >>> could also work -- add such a thing to JGroups and then AS,
> EJB3,
> > >>> Hibernate could use it for the few RPCs they make
> > >>> (03:06:45 PM) paul_m_ferraro: I see
> > >>> (03:06:57 PM) paul_m_ferraro: damn - its 4
> > >>> (03:07:07 PM) paul_m_ferraro: today's my day to pick up marcus
> > from
> > >>> preschool
> > >>> (03:07:19 PM) besYIM: that's always fun :). enjoy!
> > >>>
> > >> --
> > >> Brian Stansberry
> > >> Lead, AS Clustering
> > >> JBoss by Red Hat
> > >>
> >
> > --
> > Bela Ban
> > Lead JGroups / Clustering Team
> > JBoss
More information about the infinispan-dev
mailing list