Re: [infinispan-dev] [jgroups-dev] Race condition with RspFilter
by galder＠jboss.org
What values does RspFilerResult contain exactly?
----- "Bela Ban" <belaban(a)yahoo.com> wrote:
> Yes, that's something workable.
>
> Is everybody (especially the Infinispan team) fine with this new
> interface ? I'd like to avoid having to introduce RspFilter3 soon ...
> :-)
>
>
>
>
> Galder Zamarreno wrote:
> > Hmmm, maybe you could have a RspFilter2 interface with this and
> deprecate RspFilter?
> >
> > ----- "Bela Ban" <belaban(a)yahoo.com> wrote:
> >
> >
> >> Problem is this is an API change...
> >>
> >> Vladimir Blagojevic wrote:
> >>
> >>> Brian and I had a brief chat on IIRC regarding this issue. How
> about
> >>>
> >> having only one method in RspFilter:
> >>
> >>> RspFilerResult responseReceived(Object response, Address sender);
> >>>
> >>> RspFilterResult is an enum with four states where each state is
> >>>
> >> essentially a combination of two boolean variables: validResponse
> and
> >> needMoreResponses.
> >>
> >>> Cheers
> >>> On 2010-04-15, at 6:29 PM, Brian Stansberry wrote:
> >>>
> >>>
> >>>
> >>>> I think the RspFilter handling has a small race. (Writing this
> has
> >>>>
> >> made
> >>
> >>>> me think it's very small, but since it's 1/2 written...)
> >>>>
> >>>> Consider this simple RspFilter implementation that signals that
> no
> >>>>
> >>>> further responses are needed once a non-null response is
> received:
> >>>>
> >>>> public class NonNullFilter implements RspFilter
> >>>> {
> >>>> private volatile boolean validResponse;
> >>>>
> >>>> public boolean isAcceptable(Object response, Address sender)
> >>>> {
> >>>> if (response != null)
> >>>> {
> >>>> validResponse = true;
> >>>> }
> >>>>
> >>>> return true;
> >>>> }
> >>>>
> >>>> public boolean needMoreResponses()
> >>>> {
> >>>> return !(validResponse);
> >>>> }
> >>>> }
> >>>>
> >>>> I think that captures the basic use case of RspFilter.
> >>>>
> >>>> The handling of response is like this in
> >>>>
> >> GroupRequest.responseReceived():
> >>
> >>>> boolean responseReceived=false;
> >>>> if(!rsp.wasReceived()) {
> >>>> if((responseReceived=(rsp_filter == null) ||
> >>>> rsp_filter.isAcceptable(response_value, sender)))
> >>>> rsp.setValue(response_value);
> >>>> rsp.setReceived(responseReceived);
> >>>> }
> >>>>
> >>>> lock.lock();
> >>>> try {
> >>>> if(responseReceived)
> >>>> num_received++;
> >>>> done=rsp_filter == null? responsesComplete() :
> >>>> !rsp_filter.needMoreResponses();
> >>>> if(responseReceived || done)
> >>>> completed.signalAll(); // wakes up execute()
> >>>>
> >>>> Now imagine 2 responses arriving nearly concurrently, with
> thread
> >>>>
> >> T1
> >>
> >>>> carrying a response value of null, and T2 carrying a non-null
> >>>>
> >> response.
> >>
> >>>> 1) T1 calls isAcceptable(), validResponse == false
> >>>> 2) T1 calls rsp.setValue()
> >>>> 3) T2 calls isAcceptable(), validResponse == true
> >>>> 4) T1 calls needMoreResponses, gets "false" since validResponse
> ==
> >>>>
> >> true
> >>
> >>>> 5) T1 calls completed.signalAll(), wakes up caller
> >>>> 6) Caller thread wakes up
> >>>> 7) Caller thread processes response list, doesn't see the T2
> value
> >>>>
> >> as
> >>
> >>>> it's not set yet
> >>>> 8) T2 calls rsp.setValue().
> >>>>
> >>>> Now that's pretty improbable, i.e. that steps 4,5,6,7 all execute
>
> >>>> between 3 and 8. But it's a race.
> >>>>
> >>>> A semi-related thing is that NonNullFilter.isAcceptable() always
> >>>>
> >> returns
> >>
> >>>> true. Seems counter-intuitive, why not return "false" if
> response
> >>>>
> >> ==
> >>
> >>>> null? Reason is that num_received is only incremented if
> >>>>
> >> isAcceptable()
> >>
> >>>> returns true. Effect is that if isAcceptable() doesn't always
> >>>>
> >> return
> >>
> >>>> true, if only "null" responses are received
> >>>> NonNullFilter.needMoreResponses() will never return false, *and*
>
> >>>> GroupRequest.responsesComplete() will never return true! The
> >>>>
> >> caller
> >>
> >>>> thread will just wait until timeout
> >>>>
> >>>> I wonder if in 3.0 a simpler RspFilter API would just be a
> single
> >>>>
> >> method:
> >>
> >>>> boolean needMoreResponses((Object response, Address sender)
> >>>>
> >>>> That would make it easier to avoid the race. Is there a use case
> >>>>
> >> for not
> >>
> >>>> marking a Rsp as received? That's what the separate
> >>>> isResponseAcceptable() method allows.
> >>>>
> >>>> Apologies if this has been discussed before; I have a vague
> feeling
> >>>>
> >> it
> >>
> >>>> has come up.
> >>>>
> >>>> --
> >>>> Brian Stansberry
> >>>> Lead, AS Clustering
> >>>> JBoss by Red Hat
> >>>>
> >>>>
> >>>>
> >>
> ------------------------------------------------------------------------------
> >>
> >>>> Download Intel® Parallel Studio Eval
> >>>> Try the new software tools for yourself. Speed compiling, find
> >>>>
> >> bugs
> >>
> >>>> proactively, and fine-tune applications for parallel
> performance.
> >>>> See why Intel Parallel Studio got high marks during beta.
> >>>> http://p.sf.net/sfu/intel-sw-dev
> >>>> _______________________________________________
> >>>> Javagroups-development mailing list
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>
> ------------------------------------------------------------------------------
> >>
> >>> Download Intel® Parallel Studio Eval
> >>> Try the new software tools for yourself. Speed compiling, find
> bugs
> >>> proactively, and fine-tune applications for parallel performance.
> >>> See why Intel Parallel Studio got high marks during beta.
> >>> http://p.sf.net/sfu/intel-sw-dev
> >>> _______________________________________________
> >>> Javagroups-development mailing list
> >>>
> >>>
> >>>
> >>>
> >> --
> >> Bela Ban
> >> Lead JGroups / JBoss Clustering team
> >> JBoss - a division of Red Hat
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Download Intel® Parallel Studio Eval
> >> Try the new software tools for yourself. Speed compiling, find
> bugs
> >> proactively, and fine-tune applications for parallel performance.
> >> See why Intel Parallel Studio got high marks during beta.
> >> http://p.sf.net/sfu/intel-sw-dev
> >> _______________________________________________
> >> Javagroups-development mailing list
> >>
> >
> >
>
> --
> Bela Ban
> Lead JGroups / Clustering Team
> JBoss
16 years, 1 month
NPE ... heeeeelp
by Philippe Van Dyck
Anyone ?
Using Atomikos Session manager :
21590604 [Atomikos:245] ERROR
org.infinispan.transaction.xa.TransactionXaAdapter - Error while
processing PrepareCommand
java.lang.NullPointerException
at org.infinispan.interceptors.CacheStoreInterceptor.prepareCacheLoader(CacheStoreInterceptor.java:240)
at org.infinispan.interceptors.CacheStoreInterceptor.visitPrepareCommand(CacheStoreInterceptor.java:167)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:105)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:48)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:91)
at org.infinispan.interceptors.DistTxInterceptor.visitPrepareCommand(DistTxInterceptor.java:59)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:105)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:120)
at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:105)
at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:120)
at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:269)
at org.infinispan.transaction.xa.TransactionXaAdapter.prepare(TransactionXaAdapter.java:87)
at org.infinispan.transaction.xa.TransactionXaAdapter.commit(TransactionXaAdapter.java:97)
at com.atomikos.datasource.xa.XAResourceTransaction.commit(XAResourceTransaction.java:967)
at com.atomikos.icatch.imp.CommitMessage.send(CommitMessage.java:94)
at com.atomikos.icatch.imp.PropagationMessage.submit(PropagationMessage.java:86)
at com.atomikos.icatch.imp.Propagator$PropagatorThread.run(Propagator.java:62)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
16 years, 1 month
Re: [infinispan-dev] Getting rid of GZipInputStream
by galder＠jboss.org
Hi Phillipe,
See below:
----- "Philippe Van Dyck" <pvdyck(a)gmail.com> wrote:
> On Mon, Apr 19, 2010 at 12:36 PM, <galder(a)jboss.org> wrote:
>
> > Hi Philippe,
> >
> > So, to sum up, trying to fix
> https://jira.jboss.org/jira/browse/ISPN-133won't solve your problem,
> correct? Did you test this out by any chance?
> >
>
> Yep, "Basically InputStream#available() is utterly broken.".
> GzipIbputStream's implementation is :
> public int available() throws IOException {
> ensureOpen();
> if (reachEOF) {
> return 0;
> } else {
> return 1;
> }
> }
>
> You definitely cannot use the return value to set your buffer size.
Interesting. It seems like it's not only Apache's HTTP client library that did not care much about available() implementations, but also JDK classes as well. I'd rather read on 1k batches than 1 byte batches. See things like this, I'm even less inclined to urgently look into https://jira.jboss.org/jira/browse/ISPN-133.
>
>
> > Did you test Bzip2Compressor streams to see if that worked?
> >
> >
> Yes, perfectly, one more dependency to add to commons compression...
> any
> problem with that ?
I suppose that's a dependency to the cloud cache store module, correct? If so, I'm fine with it :)
>
> The updated CloudCacheStore with bzip2 compression is ready...
Cool, thanks very much for working on this!!
>
> Cheers,
>
> Phil
>
>
> > Cheers,
> >
> > ----- "philippe van dyck" <pvdyck(a)gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > the problem is in the JDK and linked to a 10 years old -never
> fixed-
> > > bug in GZipInputStream...
> > > I propose to get rid of GZipInputStream and use Apache Commons
> > > Bzip2Compressor streams in the CloudCacheStore.
> > >
> > > (And BTW, using len=inputStream.available() is a very bad idea
> with
> > > GZipInputStream...)
> > >
> > > WDYT ?
> > >
> > > phil
> > >
> > > Début du message réexpédié :
> > >
> > > > De : Philippe Van Dyck <pvdyck(a)gmail.com>
> > > > Date : 16 avril 2010 10:45:19 HAEC
> > > > À : infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> > > > Objet : Bug : read after end of stream @ AbstractMarshaller
> > > >
> > > > Hi all,
> > > >
> > > > since I use an InflaterInputStream to send objects to S3 using
> > > JClouds Blobstore, I need a very strict management of streams.
> > > > In AbstractMarshaller, while ((bytesRead =
> inputStream.read(buf, 0,
> > > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after
> the
> > > stream's end, waiting for '-1' to happen.
> > > > You cannot do that with a GZIPInputStream because you will get
> a
> > > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > > Should I file a bug or correct the code ?
> > > >
> > > > phil
> > > Début du message réexpédié :
> > >
> > > > De : Philippe Van Dyck <pvdyck(a)gmail.com>
> > > > Date : 16 avril 2010 10:52:22 HAEC
> > > > À : infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> > > > Objet : Réexp : Bug : read after end of stream @
> AbstractMarshaller
> > > >
> > > > Ok, I took a closer look.
> > > > Actually, it is related to HTTPCORE-199, and this bug is fixed.
> > > > What about using int len = inputStream.available(); in
> > > AbstractMarshaller again ?
> > > >
> > > > phil
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Philippe Van Dyck <pvdyck(a)gmail.com>
> > > > Date: Fri, Apr 16, 2010 at 10:45 AM
> > > > Subject: Bug : read after end of stream @ AbstractMarshaller
> > > > To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> > > >
> > > >
> > > > Hi all,
> > > >
> > > > since I use an InflaterInputStream to send objects to S3 using
> > > JClouds Blobstore, I need a very strict management of streams.
> > > > In AbstractMarshaller, while ((bytesRead =
> inputStream.read(buf, 0,
> > > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after
> the
> > > stream's end, waiting for '-1' to happen.
> > > > You cannot do that with a GZIPInputStream because you will get
> a
> > > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > > Should I file a bug or correct the code ?
> > > >
> > > > phil
> > > >
> > > Début du message réexpédié :
> > >
> > > > De : Philippe Van Dyck <pvdyck(a)gmail.com>
> > > > Date : 16 avril 2010 11:50:54 HAEC
> > > > À : infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> > > > Objet : Réexp : Bug : read after end of stream @
> AbstractMarshaller
> > > >
> > > > Ok, JDK Bug
> > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6519463
> > > >
> > > > The workaround is to simply ignore (!) the EOFException...
> > > >
> > > > WDYT ?
> > > >
> > > > phil
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Philippe Van Dyck <pvdyck(a)gmail.com>
> > > > Date: Fri, Apr 16, 2010 at 10:52 AM
> > > > Subject: Fwd: Bug : read after end of stream @
> AbstractMarshaller
> > > > To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> > > >
> > > >
> > > > Ok, I took a closer look.
> > > > Actually, it is related to HTTPCORE-199, and this bug is fixed.
> > > > What about using int len = inputStream.available(); in
> > > AbstractMarshaller again ?
> > > >
> > > > phil
> > > >
> > > >
> > > > ---------- Forwarded message ----------
> > > > From: Philippe Van Dyck <pvdyck(a)gmail.com>
> > > > Date: Fri, Apr 16, 2010 at 10:45 AM
> > > > Subject: Bug : read after end of stream @ AbstractMarshaller
> > > > To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> > > >
> > > >
> > > > Hi all,
> > > >
> > > > since I use an InflaterInputStream to send objects to S3 using
> > > JClouds Blobstore, I need a very strict management of streams.
> > > > In AbstractMarshaller, while ((bytesRead =
> inputStream.read(buf, 0,
> > > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after
> the
> > > stream's end, waiting for '-1' to happen.
> > > > You cannot do that with a GZIPInputStream because you will get
> a
> > > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > > Should I file a bug or correct the code ?
> > > >
> > > > phil
> > > >
> > > >
> > >
> > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev(a)lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
16 years, 1 month
Getting rid of GZipInputStream
by philippe van dyck
Hi all,
the problem is in the JDK and linked to a 10 years old -never fixed- bug in GZipInputStream...
I propose to get rid of GZipInputStream and use Apache Commons Bzip2Compressor streams in the CloudCacheStore.
(And BTW, using len=inputStream.available() is a very bad idea with GZipInputStream...)
WDYT ?
phil
Début du message réexpédié :
> De : Philippe Van Dyck <pvdyck(a)gmail.com>
> Date : 16 avril 2010 10:45:19 HAEC
> À : infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> Objet : Bug : read after end of stream @ AbstractMarshaller
>
> Hi all,
>
> since I use an InflaterInputStream to send objects to S3 using JClouds Blobstore, I need a very strict management of streams.
> In AbstractMarshaller, while ((bytesRead = inputStream.read(buf, 0, buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after the stream's end, waiting for '-1' to happen.
> You cannot do that with a GZIPInputStream because you will get a "java.io.EOFException: Unexpected end of ZLIB input stream".
> Should I file a bug or correct the code ?
>
> phil
Début du message réexpédié :
> De : Philippe Van Dyck <pvdyck(a)gmail.com>
> Date : 16 avril 2010 10:52:22 HAEC
> À : infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> Objet : Réexp : Bug : read after end of stream @ AbstractMarshaller
>
> Ok, I took a closer look.
> Actually, it is related to HTTPCORE-199, and this bug is fixed.
> What about using int len = inputStream.available(); in AbstractMarshaller again ?
>
> phil
>
> ---------- Forwarded message ----------
> From: Philippe Van Dyck <pvdyck(a)gmail.com>
> Date: Fri, Apr 16, 2010 at 10:45 AM
> Subject: Bug : read after end of stream @ AbstractMarshaller
> To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
>
>
> Hi all,
>
> since I use an InflaterInputStream to send objects to S3 using JClouds Blobstore, I need a very strict management of streams.
> In AbstractMarshaller, while ((bytesRead = inputStream.read(buf, 0, buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after the stream's end, waiting for '-1' to happen.
> You cannot do that with a GZIPInputStream because you will get a "java.io.EOFException: Unexpected end of ZLIB input stream".
> Should I file a bug or correct the code ?
>
> phil
>
Début du message réexpédié :
> De : Philippe Van Dyck <pvdyck(a)gmail.com>
> Date : 16 avril 2010 11:50:54 HAEC
> À : infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
> Objet : Réexp : Bug : read after end of stream @ AbstractMarshaller
>
> Ok, JDK Bug http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6519463
>
> The workaround is to simply ignore (!) the EOFException...
>
> WDYT ?
>
> phil
>
> ---------- Forwarded message ----------
> From: Philippe Van Dyck <pvdyck(a)gmail.com>
> Date: Fri, Apr 16, 2010 at 10:52 AM
> Subject: Fwd: Bug : read after end of stream @ AbstractMarshaller
> To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
>
>
> Ok, I took a closer look.
> Actually, it is related to HTTPCORE-199, and this bug is fixed.
> What about using int len = inputStream.available(); in AbstractMarshaller again ?
>
> phil
>
>
> ---------- Forwarded message ----------
> From: Philippe Van Dyck <pvdyck(a)gmail.com>
> Date: Fri, Apr 16, 2010 at 10:45 AM
> Subject: Bug : read after end of stream @ AbstractMarshaller
> To: infinispan -Dev List <infinispan-dev(a)lists.jboss.org>
>
>
> Hi all,
>
> since I use an InflaterInputStream to send objects to S3 using JClouds Blobstore, I need a very strict management of streams.
> In AbstractMarshaller, while ((bytesRead = inputStream.read(buf, 0, buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after the stream's end, waiting for '-1' to happen.
> You cannot do that with a GZIPInputStream because you will get a "java.io.EOFException: Unexpected end of ZLIB input stream".
> Should I file a bug or correct the code ?
>
> phil
>
>
16 years, 1 month
Re: [infinispan-dev] MyRpcDispatcher
by Galder Zamarreno
No, there aren't. I ended up going down the route of having a cache for the Hot Rod view information. It was actually a lot simpler since nodes had to share Hot Rod view information.
----- "Bela Ban" <bela(a)jboss.com> wrote:
> Are there any unit tests you've written ?
>
> galder(a)jboss.org wrote:
> > Hi Paul,
> >
> > I'm not sure what method Bela is talking about. The only thing I
> suggested is having some kind of scope getter method that allows
> updating the handlers map.
> >
> > Paul, let me know when you've committed this stuff and I'll try it
> out. I'm also investigating Bela's suggestion of having a separate
> cache.
> >
> > Cheers,
> >
> > ----- "Paul Ferraro" <paul.ferraro(a)redhat.com> wrote:
> >
> >
> >> Galder,
> >>
> >> To which method is Bela referring?
> >>
> >> Otherwise, I'm ready to check in the org.jgroups.blocks.mux
> package.
> >> There is a small change required in
> >> MessageDispatcher.setChannel(...):
> >>
> >> channel.setUpHandler(prot_adapter);
> >>
> >> needs to be replaced by:
> >>
> >> if (channel.getUpHandler() == null)
> >> channel.setUpHandler(prot_adapter);
> >>
> >> This is need to prevent each new RpcDispatcher/MessageDispatcher
> from
> >> overwriting the preexisting multiplexing UpHandler.
> >> I can't envision this breaking any existing use case, but I wanted
> to
> >> verify before committing.
> >>
> >> So, to review, usage will now look like:
> >>
> >> Channel c = new JChannel(...);
> >>
> >> // RpcDispatcher d = new RpcDispatcher(c, null, null, target);
> >> // c.setUpHandler(new MuxUpHandler(d.getProtocolAdapter());
> >>
> >> c.setUpHandler(new MuxUpHandler());
> >>
> >> RpcDispatcher d1 = new MuxRpcDispatcher((short) 1, c, null, null,
> >> target);
> >> RpcDispatcher d1 = new MuxRpcDispatcher((short) 2, c, null, null,
> >> target);
> >>
> >> c.connect(...);
> >>
> >> Paul
> >>
> >> On Mon, 2010-04-12 at 13:16 +0200, Bela Ban wrote:
> >>
> >>> I understand all of the new code is in the new package
> >>> org.jgroups.blocks.scope. If that's the case, I suggest add and
> >>>
> >> commit
> >>
> >>> your changes so you can experiment with them.
> >>>
> >>> I also recall Galder needed to change some method from private to
>
> >>> protected, why don't you go ahead and do this too ?
> >>>
> >> <snip>
> >>
>
> --
> Bela Ban
> Lead JGroups / JBoss Clustering team
> JBoss - a division of Red Hat
16 years, 1 month
Re: [infinispan-dev] MyRpcDispatcher
by galder＠jboss.org
Hi Paul,
I'm not sure what method Bela is talking about. The only thing I suggested is having some kind of scope getter method that allows updating the handlers map.
Paul, let me know when you've committed this stuff and I'll try it out. I'm also investigating Bela's suggestion of having a separate cache.
Cheers,
----- "Paul Ferraro" <paul.ferraro(a)redhat.com> wrote:
> Galder,
>
> To which method is Bela referring?
>
> Otherwise, I'm ready to check in the org.jgroups.blocks.mux package.
> There is a small change required in
> MessageDispatcher.setChannel(...):
>
> channel.setUpHandler(prot_adapter);
>
> needs to be replaced by:
>
> if (channel.getUpHandler() == null)
> channel.setUpHandler(prot_adapter);
>
> This is need to prevent each new RpcDispatcher/MessageDispatcher from
> overwriting the preexisting multiplexing UpHandler.
> I can't envision this breaking any existing use case, but I wanted to
> verify before committing.
>
> So, to review, usage will now look like:
>
> Channel c = new JChannel(...);
>
> // RpcDispatcher d = new RpcDispatcher(c, null, null, target);
> // c.setUpHandler(new MuxUpHandler(d.getProtocolAdapter());
>
> c.setUpHandler(new MuxUpHandler());
>
> RpcDispatcher d1 = new MuxRpcDispatcher((short) 1, c, null, null,
> target);
> RpcDispatcher d1 = new MuxRpcDispatcher((short) 2, c, null, null,
> target);
>
> c.connect(...);
>
> Paul
>
> On Mon, 2010-04-12 at 13:16 +0200, Bela Ban wrote:
> > I understand all of the new code is in the new package
> > org.jgroups.blocks.scope. If that's the case, I suggest add and
> commit
> > your changes so you can experiment with them.
> >
> > I also recall Galder needed to change some method from private to
> > protected, why don't you go ahead and do this too ?
> <snip>
16 years, 1 month
Proposed fix for unstarted cache state transfer requests [ISPN-399]
by galder＠redhat.com
Hi,
Re: https://jira.jboss.org/jira/browse/ISPN-399
This has been halting my progress with https://jira.jboss.org/jira/browse/ISPN-384 but I think I've found a way to fix the issue. Before that though, let me explain the approaches that I tried that would not work:
- When a state transfer request is received for a cache that has not started, one option might have been to start the cache there and then. However, this won't work since clients might be starting caches with programatically defined configurations, hence InboundInvocationHandlerImpl can't have any knowledge of that.
- The most sensible solution IMO would be for the state provider to simply provide empty state and the receiver to get on with it. Initially I thought that I could just get InboundInvocationHandlerImpl to get hold of the marshaller, start an object output on the passed stream and write a false boolean indicating that the state provider cannot provide state. However, this has a different meaning and the receiver throws an Exception when it sees such thing.
- Finally, the only solution I could find was to add another boolean to the state provider/receiver protocol that indicates whether the state transfer manager for that cache is started. So, if the state provider found the cache was not started, it would do the same, write a false boolean. On the receiver side, we'd read the started flag first and then read the rest of the existing protocol, i.e. canProviderState...etc.
Although the solution is relatively simple, it has some caveats including change of protocol format for state transfer which if we want to maintain backwards compatibility with 4.0, requires adding versioning to the state transfer protocol itself, something we haven't been doing so far. IOW, we do have version at the marshaller level, but there's no versioning for the collection of calls that StateTransferManagerImpl does to figure out the state. In spite of all this, we might be able to workaround this if we're able to change Marshaller.startObjectInput and return not only ObjectInput, but also the versionId read, i.e. MyObjectInput that extends ObjectInput and contains ObjectInput and the version. StateTransferManagerImpl could do a hacky cast and figure out the version of the remote node and hence switch between the different protocol versions. To avoid such hacky things in the future, including 4.1, StateTransferManagerImpl could also write the version before writing anything else.
The other caveat is that InboundInvocationHandlerImpl ends up having logic wrt to responding to state generation, hence leading to protocol information escaping StateTransferManagerImpl. However, there's no StateTransferManagerImpl for the unstarted cache, so not much that I can do there, unless the protocol itself is abstracted to a separate class that is independent of the per cache StateTransferManagerImpl component.
I've attached a patch to the jira with v1 of the fix. Note that it does not contain any of my suggestions wrt supporting multiversioned state transfer protocol.
Thoughts?
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
16 years, 1 month
Components and their fields
by Vladimir Blagojevic
Hi,
Shouldn't we strive to minimize the number of fields in composite components that we create. I am working with StateTransferManagerImpl, DistributionManagerImpl and they both have so many fields it is overwhelming :)
Why don't we inject AdvancedCache instead and then get references through it?
Regards,
Vladimir
16 years, 1 month
Re: [infinispan-dev] MyRpcDispatcher
by Galder Zamarreno
Adding Infinispan dev list.
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
----- galder(a)redhat.com wrote:
> As a FYI:
>
> I'm currently testing this and it's interesting to see that the order
> of 1) and 2) as below matters:
>
> 1) disp1=new MyRpcDispatcher((short)100, c1, null, null, new
> Server("foo1()"));
> disp2=new MyRpcDispatcher((short)0, c1, null, null, new
> Server("foo2()"));
> disp3=new MyRpcDispatcher((short)150, c1, null, null, new
> Server("foo3()"));
>
> 2) MyUpHandler handler=new MyUpHandler();
> c1.setUpHandler(handler);
>
> The MyRpcDispatcher constructor relies on MessageDispatcher ctror that
> does:
>
> prot_adapter=new ProtocolAdapter();
> ...
> channel.setUpHandler(prot_adapter);
>
> And after all those ctros have executed, we override the handler in
> 2). If you execute 2) then 1), all messages end up in the last
> uphandler set, which means that all calls go to foo3(). The reason I'm
> looking into this is because from a consumer perspective, I'm trying
> to understand what Hot Rod server would need to give to Infinispan or
> whoever managers the channel.
>
> Note that after 2) comes:
>
> 3) handler.put((short)100, disp1.getProtocolAdapter());
> handler.put((short)0, disp2.getProtocolAdapter());
> handler.put((short)150, disp3.getProtocolAdapter());
>
> Bearing all this mind, I see Hot Rod server providing its own
> Rpcdispatcher in 1) and then 3). 3) might not need to rely on Hot Rod
> server entire if out MyRpcDispatcher, whoever executes 3) can figure
> out the scope number. In other words 3) could be make to look like
> this assuming that disp1, disp2 and disp3 are defined as
> MyRpcDispatcher rather than RpcDispatcher
>
> handler.put(disp1.scope, disp1.getProtocolAdapter());
> handler.put(disp2.scope, disp2.getProtocolAdapter());
> handler.put(disp3.scope, disp3.getProtocolAdapter());
>
> My investigation continues...
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
>
> ----- "Bela Ban" <bban(a)redhat.com> wrote:
>
> > OK, why don't you take a look ? JGRP-1177 has sample code attached
> > showing how to use this.
> >
> > I'll respond in more detail tomorrow
> >
> > galder(a)redhat.com wrote:
> > > Indeed, I think I need this as well for the Hot Rod use case. Hot
> > Rod servers need to know which other Hot Rod servers are running in
> > the cluster so that they can forward the corresponding information
> > back to clients for load balancing, failover...etc.
> > >
> > > I don't want to polute the core cache channel code having to deal
> > with Hot Rod rpcs.
> > >
> > > Also, a separate channel over a shared transport does not make
> sense
> > since the Hot Rod is tightly linked to the core cache channel that
> > provides data replication capabilities amongst other things. IOW,
> if
> > Hot Rod used a separate channel for this, you could have scenarios
> > where a Hot Rod servers thinks that HotRodServer1, HotRodServer2,
> and
> > HotRodServer3 are the hot rod servers available in the cluster, but
> > underneath the core cache channel would only be formed of
> CoreCache1
> > and CoreCache2, with CoreCache3 isolated. This would be bad since a
> > client could send requests to HotRodServer3 expecting it to
> replicate
> > to 1 or 2 but that would not happen. If the same view was used,
> > clients would realise that HotRodServer3 is not part of the cluster
> > any more.
> > >
> > > I'll check the current proposed solution for
> > https://jira.jboss.org/jira/browse/JGRP-1177 tomorrow and comment
> > further.
> > >
> > > Bela, when do u expect to have something that I can test?
> > >
> > > Cheers,
> > >
> > > p.s. I'm adding cluster-dev list to discussion since this is a
> true
> > inter cluster project concern.
> > > --
> > > Galder Zamarreño
> > > Sr. Software Engineer
> > > Infinispan, JBoss Cache
> > >
> > > ----- "Brian Stansberry" <brian.stansberry(a)redhat.com> wrote:
> > >
> > >
> > >> Paul,
> > >>
> > >> Was just chatting with Galder and it seems Infinispan internally
> > could
> > >>
> > >> also have a use for this. (Their HotRodServers would want to
> make
> > RPCs
> > >>
> > >> but he doesn't want to corrupt the core infinispan usage with
> > that.)
> > >>
> > >> Can you pull together your thoughts on this, respond to Bela,
> bring
> > in
> > >>
> > >> Galder and get this pushed through? Galder's also the most
> > >> knowledgable
> > >> guy on the Hibernate/ISPN integration, which is the other big
> use
> > case
> > >>
> > >> for this.
> > >>
> > >> Thanks,
> > >>
> > >> Brian
> > >>
> > >> On 04/06/2010 03:48 PM, Brian Stansberry wrote:
> > >>
> > >>> Below is the transcript from our chat.
> > >>>
> > >>> Only other thoughts I had on this were:
> > >>>
> > >>> 1) If MyUpHandler exposed a setter for defaultHandler instead
> of
> > >>>
> > >> using
> > >>
> > >>> scope 0 that would be good.
> > >>>
> > >>> 2) If JGroups shipped an UpHandler impl that sent a Message
> with
> > a
> > >>> marker object like NoHandlerForRpc, whoever's responsible for
> > >>>
> > >> setting
> > >>
> > >>> MyUpHandler on the channel could also set that default handler.
> > As
> > >>>
> > >> a
> > >>
> > >>> convenience an overloaded constructor on MyUpHandler could do
> > this
> > >>>
> > >> for
> > >>
> > >>> you based on a boolean param.
> > >>>
> > >>> 3) If JGroups shipped a RspFilter impl that returned false to
> > >>> isAcceptable(Object response, Address sender) if response
> > >>>
> > >> instanceof
> > >>
> > >>> NoHandlerForRpc, then code that was concerned about getting
> > >>> NoHandlerForRpc could pass that to RpcDispatcher call remote
> > >>>
> > >> methods.
> > >>
> > >>> 4) The impl of 3) above should also accept an optional delegate
> > >>>
> > >> whose
> > >>
> > >>> methods would be invoked if the response isn't an instance of
> > >>> NoHandlerForRpc. That could be used in case whatever was making
> > the
> > >>>
> > >> RPC
> > >>
> > >>> also wanted to filter responses some other way.
> > >>>
> > >>> The above plus what we discussed below I think pretty nicely
> > >>> encapsulates the guts of this inside JGroups code. The users of
> > >>>
> > >> this
> > >>
> > >>> stuff would then have to:
> > >>>
> > >>> A) instantiate the channel (if not already done)
> > >>>
> > >>> B) instantiate and set the MyUpHandler and set up the default
> > >>>
> > >> handler
> > >>
> > >>> (if not already done)
> > >>>
> > >>> C) Register their RPC dispatcher
> > >>>
> > >>> D) Pass in the channel to Infinispan, which can be clueless
> about
> > >>>
> > >> all of
> > >>
> > >>> this if they so choose.
> > >>>
> > >>> E) When making any RPCs, pass in the RspFilter impl described
> > >>>
> > >> above.
> > >>
> > >>> If the Channel already existed (e.g. a second service sharing a
> > >>> CacheManager), then the user service would need to figure out a
> > way
> > >>>
> > >> to
> > >>
> > >>> get ahold of it (does Infinispan expose it)? and then do C and
> E
> > >>>
> > >> above.
> > >>
> > >>> I think the latter scenario should discouraged though.
> Different
> > >>> services sharing the same channel is always problematic. Bela
> > >>>
> > >> convinced
> > >>
> > >>> me that by default the different AS services should use
> different
> > >>> CacheManager's/Channels, so all of the above would just be
> needed
> > if
> > >>>
> > >> we
> > >>
> > >>> want to support CacheManager sharing between
> > >>>
> > >> HTTP/SFSB/Hib/HAPartition.
> > >>
> > >>> (02:07:31 PM) besYIM: this class extends ReceiverAdapter, but
> > AFAICT
> > >>>
> > >> it
> > >>
> > >>> doesn't need to; it doesn't do anything w/ that API
> > >>> (02:08:21 PM) paul_m_ferraro: I hadn't even noticed that
> > >>> (02:09:11 PM) besYIM: good; so that means I probably didn't
> miss
> > >>> anything :)
> > >>> (02:10:15 PM) besYIM: the part I don't see is where MyUpHandler
> > >>> put/remove get called
> > >>> (02:10:30 PM) besYIM: ah, in start
> > >>> (02:10:37 PM) paul_m_ferraro: yep
> > >>> (02:12:28 PM) besYIM: this would need to be used from 1) AS 2)
> > >>>
> > >> Hibernate
> > >>
> > >>> 2LC provider 3) Infinispan
> > >>> (02:13:14 PM) paul_m_ferraro: why not put in jgroups directly?
> > >>> (02:13:56 PM) besYIM: yeah, i expect so
> > >>> (02:13:58 PM) paul_m_ferraro: in org.jgroups.blocks?
> > >>> (02:14:02 PM) besYIM: yep
> > >>> (02:14:17 PM) besYIM: i was thinking about what clients would
> > need
> > >>>
> > >> to be
> > >>
> > >>> adapted to use it
> > >>> (02:15:33 PM) besYIM: i.e. what ISPN would have to change to
> deal
> > >>>
> > >> with
> > >>
> > >>> the fact this *may* be there, but typically isn't
> > >>> (02:17:15 PM) paul_m_ferraro: oh - I see you point
> > >>> (02:17:19 PM) paul_m_ferraro: your
> > >>> (02:17:22 PM) besYIM: i'm trying to figure where that
> > >>> MyRpcDispatcher.createRequestCorrelator() method gets invoked
> > >>> (02:17:40 PM) besYIM: since that method smells like a nice hook
> > to
> > >>>
> > >> hide
> > >>
> > >>> all this :)
> > >>> (02:18:21 PM) paul_m_ferraro: infinispan uses their own
> > >>>
> > >> RpcDispatcher
> > >>
> > >>> extension, right?
> > >>> (02:18:33 PM) besYIM: yeah, i believe so
> > >>> (02:19:07 PM) paul_m_ferraro: so the overridden UpHandler would
> > >>>
> > >> just
> > >>
> > >>> fall through to the default behavior if no scope header exists
> > >>> (02:20:23 PM) paul_m_ferraro: the question is, how to override
> > >>> infinispan so that it too can be multiplexed by scope
> > >>> (02:20:31 PM) paul_m_ferraro: no?
> > >>> (02:20:43 PM) besYIM: yep
> > >>> (02:21:33 PM) besYIM: the createRequestCorrelatorMethod seems
> to
> > be
> > >>>
> > >> the way
> > >>
> > >>> (02:21:42 PM) besYIM: well, maybe
> > >>> (02:22:20 PM) besYIM: if it had knowledge as whether the
> > UpHandler
> > >>>
> > >> was
> > >>
> > >>> MyUpHandler it could change the type of RequestCorrelator it
> > >>>
> > >> creates
> > >>
> > >>> (02:22:37 PM) besYIM: std UpHandler, just create the std one
> > >>> (02:22:47 PM) besYIM: special one, create MyRequestCorrelator
> > >>> (02:25:49 PM) paul_m_ferraro: so where would Infinispan get
> it's
> > >>>
> > >> scope
> > >>
> > >>> from?
> > >>> (02:26:14 PM) besYIM: "well known scopes" wiki page?
> > >>> (02:26:45 PM) besYIM: i suppose adding it to the ISPN config
> > would
> > >>>
> > >> be
> > >>
> > >>> better
> > >>> (02:28:28 PM) besYIM: ah, i found the call to
> > >>>
> > >> createRequestCorrelator --
> > >>
> > >>> in MessageDispatcher.start()
> > >>> (02:31:16 PM) besYIM: none of the params passed to that method
> > >>>
> > >> provide
> > >>
> > >>> any info on the type of the UpHandler, but
> > MessageDispatcher.channel
> > >>>
> > >> is
> > >>
> > >>> protected, so the method impl could just look at the Channel
> > >>> (02:32:52 PM) paul_m_ferraro: I should think the
> MyRpcDispatcher
> > >>>
> > >> should
> > >>
> > >>> also register itself w/the MyUpHandler, if its channel used it
> > >>> (02:33:02 PM) besYIM: yep
> > >>> (02:33:44 PM) besYIM: this could all be part of the std
> > >>>
> > >> RpcDispatcher
> > >>
> > >>> (02:34:43 PM) paul_m_ferraro: yep - and can use a default
> scope,
> > if
> > >>>
> > >> none
> > >>
> > >>> specified
> > >>> (02:35:40 PM) besYIM: yeah, so infinispan could deal with
> > >>>
> > >> configuring
> > >>
> > >>> scope or not
> > >>> (02:40:38 PM) besYIM: the other concern i had was lifecycle
> > issues
> > >>>
> > >> --
> > >>
> > >>> i.e. transient issues with a channel not disconnected but a
> > handler
> > >>> removed e.g. during service undeployment
> > >>> (02:41:34 PM) besYIM: or deployment
> > >>> (02:42:52 PM) paul_m_ferraro: so, for example, a message
> arrives
> > >>>
> > >> w/scope
> > >>
> > >>> X but scope X is no longer registered w/the UpHandler?
> > >>> (02:43:16 PM) besYIM: or hasn't yet been registered
> > >>> (02:43:52 PM) besYIM: like lets assume to make it hard that
> HTTP
> > >>> session, SFSB, Hib 2LC will all use the same CacheManager and
> > thus
> > >>>
> > >> same
> > >>
> > >>> Channel
> > >>> (02:44:44 PM) besYIM: nodes A and B are fully started; node C
> is
> > >>>
> > >> coming
> > >>
> > >>> on line. It deploys a distributable webapp, so cache is
> started,
> > >>>
> > >> Channel
> > >>
> > >>> connected, etc
> > >>> (02:45:10 PM) besYIM: but there is no handler for the SFSB and
> > Hib
> > >>>
> > >> 2LC
> > >>
> > >>> stuff yet because those aren't deployed yet
> > >>> (02:46:11 PM) besYIM: HAPartition deals with this via
> > >>>
> > >> NoHandlerForRPC
> > >>
> > >>> (02:51:02 PM) paul_m_ferraro: Well, this is the responsibility
> of
> > >>> whomever created the scoped UpHandler
> > >>> (02:51:57 PM) paul_m_ferraro: bela's impl throws a
> > >>> IllegalArgumentException - but it should be something more
> > specific
> > >>> (02:52:01 PM) besYIM: yeah
> > >>> (02:52:31 PM) besYIM: but if it throws an exception I believe
> > that
> > >>>
> > >> will
> > >>
> > >>> propagate back to the MyRequestCorrelator who sent the RPC and
> > then
> > >>>
> > >> get
> > >>
> > >>> thrown
> > >>> (02:52:32 PM) paul_m_ferraro: and perhaps infinispan should
> allow
> > a
> > >>> pluggable handler for this?
> > >>> (02:53:02 PM) paul_m_ferraro: so those cache users that want to
> > >>>
> > >> ignore
> > >>
> > >>> it can, or retry it, etc.
> > >>> (02:53:13 PM) besYIM: I don't think Infinispan will have a
> > problem
> > >>> (02:53:24 PM) paul_m_ferraro: retry the operation, if possible
> > >>> (02:53:49 PM) besYIM: Infinispan is the one who actually calls
> > >>> Channel.connect, so they are always registered first
> > >>> (02:54:24 PM) besYIM: it's the specialized handlers we'd add
> for
> > >>>
> > >> SFSB
> > >>
> > >>> ownership, or the Hibernate eviction thing that would be
> missing
> > >>> (02:59:17 PM) besYIM: i was looking at the org.jgroups.Request
> > >>>
> > >> class
> > >>
> > >>> (02:59:58 PM) besYIM: it's receiveResponse method is a
> > centralized
> > >>>
> > >> place
> > >>
> > >>> that could deal with something like a NoHandlerForRPC
> > >>> (03:00:10 PM) besYIM: i.e. discard it but mark the response
> > >>>
> > >> received
> > >>
> > >>> (03:02:41 PM) besYIM: a simple impl of
> > org.jgroups.blocks.RspFilter
> > >>> could also work -- add such a thing to JGroups and then AS,
> EJB3,
> > >>> Hibernate could use it for the few RPCs they make
> > >>> (03:06:45 PM) paul_m_ferraro: I see
> > >>> (03:06:57 PM) paul_m_ferraro: damn - its 4
> > >>> (03:07:07 PM) paul_m_ferraro: today's my day to pick up marcus
> > from
> > >>> preschool
> > >>> (03:07:19 PM) besYIM: that's always fun :). enjoy!
> > >>>
> > >> --
> > >> Brian Stansberry
> > >> Lead, AS Clustering
> > >> JBoss by Red Hat
> > >>
> >
> > --
> > Bela Ban
> > Lead JGroups / Clustering Team
> > JBoss
16 years, 1 month
