[infinispan-dev] Getting rid of GZipInputStream

Philippe Van Dyck pvdyck at gmail.com
Mon Apr 19 10:39:58 EDT 2010 

On Mon, Apr 19, 2010 at 3:14 PM, <galder at jboss.org> wrote:

> Hi Phillipe,
>
> See below:
>
> ----- "Philippe Van Dyck" <pvdyck at gmail.com> wrote:
>
> > On Mon, Apr 19, 2010 at 12:36 PM, <galder at jboss.org> wrote:
> >
> > > Hi Philippe,
> > >
> > > So, to sum up, trying to fix
> > https://jira.jboss.org/jira/browse/ISPN-133won't solve your problem,
> > correct? Did you test this out by any chance?
> > >
> >
> > Yep, "Basically InputStream#available() is utterly broken.".
> > GzipIbputStream's implementation is :
> >  public int available() throws IOException {
> >         ensureOpen();
> >         if (reachEOF) {
> >             return 0;
> >         } else {
> >             return 1;
> >         }
> >     }
> >
> > You definitely cannot use the return value to set your buffer size.
>
> Interesting. It seems like it's not only Apache's HTTP client library that
> did not care much about available() implementations, but also JDK classes as
> well. I'd rather read on 1k batches than 1 byte batches. See things like
> this, I'm even less inclined to urgently look into
> https://jira.jboss.org/jira/browse/ISPN-133.
>


I even think you could close it !


>
> >
> >
> > > Did you test Bzip2Compressor streams to see if that worked?
> > >
> > >
> > Yes, perfectly, one more dependency to add to commons compression...
> > any
> > problem with that ?
>
> I suppose that's a dependency to the cloud cache store module, correct? If
> so, I'm fine with it :)
>
> >
> > The updated CloudCacheStore with bzip2 compression is ready...
>
> Cool, thanks very much for working on this!!
>

Am I suppose to file a bug ? To what Jira issue do you want me to link the
svn commit ?


Phil



> >
> > Cheers,
> >
> > Phil
> >
> >
> > > Cheers,
> > >
> > > ----- "philippe van dyck" <pvdyck at gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > the problem is in the JDK and linked to a 10 years old -never
> > fixed-
> > > > bug in GZipInputStream...
> > > > I propose to get rid of GZipInputStream and use Apache Commons
> > > > Bzip2Compressor streams in the CloudCacheStore.
> > > >
> > > > (And BTW, using len=inputStream.available() is a very bad idea
> > with
> > > > GZipInputStream...)
> > > >
> > > > WDYT ?
> > > >
> > > > phil
> > > >
> > > > Début du message réexpédié :
> > > >
> > > > > De : Philippe Van Dyck <pvdyck at gmail.com>
> > > > > Date : 16 avril 2010 10:45:19 HAEC
> > > > > À : infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > > > Objet : Bug : read after end of stream @ AbstractMarshaller
> > > > >
> > > > > Hi all,
> > > > >
> > > > > since I use an InflaterInputStream to send objects to S3 using
> > > > JClouds Blobstore, I need a very strict management of streams.
> > > > > In AbstractMarshaller,  while ((bytesRead =
> > inputStream.read(buf, 0,
> > > > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after
> > the
> > > > stream's end, waiting for '-1' to happen.
> > > > > You cannot do that with a GZIPInputStream because you will get
> > a
> > > > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > > > Should I file a bug or correct the code ?
> > > > >
> > > > > phil
> > > > Début du message réexpédié :
> > > >
> > > > > De : Philippe Van Dyck <pvdyck at gmail.com>
> > > > > Date : 16 avril 2010 10:52:22 HAEC
> > > > > À : infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > > > Objet : Réexp : Bug : read after end of stream @
> > AbstractMarshaller
> > > > >
> > > > > Ok, I took a closer look.
> > > > > Actually, it is related to HTTPCORE-199, and this bug is fixed.
> > > > > What about using int len = inputStream.available(); in
> > > > AbstractMarshaller again ?
> > > > >
> > > > > phil
> > > > >
> > > > > ---------- Forwarded message ----------
> > > > > From: Philippe Van Dyck <pvdyck at gmail.com>
> > > > > Date: Fri, Apr 16, 2010 at 10:45 AM
> > > > > Subject: Bug : read after end of stream @ AbstractMarshaller
> > > > > To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > > >
> > > > >
> > > > > Hi all,
> > > > >
> > > > > since I use an InflaterInputStream to send objects to S3 using
> > > > JClouds Blobstore, I need a very strict management of streams.
> > > > > In AbstractMarshaller,  while ((bytesRead =
> > inputStream.read(buf, 0,
> > > > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after
> > the
> > > > stream's end, waiting for '-1' to happen.
> > > > > You cannot do that with a GZIPInputStream because you will get
> > a
> > > > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > > > Should I file a bug or correct the code ?
> > > > >
> > > > > phil
> > > > >
> > > > Début du message réexpédié :
> > > >
> > > > > De : Philippe Van Dyck <pvdyck at gmail.com>
> > > > > Date : 16 avril 2010 11:50:54 HAEC
> > > > > À : infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > > > Objet : Réexp : Bug : read after end of stream @
> > AbstractMarshaller
> > > > >
> > > > > Ok, JDK Bug
> > > > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6519463
> > > > >
> > > > > The workaround is to simply ignore (!) the EOFException...
> > > > >
> > > > > WDYT ?
> > > > >
> > > > > phil
> > > > >
> > > > > ---------- Forwarded message ----------
> > > > > From: Philippe Van Dyck <pvdyck at gmail.com>
> > > > > Date: Fri, Apr 16, 2010 at 10:52 AM
> > > > > Subject: Fwd: Bug : read after end of stream @
> > AbstractMarshaller
> > > > > To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > > >
> > > > >
> > > > > Ok, I took a closer look.
> > > > > Actually, it is related to HTTPCORE-199, and this bug is fixed.
> > > > > What about using int len = inputStream.available(); in
> > > > AbstractMarshaller again ?
> > > > >
> > > > > phil
> > > > >
> > > > >
> > > > > ---------- Forwarded message ----------
> > > > > From: Philippe Van Dyck <pvdyck at gmail.com>
> > > > > Date: Fri, Apr 16, 2010 at 10:45 AM
> > > > > Subject: Bug : read after end of stream @ AbstractMarshaller
> > > > > To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > > >
> > > > >
> > > > > Hi all,
> > > > >
> > > > > since I use an InflaterInputStream to send objects to S3 using
> > > > JClouds Blobstore, I need a very strict management of streams.
> > > > > In AbstractMarshaller,  while ((bytesRead =
> > inputStream.read(buf, 0,
> > > > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after
> > the
> > > > stream's end, waiting for '-1' to happen.
> > > > > You cannot do that with a GZIPInputStream because you will get
> > a
> > > > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > > > Should I file a bug or correct the code ?
> > > > >
> > > > > phil
> > > > >
> > > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > infinispan-dev mailing list
> > > > infinispan-dev at lists.jboss.org
> > > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> > >
> > > _______________________________________________
> > > infinispan-dev mailing list
> > > infinispan-dev at lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/infinispan-dev
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100419/f852541e/attachment-0001.html

More information about the infinispan-dev mailing list