[infinispan-dev] Getting rid of GZipInputStream

Philippe Van Dyck pvdyck at gmail.com
Mon Apr 19 07:20:31 EDT 2010


On Mon, Apr 19, 2010 at 12:36 PM, <galder at jboss.org> wrote:

> Hi Philippe,
>
> So, to sum up, trying to fix https://jira.jboss.org/jira/browse/ISPN-133won't solve your problem, correct? Did you test this out by any chance?
>

Yep, "Basically InputStream#available() is utterly broken.".
GzipIbputStream's implementation is :
 public int available() throws IOException {
        ensureOpen();
        if (reachEOF) {
            return 0;
        } else {
            return 1;
        }
    }

You definitely cannot use the return value to set your buffer size.


> Did you test Bzip2Compressor streams to see if that worked?
>
>
Yes, perfectly, one more dependency to add to commons compression... any
problem with that ?

The updated CloudCacheStore with bzip2 compression is ready...

Cheers,

Phil


> Cheers,
>
> ----- "philippe van dyck" <pvdyck at gmail.com> wrote:
>
> > Hi all,
> >
> > the problem is in the JDK and linked to a 10 years old -never fixed-
> > bug in GZipInputStream...
> > I propose to get rid of GZipInputStream and use Apache Commons
> > Bzip2Compressor streams in the CloudCacheStore.
> >
> > (And BTW, using len=inputStream.available() is a very bad idea with
> > GZipInputStream...)
> >
> > WDYT ?
> >
> > phil
> >
> > Début du message réexpédié :
> >
> > > De : Philippe Van Dyck <pvdyck at gmail.com>
> > > Date : 16 avril 2010 10:45:19 HAEC
> > > À : infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > Objet : Bug : read after end of stream @ AbstractMarshaller
> > >
> > > Hi all,
> > >
> > > since I use an InflaterInputStream to send objects to S3 using
> > JClouds Blobstore, I need a very strict management of streams.
> > > In AbstractMarshaller,  while ((bytesRead = inputStream.read(buf, 0,
> > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after the
> > stream's end, waiting for '-1' to happen.
> > > You cannot do that with a GZIPInputStream because you will get a
> > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > Should I file a bug or correct the code ?
> > >
> > > phil
> > Début du message réexpédié :
> >
> > > De : Philippe Van Dyck <pvdyck at gmail.com>
> > > Date : 16 avril 2010 10:52:22 HAEC
> > > À : infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > Objet : Réexp : Bug : read after end of stream @ AbstractMarshaller
> > >
> > > Ok, I took a closer look.
> > > Actually, it is related to HTTPCORE-199, and this bug is fixed.
> > > What about using int len = inputStream.available(); in
> > AbstractMarshaller again ?
> > >
> > > phil
> > >
> > > ---------- Forwarded message ----------
> > > From: Philippe Van Dyck <pvdyck at gmail.com>
> > > Date: Fri, Apr 16, 2010 at 10:45 AM
> > > Subject: Bug : read after end of stream @ AbstractMarshaller
> > > To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > >
> > >
> > > Hi all,
> > >
> > > since I use an InflaterInputStream to send objects to S3 using
> > JClouds Blobstore, I need a very strict management of streams.
> > > In AbstractMarshaller,  while ((bytesRead = inputStream.read(buf, 0,
> > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after the
> > stream's end, waiting for '-1' to happen.
> > > You cannot do that with a GZIPInputStream because you will get a
> > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > Should I file a bug or correct the code ?
> > >
> > > phil
> > >
> > Début du message réexpédié :
> >
> > > De : Philippe Van Dyck <pvdyck at gmail.com>
> > > Date : 16 avril 2010 11:50:54 HAEC
> > > À : infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > > Objet : Réexp : Bug : read after end of stream @ AbstractMarshaller
> > >
> > > Ok, JDK Bug
> > http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6519463
> > >
> > > The workaround is to simply ignore (!) the EOFException...
> > >
> > > WDYT ?
> > >
> > > phil
> > >
> > > ---------- Forwarded message ----------
> > > From: Philippe Van Dyck <pvdyck at gmail.com>
> > > Date: Fri, Apr 16, 2010 at 10:52 AM
> > > Subject: Fwd: Bug : read after end of stream @ AbstractMarshaller
> > > To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > >
> > >
> > > Ok, I took a closer look.
> > > Actually, it is related to HTTPCORE-199, and this bug is fixed.
> > > What about using int len = inputStream.available(); in
> > AbstractMarshaller again ?
> > >
> > > phil
> > >
> > >
> > > ---------- Forwarded message ----------
> > > From: Philippe Van Dyck <pvdyck at gmail.com>
> > > Date: Fri, Apr 16, 2010 at 10:45 AM
> > > Subject: Bug : read after end of stream @ AbstractMarshaller
> > > To: infinispan -Dev List <infinispan-dev at lists.jboss.org>
> > >
> > >
> > > Hi all,
> > >
> > > since I use an InflaterInputStream to send objects to S3 using
> > JClouds Blobstore, I need a very strict management of streams.
> > > In AbstractMarshaller,  while ((bytesRead = inputStream.read(buf, 0,
> > buf.length)) != -1) bytes.write(buf, 0, bytesRead) will read after the
> > stream's end, waiting for '-1' to happen.
> > > You cannot do that with a GZIPInputStream because you will get a
> > "java.io.EOFException: Unexpected end of ZLIB input stream".
> > > Should I file a bug or correct the code ?
> > >
> > > phil
> > >
> > >
> >
> >
> > _______________________________________________
> > infinispan-dev mailing list
> > infinispan-dev at lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/infinispan-dev/attachments/20100419/bba5d319/attachment-0001.html 


More information about the infinispan-dev mailing list