reusing infinispan's marshalling
by Adrian Nistor
Hi list!
I've been pondering about re-using the marshalling machinery of
Infinispan in another project, specifically in ProtoStream, where I'm
planning to add it as a test scoped dependency so I can create a
benchmark to compare marshalling performace. I'm basically interested
in comparing ProtoStream and Infinispan's JBoss Marshalling based
mechanism. Comparing against plain JBMAR, without using the
ExternalizerTable and Externalizers introduced by Infinispan is not
going to get me accurate results.
But how? I see the marshaling is spread across infinispan-commons and
infinispan-core modules.
Thanks!
Adrian
11 years, 8 months
Re: [infinispan-dev] [jboss-as7-dev] Module jars dissapearing leaving empty classes/ folders and errors
by Galder Zamarreño
On 04 Feb 2014, at 10:38, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
> It is almost certainly something to do with this:
>
> <module-def name="org.infinispan.server.rest" src="${infinispan.server.modules.dir}">
>
> <maven-resource-with-classifier group="org.infinispan" artifact="infinispan-server-rest" classifier="classes" />
>
> </module-def>
>
> I guess sometimes the classes artefact is being attached as a reference to the classes directory, rather than a reference to a jar, which causes the issue.
Here’s a gist with a subset of the build log [1]. When it works fine, it’s copying a jar, when it’s not, it’s copying an empty folder.
However, this is not only happening for the org.infinispan.server.rest module, others show the same issue [2]. What seems to be a pattern is that it only happens with modules that are built by us, it’s not happening for modules coming with the base AS/WF instance.
I’ve traced back and this might be due to build failures that are not producing the right jars [3].
@Stuart, this is really our problem. Sorry for the inconvenience!
[1] https://gist.github.com/galderz/b9286f385aad1316df51
[2] https://gist.github.com/galderz/9e6a9bd9b18b805db323
[3] https://gist.github.com/galderz/6ab662a1027cd96cbd8c
>
> Stuart
>
>
> On Tue, Feb 4, 2014 at 11:14 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
>
> On 04 Feb 2014, at 10:01, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
>
> >
> >
> >
> > On Tue, Feb 4, 2014 at 11:00 AM, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
> > Yes, there is nothing in the server code that modified the modules directory.
> >
> > Well, except for the new patching stuff, but that is not really relevant here.
>
> The testsuite AS/WF builds are built out of the distribution build, which shows the same problem. The distribution we build uses the scripts we got from AS [1].
>
> Do you see anything in there that could be causing this? We are using maven-antrun-plugin version 1.3, and take into account the lib.xml in [2].
>
> Finally, do you have any suggestions on changes we could make to these files to further debug the issue?
>
> Thanks a lot for your help!
>
> [1] https://github.com/infinispan/infinispan/blob/master/server/integration/b...
> [2] https://github.com/infinispan/infinispan/blob/master/server/integration/b...
>
> >
> > Stuart
> >
> >
> > Stuart
> >
> >
> > On Tue, Feb 4, 2014 at 10:56 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
> >
> > On 04 Feb 2014, at 09:37, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
> >
> > > This looks like an issue with your environment. The modules directory is static. Wildfly does not contain any code that messes with it. I would say the culprit is probably something in either your build process or your test suite.
> >
> > Correction, this is happening with AS 7.2.0.Final (Wildfly 8 used somewhere else). I guess your answer still applies?
> >
> > Cheers,
> >
> > >
> > > Stuart
> > >
> > >
> > > On Tue, Feb 4, 2014 at 10:21 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
> > > Hi all,
> > >
> > > We’re having issues with our Infinispan Server integration tests, which run within Wildfly 8.0.0.Beta1 (as I’m typing I’m wondering if we should just upgrade it to see if this goes away...?).
> > >
> > > Quite often some of the runs fail with error message [1].
> > >
> > > Having looked at the build environment when a run fails, you see this:
> > >
> > > --
> > > $ ls modules/system/layers/base/org/infinispan/server/rest/main
> > > drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<— a directory??)
> > > -rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index
> > > -rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml
> > >
> > > $ ls modules/system/layers/base/org/infinispan/server/rest/main/classes
> > > drwxrwxr-x 2 g staff 68B Feb 3 18:41 .
> > > drwxrwxr-x 5 g staff 170B Feb 3 18:41 ..
> > >
> > > $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
> > > <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
> > > ...
> > > <resource-root path="classes"/>
> > > ...
> > >
> > > This is completely different to what happens with a successful run:
> > >
> > > --
> > > $ ls modules/system/layers/base/org/infinispan/server/rest/main
> > > -rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar (<— a jar file!)
> > > -rw-r--r-- 1 g staff 278B Feb 3 19:40 infinispan-classes.jar.index
> > > -rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml
> > >
> > > $ jar tf modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar | grep ExtendedHeaders
> > > org/infinispan/rest/configuration/ExtendedHeaders.class
> > >
> > > $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
> > > <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
> > > ...
> > > <resource-root path="infinispan-classes.jar”/>
> > > —
> > >
> > > Anyone can explain what is going on here? Does it ring a bell to anyone? Is this a known Wildfly issue by any chance?
> > >
> > > [1] https://gist.github.com/galderz/bd74cebfc840ef3ae284
> > >
> > > Cheers,
> > > --
> > > Galder Zamarreño
> > > galder(a)redhat.com
> > > twitter.com/galderz
> > >
> > > Project Lead, Escalante
> > > http://escalante.io
> > >
> > > Engineer, Infinispan
> > > http://infinispan.org
> > >
> > >
> > > _______________________________________________
> > > jboss-as7-dev mailing list
> > > jboss-as7-dev(a)lists.jboss.org
> > > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> > >
> >
> >
> > --
> > Galder Zamarreño
> > galder(a)redhat.com
> > twitter.com/galderz
> >
> > Project Lead, Escalante
> > http://escalante.io
> >
> > Engineer, Infinispan
> > http://infinispan.org
> >
> >
> >
>
>
> --
> Galder Zamarreño
> galder(a)redhat.com
> twitter.com/galderz
>
> Project Lead, Escalante
> http://escalante.io
>
> Engineer, Infinispan
> http://infinispan.org
>
>
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
11 years, 8 months
Re: [infinispan-dev] [jboss-as7-dev] Module jars dissapearing leaving empty classes/ folders and errors
by Galder Zamarreño
On 04 Feb 2014, at 10:01, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
>
>
>
> On Tue, Feb 4, 2014 at 11:00 AM, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
> Yes, there is nothing in the server code that modified the modules directory.
>
> Well, except for the new patching stuff, but that is not really relevant here.
The testsuite AS/WF builds are built out of the distribution build, which shows the same problem. The distribution we build uses the scripts we got from AS [1].
Do you see anything in there that could be causing this? We are using maven-antrun-plugin version 1.3, and take into account the lib.xml in [2].
Finally, do you have any suggestions on changes we could make to these files to further debug the issue?
Thanks a lot for your help!
[1] https://github.com/infinispan/infinispan/blob/master/server/integration/b...
[2] https://github.com/infinispan/infinispan/blob/master/server/integration/b...
>
> Stuart
>
>
> Stuart
>
>
> On Tue, Feb 4, 2014 at 10:56 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
>
> On 04 Feb 2014, at 09:37, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
>
> > This looks like an issue with your environment. The modules directory is static. Wildfly does not contain any code that messes with it. I would say the culprit is probably something in either your build process or your test suite.
>
> Correction, this is happening with AS 7.2.0.Final (Wildfly 8 used somewhere else). I guess your answer still applies?
>
> Cheers,
>
> >
> > Stuart
> >
> >
> > On Tue, Feb 4, 2014 at 10:21 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
> > Hi all,
> >
> > We’re having issues with our Infinispan Server integration tests, which run within Wildfly 8.0.0.Beta1 (as I’m typing I’m wondering if we should just upgrade it to see if this goes away...?).
> >
> > Quite often some of the runs fail with error message [1].
> >
> > Having looked at the build environment when a run fails, you see this:
> >
> > --
> > $ ls modules/system/layers/base/org/infinispan/server/rest/main
> > drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<— a directory??)
> > -rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index
> > -rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml
> >
> > $ ls modules/system/layers/base/org/infinispan/server/rest/main/classes
> > drwxrwxr-x 2 g staff 68B Feb 3 18:41 .
> > drwxrwxr-x 5 g staff 170B Feb 3 18:41 ..
> >
> > $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
> > <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
> > ...
> > <resource-root path="classes"/>
> > ...
> >
> > This is completely different to what happens with a successful run:
> >
> > --
> > $ ls modules/system/layers/base/org/infinispan/server/rest/main
> > -rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar (<— a jar file!)
> > -rw-r--r-- 1 g staff 278B Feb 3 19:40 infinispan-classes.jar.index
> > -rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml
> >
> > $ jar tf modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar | grep ExtendedHeaders
> > org/infinispan/rest/configuration/ExtendedHeaders.class
> >
> > $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
> > <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
> > ...
> > <resource-root path="infinispan-classes.jar”/>
> > —
> >
> > Anyone can explain what is going on here? Does it ring a bell to anyone? Is this a known Wildfly issue by any chance?
> >
> > [1] https://gist.github.com/galderz/bd74cebfc840ef3ae284
> >
> > Cheers,
> > --
> > Galder Zamarreño
> > galder(a)redhat.com
> > twitter.com/galderz
> >
> > Project Lead, Escalante
> > http://escalante.io
> >
> > Engineer, Infinispan
> > http://infinispan.org
> >
> >
> > _______________________________________________
> > jboss-as7-dev mailing list
> > jboss-as7-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
> >
>
>
> --
> Galder Zamarreño
> galder(a)redhat.com
> twitter.com/galderz
>
> Project Lead, Escalante
> http://escalante.io
>
> Engineer, Infinispan
> http://infinispan.org
>
>
>
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
11 years, 8 months
Re: [infinispan-dev] [jboss-as7-dev] Module jars dissapearing leaving empty classes/ folders and errors
by Galder Zamarreño
On 04 Feb 2014, at 09:37, Stuart Douglas <stuart.w.douglas(a)gmail.com> wrote:
> This looks like an issue with your environment. The modules directory is static. Wildfly does not contain any code that messes with it. I would say the culprit is probably something in either your build process or your test suite.
Correction, this is happening with AS 7.2.0.Final (Wildfly 8 used somewhere else). I guess your answer still applies?
Cheers,
>
> Stuart
>
>
> On Tue, Feb 4, 2014 at 10:21 AM, Galder Zamarreño <galder(a)redhat.com> wrote:
> Hi all,
>
> We’re having issues with our Infinispan Server integration tests, which run within Wildfly 8.0.0.Beta1 (as I’m typing I’m wondering if we should just upgrade it to see if this goes away...?).
>
> Quite often some of the runs fail with error message [1].
>
> Having looked at the build environment when a run fails, you see this:
>
> --
> $ ls modules/system/layers/base/org/infinispan/server/rest/main
> drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<— a directory??)
> -rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index
> -rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml
>
> $ ls modules/system/layers/base/org/infinispan/server/rest/main/classes
> drwxrwxr-x 2 g staff 68B Feb 3 18:41 .
> drwxrwxr-x 5 g staff 170B Feb 3 18:41 ..
>
> $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
> <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
> ...
> <resource-root path="classes"/>
> ...
>
> This is completely different to what happens with a successful run:
>
> --
> $ ls modules/system/layers/base/org/infinispan/server/rest/main
> -rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar (<— a jar file!)
> -rw-r--r-- 1 g staff 278B Feb 3 19:40 infinispan-classes.jar.index
> -rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml
>
> $ jar tf modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar | grep ExtendedHeaders
> org/infinispan/rest/configuration/ExtendedHeaders.class
>
> $ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
> <module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
> ...
> <resource-root path="infinispan-classes.jar”/>
> —
>
> Anyone can explain what is going on here? Does it ring a bell to anyone? Is this a known Wildfly issue by any chance?
>
> [1] https://gist.github.com/galderz/bd74cebfc840ef3ae284
>
> Cheers,
> --
> Galder Zamarreño
> galder(a)redhat.com
> twitter.com/galderz
>
> Project Lead, Escalante
> http://escalante.io
>
> Engineer, Infinispan
> http://infinispan.org
>
>
> _______________________________________________
> jboss-as7-dev mailing list
> jboss-as7-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
11 years, 8 months
Module jars dissapearing leaving empty classes/ folders and errors
by Galder Zamarreño
Hi all,
We’re having issues with our Infinispan Server integration tests, which run within Wildfly 8.0.0.Beta1 (as I’m typing I’m wondering if we should just upgrade it to see if this goes away...?).
Quite often some of the runs fail with error message [1].
Having looked at the build environment when a run fails, you see this:
--
$ ls modules/system/layers/base/org/infinispan/server/rest/main
drwxrwxr-x 2 g staff 68B Feb 3 18:41 classes (<— a directory??)
-rw-r--r-- 1 g staff 1B Feb 3 18:41 classes.index
-rw-r--r-- 1 g staff 2.1K Feb 3 18:41 module.xml
$ ls modules/system/layers/base/org/infinispan/server/rest/main/classes
drwxrwxr-x 2 g staff 68B Feb 3 18:41 .
drwxrwxr-x 5 g staff 170B Feb 3 18:41 ..
$ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
<module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
...
<resource-root path="classes"/>
...
This is completely different to what happens with a successful run:
--
$ ls modules/system/layers/base/org/infinispan/server/rest/main
-rw-r--r-- 1 g staff 103K Feb 3 19:40 infinispan-classes.jar (<— a jar file!)
-rw-r--r-- 1 g staff 278B Feb 3 19:40 infinispan-classes.jar.index
-rw-r--r-- 1 g staff 2.1K Feb 3 19:40 module.xml
$ jar tf modules/system/layers/base/org/infinispan/server/rest/main/infinispan-classes.jar | grep ExtendedHeaders
org/infinispan/rest/configuration/ExtendedHeaders.class
$ more modules/system/layers/base/org/infinispan/server/rest/main/module.xml
<module xmlns="urn:jboss:module:1.1" name="org.infinispan.server.rest”>
...
<resource-root path="infinispan-classes.jar”/>
—
Anyone can explain what is going on here? Does it ring a bell to anyone? Is this a known Wildfly issue by any chance?
[1] https://gist.github.com/galderz/bd74cebfc840ef3ae284
Cheers,
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
11 years, 8 months
Kyro performance (Was: reusing infinispan's marshalling)
by Sanne Grinovero
Changing the subject, as Adrian will need a reply to his (more
important) question.
I don't think we should go shopping for different marshaller
implementations, especially given other priorities.
I've been keeping an eye on Kryo since a while and it looks very good
indeed, but JBMarshaller is serving us pretty well and I'm loving its
reliability.
If we need more speed in this area, I'd rather see us perform some
very accurate benchmark development and try to understand why Kyro is
faster than JBM (if it really is), and potentially improve JBM.
For example as I've already suggested, it's using an internal
identityMap to detect graphs, and often we might not need that, or
also it would be nice to refactor it to write to an existing byte
stream rather than having it allocate internal buffers, and finally we
might want a "stateless edition" so to get rid of need for pooling of
JBMar instances.
-- Sanne
On 31 January 2014 16:29, Vladimir Blagojevic <vblagoje(a)redhat.com> wrote:
> Not 100% related to what you are asking about but have a look at this
> post and the discussion that "erupted":
>
> http://gridgain.blogspot.ca/2012/12/java-serialization-good-fast-and-fast...
>
> Vladimir
> On 1/30/2014, 7:13 AM, Adrian Nistor wrote:
>> Hi list!
>>
>> I've been pondering about re-using the marshalling machinery of
>> Infinispan in another project, specifically in ProtoStream, where I'm
>> planning to add it as a test scoped dependency so I can create a
>> benchmark to compare marshalling performace. I'm basically interested
>> in comparing ProtoStream and Infinispan's JBoss Marshalling based
>> mechanism. Comparing against plain JBMAR, without using the
>> ExternalizerTable and Externalizers introduced by Infinispan is not
>> going to get me accurate results.
>>
>> But how? I see the marshaling is spread across infinispan-commons and
>> infinispan-core modules.
>>
>> Thanks!
>> Adrian
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
11 years, 8 months
