API changes in Habanero
by Manik Surtani
Guys,
Here are a few major refactorings I've done on HEAD (checked in
already). Some of these are on the API level and do affect the way
interceptors and other subsystems interact with the node structure,
hence the email.
* Got rid of the horrible BypassInterceptorChain malarky when
interacting with nodes and you don't want calls up the interceptor chain
* Used by interceptors themselves, as well as other internal
subsystems including state transfer and the cache itself.
* BypassInterceptorChain will still exist as an Option, as there are
some valid use cases for this.
* This option is now SOLELY handled by the
InvocationContextInterceptor, which directs the call to the last
interceptor in the chain if the option is present
* Not handled internally in Node implementation methods anymore
* Replaced with a bunch of XXXDirect() methods on NodeSPI for direct
interaction/bypassing interceptor chains
* E.g., Node.getChild(Fqn f) goes up the interceptor chain,
NodeSPI.getChildDirect(Fqn f) operates directly on the node.
* Benefits: much easier to read, understand and maintain, more
performant.
* Got rid of Node.getNodeSPI()
* NodeSPI interface can now be easily obtained since all methods on
NodeSPI which would otherwise return Node now return NodeSPI
* Methods on Cache (such as getRoot()) which would return Node are
overridden in CacheSPI to return NodeSPI.
* Gives interceptors and the like access to NodeSPI without letting
this interface escape to the user API.
* There are still a very small handful of cases where direct casts
are necessary, but this is very internalised (within UnversionedNode
and WorkspaceNode, for example) and a bunch of unit tests (for closer
inspection)
* In general, NodeSPI is now much tighter and can only officially be
obtained from another NodeSPI or a CacheSPI.
* Got rid of NodeSPI.getRawData() - superfluous now that we have
NodeSPI.getDataDirect()
This stuff should give us a much more robust data model and user/SPI
interface for the nodes and caches.
Please let me know what you think or if you have any issues/further
suggestions.
Cheers,
--
Manik Surtani
Lead, JBoss Cache
JBoss, a division of Red Hat
Email: manik(a)jboss.org
Telephone: +44 7786 702 706
MSN: manik(a)surtani.org
Yahoo/AIM/Skype: maniksurtani
17 years, 4 months
RE: [jbosscache-dev] JBCACHE-840
by Vladimir Blagojevic
Implemented o.j.c.factories.UnitTestCacheFactory. For now, I just
converted state transfer tests. Works as planned and will continue test
conversion after alpha2.
Thanks for your input and advice. Cheers.
17 years, 4 months
JMX interfaces for Habanero
by Manik Surtani
Guys,
(Particularly Brian)
Are we happy with the current JMX interfaces on Habanero? Is there
any more work to be done on this, or can I close JBCACHE-768?
Cheers,
--
Manik Surtani
Lead, JBoss Cache
JBoss, a division of Red Hat
Email: manik(a)jboss.org
Telephone: +44 7786 702 706
MSN: manik(a)surtani.org
Yahoo/AIM/Skype: maniksurtani
17 years, 4 months
RE: [jbosscache-dev] VAM in JDBCCacheLoader : performance numbers andanalysis
by Brian Stansberry
jbosscache-dev-bounces(a)lists.jboss.org wrote:
> On 28 Dec 2006, at 00:05, Galder Zamarreno wrote:airs here.
<snip>
>> Looking a bit deeper with the profiler, I have seen that
>> TreeCacheMarshaller200 takes ages marshalling a map containing string
>> key/value pairs. Is this expected? We're adding quite a few things to
>> a simple String/String map, such as versioning and reference
>> information. Compared to standard writeObject in HashMap,
>> TreeCacheMarshaller200 spends quite a bit of time mantaining/
>> updating a reference map.
>
> It is possible that such a reference map is meaningless in
> the case of writing to a cache loader. It makes sense
> streaming on the wire for an RPC call though. So maybe there
> needs to be an overloaded form of objectTo|FromByteBuffer
> that could disable the ref map?
>
Isn't that map used to ensure only a single copy of a
multiple-referenced object survives serialization? I'd think correct
behavior for that would be critical no matter where it's done.
- Brian
17 years, 4 months
RE: [jbosscache-dev] JBCACHE-905 - code coverage for 1.4.1
by Galder Zamarreno
I managed to get it sorted passing lib.dir in build.xml as system property to the tests. In the test, I'm trying to create a CL without parent CL. I can only use jar files for this as the classes directory for the IDE could be different depending on the person. Alternatively, I guess I could maybe use javassist to generate something on the fly.
The test did not work when the test was executed from a different directory to the JBC root one, for example for CC. By passing the lib dir as system property, I can check whether it's set.
If set, use the system property, quite likely build.xml run. If not set, quite likely IDE run which allows lib dir to be hardcoded.
It's a bit hacky, I admit, any better ideas?
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
IT executives: Red Hat still #1 for value http://www.redhat.com/promo/vendor/
-----Original Message-----
From: jbosscache-dev-bounces(a)lists.jboss.org [mailto:jbosscache-dev-bounces@lists.jboss.org] On Behalf Of Galder Zamarreno
Sent: 28 December 2006 13:23
To: Manik Surtani; Hany Mesha
Cc: jbosscache-dev(a)lists.jboss.org
Subject: RE: [jbosscache-dev] JBCACHE-905 - code coverage for 1.4.1
I added this unit test. I'm working on the fix :)
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
IT executives: Red Hat still #1 for value http://www.redhat.com/promo/vendor/
-----Original Message-----
From: jbosscache-dev-bounces(a)lists.jboss.org [mailto:jbosscache-dev-bounces@lists.jboss.org] On Behalf Of Manik Surtani
Sent: 19 December 2006 11:32
To: Hany Mesha
Cc: jbosscache-dev(a)lists.jboss.org
Subject: Re: [jbosscache-dev] JBCACHE-905 - code coverage for 1.4.1
Looking at
http://cruisecontrol.jboss.com/cc/artifacts/jboss-cache-
testsuite-140/20061219021805/results/index.html
I only see one test failing
(org.jboss.cache.marshall.RedeploymentEmulationTest) but I suspect
this has to do with a bug in the test's setup and teardown, since
when run on its own, it passes.
--
Manik Surtani
Lead, JBoss Cache
JBoss, a division of Red Hat
Email: manik(a)jboss.org
Telephone: +44 7786 702 706
MSN: manik(a)surtani.org
Yahoo/AIM/Skype: maniksurtani
On 19 Dec 2006, at 04:08, Hany Mesha wrote:
> Hi all,
>
> I've run Clover against Branch_JBossCache_1_4_0 and the initial
> generated report and statistics are posted on the wiki at
> http://wiki.jboss.org/wiki/Wiki.jsp?
> page=JBossCache14xCodeCoverageAnalysis
>
> We have an overall coverage of 89.6 % which is pretty good number for
> initial report considering that there are test cases failing. If those
> test cases get fixed, I'll be able to produce more accurate report and
> therefore I'll be able to identify areas that are not covered in the
> test suite and write test cases to cover those in the next few days
> before releasing 1.4.1 GA.
>
> Please let me know if you're working on fixing one or more test
> case in
> the 1.4 branch and when you think you'll check that fix in?
>
> Thanks,
>
> -Hany
>
>
>
> Hany M. Mesha
> Sr. Software Engineer, Consultant
> Novell Identity Management Engineering
> Toronto, Canada
> hmesha(a)novell.com
> Mobile: 416-456-6945
> skype: hanymesha
>
> Novell, Inc.
> SUSE® Linux Enterprise 10
> Your Linux is ready
> http://www.novell.com/linux
>
>
>
> _______________________________________________
> jbosscache-dev mailing list
> jbosscache-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/jbosscache-dev
_______________________________________________
jbosscache-dev mailing list
jbosscache-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jbosscache-dev
_______________________________________________
jbosscache-dev mailing list
jbosscache-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jbosscache-dev
17 years, 4 months
VAM in JDBCCacheLoader : performance numbers and analysis
by Galder Zamarreno
Hi,
I've been doing some performance tests for JDBCCacheLoader with both normal Java Serialization and VersionAwareMarshaller, using the existing JDBCCacheLoaderPerfTest.
The results I got back were quite suprising. Executing a test consisting of 10 threads (each thread interacts with a single node) and each thread adds 100 key/value pairs to their node. Tests showed that normal serialization was something like 15 times faster than VAM when it came to writes. For reads, normal serialization was still 4 times faster. We're talking about String/String key value pairs here.
Looking a bit deeper with the profiler, I have seen that TreeCacheMarshaller200 takes ages marshalling a map containing string key/value pairs. Is this expected? We're adding quite a few things to a simple String/String map, such as versioning and reference information. Compared to standard writeObject in HashMap, TreeCacheMarshaller200 spends quite a bit of time mantaining/updating a reference map.
In this case, for a single put, each key and value values are different (key0/value0 key1/value1 key2/value2...), maintaining references becomes an extra overhead as the reference maps are created for each request to marshall the original map. However, I can see the point why it was done (avoid extra serialization and reduce the amount of data in the wire).
I had a quick thought about caching reference data so that it survived several requests to marshal data, but it's probably a bad idea as this cache would become a contention for concurrent marshalling and increase the memory requirements. Has anyone been this route before? I think it might only produce positive results in such specific test as this.
I tried JBoss serialization and the results were even worse. What type of Objects is JB serialization recommended for? Is it user defined ones?
I have attached JProfiler files from all three tests and also a CVS diff in case anyone wants to have a quick glance. Note that even though there's 1000 calls to marshallMap, as threads add new key/value pairs, the data read-to-be-updated increases and hence the number of items to iterate through in the map.
FYI, useRegionBasedMarshalling is set to false.
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
IT executives: Red Hat still #1 for value http://www.redhat.com/promo/vendor/
17 years, 4 months
RE: [jbosscache-dev] JBCACHE-840
by Vladimir Blagojevic
Yeah I know:) It is not a small job. I'll prepare this mechanism and
then right after alpha2 I will convert cache construction.
> -----Original Message-----
> From: Manik Surtani [mailto:manik@jboss.org]
> Sent: Wednesday, January 03, 2007 11:10 AM
> To: Vladimir Blagojevic
> Cc: jbosscache-dev(a)lists.jboss.org
> Subject: Re: [jbosscache-dev] JBCACHE-840
>
> You'd need to refactor all the unit tests to use this
> mechanism of cache construction though ... is this for after alpha2?
> --
> Manik Surtani
>
> Lead, JBoss Cache
> JBoss, a division of Red Hat
>
> Email: manik(a)jboss.org
> Telephone: +44 7786 702 706
> MSN: manik(a)surtani.org
> Yahoo/AIM/Skype: maniksurtani
>
>
>
> On 3 Jan 2007, at 15:38, Vladimir Blagojevic wrote:
>
> > Just an update on what I am doing regarding JBCACHE-840. The best
> > approach seems to be to create UnitTestCacheFactory with following
> > signatures:
> >
> > Cache createCache(Configuration c);
> > Configuration createConfiguration(int type);
> //type=sync|async Cache
> > createCache(int type); //type=sync|async
> >
> > UnitTestCacheFactory will have inner parser class that that is a
> > subclass of XmlConfigurationParser. This subclass will read cache
> > configuration file which will have all supported jgroups stacks.
> > Something similar to mux stacks.
> >
> > When invoking create methods of UnitTestCacheFactory jvm parameter
> > will be consulted to decide whether to crate udp, tcp,
> tcp-nio stack
> > and so on.
> >
> > If you have some suggestions shoot back.
> >
> >
> > _______________________________________________
> > jbosscache-dev mailing list
> > jbosscache-dev(a)lists.jboss.org
> > https://lists.jboss.org/mailman/listinfo/jbosscache-dev
>
>
>
17 years, 4 months
RE: [jbosscache-dev] JBCACHE-840
by Vladimir Blagojevic
Just an update on what I am doing regarding JBCACHE-840. The best
approach seems to be to create UnitTestCacheFactory with following
signatures:
Cache createCache(Configuration c);
Configuration createConfiguration(int type); //type=sync|async
Cache createCache(int type); //type=sync|async
UnitTestCacheFactory will have inner parser class that that is a
subclass of XmlConfigurationParser. This subclass will read cache
configuration file which will have all supported jgroups stacks.
Something similar to mux stacks.
When invoking create methods of UnitTestCacheFactory jvm parameter will
be consulted to decide whether to crate udp, tcp, tcp-nio stack and so
on.
If you have some suggestions shoot back.
17 years, 4 months
CacheLoader question
by Ben Wang
Manik,
It doesn't quite say explicitly in the new CacheSPI spec. But does the api:
getCacheLoaderManager();
1. Can return null if not configured
2. Only return not null if cache loader config is specified?
Thanks,
-Ben
17 years, 4 months
JBCACHE-840
by Vladimir Blagojevic
http://jira.jboss.com/jira/browse/JBCACHE-840
Background:
All junit tests should have a system property (with a default if the
system
property is not defined) to use a certain JGroups protocol stack. Thus,
for
example, we can run whole JBC testsuite with Jgroups tcp stack. We
should
have multiple tests targets that run testsuite with tcp,udp, and other
supported protocols.
Current state of affairs:
All our non-local mode unit tests use configuration files defined in
META-INF
directory. We use half a dozen configuration files such as
replSync-service.xml,
replAsync-service.xml, replSync-passivation-service.xml and so on.
It is hard to keep track of all these configuration files and coordinate
that
all of them have the same Jgroups configuration element. Thus
implementation
of JBCACHE-840 becomes almost impossible.
Solution:
1) Use only replSync-service.xml and replAsync-service.xml
Having all our non-local tests rely on these two files
allows us to have a strong guarantee that all non-local tests
pass with these two specific configuration files. It also
constraints all non-local test to only two files.
2) Remove use of other sync and async configuration files
In some tests we use variation of replSync-service.xml and
replAsync-service.xml that have additional configuration
elements such as passivation, eviction etc. Use of these
configuration files can be supplemented by use of
replSync-service.xml
and replAsync-service.xml with addition of programmatic injection of
eviction, passivation and other elements during test setup.
3) Add tests targets for non udp jgroups configs
Having 1) and 2) completed allows us to modify
JGroups stack element in replSync-service.xml and
replAsync-service.xml and run the entire test suite with
a specific JGroups protocol stack such as tcp.
We can modify all our test targets to include specific
parameter indicating JGroups stack to be used (i.e udp|tcp)
Based on this parameter we can create specific replSync-service.xml
and
replAsync-service.xml prior to execution of any test build
target.
Do you guys have some comments, suggestions and such?
Regards,
Vladimir
17 years, 4 months
