Storing k, v at root node of marshaled region and cache loaders.
by Galder Zamarreno
Hi all,
Yesterday while looking at a support case, I discovered an awkward
behavior when it came to root region based marshalling nodes and cache
loaders.
Let's say you have a cache configured with a cache loader and have a
region marshaled called starting at /me. I have created an MBean so that
I can register/unregister classloader and activate/deactivate the region
upon deployment/undeployment of the classes I wanna put under /me.
Now, let's say that I put a k,v in /me which will get stored in the
cache loader. If I know touch the deployment archive to force a
redeployment and check the contents of the cache, they're empty. If I
try to get the k I put under /me, it'll return nothing. Why?
Simple: The cache instance is configured to preload from root (/) but
when the region is activated, /me node is created. When the preloading
phase comes, a check is made to see whether the node needs to be loaded:
private boolean mustLoad(DataNode n, Object key)
{
if (log.isTraceEnabled())
{
log.trace("mustLoad called with key=" + key + " and datanode="
+ n);
}
return n == null ||
(n.containsKey(TreeCache.UNINITIALIZED) && (key == null
|| !n.containsKey(key)));
}
The problem is that nothing will be preloaded for at least the /me node,
because the node is not null, and it's not TreeCache.UNINITIALIZED. Any
child nodes under the region node do not have issues like this.
This is likely a bug: maybe when the region is activated, the node
should probably be created as TreeCache.UNINITIALIZED? I think this
would work).
It's true that you could say that putting data at the root node of the
region is not a good idea regardless, but I think deep down this is a bug.
Thoughts?
p.s. This is at least present in 1.4.x, need to check trunk. However,
wanted to gather some thoughts before I continue debugging/fixing it.
--
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
16 years, 1 month
Lets ditch docbook (long-term)
by Jason T. Greene
This week a community contributor sent me some a very tiny correction to
the POJO Cache documentation (the word you had an additional y, "yyou").
Updating the docs for such a simple fix is a royal pain in the ass, and
required me merging it to 3 different project branches, regenerating
documentation using maven (which failed btw, since maven is a pos, and I
had to update an already released tag), and finally committing the new
docs to the CMS SVN.
This process combined with the unfriendly docbook format basically
discourages the docs from being properly maintained.
IMO a good wiki, like mediawiki, is superior to docbook for community
documentation, since its easy to update, easy to use, and allows
external community members to contribute updates without getting code
svn access.
I think we should move our project docs to something like mediawiki, and
save docbook for situations when you actually want printed documentation
(like EAP).
--
Jason T. Greene
JBoss, a division of Red Hat
16 years, 1 month
Code coverage
by Manik Surtani
I've added EMMA code coverage to the JBC support POMs. This is
disabled by default since it entails adding interceptors to tests and
makes the test suite run slower than usual. Instead it can be run
manually whenever new features are added or there are significant code
changes, etc. to ensure we have a decent level of code coverage.
This is formally a part of jbosscache-common 1.3, but recent 1.3-
SNAPSHOT versions have this as well.
You can enable it by specifying the "codeCoverage" profile. If you
run the "site" goal, it will generate HTML reports based on the tests
run.
E.g.,
$ mvn site -PcodeCoverage
will run the default test suite (tests in the "unit" group in
jbosscache-core, for example, or "functional" in jbosscache-pojo).
In jbosscache-core, other test suites can be selected by specifying
the necessary profile.
E.g.,
$ mvn site -PcodeCoverage,test-functional
Please see README-Maven.txt for more details.
Reports are in
target/site/index.html
which has links to EMMA code coverage reports, etc.
Enjoy!
Manik
--
Manik Surtani
Lead, JBoss Cache
manik(a)jboss.org
16 years, 2 months
2.2.0.CR2
by Manik Surtani
Guys,
We hope to cut a 2.2.0.CR2 release early next week (Monday/Tuesday
timeframe). This is pretty much ready for GA, but we'd like it to go
through another CR cycle to catch any last-minute issues, considering
the degree of refactoring in this release.
Jason, do you also want to cut an early preview of array interception
in POJO Cache?
Brian, we will test this against the Hibernate test suite - trunk runs
well so I have no reason to expect problems with 2.2.0.CR2. Anything
else we should test against? Can the http session replication code be
easily tested with JBC snapshots?
Cheers
--
Manik Surtani
Lead, JBoss Cache
manik(a)jboss.org
16 years, 2 months
Re: new utility classes for removing Thread.sleeps
by Manik Surtani
Looks good. A couple of implementation notes:
1. ReplicationListener constructor - getting components from the cache
TestingUtil.extractField() is brittle (since it relies on field name)
and could break with even the slightest refactoring. Better to use
TestingUtil.extractComponentRegistry() and
ComponentRegistry.getComponent() to get the component you want. Much
safer.
2. ReplicationListener constructor - updating components in the cache
Again rather than using TestingUtil.replaceField() - equally brittle -
try using componentRegistry.registerComponent() to register a new
component, and componentRegistry.rewire() if the cache is already
running to ensure the new component is re-injected everywhere it is
needed.
3. How does this behave with sync replication? The javadocs say it
doesn't make any sense, but it may be done in base test classes, such
that the test could have subclasses which use sync and async
replication. I'm guessing waitForReplication() will always return
immediately with sync replication?
Cheers,
Manik
PS: cc'ing jbosscache-dev - discussions like this should be on the
list, just in case anyone else is interested. :-)
On 29 May 2008, at 08:41, Mircea Markus wrote:
> I've just migrated o.j.c.replicated.AsyncReplTest to the new
> listener mechanism.
> Can you please take a look and let me know what do you think.
>
> Cheers,
> Mircea
>
> Brian Stansberry wrote:
>> Try o.j.c.replicated.AsyncReplTest
>>
>> Mircea Markus wrote:
>>> Sorry, wrong button :)
>>> Can you point me to such a class? I'll continue refactoring on Th
>>> and I'll start with one of these for convenience. The plan is to
>>> remove (almost) all 'sleeps' from code.
>>>
>>> Mircea Markus wrote:
>>>> Can you point me
>>>>
>>>> Brian Stansberry wrote:
>>>>> Thanks. Are you planning to add one for the ordinary REPL_ASYNC
>>>>> case; i.e. no replication queue? That's the one where I end up
>>>>> adding Thread.sleep() the most -- waiting to ensure the async
>>>>> message has arrived on the remote node before testing failover.
>>>>>
>>>>> Mircea Markus wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Take a look at
>>>>>> org.jboss.cache.util.internals.EvictionController and
>>>>>> org.jboss.cache.util.internals.ReplicationQueueNotifier.
>>>>>> Right now they are only being used in
>>>>>> StateTransferConcurrencyTest, I will change more tests to use
>>>>>> them later on this week. Any feedback is welcomed!
>>>>>> Cheers,
>>>>>> Mircea
>>>>>
>>
--
Manik Surtani
Lead, JBoss Cache
manik(a)jboss.org
16 years, 2 months
Documentation page corrected
by Jason T. Greene
The last update to the doc index had broken links to a non-existent
release of POJO Cache (2.1.1). I have corrected this to point to the
current stable version (2.1.0).
--
Jason T. Greene
JBoss, a division of Red Hat
16 years, 2 months
Re: Session Clustering for SIP
by Brian Stansberry
For the mailing list, see https://lists.jboss.org/mailman/listinfo
Jean Deruelle wrote:
> Hi Brian and all jbosscache team,
>
> Yes this is what I meant having the same benefit as if my class was
> annotated with @Replicable, maybe on SET_AND_GET too.
>
> By the way I cannot find any way to subscribe to the jbosscache-dev list
> either through http://post-office.corp.redhat.com/mailman/listinfo/ or
> http://www.redhat.com/mailman/listinfo ?
>
> Thanks
> Jean
>
>
> Brian Stansberry wrote:
>> Copying to jbosscache-dev list. Jean, since you're doing caching work,
>> suggest you subscribe to that list if you haven't already.
>>
>> I think your suggestion is definitely worth considering. If I
>> understand you correctly, you're basically proposing something like
>> adding a replication-trigger config to JBC, whereby with
>> SET_AND_NON_PRIMITIVE_GET, JBC would track the type of any read and if
>> not primitive, generate a cluster-wide write at transaction commit.
>>
>> Jean Deruelle wrote:
>>> Hi Brian,
>>>
>>> Thinking a bit more about this.
>>> You seems to have achieved in http sessions what is not possible to
>>> do with Jboss cache natively and that is to me a drawback for Jboss
>>> Cache ease of use.
>>>
>>> It is monitor automatically any changes to an object even if its
>>> bytecode has not been enhanced by JBoss AOP...
>>> Do you think it would not be better to integrate your code into Jboss
>>> Cache this could help us to have a cleaner code and benefit others ?
>>> Maybe with a note saying that it's less performing than if you
>>> preprocess your classes with Jboss AOP (if my guess that it is less
>>> performing is right...)
>>>
>>> wdyt ?
>>> Best regards
>>> Jean
>>>
>>> Brian Stansberry wrote:
>>>> Jean Deruelle wrote:
>>>>> Brian Stansberry wrote:
>>>>> <snip/>
>>>>>>
>>>>>>>>> As a first version, I intend to try to keep things as simple as
>>>>>>>>> possible so my plan is "just" to store the complete SipSession
>>>>>>>>> object I have into the cache and let TreeCacheAop do the job
>>>>>>>>> when attributes are added/modified/removed to replicate to
>>>>>>>>> other node(s).
>>>>>>>>>
>>>>>>>>> I was looking at the AttributeBasedClusteredSession
>>>>>>>>> implementation and one thing I fail to understand is why it is
>>>>>>>>> using a lot of transient map to handle session attributes (all
>>>>>>>>> attibutes, only modified, only removed Maps) instead of letting
>>>>>>>>> the Cache handling that for you ? Even for replication it seems
>>>>>>>>> to be done by hand through the storeSession method of the
>>>>>>>>> JbossCacheManager class and the SnapshotManager class ?
>>>>>>>>>
>>>>>>>>
>>>>>>>> This mostly relates to not forcing the user to have to remember
>>>>>>>> to call setAttribute if they modify an attribute:
>>>>>>>>
>>>>>>>> Set set = session.getAttribute("set");
>>>>>>>> set.add(someObject);
>>>>>>>> // oops, forgot to call setAttribute, updated set doesn't
>>>>>>>> replicate!
>>>>>>>>
>>>>>>> I thought and still think this is not needed with TreeCacheAop in
>>>>>>> doing putObject instead of put (see
>>>>>>> http://www.jboss.org/file-access/default/members/jbosscache/freezone/docs...)
>>>>>>>
>>>>>>> and it seems that it is already used for the
>>>>>>> FieldBasedClusteredSession but that there is an overhead there of
>>>>>>> creating observer for the pojo although the TreeCacheAop does the
>>>>>>> exact same thing for you (at least that is my understanding of
>>>>>>> TreeCacheAop as defined in the link above "Note that a user will
>>>>>>> only need to issue this call once for each POJO (think of it as
>>>>>>> attaching POJO to cache management). Once it is executed,
>>>>>>> TreeCacheAop will assign an interceptor for the pojo instance and
>>>>>>> its sub-objects." but I start to think that my assumption is
>>>>>>> wrong here... can you confirm ?)
>>>>>>>
>>>>>>
>>>>>> To use putObject you have to use FieldBasedClusteredSession, and
>>>>>> PojoCache (fka TreeCacheAop) can only add the interceptor, detect
>>>>>> changes to the object etc. if the attribute's class has been
>>>>>> bytecode enhanced by JBoss AOP. Not a default solution, since
>>>>>> most users will not take the steps needed to bytecode enhance
>>>>>> their classes.
>>>>>>
>>>>>> If you use putObject and the object is not bytecode enhanced,
>>>>>> PojoCache has no idea if you subsequently change the object. So
>>>>>> you're back to the issue of needing to deal with user forgetting
>>>>>> to call setAttribute.
>>>>> Ok it's clearer now and it all makes sense :-)
>>>>> I guess even with JbossCache 2.x and Jboss 5.x we will need to deal
>>>>> with the same thing then, right ?
>>>>>
>>>>
>>>> Yes.
>>>>
>>>>> I was wondering if one could leverage the jboss-web.xml with new
>>>>> tags in it to state which are the classes he wants to be
>>>>> replicated. (with potentially pattern macthing)
>>>>> On those classes, at load time, JbossAS would add automatically the
>>>>> necessary Jboss cache annotation
>>>>> (@org.jboss.cache.pojo.annotation.Replicable) through the javassist
>>>>> library and the classes will be aspectized with load time weaving
>>>>> and thus the full features of PojoCache could be used with no extra
>>>>> code.
>>>>> But I guess the problem here is that it still add some burden on
>>>>> the end developer to describe things in jboss-web.xml and that
>>>>> could be error prone so not an ideal solution neither I guess
>>>>>
>>>>
>>>> If a user doesn't want to annotate their classes, they can deploy a
>>>> jboss-aop.xml file to get the load time weaving. Adding the facility
>>>> you describe in jboss-web.xml might make the syntax a bit easier,
>>>> but it's doable now.
>>>>
>>>>
>>>
>>>
>>
>
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
16 years, 2 months
Re: Session Clustering for SIP
by Brian Stansberry
Copying to jbosscache-dev list. Jean, since you're doing caching work,
suggest you subscribe to that list if you haven't already.
I think your suggestion is definitely worth considering. If I understand
you correctly, you're basically proposing something like adding a
replication-trigger config to JBC, whereby with
SET_AND_NON_PRIMITIVE_GET, JBC would track the type of any read and if
not primitive, generate a cluster-wide write at transaction commit.
Jean Deruelle wrote:
> Hi Brian,
>
> Thinking a bit more about this.
> You seems to have achieved in http sessions what is not possible to do
> with Jboss cache natively and that is to me a drawback for Jboss Cache
> ease of use.
>
> It is monitor automatically any changes to an object even if its
> bytecode has not been enhanced by JBoss AOP...
> Do you think it would not be better to integrate your code into Jboss
> Cache this could help us to have a cleaner code and benefit others ?
> Maybe with a note saying that it's less performing than if you
> preprocess your classes with Jboss AOP (if my guess that it is less
> performing is right...)
>
> wdyt ?
> Best regards
> Jean
>
> Brian Stansberry wrote:
>> Jean Deruelle wrote:
>>> Brian Stansberry wrote:
>>> <snip/>
>>>>
>>>>>>> As a first version, I intend to try to keep things as simple as
>>>>>>> possible so my plan is "just" to store the complete SipSession
>>>>>>> object I have into the cache and let TreeCacheAop do the job when
>>>>>>> attributes are added/modified/removed to replicate to other node(s).
>>>>>>>
>>>>>>> I was looking at the AttributeBasedClusteredSession
>>>>>>> implementation and one thing I fail to understand is why it is
>>>>>>> using a lot of transient map to handle session attributes (all
>>>>>>> attibutes, only modified, only removed Maps) instead of letting
>>>>>>> the Cache handling that for you ? Even for replication it seems
>>>>>>> to be done by hand through the storeSession method of the
>>>>>>> JbossCacheManager class and the SnapshotManager class ?
>>>>>>>
>>>>>>
>>>>>> This mostly relates to not forcing the user to have to remember to
>>>>>> call setAttribute if they modify an attribute:
>>>>>>
>>>>>> Set set = session.getAttribute("set");
>>>>>> set.add(someObject);
>>>>>> // oops, forgot to call setAttribute, updated set doesn't replicate!
>>>>>>
>>>>> I thought and still think this is not needed with TreeCacheAop in
>>>>> doing putObject instead of put (see
>>>>> http://www.jboss.org/file-access/default/members/jbosscache/freezone/docs...)
>>>>>
>>>>> and it seems that it is already used for the
>>>>> FieldBasedClusteredSession but that there is an overhead there of
>>>>> creating observer for the pojo although the TreeCacheAop does the
>>>>> exact same thing for you (at least that is my understanding of
>>>>> TreeCacheAop as defined in the link above "Note that a user will
>>>>> only need to issue this call once for each POJO (think of it as
>>>>> attaching POJO to cache management). Once it is executed,
>>>>> TreeCacheAop will assign an interceptor for the pojo instance and
>>>>> its sub-objects." but I start to think that my assumption is wrong
>>>>> here... can you confirm ?)
>>>>>
>>>>
>>>> To use putObject you have to use FieldBasedClusteredSession, and
>>>> PojoCache (fka TreeCacheAop) can only add the interceptor, detect
>>>> changes to the object etc. if the attribute's class has been
>>>> bytecode enhanced by JBoss AOP. Not a default solution, since most
>>>> users will not take the steps needed to bytecode enhance their classes.
>>>>
>>>> If you use putObject and the object is not bytecode enhanced,
>>>> PojoCache has no idea if you subsequently change the object. So
>>>> you're back to the issue of needing to deal with user forgetting to
>>>> call setAttribute.
>>> Ok it's clearer now and it all makes sense :-)
>>> I guess even with JbossCache 2.x and Jboss 5.x we will need to deal
>>> with the same thing then, right ?
>>>
>>
>> Yes.
>>
>>> I was wondering if one could leverage the jboss-web.xml with new tags
>>> in it to state which are the classes he wants to be replicated. (with
>>> potentially pattern macthing)
>>> On those classes, at load time, JbossAS would add automatically the
>>> necessary Jboss cache annotation
>>> (@org.jboss.cache.pojo.annotation.Replicable) through the javassist
>>> library and the classes will be aspectized with load time weaving and
>>> thus the full features of PojoCache could be used with no extra code.
>>> But I guess the problem here is that it still add some burden on the
>>> end developer to describe things in jboss-web.xml and that could be
>>> error prone so not an ideal solution neither I guess
>>>
>>
>> If a user doesn't want to annotate their classes, they can deploy a
>> jboss-aop.xml file to get the load time weaving. Adding the facility
>> you describe in jboss-web.xml might make the syntax a bit easier, but
>> it's doable now.
>>
>>
>
>
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
16 years, 2 months
Re: Session Clustering for SIP
by Brian Stansberry
Copying to jbosscache-dev list. Jean, since you're doing caching work,
suggest you subscribe to that list if you haven't already.
I think your suggestion is definitely worth considering. If I understand
you correctly, you're basically proposing something like adding a
replication-trigger config to JBC, whereby with
SET_AND_NON_PRIMITIVE_GET, JBC would track the type of any read and if
not primitive, generate a cluster-wide write at transaction commit.
Jean Deruelle wrote:
> Hi Brian,
>
> Thinking a bit more about this.
> You seems to have achieved in http sessions what is not possible to do
> with Jboss cache natively and that is to me a drawback for Jboss Cache
> ease of use.
>
> It is monitor automatically any changes to an object even if its
> bytecode has not been enhanced by JBoss AOP...
> Do you think it would not be better to integrate your code into Jboss
> Cache this could help us to have a cleaner code and benefit others ?
> Maybe with a note saying that it's less performing than if you
> preprocess your classes with Jboss AOP (if my guess that it is less
> performing is right...)
>
> wdyt ?
> Best regards
> Jean
>
> Brian Stansberry wrote:
>> Jean Deruelle wrote:
>>> Brian Stansberry wrote:
>>> <snip/>
>>>>
>>>>>>> As a first version, I intend to try to keep things as simple as
>>>>>>> possible so my plan is "just" to store the complete SipSession
>>>>>>> object I have into the cache and let TreeCacheAop do the job when
>>>>>>> attributes are added/modified/removed to replicate to other node(s).
>>>>>>>
>>>>>>> I was looking at the AttributeBasedClusteredSession
>>>>>>> implementation and one thing I fail to understand is why it is
>>>>>>> using a lot of transient map to handle session attributes (all
>>>>>>> attibutes, only modified, only removed Maps) instead of letting
>>>>>>> the Cache handling that for you ? Even for replication it seems
>>>>>>> to be done by hand through the storeSession method of the
>>>>>>> JbossCacheManager class and the SnapshotManager class ?
>>>>>>>
>>>>>>
>>>>>> This mostly relates to not forcing the user to have to remember to
>>>>>> call setAttribute if they modify an attribute:
>>>>>>
>>>>>> Set set = session.getAttribute("set");
>>>>>> set.add(someObject);
>>>>>> // oops, forgot to call setAttribute, updated set doesn't replicate!
>>>>>>
>>>>> I thought and still think this is not needed with TreeCacheAop in
>>>>> doing putObject instead of put (see
>>>>> http://www.jboss.org/file-access/default/members/jbosscache/freezone/docs...)
>>>>>
>>>>> and it seems that it is already used for the
>>>>> FieldBasedClusteredSession but that there is an overhead there of
>>>>> creating observer for the pojo although the TreeCacheAop does the
>>>>> exact same thing for you (at least that is my understanding of
>>>>> TreeCacheAop as defined in the link above "Note that a user will
>>>>> only need to issue this call once for each POJO (think of it as
>>>>> attaching POJO to cache management). Once it is executed,
>>>>> TreeCacheAop will assign an interceptor for the pojo instance and
>>>>> its sub-objects." but I start to think that my assumption is wrong
>>>>> here... can you confirm ?)
>>>>>
>>>>
>>>> To use putObject you have to use FieldBasedClusteredSession, and
>>>> PojoCache (fka TreeCacheAop) can only add the interceptor, detect
>>>> changes to the object etc. if the attribute's class has been
>>>> bytecode enhanced by JBoss AOP. Not a default solution, since most
>>>> users will not take the steps needed to bytecode enhance their classes.
>>>>
>>>> If you use putObject and the object is not bytecode enhanced,
>>>> PojoCache has no idea if you subsequently change the object. So
>>>> you're back to the issue of needing to deal with user forgetting to
>>>> call setAttribute.
>>> Ok it's clearer now and it all makes sense :-)
>>> I guess even with JbossCache 2.x and Jboss 5.x we will need to deal
>>> with the same thing then, right ?
>>>
>>
>> Yes.
>>
>>> I was wondering if one could leverage the jboss-web.xml with new tags
>>> in it to state which are the classes he wants to be replicated. (with
>>> potentially pattern macthing)
>>> On those classes, at load time, JbossAS would add automatically the
>>> necessary Jboss cache annotation
>>> (@org.jboss.cache.pojo.annotation.Replicable) through the javassist
>>> library and the classes will be aspectized with load time weaving and
>>> thus the full features of PojoCache could be used with no extra code.
>>> But I guess the problem here is that it still add some burden on the
>>> end developer to describe things in jboss-web.xml and that could be
>>> error prone so not an ideal solution neither I guess
>>>
>>
>> If a user doesn't want to annotate their classes, they can deploy a
>> jboss-aop.xml file to get the load time weaving. Adding the facility
>> you describe in jboss-web.xml might make the syntax a bit easier, but
>> it's doable now.
>>
>>
>
>
--
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
brian.stansberry(a)redhat.com
16 years, 2 months
Refactoring (deprecating) NodeSPI#getLock()
by Manik Surtani
Currently locking takes place all over the codebase, pretty messy.
1. Locks reside in a Node, and is exposed via NodeSPI.getLock()
2. Some interceptors and the state transfer manager acquire and
release locks using methods on NodeLock from NodeSPI.getLock().
3. Others release locks by using helper methods in TransactionEntry,
InvocationContext, TransactionTable and TransactionUtil.
4. NodeLock has methods for acquiring and releasing locks, both
individually and recursively.
I propose that we lose all this bloat and profileration of logic by
writing a single LockManager, which would expose a few simple methods
to acquire and release locks. All lock acquisitions/releases would go
through this LockManager.
Specifically, I want to deprecate NodeSPI#getLock(). I think code
should not have direct access to locks, and should always go through
the LockManager. The reason for this is to allow easy migration to
more interesting locking strategies in future, especially not having a
lock being a part of a node (this has caused us to introduce all sorts
of inefficiencies to prevent concurrent eviction/deletion/creation
events) and to lock based on Fqn. Even allows us to easily introduce
StripedLocks or distributed locks at a later date.
It also helps us write specific LockManager implementations that just
uses standard JDK util.concurrent locks, which is what I need for MVCC
in 3.x.
What are your thoughts on this?
Cheers,
--
Manik Surtani
Lead, JBoss Cache
manik(a)jboss.org
16 years, 2 months