new jdbc cache loader (RedcdRecJDBCCacheLoader) - backward compatibility
by Mircea Markus
Hi,
I've finished testing and the new cache loader impl
(RedcdRecJDBCCacheLoader) works fine on existing data persisted by the prev
cache loader impl(JDBCCacheLoader).
Encountered issues:
JDBCCacheLoader does not persists the root. When one performs a put("a/b/c")
on an empty loader, all the parent nodes should be created, i.e. 'a/b', 'a'
and 'a' and '/' - root is skipped currently.
RedcdRecJDBCCacheLoader relies on root being persisted, so it performs the
following logic in start:
public void start() throws Exception
{
super.start();
if (!exists(Fqn.ROOT) && getNodeCount() > 0)
{
put(Fqn.ROOT, new HashMap());
}
}
The bug still exists in JDBCCacheLoader - I'll take a look to fix it. As
Manik said, the fix shouldn't break the compatibility.
Cheers,
Mircea
17 years, 10 months
Dependency on concurrent jar
by Vladimir Blagojevic
Hey,
Why don't we remove dependency on concurrent.jar? Can we? My search
returns only dozen tests that still use it and one source file
PojoCacheImpl.java.
Cheers.
17 years, 10 months
RE: [jbosscache-dev] PojoCache annotations and marker interfaces
by Brian Stansberry
This is checked in and the tag moved. Thanks. :)
Rajesh Rajasekaran wrote:
> I haven't started on the release.
> I was planning to kickoff the testsuites on the weekend.
> But I can wait until you get your fixes in.
> Let me know once it is retagged.
>
>
> -----Original Message-----
> From: Brian Stansberry
> Sent: Saturday, February 03, 2007 7:58 AM
> To: Rajesh Rajasekaran
> Cc: 'Manik Surtani'
> Subject: FW: [jbosscache-dev] PojoCache annotations and
> marker interfaces
>
> Rajesh - have you started on this on 1.4.1.SP1? If yes, the
> following is not a big deal and I'll skip it; if no I'll move the tag.
>
> Manik Surtani wrote:
>> On 2 Feb 2007, at 20:23, Brian Stansberry wrote:
>>
>>> I see that in 1.4.x, we have this lovely situation:
>>>
>>> jboss-cache.jar includes the
> o.j.c.aop.(InstanceOf)AopMarker marker
>>> interfaces. jboss-cache-jdk5.jar also includes the
>>> o.j.c.aop.annotation.(InstanceOf)AopMarker annotations, but does not
>>> include the marker interfaces.
>>>
>>> Thus, in AS 4.2, if I want to support both annotation styles for
>>> FIELD granularity replication, there is no JBC jar that I can
>>> integrate to allow it. I'd like to support both, since it seems user
>>> friendly to support the different methods the PojoCache examples
>>> use. (Note: we're deprecating the redundant
>>> o.j.web.tomcat.tc5.session.AopMarker marker interfaces, so good
>>> support of the standard PojoCache markers is even more important.)
>>>
>>> If 1.4.1.SP1 isn't in QA yet (I saw another fix this AM) can we fix
>>> this? I see no reason not to include the marker interfaces in the
>>> jboss-cache-jdk5.jar. It's not like they don't work in JDK 5 or
>>> something.
>>>
>>
>> I believe QA on this HAS started. The task's been assigned to
>> Rajesh, although I don't think he's started on the release process
>> yet. If you think this fix is important enough and quick/easy
>> enough to get in before Monday AM, go ahead and do it and move the
>> tag for the affected files. Just remember to ping Rajesh and tell
>> him, in
>> case he's checked out the srcs already.
>>
>>
>>
>>> This isn't critical, but this whole area is really user unfriendly
>>> (don't get me started on the atrocious aopc) and doing what we can
>>> to make it easier is a good thing.
>>>
>>> Brian Stansberry
>>> Lead, AS Clustering
>>> JBoss, a division of Red Hat
>>> Ph: 510-396-3864
>>> skype: bstansberry
>>>
>>> _______________________________________________
>>> jbosscache-dev mailing list
>>> jbosscache-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jbosscache-dev
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
Ph: 510-396-3864
skype: bstansberry
17 years, 10 months
new JDBC cache loader impl ready / final figures and design
by Mircea Markus
Hi,
Attached is a comparison between old and new impl on JDBC cache loading.
Basically recursion was eliminated in the case of loadState and remove, and
the put had a 50% optimization. All other operation are more or less a
single db call, so nothing can really be programmatically improved.
The second image contains the design. Any comments are extremely welcomed :)
Nothing is committed yet on CVS.
An open issue is the backward compatibility - working on that.
Cheers,
Mircea
P.S.
How I've tested the performance (phps more easy to understand it from code
:-)
a tree with a TREE_DEPTH depth is created. Each node in the tree has a
CHILDREN number of children. By default I've used TREE_DEPTH=7 and
CHILDREN=3 which gave a total number of 3280 children. A node is selected at
each depth from disjunct subtrees. ( i.e. in the benchmark used 7 nodes
are selected). On all those nodes a certain operation is performed:
loadState, remove etc. For benchmarking the put operation, the in memory
tree nodes are Collections.shuffle(List<Fqn>), then they are (randomly)
added to the database/classLoader.put. This procedure was repeated sever
times. Also additional testing was performed in the case of put for deep
trees (100 - 200 nodes)
17 years, 10 months
PojoCache annotations and marker interfaces
by Brian Stansberry
I see that in 1.4.x, we have this lovely situation:
jboss-cache.jar includes the o.j.c.aop.(InstanceOf)AopMarker marker
interfaces.
jboss-cache-jdk5.jar also includes the
o.j.c.aop.annotation.(InstanceOf)AopMarker annotations, but does not
include the marker interfaces.
Thus, in AS 4.2, if I want to support both annotation styles for FIELD
granularity replication, there is no JBC jar that I can integrate to
allow it. I'd like to support both, since it seems user friendly to
support the different methods the PojoCache examples use. (Note: we're
deprecating the redundant o.j.web.tomcat.tc5.session.AopMarker marker
interfaces, so good support of the standard PojoCache markers is even
more important.)
If 1.4.1.SP1 isn't in QA yet (I saw another fix this AM) can we fix
this? I see no reason not to include the marker interfaces in the
jboss-cache-jdk5.jar. It's not like they don't work in JDK 5 or
something.
This isn't critical, but this whole area is really user unfriendly
(don't get me started on the atrocious aopc) and doing what we can to
make it easier is a good thing.
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
Ph: 510-396-3864
skype: bstansberry
17 years, 10 months
Optimising node storage in memory
by Manik Surtani
Guys,
Playing around with looking at how efficient (or not) it is to store
nodes as children of other nodes. While this makes logical sense, on
an implementation level walking the tree sucks - O(n) where n is the
depth of the tree. This works great for small trees, but as depth
increases, so does the lookup time.
Since all locking happens on the interceptor level (based on a
logical Fqn) isn't the tree structure effectively maintained here?
What if Node objects weren't directly linked, but maintained in a
simple HashMap, keyed on Fqn? Instant constant-time lookups,
regardless of tree depth. Sure, it will mean tuning the HashMap for
the mem footprint/speed tradeoff, maybe even a custom Map impl tuned
for this purpose.
And in the initial part the fix should be easy - just replace
CacheImpl.peek() with a map lookup, and _remove() and _put() with a
map put.
Will think about this some more, and post later, but for now I wrote
a quick test class that looped through node creation, lookup and
deletion through arbitrary depths and these are what the numbers look
like.
Starting test, with 10000 nodes evenly spread out across the tree.
All times in millis.
Testing with depth 10
Node Creation 7924
Node Retrieval 1141
Node Deletion 889
Testing with depth 20
Node Creation 16372
Node Retrieval 2072
Node Deletion 1524
Testing with depth 30
Node Creation 31440
Node Retrieval 3010
Node Deletion 2303
Testing with depth 40
Node Creation 52238
Node Retrieval 4172
Node Deletion 3184
Testing with depth 50
Node Creation 80781
Node Retrieval 35788
Node Deletion 34107
Arbitrary depths may not be very useful with current use cases but
thinking of scaling to larger and larger grids and dealing with ever
larger data volumes, this may be a crucial optimisation. Thoughts
and comments?
Cheers,
Manik
FYI, the script to run is tests/scripts/loopFlatteningTest.sh in
HEAD. Usage: loopFlatteningTest.sh <space-separated list of depths
to test>, e.g., loopFlatteningTest.sh 10 20 30 40 50
--
Manik Surtani
Lead, JBoss Cache
JBoss, a division of Red Hat
Email: manik(a)jboss.org
Telephone: +44 7786 702 706
MSN: manik(a)surtani.org
Yahoo/AIM/Skype: maniksurtani
17 years, 10 months
