RE: [jbosscache-dev] Optimising node storage in memory
by Galder Zamarreno
You said replacing _remove() with a map put??
Would you have intermediate empty nodes? So, if you have /a/b/c/d, what would the Map look like? Just an entry for /a/b/c/d? or entry for /, entry for /a,...etc and finally entry in the map with /a/b/c/d?
Speed of removals and additions would certainly depend on this. Without intermediate nodes, they would be O(1), but how would you handle parental locks? You could have had two maps, one only with populated nodes so that you can get O(1) for additions, removals and lookups, and another map with current locking information where all nodes would be present, both intermediate and populated nodes?
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
-----Original Message-----
From: jbosscache-dev-bounces(a)lists.jboss.org [mailto:jbosscache-dev-bounces@lists.jboss.org] On Behalf Of Manik Surtani
Sent: 02 February 2007 18:18
To: jbosscache-dev(a)lists.jboss.org
Subject: [jbosscache-dev] Optimising node storage in memory
Guys,
Playing around with looking at how efficient (or not) it is to store
nodes as children of other nodes. While this makes logical sense, on
an implementation level walking the tree sucks - O(n) where n is the
depth of the tree. This works great for small trees, but as depth
increases, so does the lookup time.
Since all locking happens on the interceptor level (based on a
logical Fqn) isn't the tree structure effectively maintained here?
What if Node objects weren't directly linked, but maintained in a
simple HashMap, keyed on Fqn? Instant constant-time lookups,
regardless of tree depth. Sure, it will mean tuning the HashMap for
the mem footprint/speed tradeoff, maybe even a custom Map impl tuned
for this purpose.
And in the initial part the fix should be easy - just replace
CacheImpl.peek() with a map lookup, and _remove() and _put() with a
map put.
Will think about this some more, and post later, but for now I wrote
a quick test class that looped through node creation, lookup and
deletion through arbitrary depths and these are what the numbers look
like.
Starting test, with 10000 nodes evenly spread out across the tree.
All times in millis.
Testing with depth 10
Node Creation 7924
Node Retrieval 1141
Node Deletion 889
Testing with depth 20
Node Creation 16372
Node Retrieval 2072
Node Deletion 1524
Testing with depth 30
Node Creation 31440
Node Retrieval 3010
Node Deletion 2303
Testing with depth 40
Node Creation 52238
Node Retrieval 4172
Node Deletion 3184
Testing with depth 50
Node Creation 80781
Node Retrieval 35788
Node Deletion 34107
Arbitrary depths may not be very useful with current use cases but
thinking of scaling to larger and larger grids and dealing with ever
larger data volumes, this may be a crucial optimisation. Thoughts
and comments?
Cheers,
Manik
FYI, the script to run is tests/scripts/loopFlatteningTest.sh in
HEAD. Usage: loopFlatteningTest.sh <space-separated list of depths
to test>, e.g., loopFlatteningTest.sh 10 20 30 40 50
--
Manik Surtani
Lead, JBoss Cache
JBoss, a division of Red Hat
Email: manik(a)jboss.org
Telephone: +44 7786 702 706
MSN: manik(a)surtani.org
Yahoo/AIM/Skype: maniksurtani
_______________________________________________
jbosscache-dev mailing list
jbosscache-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/jbosscache-dev
19 years, 2 months
reworking CacheMarshaller200.marshallObject() to a switch statement?
by Galder Zamarreno
Looking at CacheMarshaller200.marshallObject, we seem to have quite a big if chain. For BETA2, I was thinking that we could create a static Map<Class, int> containing class names and magic numbers, similar to the work we did for MethodDeclarations.
public static final Map COLLECTIONS = new HashMap();
static
{
COLLECTIONS.put(ArrayList.class, MAGICNUMBER_ARRAY_LIST);
}
...
int i = COLLECTIONS.get(o.getClass());
switch(i)
{
Case MAGICNUMBER_ARRAY_LIST: ...
}
Also, the instanceof if statements could be translated into this, specially bearing in mind that when we unmarshall, we create new instances of the class we compared for in the instanceof statement, so we would be back to the problem we had with the customer collections.
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
19 years, 2 months
RE: Inputs on aopc
by Brian Stansberry
Kabir Khan wrote:
>> <snip>
>>
>>>> aopclasspath -- Huh???
>>> A folder to the annotated classes
>>
>> How is that different from src or classpath?
>
> By default we do not browse for annotations (it is
> time-consuming to look inside every file), so if you have
> annotations you want parsed you need to specify them.
>
OK, that makes sense. When I reread the doc and actually followed the
link to the "Annotation Bindings" chapter I figured that was it. ;) But
this is confusing:
"aopclasspath - This should mirror your class path and contain all
JARs/directories..."
The "mirror class path" thing threw me off, as to me "mirror" implies
"duplicate". Perhaps
"aopclasspath -- A subset of the JARs/directories in classpath or
classpathref that includes all JARs/directories that may have annotated
aspects (See Chapter "Annotation Bindings"). The AOPC compiler will
browse each class file in this path to determine if any of them are
annotated with @Aspect. Browsing for annotations is time consuming, so
only classes in the JARs/directories listed in aopclasspath will be
scanned."
The next sentence "This gets used for the jboss.aop.class.path optional
system property which is described in the "Command Line" section." is
really confusing. Is it necessary? (No need to reply to that question.
Or to any of this :) )
> btw src, (which I agree is craply named, but created before
> my time) specifies the files that you want to transform -
> there may be a LOT of other things on the classpath which you do not
> want to transform...
>
Yeah, it's pretty clear what it is and how to use it (other than the
docs should say "A directory containing class files to be transformed",
not just "containing files". There's just a mental disconnect because
of the name that I expect affects people looking at the PojoCache
examples.
> IIRC you can specify multiple src entries, but I agree using
> includes would be nicer
Yep, you can have multiple src entries. You've got includes as well;
it's just that for some reason they are at the same level as src rather
than being nested.
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
Ph: 510-396-3864
skype: bstansberry
19 years, 2 months
RE: Inputs on aopc
by Brian Stansberry
Kabir Khan wrote:
> I've created a JIRA issue for these
>
> http://jira.jboss.com/jira/browse/JBAOP-355
>
Great. :)
>> -----Original Message-----
>> From: Brian Stansberry [mailto:brian.stansberry@jboss.com]
<snip>
>> aopclasspath -- Huh???
> A folder to the annotated classes
How is that different from src or classpath?
- Brian
19 years, 2 months
RE: [jbosscache-dev] PojoCache annotations and marker interfaces
by Brian Stansberry
FYI on a issue related to the 2 different jar versions:
Starting in 1.4.0 when the 2 jar versions appeared, we (aka me) began
checking both versions of the jar into the relevant release's lib dir in
repository.jboss.com. This was not correct -- by doing that I put both
jars on the AS build classpath. The build would only copy
jboss-cache.jar into the AS distribution, but both would be on the
classpath during build.
4.2 is JDK 5 based, so now jboss-cache-jdk50.jar is pulled into the
distro. The other shouldn't be on the classpath during build, as having
it there masks errors. Unfortunately, AFAICT the only way to not have it
be on the classpath is to not have it in the repository. So, beginning
with 1.4.1.SP1 I'm not going to put the jboss-cache.jar in the
repository anymore.
This is cleaned up in 2.0 where the PojoCache and standard cache stuff
are in separate jars. No more having two jars with the same stuff except
a few differences. :)
jbosscache-dev-bounces(a)lists.jboss.org wrote:
> Kabir Khan wrote:
>> At the AOP-level, the flavour of JBoss AOP deployed determines if we
>> look for annotationc'ed interfaces (jdk 1.4), or real annotations
>> (jdk 5), so there is not a lot that can be done in this regard at
>> present...
>>
>
> I'm not sure I follow. In the 4.2 testsuite I had a
> jboss-aop.xml that included prepare expressions and
> SubjectImpl mixin introductions with pointcuts for both
> @org.jboss.cache.aop.AopMarker (the marker interface) and
> @org.jboss.cache.aop.annotation.PojoCacheable annotations.
> Different test classes used each approach. For the classes
> that used the marker interfaces, I first used annotationc.
> Then aopc was run against all the classes.
>
> This all worked fine until I found that the AS build had both
> flavors of jboss-cache.jar on the classpath. When I fixed
> that, then annotationc failed because the marker interfaces
> weren't on the classpath anymore.
> But when all the classes were there, it all worked fine.
>
>> If you have some suggestions for aopc, please send them my way :-)
>> I've been thinking of tidying it up before we get AOP 2.0.0.GA out
>>
>
> Sure; will do. Sorry for the rant. The missing classes thing pissed me
> off and aopc got some spillover venom.
>
> - Brian
>
>>> -----Original Message-----
>>> From: jbosscache-dev-bounces(a)lists.jboss.org
>>> [mailto:jbosscache-dev-bounces@lists.jboss.org] On Behalf Of Brian
>>> Stansberry Sent: 02 February 2007 20:24
>>> To: jbosscache-dev(a)lists.jboss.org
>>> Subject: [jbosscache-dev] PojoCache annotations and marker
>>> interfaces
>>>
>>> I see that in 1.4.x, we have this lovely situation:
>>>
>>> jboss-cache.jar includes the o.j.c.aop.(InstanceOf)AopMarker marker
>>> interfaces. jboss-cache-jdk5.jar also includes the
>>> o.j.c.aop.annotation.(InstanceOf)AopMarker annotations, but does not
>>> include the marker interfaces.
>>>
>>> Thus, in AS 4.2, if I want to support both annotation styles for
>>> FIELD granularity replication, there is no JBC jar that I can
>>> integrate to allow it. I'd like to support both, since it seems user
>>> friendly to support the different methods the PojoCache examples
>>> use. (Note: we're deprecating the redundant
>> o.j.web.tomcat.tc5.session.AopMarker marker
>>> interfaces, so good support of the standard PojoCache markers is
>>> even more important.)
>>>
>>> If 1.4.1.SP1 isn't in QA yet (I saw another fix this AM) can we fix
>>> this? I see no reason not to include the marker interfaces in the
>>> jboss-cache-jdk5.jar. It's not like they don't work in JDK 5 or
>>> something.
>>>
>>> This isn't critical, but this whole area is really user unfriendly
>>> (don't get me started on the atrocious aopc) and doing what we can
>>> to make it easier is a good thing.
19 years, 2 months
Inputs on aopc
by Brian Stansberry
Hey Kabir,
Some hopefully more constructive inputs on aopc:
1) It appears that if any jboss-aop.xml is on the classpath defined by
the "classpath" element, it gets applied. In the testsuite, I used a
broad classpath, and it picks up a file with the same bindings as the
file I name in aoppath. This causes aopc to fail; javaassist complains
about seeing the same field twice. The workaround is to not specify
anything in aoppath. But that's confusing. Seems to me that if aoppath
is used, only bindings specified on the files in that path should be
applied.
2) When trying to figure the above out, I look at the Reference Guide
and then get really confused.
classpath/classpathref -- simple enough.
src -- isn't really source (i.e. .java) it's class files. It's
basically a subset of classpath, right? This isn't spelled out clearly
in the docs and the name of the element is confusing.
aopclasspath -- Huh???
3) Also, having "include" as a nested element within src would be more
consistent with normal ant usage.
Cheers,
Brian Stansberry
Lead, AS Clustering
JBoss, a division of Red Hat
Ph: 510-396-3864
skype: bstansberry
19 years, 2 months
RE: [jbosscache-dev] PojoCache annotations and marker interfaces
by Brian Stansberry
Kabir Khan wrote:
> At the AOP-level, the flavour of JBoss AOP deployed
> determines if we look for annotationc'ed interfaces (jdk
> 1.4), or real annotations (jdk 5), so there is not a lot that
> can be done in this regard at present...
>
I'm not sure I follow. In the 4.2 testsuite I had a jboss-aop.xml that
included prepare expressions and SubjectImpl mixin introductions with
pointcuts for both @org.jboss.cache.aop.AopMarker (the marker interface)
and @org.jboss.cache.aop.annotation.PojoCacheable annotations.
Different test classes used each approach. For the classes that used the
marker interfaces, I first used annotationc. Then aopc was run against
all the classes.
This all worked fine until I found that the AS build had both flavors of
jboss-cache.jar on the classpath. When I fixed that, then annotationc
failed because the marker interfaces weren't on the classpath anymore.
But when all the classes were there, it all worked fine.
> If you have some suggestions for aopc, please send them my
> way :-) I've been thinking of tidying it up before we get AOP
> 2.0.0.GA out
>
Sure; will do. Sorry for the rant. The missing classes thing pissed me
off and aopc got some spillover venom.
- Brian
>> -----Original Message-----
>> From: jbosscache-dev-bounces(a)lists.jboss.org
>> [mailto:jbosscache-dev-bounces@lists.jboss.org] On Behalf Of Brian
>> Stansberry Sent: 02 February 2007 20:24
>> To: jbosscache-dev(a)lists.jboss.org
>> Subject: [jbosscache-dev] PojoCache annotations and marker interfaces
>>
>> I see that in 1.4.x, we have this lovely situation:
>>
>> jboss-cache.jar includes the o.j.c.aop.(InstanceOf)AopMarker marker
>> interfaces. jboss-cache-jdk5.jar also includes the
>> o.j.c.aop.annotation.(InstanceOf)AopMarker annotations, but does not
>> include the marker interfaces.
>>
>> Thus, in AS 4.2, if I want to support both annotation styles for
>> FIELD granularity replication, there is no JBC jar that I can
>> integrate to allow it. I'd like to support both, since it seems user
>> friendly to support the different methods the PojoCache examples
>> use. (Note: we're deprecating the redundant
> o.j.web.tomcat.tc5.session.AopMarker marker
>> interfaces, so good support of the standard PojoCache markers is
>> even more important.)
>>
>> If 1.4.1.SP1 isn't in QA yet (I saw another fix this AM) can we fix
>> this? I see no reason not to include the marker interfaces in the
>> jboss-cache-jdk5.jar. It's not like they don't work in JDK 5 or
>> something.
>>
>> This isn't critical, but this whole area is really user unfriendly
>> (don't get me started on the atrocious aopc) and doing what we can to
>> make it easier is a good thing.
>>
>> Brian Stansberry
>> Lead, AS Clustering
>> JBoss, a division of Red Hat
>> Ph: 510-396-3864
>> skype: bstansberry
>>
>> _______________________________________________
>> jbosscache-dev mailing list
>> jbosscache-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/jbosscache-dev
19 years, 2 months
