I've been profiling JBoss AS web session replication, and one of the significant hits I'm seeing is from calls to File.exists() from FileCacheLoader (see attached). In turn, those calls are due calls from LegacyActivationInterceptor, particularly the removeNodeFromCacheLoader() method, which is called with every invocation.[1] Many of these calls are on the most critical path[2] so speeding them can have large implications for overall cluster performance. I'm wondering if we can be smarter here and avoid most calls to removeNodeFromCacheLoader()? Is it a correct statement that, logically, with passivation the only time it would make sense to remove a node from the cache loader is if that invocation loaded data from the cache loader? The node remove is allowed because the in-memory data has become complete; the only way previously incomplete data can become complete is if a request has loaded data. So, if we can pass back to the activation interceptor info on whether a load has occurred (e.g. via a simple boolean flag in InvocationContext), we can avoid most calls to CacheLoader.exists(). Thoughts?

[1] ActivationInterceptor would have the same problem if I were using MVCC. [2] Up calls from JGroups. NAKACK only allows one such call per peer at a time, and FC blocks web request threads based on the handling of these calls, so the speed of these calls is the most critical path in the whole system. -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat brian.stansberry(a)redhat.com < websessreplhotspots .png>_______________________________________________ jbosscache-dev mailing list jbosscache-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jbosscache-dev

Manik Surtani wrote: > On 24 Nov 2008, at 17:45, Brian Stansberry wrote: >> I've been profiling JBoss AS web session replication, and one of >> the significant hits I'm seeing is from calls to File.exists() >> from FileCacheLoader (see attached). In turn, those calls are due >> calls from LegacyActivationInterceptor, particularly the >> removeNodeFromCacheLoader() method, which is called with every >> invocation.[1] Many of these calls are on the most critical >> path[2] so speeding them can have large implications for overall >> cluster performance. >> >> I'm wondering if we can be smarter here and avoid most calls to >> removeNodeFromCacheLoader()? >> >> Is it a correct statement that, logically, with passivation the >> only time it would make sense to remove a node from the cache >> loader is if that invocation loaded data from the cache loader? >> The node remove is allowed because the in-memory data has become >> complete; the only way previously incomplete data can become >> complete is if a request has loaded data. So, if we can pass back >> to the activation interceptor info on whether a load has occurred >> (e.g. via a simple boolean flag in InvocationContext), we can >> avoid most calls to CacheLoader.exists(). >> >> Thoughts? > Makes sense. See > https://jira.jboss.org/jira/browse/JBCACHE-1446 That helps. I see now though that there are also calls via the tx prepare processing -- LegacyActivationInterceptor $ActivationModificationsBuilder.handlePutCommand() (see attached). I'm not sure exactly what that logic is doing, so I won't try to comment on whether tracking whether nodes were loaded in the TransactionContext makes sense. :)

I'm working to eliminate the biggest hotspot in the attached graphic; once I do I'll test w/ CR1 vs CR2 to get a feel for how much JBCACHE-1446 improved things. ...done... w/ 3.0.1.CR1: 3677 req/sec w/ 3.0.1.CR2: 3955 req/sec no CL at all: 4930 req/sec (w/ CR2; CL config is commented out)

Manik Surtani wrote: > On 26 Nov 2008, at 04:28, Brian Stansberry wrote: >> I'm not sure exactly what that logic is doing, so I won't try to >> comment on whether tracking whether nodes were loaded in the >> TransactionContext makes sense. :) > The purpose of this visitor is to build a List<Modification> from a > List<WriteCommand> of commands that have happened in the > transaction, which then gets passed to the CacheLoader.prepare() > call. > It does make sense to track load information in the transaction > context as well, so let me re-open the JIRA. >> I'm working to eliminate the biggest hotspot in the attached >> graphic; once I do I'll test w/ CR1 vs CR2 to get a feel for how >> much JBCACHE-1446 improved things. >> >> ...done... >> >> w/ 3.0.1.CR1: 3677 req/sec >> w/ 3.0.1.CR2: 3955 req/sec >> no CL at all: 4930 req/sec (w/ CR2; CL config is commented out) w/ latest snapshot: 4476 req/sec

:-) No time today to look at profiler to see if there's any obvious explanation for the diff vs. no CL at all. > Not bad. So a 7.5% increase in throughput when moving to CR2. > Cheers > Manik -- Brian Stansberry Lead, AS Clustering JBoss, a division of Red Hat brian.stansberry(a)redhat.com