Eviction overhaul
by Manik Surtani
Hello all.
I have finished my work with the eviction code in Infinispan, here is a
summary of what has happened.
From a user perspective (including API and configuration) as well as
a design overview, please have a look at
http://www.jboss.org/community/docs/DOC-13449
From an implementation perspective, have a look at the srcs of
FIFODataContainer and LRUDataContainer. These two classes are where
everything happens. The javadocs should explain the details, but in a
nutshell you can expect constant time operations for all puts, gets,
removes, iterations. :-)
Feedback on the impls would be handy. :-)
Cheers
--
Manik Surtani
Lead, JBoss Cache
http://www.jbosscache.org
manik(a)jboss.org
16 years, 11 months
ReplicateCommand
by Mircea Markus
Hi,
Some thought on ReplicateCommand.
It is also used to replicate a single command, which is inefficient, as
2 objects are unnecessarily created: the ReplicateCommand itself and an
array holding only one object.
Why not replicated the aggregated command directly?
Another thing about the name: even though it is correct, it sounds very
much like ReplicableCommand. I suggest renaming to CompositeCommand, or
ContainerCommand.
wdyt?
Cheers,
Mircea
17 years
Disabled tests
by Manik Surtani
So these are the tests currently disabled in trunk:
Gemini:trunk manik$ grep -R "Test.*enabled[ ]*=[ ]*false" * | grep -
v .svn | sed -e "s/:.*//"
src/test/java/org/infinispan/loader/jdbc/PooledConnectionFactoryTest.java
src/test/java/org/infinispan/loader/s3/S3CacheStoreIntegrationTest.java
src/test/java/org/infinispan/profiling/AbstractProfileTest.java
src/test/java/org/infinispan/profiling/MemConsumptionTest.java
src/test/java/org/infinispan/profiling/ProfileTest.java
src/test/java/org/infinispan/profiling/ProfileTest.java
src/test/java/org/infinispan/profiling/ProfileTest.java
src/test/java/org/infinispan/profiling/ProfileTestSlave.java
src/test/java/org/infinispan/profiling/TreeProfileTest.java
src/test/java/org/infinispan/test/fwk/TestNameVerifier.java
I think the only relevant ones are:
src/test/java/org/infinispan/loader/jdbc/PooledConnectionFactoryTest.java
src/test/java/org/infinispan/loader/s3/S3CacheStoreIntegrationTest.java
Mircea and Adrian - these are yours, I believe?
Cheers
--
Manik Surtani
Lead, JBoss Cache
http://www.jbosscache.org
manik(a)jboss.org
17 years
JMX stuff in infinispan
by Mircea Markus
implementation finished.
Following features were added, compared with JBossCache:
1) jmx domain care be specified now.
2) an MBeanServer lookup can be configured, to lookup the server on
which the components will be registered - in prev version the
PlatformMBeanServer was hardcoded (now it's only an default)
3) missing unit tests were added for all the annotated MBeans
4) jmx is now configured in two places:
- in global section: to enable exposure of shared information (rpc
manager info, cache manager info)
e.g <globalJmxStatistics enabled="true" jmxDomain="infinispan"
mBeanServerLookup="org.infinispan.jmx.PerThreadMBeanServerLookup" />
- for each cache, where cache specific info is configured(mainly
interceptors, which now are cache specific)
e.g. <jmxStatistics enabled="false"/>
Cheers,
Mircea
17 years
test failing on win
by Mircea Markus
Hi,
There are about 10 tests failing on WIN. On Linux there everything
passes, I'm looking into it.
Failed tests:
testUpdatingLastUsed(org.infinispan.container.SimpleDataContainerTest)
testUpdatingLastUsed(org.infinispan.container.LRUDataContainerTest)
testUpdatingLastUsed(org.infinispan.container.FIFODataContainerTest)
testLoadAndStoreWithLifespan(org.infinispan.loader.dummy.DummyInMemoryCacheStoreTest)
testLoadAndStoreWithLifespanAndIdle(org.infinispan.loader.dummy.DummyInMemoryCacheStoreTest)
testLoadAndStoreWithLifespan(org.infinispan.loader.decorators.ChainingCacheLoaderTest)
testLoadAndStoreWithLifespanAndIdle(org.infinispan.loader.jdbc.stringbased.JdbcStringBasedCacheStoreTest)
testLoadAndStoreWithIdle(org.infinispan.loader.bdbje.BdbjeCacheStoreIntegrationTest)
testLoadAndStoreWithLifespanAndIdle(org.infinispan.loader.jdbc.JdbcBinaryCacheStoreTest)
testLoadAndStoreWithIdle(org.infinispan.loader.jdbc.JdbcBinaryCacheStoreTest)
testLoadAndStoreWithLifespanAndIdle(org.infinispan.loader.s3.S3CacheStoreIntegrationTest)
testLoadAndStoreWithLifespanAndIdle(org.infinispan.loader.file.FileCacheStoreTest)
testLoadAndStoreWithIdle(org.infinispan.loader.file.FileCacheStoreTest)
testSTWithThirdWritingNonTxCache(org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest)
testSTWithWritingNonTxThread(org.infinispan.statetransfer.StateTransferCacheLoaderFunctionalTest)
Tests run: 618, Failures: 15, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] There are test failures.
Please refer to
C:\jboss\infinispan\zzzz_trunk_pristine\target\surefire-reports for the
individual test results.
[INFO]
------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3 minutes 11 seconds
[INFO] Finished at: Tue Mar 31 10:54:07 EEST 2009
[INFO] Final Memory: 61M/114M
[INFO]
------------------------------------------------------------------------
C:\jboss\infinispan\zzzz_trunk_pristine>svn status
? target
? infinispan.log
? Infinispan-BdbjeCacheStore
C:\jboss\infinispan\zzzz_trunk_pristine>svn update
At revision 49.
17 years
StateTransferControlCommand initialisation?
by Galder Zamarreno
Hi,
I'm looking at RemoteCommandFactory in Infinispan as part as mapping the
marshalling stuff to JBoss Marshalling and here's something that has
confused me:
fromStream() method says:
* Creates an un-initialized command. Un-initialized in the sense that
parameters will be set, but any components
* specific to the cache in question will not be set.
* <p/>
* You would typically set these parameters using {@link
org.infinispan.commands.CommandsFactory#initializeReplicableCommand(ReplicableCommand)}
* <p/>
And then you do the following:
case StateTransferControlCommand.METHOD_ID:
command = new StateTransferControlCommand();
((StateTransferControlCommand) command).init(rpcManager);
break;
Shouldn't ((StateTransferControlCommand) command).init(rpcManager) go
into
org.infinispan.commands.CommandsFactory#initializeReplicableCommand(ReplicableCommand).
initializeReplicableCommand() javadoc says:
* Initializes a {@link org.infinispan.commands.ReplicableCommand} read from
* a data stream with components specific to the target cache instance.
The rpcManager seems to be one those components?
Cheers,
--
Galder Zamarreño
Sr. Software Maintenance Engineer
JBoss, a division of Red Hat
17 years
Eviction in Infinispan
by Manik Surtani
Hello all.
So as you know eviction changed pretty substantially from JBC3 but the
overall design still remained the same. I'm proposing a radical
change to the design, which we briefly discussed in Neuchatel.
Here's a recap of what has changed already since JBC3:
1. ExpirationAlgorithm and ElementSizeAlgorithm dropped,
unnecessary. Leaves us with {FIFO, LRU, MRU, LFU}
2. Eviction queue design improved to O(1) for most operations on most
algorithms, except LFU (O(log n))
3. Expiration treated as a top-level construct where lifespans for
entries can be provided on a per-entry basis
4. Eviction watermarks are now exclusively size-based (numElements)
and not time-based.
5. MinTTL still supported.
Now here is how I want to change things further:
1. Drop MRU and LFU. I don't see MRU being useful except in very
specific edge cases. LRU somewhat useful but very expensive.
2. Re-introduce a time-based watermark. Brian, this is where your
thoughts are most useful - how do you see this being used? In
passivation?
3. Get rid of the eviction queues. Unnecessary duplication of keys.
The core datastructure can be refactoreed to provide an efficient,
threadsafe lookup for entries, where entries are also ordered for
eviction.
4. To support 3. and make it as efficient as possible, eviction may
only be an approximation. (e.g., if items 1 - 10 are added in order,
and 3 need to be removed based on FIFO, we may lose 7, 9 and 10
instead of 8, 9 and 10.). How much of a concern is this? (typical
tradeoff of accuracy vs efficiency)
Cheers
--
Manik Surtani
Lead, JBoss Cache
http://www.jbosscache.org
manik(a)jboss.org
17 years
Problems with test suite reports
by Manik Surtani
Mircea
I think this could be an issue around why Dominik forked Surefire on
JBC (we've used the stock Surefire plugin on Infinispan).
Basically, you have no visibility on failures in the Hudson reports:
http://hudson.jboss.org/hudson/view/HORIZON/job/Infinispan-trunk-JDK5/org...
This just identifies test method name, but not class name. It means
you need to dig through stack traces to get details. No biggie, but
would be much nicer if this was more explicit. :-)
Any idea why this is the case or how it could be fixed easily?
Cheers
--
Manik Surtani
Lead, JBoss Cache
http://www.jbosscache.org
manik(a)jboss.org
17 years
JDK 6 and Infinispan
by Adrian Cole
Hi, devs.
As you may know, JDK 5 is officially end of life this October. As that time
is not too far away, I thought we could consider how we want to address the
issue.
As I see it, we have a few choices:
1. set default binary to JDK 6 and produce a separate mvn artifact with
classifier jdk15 for compatibility.
2. set default binary to JDK 5 and produce a separate mvn artifact with
classifier jdk6 for compatibility.
3. do nothing and only produce a JDK 5 jar.
What are your thoughts?
-Adrian Cole
17 years
