Infinispan Server playing catch up :(
by Galder Zamarreño
Hey,
Infinispan Server CI is failing because REST cache store is not installed. I guess we need to modify the CI script to build REST cache store before hand too? [1]
Also, while trying to replicate some JIRAs in Server, I've spotted two errors [2] and once that was fixed by adding the dependency to the testsuite pom, then [3]. The fix is simple, just add these dependencies to testsuite/example-configs/pom.xml:
<dependency>
<groupId>org.infinispan.protostream</groupId>
<artifactId>protostream</artifactId>
</dependency>
<dependency>
<groupId>org.infinispan</groupId>
<artifactId>infinispan-remote-query-client</artifactId>
</dependency>
However, this is very frustrating and slows down resolution of other issues. This fix is something that should have done when remote querying was added, but it wasn't because the CI/PR integration didn't caught it.
So, we need to rethink CI/PR integration in such way that whenever a PR is sent to infinispan/infinispan, all potentially depending CIs need to run, which are:
- infinispan/infinispan
- infinispan/infinispan-server
- infinispan/cachestore-*
Until that happens, infinispan/infinispan-server and infinispan/cachestore-* are always gonna be playing catch up :(
WRT REST cache store dependency miss, when a PR is sent to infinispan/infinispan-server, it probably needs to build (but not test) infinispan/infinispan and infinispan/cachestore-* to make sure all the latest artifacts are available.
Does this make sense? Is this doable in our TeamCity installation?
Cheers,
[1] http://ci.infinispan.org/viewLog.html?buildId=3141&buildTypeId=bt11&tab=b...
[2] https://gist.github.com/galderz/61985831e87780cb2ca2
[3] https://gist.github.com/galderz/6897fd8ddfa187754b36
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
10 years, 9 months
Cache store for MapDB (what used to be JDBM)
by Randall Hauch
Has anyone looked at writing a cache store that uses MapDB? It provides Maps, Sets and Queues backed by disk storage or off-heap memory, with MVCC and (non-JTA) transactions. The author previously wrote JDBM (multiple versions), and has recently ventured out on his own to focus on MapDB full-time. It's only at 0.9.5, but progressing quite nicely. I've been looking at it for other uses, and quite enjoy it.
http://mapdb.org
10 years, 9 months
Generify MarshalledEntryImpl ?
by Galder Zamarreño
Hi,
If MarshalledEntry is generalized with <K, V>, shouldn't MarshalledEntryImpl be too?
Imagine I have:
CacheWriter<Integer, String> boundedStore = new SingleFileStore();
Trying to write throws a unchecked assinment warning when it shouldn't since I'm using the <K, V> and types expected by the CacheWriter:
boundedStore.write(new MarshalledEntryImpl(1, "v1", null, getMarshaller()));
Cheers,
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
10 years, 9 months
SNAPSHOT dependencies on protostream
by Mircea Markus
Hi,
Whilst having dependencies on SNAPSHOT is not generally good, we do allow SNAPSHOT dependencies between our own components: e.g. cache stores' upstream depend on an SNAPSHOT of Infinispan core. I think the rule should still apply for protostream, i.e. on master, the query should be allowed to depend on protostream-SNAPSHOT, assuming protostream doesn't have external SNAPSHOT dependencies (it doesn't). Otherwise we end up releasing a new protostream whenever e.g. a bug gets fixed and query needs it - way too often.
Of course we shouldn't allow any SNAPSHOT dependencies during the release.
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 9 months
Moving docs
by Manik Surtani
Guys
So one of the proposed plans is to move Infinispan documentation out of Confluence, and store them as AsciiDoc in git alongside the code. Scripts will be provided to export AsciiDoc to HTML (for web), PDF (for download), possibly even MOBI for Kindles and eBook readers. As we have different branches for major versions of Infinispan (5.0.x, 5.1.x, etc) docs too will be stored alongside their relevant versions.
One question I have is splitting up docs. Now that we have several different repositories for Infinispan, I presume the docs too will get split up? Note that this is only for Infinispan 6.0, since prior to Infinispan 6.0, all code was in one single repo.
Any thoughts/concerns?
Cheers
Manik
--
Manik Surtani
10 years, 9 months
Replicated sync vs async caches after ISPN-2473
by Giovanni Meo
Hi infinispan-dev,
in order to chase an issue i have reported earlier i started to go through the
code path executed by a replicated sync cache when adding a key to the
ConcurrentMap. While walking through the
org.infinispan.statetransfer.StateTransferInterceptor:handleWriteCommand method
i run into a pointer to ISPN-2473.
Now what this enhancement has done is to transform the sync writes to async ones
to avoid deadlocks. Now the question i have, from a newbie point of view, and
sorry if it's trivial, what is now the difference between an async replicated
cache and a sync one? Seems after this enhancement there is not much difference
for non-transactional caches is that correct?
Thanks,
Giovanni
--
Giovanni Meo
Via del Serafico, 200 Telephone: +390651644000
00142, Roma Mobile: +393480700958
Italia Fax: +390651645917
VOIP: 8-3964000
“The pessimist complains about the wind;
the optimist expects it to change;
the realist adjusts the sails.” -- Wm. Arthur Ward
IETF credo: "Rough consensus and running code"
10 years, 9 months
Expiration element of LevelDB cache store redundant?
by Martin Gencur
Hi,
let's look at configuration of LevelDB in Infinispan Server:
https://gist.github.com/mgencur/6520948
After investigating what the <expiration> configuration element is good
for, I think it's an implementation detail and should be removed.
Every time a mortal entry is stored in the cache, it's put (in internal
format of the cache store - ExpiryEntry) in an expiryEntryQueue whose
size is specified in the configuration through "queue-size" attribute.
The path attribute (location of dbExpired) is only used when purge() is
called on the cache store. At that moment, all entries that are in the
expiryEntryQueue are flushed into dbExpired (location specified through
"path"), all the elements from dbExpired are read and if they have
expiry time shorter than current time, they are removed from the primary
cache store's location, i.e. they are purged.
IMO, the location where intermediate (ExpiryEntry) are stored is an
implementation (users don't have access to it anyway) detail and the
queue-size attribute is a black-box users will never know how to set it.
I would suggest hiding these implementation details and simplify the
configuration. There are way too many configuration elements already.
WDYT?
Thanks
Martin
10 years, 9 months
persistence thread pool
by Mircea Markus
Hi,
With the new store API it is possible to iterate over the entries in a store in parallel. Also the purging of the expired entries happen in parallel as well.
I' like to add a persistenceExecutor (global component) to define the thread pool that would be used for these parallel operations.
How does that sound?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 9 months
HotRod client dependencies in 6.0.0.Alpha4
by Tristan Tarrant
Hi all,
at the beginning of the 6.0.0 cycle I worked to reduce the number of
dependencies required by the hotrod client. Unfortunately, with the
introduction of remote querying, the number of dependencies has gone up
again:
+- org.infinispan:infinispan-commons:jar:6.0.0.Alpha4:compile
+- org.jboss.marshalling:jboss-marshalling-river:jar:1.3.15.GA:compile
+- commons-pool:commons-pool:jar:1.6:compile
+- org.infinispan:infinispan-query-dsl:jar:6.0.0.Alpha4:compile
+- org.infinispan:infinispan-remote-query-client:jar:6.0.0.Alpha4:compile
| \- com.google.protobuf:protobuf-java:jar:2.5.0:compile
+- log4j:log4j:jar:1.2.16:compile
+- net.jcip:jcip-annotations:jar:1.0:compile
we should strive to remove the unnecessary ones (jcip, log4j) and make
the ones needed for remote query optional.
What do you think ?
Tristan
10 years, 9 months
