Releasing Infinispan 6.0.0.Beta2
by Mircea Markus
The 6.0.0.Beta2 is scheduled for Friday 27 October.
Here's the list of open bugs for it, please tackle them based on priority: http://goo.gl/NMoXf4
(Adrian, the query related bugs will be further refined after reviewing them with Sanne)
It's Vladimir turn to release it this time ;)
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 8 months
Read Committed Distributed Cache Concerns
by William Burns
I was recently refactoring code dealing with isolation levels and
found how ReadCommitted is implemented and I have a few concerns I
wanted to bring up.
ReadCommitted read operations work by storing a reference to the value
from the data store in its caller's context. Thus whenever another
transaction is committed that updates the data store value any context
that has that reference now sees the latest committed value. This
works well for Local and Replicated caches since all data stores are
updated with the latest value upon completion of the transaction.
However Distributed caches only the owners see the update in their
data store and thus any non owner will still have the old value they
previously read before the commit occurred.
This seems quite inconsistent that Distributed caches run in a mix of
Repeatable Read/Read Committed depending on what node and what key you
are using.
To operate properly we could track requests similar to how it works
for L1 so we can tell non owners to clear out their context values for
values they read remotely that they haven't updated (since Read
Committed writes should return the same written value). That seems
like quite a bit of additional overhead though.
I am wondering is it worth it to try to keep Read Committed isolation
level though? It seems that Repeatable Read would be simpler and most
likely more performant as you wouldn't need all the additional remote
calls to get it to work properly. Or is it okay that we have
different isolation levels for some keys on some nodes? This could be
quite confusing if a user was using a local and remote transaction and
a transaction may not see the other's committed changes when they
expect to.
What do you guys think?
- Will
P.S.
I also found a bug with Read Committed for all caches where if you do
a write that changes the underlying InternalCacheEntry to a new type,
that reads won't see subsequent committed values. This is caused
because the underlying data is changed to a new reference and a read
would still be holding onto a reference of the old InternalCacheEntry.
This can happen when using the various overridden put methods for
example. We should have a good solution for it, but may not be
required if we find that Read Committed itself is flawed beyond
saving.
10 years, 8 months
replacing the (FineGrained)AtomicMap with grouping
by Mircea Markus
Hi,
Most of the FGAM functionality can be achieved with grouping, by using the FGAM key as a grouping key.
The single bit that seems to be missing from grouping to equivalent the functionality of FGAM is obtaining all the entries under a single group. IOW a method like:
Map<K,V> groupedKeys = Cache.getGroup(groupingKey, KeyFilter);
This can be relatively easily implemented with the same performance as an AtomicMap lookup.
Some other differences worth mentioning:
- the cache would contain more entries in the grouping API approach. Not sure if this is really a problem though.
- in order to assure REPEATABLE_READ, the AM (including values) is brought on the node that reads it (does't apply to FGAM). Not nice.
- people won't be able to lock an entire group (the equivalent of locking a AM key). I don't think this is a critical requirement, and also can be worked around. Or added as a built in function if needed.
I find the idea of dropping FGAM and only using grouping very tempting:
- there is logic duplication between Grouping and (FG)AM (the locality, fine grained locking) that would be removed
- FGAM and AM semantic is a bit ambiguous in corner cases
- having a Cache.getGroup does make sense in a general case
- reduce the code base
What do people think?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 8 months
Migration of data between cache stores
by Galder Zamarreño
Hi all,
Re: https://issues.jboss.org/browse/ISPN-3318
Re: https://docs.jboss.org/author/display/ISPN/Rolling+upgrades
I've been thinking about this migration stuff, and I came to the conclusion that for anyone that wants to migrate cache store data and are using Hot Rod or REST, they should use existing rolling upgrade system. That'll give them the ability directly (thanks to cache.keySet() returning keys in cache store) to fetch all data from the cache store, in the old format, and be able to apply to new cluster, in new format, without the need of extra code to deal with cache store lower level details.
The big question is though, how to migrate cache store data for those embedded caches. If you extrapolate what I said above, the question might really be: how to do rolling upgrades of embedded/library mode caches.
The main problem here is that you could think of configuring the target cluster with cache store details of the source cluster, but this won't work: Infinispan 6.0 cannot read Infinispan 5.x cache stores. All the classes and formats for Infinispan 5.x are gone from the source code in Infinispan 6.0.
There's three ways to resolve this:
a) Bring back Infinispan 5.x classes to 6.0 to be able to read 5.x cache stores directly from 6.0.
b) Develop some code for Infinispan 5.x to dump cache store data into a file or somewhere in a given format and then read that from 6.0.
c) Use CLI to connect from new cluster to the old cluster and fetch all keys.
I think the easiest thing would be to do c), developing a CLISourceMigrator (implements org.infinispan.upgrade.SourceMigrator) to be able to read all keys from the source embedded cluster. However, I don't see anywhere in the CLI a command that returns back all keys. Upgrade with DUMPKEYS simply calls up the local SourceMigrator.recordKnownGlobalKeyset. We'd need such a command in Infinispan 5.x and 6.x, the former to send all keys and the latter to request them.
We'd also need a CLITargetMigrator which is pretty straightforward to do since once we have all keys, it's easy to loop through them and request them using CLI's Get operation.
@Tristan, you're the rolling upgrade meister, how does this sound? If we get this right, we'd not only get cache store migration, but also embedded/library cluster rolling upgrade.
Cheers,
--
Galder Zamarreño
galder(a)redhat.com
twitter.com/galderz
Project Lead, Escalante
http://escalante.io
Engineer, Infinispan
http://infinispan.org
10 years, 8 months
Query and dynamic discovery of types
by Sanne Grinovero
For Infinispan 6.0 we decided that the following issue is a bloker:
https://issues.jboss.org/browse/ISPN-2143 - Improve how different
indexed caches sync up on new indexed types
It really is important, and I'm a bit concerned that it was moved to
CR1 as it might not be trivial: for embedded query it needs to store a
reference to class definitions (potentially with classloader
information too), and for remote queries it needs to distribute the
indexing configuration schema.
Also I'm wondering if we shouldn't go even further: in Hibernate
Search the internal complexity of handling newly appearing types
(rather than statically configured types) is very hard.
In the next version we might need to drop this, so it would be great
if we could stabilize the Infinispan Query API right now in such a way
that it won't need this feature anymore.
I proposed it years ago but it was rejected because of usability
concerns.. would you still reject the idea to simply list the indexed
types at configuration time? If so, I think we need to explore
alternative solutions.
Note that I only want to remove the dependency to dynamic _Class_
definitions being added at runtime; we will still support dynamically
defined types so in case you need extreme flexibility like with remote
queries that will work, as long as you provide a smart bridge able to
reconfigure itself based on the dynamic metadata; I think that's a
cleaner approach as you would be directly in control of it rather than
having to workaround a design which was thought for a different
purpose.
Sanne
10 years, 8 months
Deprecated API removal
by Pedro Ruivo
Hello,
The following API is going to be removed in the next release (6.0.0.Beta1):
classes:
* org.infinispan.loaders.remote.wrapper.EntryWrapper
* org.infinispan.context.FlagContainer
methods:
* StreamingMarshaller.startObjectOutput
* AdvancedCache.getCacheEntry
* AtomicMapLookup.getAtomicMap
* DistributionManager.isLocal
* TxInterceptor.setStatisticsEnabled
* GridFile.delete
* ModuleLifecycle.cacheManagerStarting
* ModuleLifecycle.cacheStarting
* DeadlockDetectingLockManager.getLocallyInterruptedTransactions
packages:
* org.infinispan.config
Please update your code accordingly.
Thank you.
Regards,
Pedro Ruivo
10 years, 9 months
JBM version in ISPN 6.0
by Mircea Markus
Hi guys,
ISPN 6.0 final is scheduled for 11 Oct.
Is there going to be an JBM 2.0 final release prior that date?
Paul, what JBM version does 6.0 integration into AS?
Cheers,
--
Mircea Markus
Infinispan lead (www.infinispan.org)
10 years, 9 months
Fluent config could be type-safe
by Lincoln Baxter, III
Thought.
It seems very easy to do things with the fluent builder API that either
have no effect or have little effect - such as:
new ConfigurationBuilder().clustered().clustered().clustered()
While this is probably not a problem. The call to `.clustered()` could
return a sub-configuration interface that would ultimately return an
intermediate interface, only declaring methods that may be used to
configure a cluster - thus making such a declaration syntactically illegal.
Once a valid configuration state on this object (and its sub-objects) is
reached, the returned interface type would present the user with a
`ClusteredConfigurationBuilder`, e.g. a ConfigurationBuilder that is
already clustered, and cannot be used to configure a cluster.
I'm fully aware of how many interfaces this would require, but I also have
a proof of concept that has shown to be very practically successful in
eliminating user error for complex configurations, and has been well worth
the effort:
https://github.com/ocpsoft/rewrite/blob/master/api-tests/src/test/java/or...
The above test does not verify invalid configuration ordering because it is
actually a compilation error and cannot be tested with JUnit! But hopefully
you get the idea.
You can see the hierarchy of this (probably simpler than infinispan)
configuration builder API here:
https://github.com/ocpsoft/rewrite/tree/master/api/src/main/java/org/ocps...
Any class that starts with "Configuration..." is part of the relevant API.
Also, you could avoid an API-breaking change if provided a
TypesafeConfigurationBuilder that simply delegates to the current
ConfigurationBuilder API.
Thoughts?
--
Lincoln Baxter, III
http://ocpsoft.org
"Simpler is better."
10 years, 9 months
