Design change in Infinispan Query
by Sanne Grinovero
Hello all,
currently Infinispan Query is an interceptor registering on the
specific Cache instance which has indexing enabled; one such
interceptor is doing all what it needs to do in the sole scope of the
cache it was registered in.
If you enable indexing - for example - on 3 different caches, there
will be 3 different Hibernate Search engines started in background,
and they are all unaware of each other.
After some design discussions with Ales for CapeDwarf, but also
calling attention on something that bothered me since some time, I'd
evaluate the option to have a single Hibernate Search Engine
registered in the CacheManager, and have it shared across indexed
caches.
Current design limitations:
A- If they are all configured to use the same base directory to
store indexes, and happen to have same-named indexes, they'll share
the index without being aware of each other. This is going to break
unless the user configures some tricky parameters, and even so
performance won't be great: instances will lock each other out, or at
best write in alternate turns.
B- The search engine isn't particularly "heavy", still it would be
nice to share some components and internal services.
C- Configuration details which need some care - like injecting a
JGroups channel for clustering - needs to be done right isolating each
instance (so large parts of configuration would be quite similar but
not totally equal)
D- Incoming messages into a JGroups Receiver need to be routed not
only among indexes, but also among Engine instances. This prevents
Query to reuse code from Hibernate Search.
Problems with a unified Hibernate Search Engine:
1#- Isolation of types / indexes. If the same indexed class is
stored in different (indexed) caches, they'll share the same index. Is
it a problem? I'm tempted to consider this a good thing, but wonder if
it would surprise some users. Would you expect that?
2#- configuration format overhaul: indexing options won't be set on
the cache section but in the global section. I'm looking forward to
use the schema extensions anyway to provide a better configuration
experience than the current <properties />.
3#- Assuming 1# is fine, when a search hit is found I'd need to be
able to figure out from which cache the value should be loaded.
3#A we could have the cache name encoded in the index, as part
of the identifier: {PK,cacheName}
3#B we actually shard the index, keeping a physically separate
index per cache. This would mean searching on the joint index view but
extracting hits from specific indexes to keep track of "which index"..
I think we can do that but it's definitely tricky.
It's likely easier to keep indexed values from different caches in
different indexes. that would mean to reject #1 and mess with the user
defined index name, to add for example the cache name to the user
defined string.
Any comment?
Cheers,
Sanne
12 years, 4 months
singleton @Listeners
by Mircea Markus
This is a problem that pops up constantly:
User: "I add a listener to my distributed/replicated cache but this gets invoked numOwners times - can I make that to be invoked only once cluster wise?"
Developer: "Yes, you can! You have to do that and that..."
What about a "singleton" attribute on the Listener? Would make the reply shorter:
Developer: "Use @Listener(singleton=true)"
Cheers,
Mircea
12 years, 10 months
Re: [infinispan-dev] Removing Infinispan dependency on the Hibernate-Infinispan module in 4.x
by Galder Zamarreño
Scott, what do you suggest doing instead then? Without the commands, evictAll invalidation won't work.
Are you suggesting that I revert back to using the cache as a notification bus so that regions are invalidated?
On Feb 8, 2012, at 4:13 PM, Scott Marlow wrote:
> http://lists.jboss.org/pipermail/infinispan-dev/2012-February/010125.html has more context.
>
> Since there are no easy/quick fixes that can be applied at this time, to remove the AS7 Infinispan dependency on the Hibernate-Infinispan module, I think we should avoid depending on the service loader way to supply the custom commands (in the Hibernate-Infinispan module), at least until this can be addressed elsewhere.
>
> I propose that the Hibernate-Infinispan second level cache should not use the Service Loader to pass custom commands into Infinispan. If we agree, I'll create a jira for this.
>
> Scott
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
13 years
Testsuite error jiras
by Galder Zamarreño
When you open JIRAs to fix randomly failing tets, i.e. https://issues.jboss.org/browse/ISPN-2102, please make sure the test is disabled too.
This avoids confusion with the tests that have been identified and the ones that have not been.
Cheers,
--
Galder Zamarreño
Sr. Software Engineer
Infinispan, JBoss Cache
13 years, 7 months
TOB/TOA integration
by Mircea Markus
Hi guys,
I've just had a chat with the CloudTM team(CC) around the integration of TOB/TOA[1] into Infinispan. Here are some points:
- the TOB and TOA code has been reviewed in detail by us. The only part missing is the state transfer integration
- there's not a lot of sense in integrating TOB/TOM over the existing state transfer as we would not back port that to 5.1 and it would be dropped in 5.2
- CloudTM would rebase the TOB/TOA work on top of the alpha NBST[2](ATM planned at the end of next week/3 Aug) and we'll integrate that
- first releases of the TOB/TOA would be marked as experimental in 5.2
How does that sound?
Cheers,
Mircea
[1] TOA used to be referred to as TOM (M from multicast). In JGroups terminology that's an Anycast, so we decided to be consistent with that and use TOAnycast.
[2] https://community.jboss.org/wiki/Non-blockingStateTransfer
13 years, 9 months
drafts, sketches and pull requests
by Sanne Grinovero
Hi all,
I see that on github there are many pull requests for Infinispan, but
a good amount of them is not to be merged, needing more feedback or
pending for other reasons.
Rather than keeping things open, shouldn't we track these on JIRA? I'd
like to interpret the "pull requests" as things that should be
reviewed for immediate merging.
Shall we close those pulls which need further work?
Cheers,
Sanne
13 years, 9 months
Double+ buffering during value marshalling
by Sanne Grinovero
While debugging around the Infinispan/JBMar integration I noticed that
Infinispan defines in-memory buffers wrapped as byte streams, and uses
JBoss Marshaller to write to these.
JBoss Marshaller also buffers writes, and flushes when needed or on
demand; I did already know of both, but just realized that these two
levels of buffering are both applied when serializing instances in a
org.infinispan.marshall.MarshalledValue; I guess we could do better
avoiding buffering and have JBMAR write straight away? It would also
mean less array resizing, as we are often able to allocate the right
size in one shot.
I gave a look into RiverMarshaller, and this didn't look like a simple
task as _byte[] buffer_ is the main element around which most of the
code revolves (in it's superclass SimpleDataOutput).
I'm wondering if SimpleDataOutput wouldn't be simpler by just writing
to a org.jboss.marshalling.ByteOutput (and have an optional buffering
implementation), or if Infinispan should use River in a different way.
Regards,
Sanne
13 years, 10 months
releasing 5.2.Alpha3
by Mircea Markus
Hi,
Unless something critical appears - not very likely but there are still some 3-4 test failures on core - NBST is ready to be released by tomorrow.
It will have a limitation though: due to CH changes, the HR clients might loose the smart routing capability - this will be addressed in the next release.
I will release 5.2.Alpha3 tomorrow PM.
Cheers,
Mircea
13 years, 10 months
NPE with legacy config in 5.2
by Ales Justin
I updated AS7 upstream master to run against Ispn upstream master.
But now I get this NPE:
I guess we should check for null "cacheLoader" here?
private static CacheLoaderConfig getLoaderConfig(LoaderConfiguration loader, CacheLoader cacheLoader) {
if (cacheLoader.getClass().isAnnotationPresent(CacheLoaderMetadata.class)) { // << ----- HERE
return Util.getInstance(cacheLoader.getClass().getAnnotation(CacheLoaderMetadata.class).configurationClass());
} else {
---
01:47:24,176 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 32) MSC00001: Failed to start service jboss.infinispan.capedwarf.dist: org.jboss.msc.service.StartException in service jboss.infinispan.capedwarf.dist: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.start() on object of type CacheLoaderManagerImpl
at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:87)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [classes.jar:1.6.0_33]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [classes.jar:1.6.0_33]
at java.lang.Thread.run(Thread.java:680) [classes.jar:1.6.0_33]
at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA.jar:2.0.0.GA]
Caused by: org.infinispan.CacheException: Unable to invoke method public void org.infinispan.loaders.CacheLoaderManagerImpl.start() on object of type CacheLoaderManagerImpl
at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:238)
at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:889)
at org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:639)
at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:628)
at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:531)
at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:174)
at org.infinispan.CacheImpl.start(CacheImpl.java:523)
at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:661)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:554)
at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:568)
at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:125)
at org.jboss.as.clustering.infinispan.DefaultEmbeddedCacheManager.getCache(DefaultEmbeddedCacheManager.java:116)
at org.jboss.as.clustering.infinispan.subsystem.CacheService.start(CacheService.java:78)
at org.jboss.as.clustering.msc.AsynchronousService$1.run(AsynchronousService.java:82)
... 4 more
Caused by: org.infinispan.CacheException: Unable to start cache loaders
at org.infinispan.loaders.CacheLoaderManagerImpl.start(CacheLoaderManagerImpl.java:161)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [classes.jar:1.6.0_33]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) [classes.jar:1.6.0_33]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) [classes.jar:1.6.0_33]
at java.lang.reflect.Method.invoke(Method.java:597) [classes.jar:1.6.0_33]
at org.infinispan.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:236)
... 17 more
Caused by: java.lang.NullPointerException
at org.infinispan.configuration.cache.LegacyConfigurationAdaptor.getLoaderConfig(LegacyConfigurationAdaptor.java:286)
at org.infinispan.configuration.cache.LegacyConfigurationAdaptor.adapt(LegacyConfigurationAdaptor.java:251)
at org.infinispan.loaders.CacheLoaderManagerImpl.createCacheLoader(CacheLoaderManagerImpl.java:281)
at org.infinispan.loaders.CacheLoaderManagerImpl.start(CacheLoaderManagerImpl.java:147)
... 22 more
13 years, 10 months
