March 2015 - infinispan-dev - Jboss List Archives

by William Burns

I am nearing completion of the new multi get command [1], allowing for more efficient retrieval of multiple keys at the same time. Thanks again to Radim for a lot of the initial work. In doing so though I want to make sure I get feedback on how we want the API to actually look, since this is much more difficult to change down the line. There are a few things in particular I wanted to discuss. 1. The actual name of the method. The main suggestions I have seen are getAll or getMany. I do like the naming of the former, however it seems a bit misleading (remember API is getAll(Set) since we are really getting a subset. So I am thinking the possibilities at this point are getAllOf, getAllFor or getMany. I am leaning maybe towards the last one (getMany). I am open to any suggestions though. 2. What should the return value be for this method. Currently this returns a Map<K, V> which makes sense when we retain these values in the local transactional context and is pretty nice and clean for end users. The other idea is to use a streaming API possibly returning an Iterable<CacheEntry<K, V>>. The streaming return value doesn't seem as intuitive to me, but opens us up for more efficient usage especially when there may be a lot of keys passed in (also results can be processed concurrently while retrieval is still occurring). I would lean towards returning the Map<K, V> value, however the next point could change that. 3. Where this method should live. Right now this method is on the BasicCache interface which is a parent of both Cache (embedded) and RemoteCache (remote). Unfortunately for remote clients to be as efficient as possible we really need a Streaming API so that we don't have multiple copies of data in memory at multiple places at the same time. For that reason I suggest we use the streaminig API for both or have a different API for remote vs embedded. Let me know what you guys think. Cheers, - Will [1] https://issues.jboss.org/browse/ISPN-2183

9 years, 9 months

7
17
0 / 0

Early Access builds for JDK 9 b55 and JDK 8u60 b08 are available on java.net

by Rory O'Donnell

Hi Galder, Early Access build for JDK 9 b55 <https://jdk9.java.net/download/> available on java.net, summary of changes are listed here <http://www.java.net/download/jdk9/changes/jdk9-b55.html> Early Access build for JDK 8u60 b08 <http://jdk8.java.net/download.html> is available on java.net, summary of changes are listed here. <http://www.java.net/download/jdk8u60/changes/jdk8u60-b08.html> Rgds,Rory -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland

9 years, 9 months

1
0
0 / 0

Grouping lucene query on Infinispan

by Raymond Rennell Douglass III

Hello all, This is my first time mailing this list, so please let me know if this is not the right place for this. I use a replicated and/or distributed cache to store some Objects and it is indexed using Infinispan's indexing via Hibernate Search. I'm using Infinispan 7.1.x. I wanted to use Lucene's GroupingSearch functionality, which I could not find anywhere in the Infinispan or Hibernate Search docs, so I used this: https://docs.jboss.org/infinispan/7.1/apidocs/org/infinispan/query/Search... I wrote code like so: SearchIntegrator searchIntegrator = Search.getSearchManager(cache).getSearchFactory(); IndexReader reader = searchIntegrator.getIndexReaderAccessor().open(ENTITY_CLASSES); IndexSearcher searcher = new IndexSearcher(reader); String groupedBy = ...; GroupingSearch groupingSearch = new GroupingSearch(groupedBy); TopGroups<BytesRef> result = groupingSearch.search(searcher, query, offset, limit); I noticed in the javadocs for the getSearchFactory method that it's deprecated (maybe just in Infinispan 7.0?). So I wanted to ask: 1. Is there are better way to perform a grouping search? 2. Will this API be available in future Infinispan versions? I personally think access to an IndexReader is very helpful for those strange edge cases to run custom Lucene queries. Thanks, -- Ray Douglass

9 years, 9 months

2
1
0 / 0

Infinispan Directory Provider moving from Hibernate Search, suggestions on names

by Gustavo Fernandes

Hi all, We are in process of migrating Hibernate Search's Infinispan Directory Provider to the Infinispan codebase. As you know, the Infinispan Lucene Directory has always been part of Infinispan, but not the layer (called 'Directory Provider') that glues it to the Hibernate Search core engine. With this move the goal is to facilitate maintenance and avoid circular dependencies between both projects. So what would be the new package name and the jboss module name for this component? I've created a preview PR [1] that is using 'org.infinispan.hibernate.search' and 'org.infinispan.directory-provider' for package and module respectively, but would like to hear more opinions. Thoughts? [1] https://github.com/infinispan/infinispan/pull/3322

9 years, 9 months

4
5
0 / 0

Status update weekly meeting

by Galder Zamarreño

Hi, Forgot to send this week's meeting log, here it is: http://transcripts.jboss.org/meeting/irc.freenode.org/infinispan/2015/inf... Cheers, -- Galder Zamarreño galder(a)redhat.com

9 years, 9 months

1
0
0 / 0

Random SocketTimeoutExceptions demonstrate we're not using Netty the right way

by Galder Zamarreño

Hi all, Summary: I've been debugging [1] and found that an unexpected operation throwing a SocketTE was being thrown due to the worker thread pool being too small, but the real problem is the fact that we can block within Netty's worker thread, something we're hoping to fix with Gustavo's work to implement [2]. The test in [1] was failing randomly as a result of this sequence of events: 1. Server configured with worker thread pool being 2 (in code, this is 2 * Runtime.getRuntime().availableProcessors()) 2. We emulate a server side operation hanging with a sleep injected by an interceptor. 3. Client gets SocketTimeoutException and retries the operation. 4. The retried operation hangs because it cannot acquire the lock. 5. Client gets SocketTimeoutException again, runs out of retries. 6. Test now executes an operation that should not throw a SocketTE but it gets it because the 2 worker threads are in use. The first is in the sleep, and the retried one waiting for the lock, so it cannot make any progress and hence ends up getting a SocketTE. You might think this can be solved by increasing the default worker thread pool size but you're just moving the problem around. You could have N operations that hang at the same time, e.g. if there's a long GC pause. So really, we need to stop blocking within the worker thread and implement [2]. I think we might have seen this issue already back in Infinispan 5, because we were already getting SocketTEs back then in another SocketTE test [3]. I wasn't able to replicate it locally at the time and solved it by ignoring it :| As far as ISPN-5314 is concerned, it can easily be solved by increasing the worker thread pool for that particular test, but we should revert that back when [2] is implemented. Cheers, [1] https://issues.jboss.org/browse/ISPN-5314 [2] https://issues.jboss.org/browse/ISPN-5083 [3] https://issues.jboss.org/browse/ISPN-2110 -- Galder Zamarreño galder(a)redhat.com

9 years, 9 months

2
2
0 / 0

Dealing with ownerless segments

by Galder Zamarreño

Hey Dan, Re: https://issues.jboss.org/browse/ISPN-5208 I've created a unit test based on Matej's feedback to replicate the issue [1]. Debugging through it, I've discovered that some topology updates are sent with segments that have no owner, hence when the key is hashed to that segment, it finds no owner for it. Looking at the logs, I see: 2015-03-24 10:29:57,855 4405 TRACE [org.infinispan.topology.CacheTopology] (remote-thread-main-NodeC-p15-t4:) Current consistent hash's routing table: 0: 0, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0, 10: 0, 11: 0, 12: 0, 13: 0, 14: 0, 15: 0, 16: 0, 17: 0, 18: 0, 19: 0, 20: 0, 21: 0, 22: 0, 23: 0, 24: 0, 25: 0, 26: 0, 27: 0, 28: 0, 29: 0, 30: 1, 31: 1, 32: 1, 33: 1, 34: 1, 35: 1, 36: 1, 37: 1, 38: 1, 39: 1, 40: 1, 41: 1, 42: 1, 43: 1, 44: 1, 45: 1, 46: 1, 47: 1, 48: 1, 49: 1, 50: 1, 51: 1, 52: 1, 53: 1, 54: 1, 55: 1, 56: 1, 57: 1, 58: 1, 59: 1 If having a segment that has no owners is valid scenario, what should we do about it? - Should the server avoid sending such topology unless all segments have owners? - Or should the client use round-robin as fallback when segment for key has no owner? - Or is this a bug? Cheers, [1] https://gist.github.com/galderz/17f1c4f21b4e7856aa6b -- Galder Zamarreño galder(a)redhat.com

9 years, 9 months

1
1
0 / 0

Cache Java 8 API proposal - First draft

by Galder Zamarreño

Hi all, We're starting to work on some prototypes for the Java 8 API coming up in Infinispan 8. I've written a design wiki for a replacement of our main embedded Cache interface that takes advantage of Java 8 improvements: https://github.com/infinispan/infinispan/wiki/Java-8-API-proposal It'd be great to hear your thoughts. Proposals for Java 8 versions for cache manager, remote cache and other external APIs are yet TBD. Regards, -- Galder Zamarreño galder(a)redhat.com

9 years, 9 months

3
6
0 / 0

FindBugs on infinispan

by Jakub Markos

Hi, out of curiosity, I ran FindBugs [1] (java static analysis tool) on infinispan. Attached are 2 output files (compressed, because the mailing list accepts only <0.5MB attachments): 1. for the release-jars-analysis.html, only the 3 main jars from 7.2.0.Alpha1 were analyzed 2. for the project-jars-analysis.html, I took all jars that were present in the infinispan project directory after a build (except the test jars) - this one has therefore more findings In both cases, FindBugs complained that it couldn't find some imported classes, so the analysis may not be 100% complete. I didn't look through the actual results much, but for example it detected an infinite loop at [2], or a self-assignment at [3]. If you want to run it yourself, you can use [4]. There is also a GUI, but I wasn't able to save the results into a html, and a maven plugin [5], but it only creates the results in an xml format for each module separately, so it's not very useful. Jakub [1] http://findbugs.sourceforge.net/ [2] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98ac... [3] https://github.com/infinispan/infinispan/blob/841c789a866745b8d48475f98ac... [4] bin/findbugs -maxHeap 4000 -effort:max -textui -progress -release infinispan -html -output infinispan-findbugs-analysis -onlyAnalyze org.infinispan.- infinispan-7.2.0.Alpha1-all/infinispan-embedded-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-embedded-query-7.2.0.Alpha1.jar infinispan-7.2.0.Alpha1-all/infinispan-remote-7.2.0.Alpha1.jar [5] http://mojo.codehaus.org/findbugs-maven-plugin/

9 years, 9 months

7
7
0 / 0

org.infinispan.notifications.cachelistener.filter vs org.infinispan.filter

by Galder Zamarreño

Hey Will, I wanted to ask you about the classes in org.infinispan.filter package. I thought we agreed to get rid of these duplicate classes before 7.0.0.Final: - org.infinispan.filter.Converter - org.infinispan.filter.KeyValueFilter - ... and related classes The reason being that we pretty much had the same classes in org.infinispan.notifications.cachelistener.filter package. Both set of classes were added in 7.0, so there's no reason why we should have left both sets around :| These latter classes provide more information, but it can confuse users to decide which filters/converters to use where and how. Am I missing something here? We would not be able to remove anything now, but we should deprecate what should not be used. >From a remote even perspective, only the org.infinispan.notifications.cachelistener.filter ones are relevant. Cheers, -- Galder Zamarreño galder(a)redhat.com

9 years, 9 months

3
4
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-dev March 2015