September 2014 - infinispan-dev - Jboss List Archives

Asynchronous cache's "void put()" call expectations changed from 6.0.0 to 6.0.1/7.0

by Galder Zamarreño

Hi all, @Paul, this might be important for WF if using async repl caches (the same I think applies to distributed async caches too) Today I’ve been trying to upgrade Infinispan version in Hibernate master from 6.0.0.Final to 7.0.0.Beta1. Overall, it’s all worked fine but there’s one test that has started failing. Essentialy, this is a clustered test for a repl async cache (w/ cluster cache loader) where a non-owner cache node does put() and immediately, on the same cache, it calls a get(). The test is failing because the get() does not see the effects of the put(), even if both operations are called on the same cache instance. According to Dan, this should have been happening since [1] was implemented, but it’s really started happening since [2] when lock delegation was enabled for replicated caches (EntryWrappingInterceptor.isUsingLockDelegation is now true whereas in 6.0.0 it was false). Not sure we set expectations in this regard, but clearly it’s big change in terms of expectations on when “void put()” completes for async repl caches. I’m not sure how we should handle this, but it definitely needs some discussion and adjuts documentation/javadoc if needed. Can we do something differently? Indepent of how we resolve this, this is the result of once again of trying to shoehole async behaviour into sync APIs. Any async caches (DIST, INV, REPL) should really be accessed exclusively via the AsyncCache API, where you can return quickly and use the future, and any listener to attach to it (a bit ala Java8’s CompletableFuture.map lambda calls) as a way to signal that the operation has completed, and then you have an API and cache mode that make sense and is consistent with how async APIs work. Right now, when a repl async cache’s “void put()” call is not very well defined. Does it return when message has been put on the network? What impact does it have in the local cache contents? Also, a very big problem of the change of behaviour is that if left like that, you are forcing users to code differently, using the same “void put()” API depending on the configuration (whether async/sync). As clearly shown by the issue above, this is very confusing. It’s a lot more logical IMO, and I’ve already sent an email on this very same topic [3] back in January, that whether a cache is sync or async should be based purely on the API used and forget about the static configuration flag on whether the cache is async or sync. Cheers, [1] https://issues.jboss.org/browse/ISPN-2772 [2] https://issues.jboss.org/browse/ISPN-3354 [3] http://lists.jboss.org/pipermail/infinispan-dev/2014-January/014448.html -- Galder Zamarreño galder(a)redhat.com twitter.com/galderz

9 years, 1 month

4
6
0 / 0

Analysis of infinispan-6.0.2 dependency on JDK-Internal APIs

by Rory O'Donnell Oracle, Dublin Ireland

Hi Galder, As part of the preparations for JDK 9, Oracle’s engineers have been analyzing open source projects like yours to understand usage. One area of concern involves identifying compatibility problems, such as reliance on JDK-internal APIs. Our engineers have already prepared guidance on migrating some of the more common usage patterns of JDK-internal APIs to supported public interfaces. The list is on the OpenJDK wiki [0], along with instructions on how to run the jdeps analysis tool yourself . As part of the ongoing development of JDK 9, I would like to encourage migration from JDK-internal APIs towards the supported Java APIs. I have prepared a report for your project rele ase infinispan-6.0.2 based on the jdeps output. The report is attached to this e-mail. For anything where your migration path is unclear, I would appreciate comments on the JDK-internal API usage patterns in the attached jdeps report - in particular comments elaborating on the rationale for them - either to me or on this mailing list. Finding suitable replacements for unsupported interfaces is not always straightforward, which is why I am reaching out to you early in the JDK 9 development cycle so you can give feedback about new APIs that may be needed to facilitate this exercise. Thank you in advance for any efforts and feedback helping us make JDK 9 better. Rgds,Rory [0] https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland

9 years, 6 months

3
5
0 / 0

On ParserRegistry and classloaders

by Sanne Grinovero

I see the ParserRegistry was changed quite a bit; in Infinispan 6 it allowed to specify a different classloader for some operations, now it only takes a classloader during construction time. For WildFly/JBoss users, it is quite common that the configuration file we want parsed is not in the same classloader that the ParserRegistry needs to lookup its own parser components (as its design uses the ServiceLoader to discover components of the parser). This is especially true when Infinispan is not used by the application directly but via another module (so I guess also critical for capedwarf). I initially though to workaround the problem using a "wrapping classloader" so that I could pass a single CL instance which would try both the deployment classloader and the Infinispan's module classloader, but - besides this being suboptimal - it doesn't work as I'm violating isolation between modules: I can get exposed to an Infinispan 6 module which contains also Parser components, which get loaded as a service but are not compatible (different class definition). I'll need these changes in the ParserRegistry reverted please. Happy to send a pull myself, but before I attempt to patch it myself could someone explain what the goal was? thanks, Sanne

9 years, 6 months

4
11
0 / 0

Feedback and requests on clustered and remote listeners

by Emmanuel Bernard

Hi all, I have had a good exchange on how someone would use clustered / remote listeners to do custom continuous query features. I have a few questions and requests to make this fully and easily doable ## Value as bytes or as objects Assuming a Hot Rod based usage and protobuf as the serialization layer. What are KeyValueFilter and Converter seeing? I assume today the bytes are unmarshalled and the Java object is provided to these interfaces. In a protobuf based storage, does that mean that the user must create the Java objects out of a protobuf compiler and deploy these classes in the classpath of each server node? Alternatively, could we pass the raw protobuf data to the KeyValueFilter and Converter? They could read the relevant properties at no deserialization cost and with lss problems related to the classloader. Thoughts? ## Synced listeners In a transactional clustered listener marked as sync. Does the transaction commits and then waits for the relevant clustered listeners to proceed before returning the hand to the Tx client? Or is there something else going on? ## oldValue and newValue I understand why the oldValue was not provided in the initial work. It requires to send more data across the network and at least double the number of values unmarshalled. But for continuous queries, being able to compare the old and the new value is critical to reduce the number of events sent to the listener. Imagine the following use case. A listener exposes the average age for a certain type of customer. You would implement it the following way. 1. Add a KeyValueFilter that - upon creation, filter out the customers of the wrong type - upon update, keep customers that - *were* of the right time but no longer are - were not of the right type but now now *are* - remains of the right type and whose age has changed - upon deletion, keep customers that *were* of the right type 2. Converter In the converter, one could send the whole customer but it would be more efficient to only send the age of the customer as well as wether it is added to or removed from the matching customers - upon creation, you send the customer age and mark it as addition - upon deletion, you send the customer age and mark it as deletion - upon update - if the customer was of the right type but no longer is, send the age as well as a deletion flag - if the customer was not of the right type but now is, send the age as well as an addition flag - if the customer age has changed, send the difference with a modification flag 3. The listener then needs to keep the total sum of all ages as well as the total number of customers of the right type. Based on the sent events, it can adjust these two counters. That requires us to be able to provide the old and new value to the KeyValueFilter and the Converter interface as well as the type of event (creation, update, deletion). If you keep the existing interfaces and their data, the data send and the memory consumed becomes much much bigger. I leave it as an exercise but I think you need to: - send *all* remove and update events regardless of the value (essentially no KeyValueFilter) - in the listener, keep a list of *all* matching keys so that you know if a new event is about a data that was already matching your criteria or not and act accordingly. BTW, you need the old and new value even if your listener returns actual matching results instead of an aggregation. More or less for the same reasons. Continuous query is about the most important use case for remote and clustered listeners and I think we should address it properly and as efficiently as possible. Adding continuous query to Infinispan will then “simply” be a matter of agreeing on the query syntax and implement the predicates as smartly as possible. With the use case I describe, I think the best approach is to merge the KVF and Converter into a single Listener like interface that is able to send or silence an event payload. But that’s guestimate. Because oldValue / newValue implies an unmarshalling overhead we might want to make it an annotation based flag on the class that is executed on each node (somewhat similar to the settings hosted on @Listener). ## includeCurrentState and very narrow filtering The existing approach is fine (send a create event notif for all existing keys and queue changes in the mean time) as long as the listener plans to consume most of these events. But in case of a big data grid, with a lot of passivated entries, the cost would become non negligible. An alternative approach is to first do a query matching the elements the listener is interested in and queue up the events until the query is fully processed. Can a listener access a cache and do a query? Should we offer such option in a more packaged way? For a listener that is only interested in keys whose value city contains Springfield, Virginia, the gain would be massive. ## Remote listener and non Java HR clients Does the API of non Java HR clients support the enlistements of listeners and attach registered keyValueFilter / Converter? Or is that planned? Just curious. Emmanuel

9 years, 6 months

5
34
0 / 0

Configure named cache in remote infinispan 6.0.2 cluster

by Radha Mohan Maheshwari

Hi all, how to pass custom named cache config to infinispan 6.0.2 server as -c option is only taking server configuration not cache config getting this exception while passing custom cache config xml 15:23:26,978 ERROR [org.jboss.as.server] (Controller Boot Thread) JBAS015956: Caught exception during boot: org.jboss.as.controller.persistence.ConfigurationPersistenceException: JBAS014676: Failed to parse configuration at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:141) [jboss-as-controller-7.2.0.Final.jar:7.2.0.Final] at org.jboss.as.server.ServerService.boot(ServerService.java:308) [jboss-as-server-7.2.0.Final.jar:7.2.0.Final] at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:188) [jboss-as-controller-7.2.0.Final.jar:7.2.0.Final] at java.lang.Thread.run(Thread.java:744) [rt.jar:1.7.0_51] Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[2,1] Message: Unexpected element '{urn:infinispan:config:6.0}infinispan' at org.jboss.staxmapper.XMLMapperImpl.processNested(XMLMapperImpl.java:108) [staxmapper-1.1.0.Final.jar:1.1.0.Final] at org.jboss.staxmapper.XMLMapperImpl.parseDocument(XMLMapperImpl.java:69) [staxmapper-1.1.0.Final.jar:1.1.0.Final] at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:133) [jboss-as-controller-7.2.0.Final.jar:7.2.0.Final] ... 3 more PFA custom config file -- Radha Mohan Maheshwari

9 years, 6 months

2
2
0 / 0

Re: [infinispan-dev] Clustering standalone Infinispan w/ WF running Infinispan

by Paul Ferraro

You should not need to use a MuxChannel. This would only be necessary if there are other EAP services sharing the channel. Using a MuxChannel allows your standalone Infinispan instance to filter these irrelevant messages. However, in JDG, there should be no other services other than Infinispan using the channel - hence the MuxChannel stuff is unnecessary. I think Dennis earlier response was spot on. EAP/JDG configures it's cache managers using a ModularClassResolver (which includes a module name along with the class name when marshalling). Your standalone Infinispan instances do not use this and therefore cannot make sense of the message body. Paul ----- Original Message ----- > From: "Kurt T Stam" <kurt.stam(a)jboss.com> > To: "Stelios Koussouris" <stkousso(a)redhat.com>, "Radoslav Husar" <rhusar(a)redhat.com> > Cc: "Galder Zamarreño" <galder(a)redhat.com>, "Paul Ferraro" <paul.ferraro(a)redhat.com>, "Richard Achmatowicz" > <rachmato(a)redhat.com>, "infinispan -Dev List" <infinispan-dev(a)lists.jboss.org> > Sent: Monday, September 29, 2014 11:39:59 AM > Subject: Re: Clustering standalone Infinispan w/ WF running Infinispan > > Thanks for following up Stelios, I think Galder is traveling the next 2 > weeks. > > So - do we need fixes on both ends then so that the boot order does not > matter? In which project(s) would we apply > there changes? Or can they be applied in the end-user's code? > > Thx, > > --Kurt > > > > On 9/26/14, 11:19 AM, Stelios Koussouris wrote: > > Hi, > > > > Rado: It is both ways. ie. if I start first the JDG Server I get the issue > > on the library mode side when I start that one. If reverse the order of > > startup I get it in the JDG Server side. > > > > Question: > > ----------------------------------------------------------------------------------------------------------------------- > > ...IMO the channel needs to be wrapped as > > org.jboss.as.clustering.jgroups.MuxChannel before passing to infinispan. > > ... > > ----------------------------------------------------------------------------------------------------------------------- > > For now that this is not being done. If I wanted to do it manually on the > > library side where I can create the protocol programmatically we are > > talking about something like this? > > > > ProtocolStackConfigurator configurator = > > ConfiguratorFactory.getStackConfigurator("jgroups-udp.xml"); > > MuxChannel channel = new MuxChannel(configurator); > > org.infinispan.remoting.transport.Transport transport = new > > org.infinispan.remoting.transport.jgroups.JGroupsTransport(channel); > > > > .... > > then replace the below > > new > > GlobalConfigurationBuilder().clusteredDefault().globalJmxStatistics().cacheManagerName("RDSCacheManager").allowDuplicateDomains(true).enable().transport().clusterName("UDM-CLUSTER").addProperty("configurationFile", > > "jgroups-udp.xml") > > > > WITH > > new > > GlobalConfigurationBuilder().clusteredDefault().globalJmxStatistics().cacheManagerName("RDSCacheManager").allowDuplicateDomains(true).enable().transport(Transport).clusterName("UDM-CLUSTER") > > > > Btw, someone mentioned that if I follow this method I need to to know the > > assigned mux ids, but that is not quite clear what it means with regards > > to the JGroupsTransport configuration > > > > Thanks, > > > > Stylianos Kousouris > > Red Hat Middleware Consultant > > > > ----- Original Message ----- > > From: "Radoslav Husar" <rhusar(a)redhat.com> > > To: "Galder Zamarreño" <galder(a)redhat.com>, "Paul Ferraro" > > <paul.ferraro(a)redhat.com> > > Cc: "Richard Achmatowicz" <rachmato(a)redhat.com>, "infinispan -Dev List" > > <infinispan-dev(a)lists.jboss.org>, "Stelios Koussouris" > > <stkousso(a)redhat.com>, "Kurt T Stam" <kurt.stam(a)jboss.com> > > Sent: Friday, 26 September, 2014 3:47:16 PM > > Subject: Re: Clustering standalone Infinispan w/ WF running Infinispan > > > > From what Stelios is telling me the question is a little bit other way > > round: he is using library mode infinispan and jgroups in EAP and > > connecting to JDG. So the question is what JDG is doing with the stack, > > not AS/WF as its infinispan/jgroups subsystem is not used. > > > > Unfortunately I don't have access to the JDG repo so I don't know what > > changes have been made there but if you are using the same jgroups > > logic, IMO the channel needs to be wrapped as > > org.jboss.as.clustering.jgroups.MuxChannel before passing to infinispan. > > > > Rado > > > > On 26/09/14 15:03, Galder Zamarreño wrote: > >> Hey Paul, > >> > >> In the last couple of days, a couple of people have encountered the > >> exception in [1] when trying to cluster a standalone Infinispan app with > >> its own JGroups configuration file with a AS/WF running Infinispan cache. > >> > >> From my POV, 3 possible causes: > >> > >> 1. Dependency mismatches between AS/WF and the standalone app. Having done > >> some quick study of Kurt’s case, apart from micro version changes, all > >> looks good. > >> > >> 2. Mismatch in the Infinispan and/or JGroups configuration file. > >> > >> 3. AS/WF puts something on the clustered wire that standalone Infinispan > >> does not expect. Are you still doing multiplexing? Could you be adding > >> extra info to the wire? > >> > >> With this email, I’m trying to get some clarification from you if the > >> issue could be due to 3rd option. If it’s either of the first two, it’s a > >> matter of digging and finding the difference, but if it’s 3rd one, it’s > >> more problematic. > >> > >> Any ideas? > >> > >> [1] https://gist.github.com/skoussou/92f062f2d0bd17168e01 > >> -- > >> Galder Zamarreño > >> galder(a)redhat.com > >> twitter.com/galderz > >> > >

9 years, 6 months

2
2
0 / 0

Early Access builds for JDK 9 b32 and JDK 8u40 b07 are available on java.net

by Rory O'Donnell Oracle, Dublin Ireland

Hi Galder, Early Access build for JDK 9 b32 <https://jdk9.java.net/download/> is available on java.net, summary of changes are listed here <http://www.java.net/download/jdk9/changes/jdk9-b32.html> Early Access build for JDK 8u40 b07 <http://jdk8.java.net/download.html> is available on java.net, summary of changes are listed here. <http://www.java.net/download/jdk8u40/changes/jdk8u40-b07.html> Rgds,Rory -- Rgds,Rory O'Donnell Quality Engineering Manager Oracle EMEA , Dublin, Ireland

9 years, 6 months

1
0
0 / 0

Assistance to write a custom CacheStore

by Sanne Grinovero

I noticed that we often have questions about how to implement a custom CacheStore. If someone had some time to write a nice guide for that, we might have some more luck in getting help to upgrade all the CacheStores which have been granted the status of "Abandonware" as defined by a user (!). https://issues.jboss.org/browse/ISPN-4751

9 years, 6 months

4
3
0 / 0

Clustering standalone Infinispan w/ WF running Infinispan

by Galder Zamarreño

Hey Paul, In the last couple of days, a couple of people have encountered the exception in [1] when trying to cluster a standalone Infinispan app with its own JGroups configuration file with a AS/WF running Infinispan cache. >From my POV, 3 possible causes: 1. Dependency mismatches between AS/WF and the standalone app. Having done some quick study of Kurt’s case, apart from micro version changes, all looks good. 2. Mismatch in the Infinispan and/or JGroups configuration file. 3. AS/WF puts something on the clustered wire that standalone Infinispan does not expect. Are you still doing multiplexing? Could you be adding extra info to the wire? With this email, I’m trying to get some clarification from you if the issue could be due to 3rd option. If it’s either of the first two, it’s a matter of digging and finding the difference, but if it’s 3rd one, it’s more problematic. Any ideas? [1] https://gist.github.com/skoussou/92f062f2d0bd17168e01 -- Galder Zamarreño galder(a)redhat.com twitter.com/galderz

9 years, 6 months

3
2
0 / 0

Lucene compatibility PR

by Sanne Grinovero

All, I will be away next week but I highly need my Lucene compatibility PR [1] to be included in next week's Infinispan release. If it's not good as-is, please take ownership of it and don't wait for me to get back before integrating it. Cheers, Sanne [1] https://github.com/infinispan/infinispan/pull/2904

9 years, 6 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-dev September 2014