Is there a better name for Metadata? Like FileMetadata maybe? Better FileMetadata or maybe FileHeader.

Where is ispn-cache-default-conf.xml used? For tests only? If not: is it possible to use a programmatic version instead and what is "It's a movie cache"? Yes in tests only so far. However it can be used as a provided default configuration. I will send maybe question to infinispan group about best configuration parameters. "It's a movie cache" it's the name for cache configured in ispn-cache-default-conf.xml. In tests in this cache are stored indexes for entity Movie. Indexes for all other entities are stored in default HSInfinispanCache.

Hi there - all looks good. Some comments: - Summary documentation - is this going to be published on a wiki page or something somewhere? Especially the Infinispan bit? I think people will find this info very useful... - CacheKey - if this class is what everything is going to be used in the cache, for performance you should cache the hashcode. Calculate it once and then cache it as an instance variable. If this class is immutable it can be done on construction, even. Infinispan uses hashcode() a lot. :-) But then again, depending on how many entries live in the cache, the overhead of an extra int for every entry may be heavy ... - LockCacheKey - is probably more performant if this is implemented as a boolean flag on CacheKey. Then you won't need to look at the class type when working out hashcodes - Have you written any stress or performance tests? Cheers Manik On 23 Aug 2009, at 22:53, Łukasz Moreń wrote: Hi, Yes, I can adjust the patch next days. I've just noticed that I send summary in not friendly format :), better one is now attached. There is explanation for yours questions below. 2009/8/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt; > Hey Lukasz,Your patch looks quite good and pass tests on my side. > > I encourage others to check out the patch before we apply it (ideally > another person form HSearch and one person from infinispan. > > > Lukasz, I have a few questions/remarks though before applying it. Can you > answer / adjust the patch? > > IndexWriterSetting > Why move to return Object in parsing from the initial int? > IndexWriterSetting has to set up MergeScheduler in IndexWriter. Before, parsing was responsible for number conversion from String to int. Now I have to parse class name, and build/return MergeScheduler from it. > > Move DPHelper#createInfinispanCacheManager to IDP > this is not something that can be shared as it creates a hard dependency > on infinispan otherwise. > > in createInfinispanCacheManager > Don't log in error the fact that xml is not used if a default config is > used. Just log in trace at best. > > Rename InfinispanCacheManagerConfigurationImpl to > DefaultInfinispanCacheManagerConfiguration or even better with a name > describing nicely the behavior of the infinispan config. > > in InfinispanIndexOutput, is it possible to get writeBytes bigger than > buffer size? If yes, does newCheck creates the appropriate numbers of > chunks? > Yes it is possible. Writing process is divided into stages, during every stage can be written max: buffer_size bytes. At the end of the stage its checked if necessary is new chunk, if so new chunk is created. > > InfinispanDirectoryProvider > put the configuration proeprties available in the > InfinispanDirectoryProvider javadoc. > > I think the default cache name should be "Hibernate Search" instead of > "HSInfinispanCache". We know it's in infinispan :) > > what's the try catch opening and closing an IW about? It looks weird. > IW is opened with create=true parameter, first index have to be initialized/created. Always next IW is opened with create=false parameter, then data is appended to exisitng index. Similar things are done in other DP's. in stop() > you don't close the CacheManager? How is that? > Yes. Should be closed. > > InfinispanCacheManagerConfigurationImpl > What does "Infinispan-Cluster" correspond to? Why this name? Shouldn't it > be "Hibernate Search cluster"? > Is it safe to override the GlobalConfiguration? What if JBoss AS use > infinispan to run? > This name is used to distinguish cluster used by HSearch - All nodes with the same name form a group. Yes, rather "Hibernate Search cluster" is better name. It is safe to modify GlobalConfiguration, there can be set up configuration for CacheManager like communication way (JGroups or something else), stack configuration for JGroups, etc.; where Configuration is used to configure specific cache. I think just the infinispan cluster name on JBoss AS have to be different from HSearch, then they will be independent. > > Why the use of DummyTransactionManagerLookup. Doesn't Infinispan guess the > right TM depending on the environment? e in JBoss As use the JBoss one etc? > I think GenericTransactionManagerLookup does that. > Yes right, I was testing it with DummyTM and forgot to change it later. > > InfinispanCacheManagerConfiguration > some javadoc on the methods would be useful. I don't know what do > implement here. > > > Is there a better name for Metadata? Like FileMetadata maybe? > Better FileMetadata or maybe FileHeader. > Where is ispn-cache-default-conf.xml used? For tests only? If not: is it > possible to use a programmatic version instead and what is "It's a movie > cache"? > Yes in tests only so far. However it can be used as a provided default configuration. I will send maybe question to infinispan group about best configuration parameters. "It's a movie cache" it's the name for cache configured in ispn-cache-default-conf.xml. In tests in this cache are stored indexes for entity Movie. Indexes for all other entities are stored in default HSInfinispanCache. > > Emmanuel > > > > Begin forwarded message: > > *From: *Łukasz Moreń <lukasz.moren(a)gmail.com&gt; > *Date: *21 août 2009 02:11:03 HAEC > *To: *Emmanuel Bernard <emmanuel(a)hibernate.org&gt; > *Subject: **GSoC patch with Infinispan Directory Provider* > > I'm sending patch and piece of documentation - not much but necessary > information are included. > There are some todos but I didn't manage to finish it yet. > I changed maven jgroups dependency to 2.8.beta2, before version was > clashed with used by infinispan. > In pom file there was dependency on hibernate common annotations > 3.2.shapshot. It should't be 3.5? > > Cheers,Lukasz > > > > > > <GSoC2009_summary.pdf>_______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org

Hi Łukasz, what are the news about the Lucene Directory? I'm very eager to test it, and have some time to help you if needed. Is there an updated patch to see? How are you testing it? Maybe I could help on that? Also we're going to need some "glue" to integrate the first part of your work (the jgroups backend) with the second part (the directory), as the jgroups backend will need to choose a single node to be used for indexwrites; if this node is removed from the cluster a new one should be elected. Manik had commented about this: "One way to do this is to use the JGroups coordinator as the master..." but since then the discussion on this was discontinued. Sanne 2009/8/26 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: > Hi, thanks for comments ant tips. I'm improving it. > Yes, I was checking with profiler tool and hashcode - even not so heavy - > was called often summary took some time. > There is one test where multiple threads can read or write from/to different > cache instances. However I think would be good to do some real test e.g. > with JMeter on sample app. > > > 2009/8/26 Manik Surtani <manik(a)jboss.org&gt; >> >> Hi there - all looks good. Some comments: >> >> Summary documentation - is this going to be published on a wiki page or >> something somewhere? Especially the Infinispan bit? I think people will >> find this info very useful... >> CacheKey - if this class is what everything is going to be used in the >> cache, for performance you should cache the hashcode. Calculate it once and >> then cache it as an instance variable. If this class is immutable it can be >> done on construction, even. Infinispan uses hashcode() a lot. :-) But >> then again, depending on how many entries live in the cache, the overhead of >> an extra int for every entry may be heavy ... >> LockCacheKey - is probably more performant if this is implemented as a >> boolean flag on CacheKey. Then you won't need to look at the class type >> when working out hashcodes >> Have you written any stress or performance tests? >> >> Cheers >> Manik >> On 23 Aug 2009, at 22:53, Łukasz Moreń wrote: >> >> Hi, >> Yes, I can adjust the patch next days. I've just noticed that I send >> summary in not friendly format :), better one is now attached. >> There is explanation for yours questions below. >> >> >> 2009/8/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt; >>> >>> Hey Lukasz, >>> Your patch looks quite good and pass tests on my side. >>> I encourage others to check out the patch before we apply it (ideally >>> another person form HSearch and one person from infinispan. >>> >>> Lukasz, I have a few questions/remarks though before applying it. Can you >>> answer / adjust the patch? >>> IndexWriterSetting >>> Why move to return Object in parsing from the initial int? >> >> IndexWriterSetting has to set up MergeScheduler in IndexWriter. Before, >> parsing was responsible for number conversion from String to int. Now I have >> to parse class name, and build/return MergeScheduler from it. >> >>> >>> Move DPHelper#createInfinispanCacheManager to IDP >>> this is not something that can be shared as it creates a hard dependency >>> on infinispan otherwise. >>> in createInfinispanCacheManager >>> Don't log in error the fact that xml is not used if a default config is >>> used. Just log in trace at best. >>> Rename InfinispanCacheManagerConfigurationImpl to >>> DefaultInfinispanCacheManagerConfiguration or even better with a name >>> describing nicely the behavior of the infinispan config. >>> in InfinispanIndexOutput, is it possible to get writeBytes bigger than >>> buffer size? If yes, does newCheck creates the appropriate numbers of >>> chunks? >> >> Yes it is possible. Writing process is divided into stages, during every >> stage can be written max: buffer_size bytes. At the end of the stage its >> checked if necessary is new chunk, if so new chunk is created. >> >>> >>> InfinispanDirectoryProvider >>> put the configuration proeprties available in the >>> InfinispanDirectoryProvider javadoc. >>> I think the default cache name should be "Hibernate Search" instead of >>> "HSInfinispanCache". We know it's in infinispan :) >>> what's the try catch opening and closing an IW about? It looks weird. >> >> IW is opened with create=true parameter, first index have to be >> initialized/created. Always next IW is opened with create=false parameter, >> then data is appended to exisitng index. Similar things are done in other >> DP's. >>> >>> in stop() >>> you don't close the CacheManager? How is that? >> >> Yes. Should be closed. >> >>> >>> InfinispanCacheManagerConfigurationImpl >>> What does "Infinispan-Cluster" correspond to? Why this name? Shouldn't it >>> be "Hibernate Search cluster"? >>> Is it safe to override the GlobalConfiguration? What if JBoss AS use >>> infinispan to run? >> >> This name is used to distinguish cluster used by HSearch - All nodes with >> the same name form a group. Yes, rather "Hibernate Search cluster" is better >> name. It is safe to modify GlobalConfiguration, there can be set up >> configuration for CacheManager like communication way (JGroups or something >> else), stack configuration for JGroups, etc.; where Configuration is used to >> configure specific cache. I think just the infinispan cluster name on JBoss >> AS have to be different from HSearch, then they will be independent. >> >>> >>> Why the use of DummyTransactionManagerLookup. Doesn't Infinispan guess >>> the right TM depending on the environment? e in JBoss As use the JBoss one >>> etc? I think GenericTransactionManagerLookup does that. >> >> Yes right, I was testing it with DummyTM and forgot to change it later. >> >>> >>> InfinispanCacheManagerConfiguration >>> some javadoc on the methods would be useful. I don't know what do >>> implement here. >>> >>> Is there a better name for Metadata? Like FileMetadata maybe? >> >> Better FileMetadata or maybe FileHeader. >>> >>> Where is ispn-cache-default-conf.xml used? For tests only? If not: is it >>> possible to use a programmatic version instead and what is "It's a movie >>> cache"? >> >> Yes in tests only so far. However it can be used as a provided default >> configuration. I will send maybe question to infinispan group about best >> configuration parameters. "It's a movie cache" it's the name for cache >> configured in ispn-cache-default-conf.xml. In tests in this cache are stored >> indexes for entity Movie. Indexes for all other entities are stored in >> default HSInfinispanCache. >>> >>> Emmanuel >>> >>> >>> Begin forwarded message: >>> >>> From: Łukasz Moreń <lukasz.moren(a)gmail.com&gt; >>> Date: 21 août 2009 02:11:03 HAEC >>> To: Emmanuel Bernard <emmanuel(a)hibernate.org&gt; >>> Subject: GSoC patch with Infinispan Directory Provider >>> I'm sending patch and piece of documentation - not much but necessary >>> information are included. >>> There are some todos but I didn't manage to finish it yet. >>> I changed maven jgroups dependency to 2.8.beta2, before version was >>> clashed with used by infinispan. >>> In pom file there was dependency on hibernate common annotations >>> 3.2.shapshot. It should't be 3.5? >>> Cheers, >>> Lukasz >>> >>> >>> >> >> <GSoC2009_summary.pdf>_______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> -- >> Manik Surtani >> manik(a)jboss.org >> Lead, Infinispan >> Lead, JBoss Cache >> http://www.infinispan.org >> http://www.jbosscache.org >> >> >> > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev >

That does sounds pretty cool. Would be nice if the lucene indexes could scale along with how people will want to use infinispan. Probably worth playing with.

Sent from my phone. On 13/09/2009, at 8:37 AM, Jeff Ramsdale <jeff.ramsdale(a)gmail.com&gt; wrote: > I'm afraid I haven't followed the Infinispan-Lucene implementation > closely, but have you looked at the Compass Project? > (http://www.compass-project.org/overview.html) It provides a > simplified interface to Lucene (optional) as well as Directory > implementations built on Terracotta, Gigaspaces and Coherence. The > latter, in particular, might be a useful guide for the Infinispan > implementation. I believe it's mature enough to have solved many of > the most difficult problems of implementing Directory on a distributed > Map. > > If someone has any experience with Compass (particularly it's > Directory implementations) I'd be interested in hearing about it... > It's Apache 2.0 licensed, btw. > > -jeff > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

if I want to play around with it ?

if I want to play around with it ?

Hi, I'm wondering if it is reasonable to have multiple threads/nodes that modifies indexes in Lucene Directory based on Infinispan? Let's assume that two nodes try to update index in this same time. First one creates IndexWriter and obtains write lock. There is high propability that second node throws LockObtainFailedException (as one IndexWriter is allowed on single index) and index is not modified. How is that? Should be always only one node that makes changes in the index? Cheers, Lukasz W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń <lukasz.moren(a)gmail.com&gt; napisał: > > Hi, > > With using JMeter I wanted to check if Infinispan dir does not crash under > heavy load in "real" use and check performance in comparison with none/other > directories. > However appeared problem when multiple IndexWriters tries to modify index > (test InfinispanDirectoryTest) - random deadlocks, and Lucene exceptions. > IndexWriter tries to access files in index that were removed before. I'm > looking into it, but not having good idea. > > Concerning the last part, I think similar thing is done in > InfinispanDirectoryProviderTest. Many threads are making changes and > searching (not checking if db is in sync with index). > If threads finish their work, with Lucene query I'm checking if index > contains as many results as expected. Maybe you meant something else? > Would be good to run each node in different VM. > >> Great ! Looking forward to it. What state are things in at the moment >> if I want to play around with it ? > > Should work with with one master(updates index) and one many slave nodes > (sends changes to master). I tried with one master and one slave (both with > jms and jgroups backend) and worked ok. Still fails if multiple nodes want > to modify index. > > I've attached patch with current version. > > Cheers, > Łukasz > > 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; >> >> Great ! Looking forward to it. What state are things in at the moment >> if I want to play around with it ? >> >> Sent from my phone. >> >> On 13/09/2009, at 7:26 PM, Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >> wrote: >> >> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: >> >> That does sounds pretty cool. Would be nice if the lucene indexes >> >> could scale along with how people will want to use infinispan. >> >> Probably worth playing with. >> > >> > Sure, this is the goal of Łukasz's work; We know compass has >> > some good Directories, but we're building our own as one based >> > on Infinispan is not yet available. >> > >> >> >> >> Sent from my phone. >> >> >> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale <jeff.ramsdale(a)gmail.com&gt; >> >> wrote: >> >> >> >>> I'm afraid I haven't followed the Infinispan-Lucene implementation >> >>> closely, but have you looked at the Compass Project? >> >>> (http://www.compass-project.org/overview.html) It provides a >> >>> simplified interface to Lucene (optional) as well as Directory >> >>> implementations built on Terracotta, Gigaspaces and Coherence. The >> >>> latter, in particular, might be a useful guide for the Infinispan >> >>> implementation. I believe it's mature enough to have solved many of >> >>> the most difficult problems of implementing Directory on a >> >>> distributed >> >>> Map. >> >>> >> >>> If someone has any experience with Compass (particularly it's >> >>> Directory implementations) I'd be interested in hearing about it... >> >>> It's Apache 2.0 licensed, btw. >> >>> >> >>> -jeff >> >>> _______________________________________________ >> >>> infinispan-dev mailing list >> >>> infinispan-dev(a)lists.jboss.org >> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> >> infinispan-dev mailing list >> >> infinispan-dev(a)lists.jboss.org >> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> >> > >> > _______________________________________________ >> > infinispan-dev mailing list >> > infinispan-dev(a)lists.jboss.org >> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev

Why not commit the Infinispan tx at lock release, instead of at indexwriter's commit?

Hi, Thanks for explanation. Maybe better I will concentrate on the first release and postpone distributed writing. There is already LockStrategy that uses Infinispan. With using it I was wrapping changes made by IndexWriter in Infinispan transaction, because of performance reasons - on lock obtaining transaction was started, on lock release transaction was commited. Hovewer Ispn transaction commit on lock release is not good idea since IndexWriter calls index commit before lock is released(and ispn transaction is committed). I was thinking to override Workspace class and getIndexWriter(start infinispan tx), commitIndexWriter (commit tx) methods to wrap IndexWrite lifecycle, but this needs few other changes. Some other ideas? Cheers, Lukasz 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; Hi Łukasz, > you've rightful concerns, because the way the IndexWriter tries to > achieve the lock > that will bring some trouble; As far as I remember we decided in this > first release > to avoid multiple writer nodes because of this reasons > (that's written in your docs?) > > Actually it shouldn't be very hard to do, as the LockStrategy is > pluggable (see changes from HSEARCH-345) > and you could implement one delegating to an Infinispan eager lock on some > key, > like the default LockStrategy takes a file lock in the index directory. > > Maybe it's simpler to support this distributed writing instead of > sending the queue to some single > (elected) node? Would be cool, as the Document Analysis effort would > be distributed, > but I have no idea if this would be more or less efficient than a > single node writing; it could > bring some huge data transfers along the wire during segments merging > (basically fetching > the whole index data at each node performing a segment merge); maybe > you'll need to > play with IndexWriter settings ( > > http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... > ) > probably need to find the sweet spot for "merge_factor". > I just saw now that MergePolicy is now re-implementable, but I hope > that won't be needed. > > Sanne > > 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: > > Hi, > > > > I'm wondering if it is reasonable to have multiple threads/nodes that > > modifies indexes in Lucene Directory based on Infinispan? Let's assume > that > > two nodes try to update index in this same time. First one creates > > IndexWriter and obtains > > write lock. There is high propability that second node throws > > LockObtainFailedException (as one IndexWriter is allowed on single > index) > > and index is not modified. How is that? Should be always only one node > that > > makes changes in > > the index? > > > > Cheers, > > Lukasz > > > > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń > > <lukasz.moren(a)gmail.com&gt; napisał: > >> > >> Hi, > >> > >> With using JMeter I wanted to check if Infinispan dir does not crash > under > >> heavy load in "real" use and check performance in comparison with > none/other > >> directories. > >> However appeared problem when multiple IndexWriters tries to modify > index > >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene > exceptions. > >> IndexWriter tries to access files in index that were removed before. > I'm > >> looking into it, but not having good idea. > >> > >> Concerning the last part, I think similar thing is done in > >> InfinispanDirectoryProviderTest. Many threads are making changes and > >> searching (not checking if db is in sync with index). > >> If threads finish their work, with Lucene query I'm checking if index > >> contains as many results as expected. Maybe you meant something else? > >> Would be good to run each node in different VM. > >> > >>> Great ! Looking forward to it. What state are things in at the moment > >>> if I want to play around with it ? > >> > >> Should work with with one master(updates index) and one many slave > nodes > >> (sends changes to master). I tried with one master and one slave (both > with > >> jms and jgroups backend) and worked ok. Still fails if multiple nodes > want > >> to modify index. > >> > >> I've attached patch with current version. > >> > >> Cheers, > >> Łukasz > >> > >> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; > >>> > >>> Great ! Looking forward to it. What state are things in at the moment > >>> if I want to play around with it ? > >>> > >>> Sent from my phone. > >>> > >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero <sanne.grinovero(a)gmail.com > > > >>> wrote: > >>> > >>> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: > >>> >> That does sounds pretty cool. Would be nice if the lucene indexes > >>> >> could scale along with how people will want to use infinispan. > >>> >> Probably worth playing with. > >>> > > >>> > Sure, this is the goal of Łukasz's work; We know compass has > >>> > some good Directories, but we're building our own as one based > >>> > on Infinispan is not yet available. > >>> > > >>> >> > >>> >> Sent from my phone. > >>> >> > >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale <jeff.ramsdale(a)gmail.com&gt; > >>> >> wrote: > >>> >> > >>> >>> I'm afraid I haven't followed the Infinispan-Lucene implementation > >>> >>> closely, but have you looked at the Compass Project? > >>> >>> (http://www.compass-project.org/overview.html) It provides a > >>> >>> simplified interface to Lucene (optional) as well as Directory > >>> >>> implementations built on Terracotta, Gigaspaces and Coherence. The > >>> >>> latter, in particular, might be a useful guide for the Infinispan > >>> >>> implementation. I believe it's mature enough to have solved many > of > >>> >>> the most difficult problems of implementing Directory on a > >>> >>> distributed > >>> >>> Map. > >>> >>> > >>> >>> If someone has any experience with Compass (particularly it's > >>> >>> Directory implementations) I'd be interested in hearing about > it... > >>> >>> It's Apache 2.0 licensed, btw. > >>> >>> > >>> >>> -jeff > >>> >>> _______________________________________________ > >>> >>> infinispan-dev mailing list > >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >> _______________________________________________ > >>> >> infinispan-dev mailing list > >>> >> infinispan-dev(a)lists.jboss.org > >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >> > >>> > > >>> > _______________________________________________ > >>> > infinispan-dev mailing list > >>> > infinispan-dev(a)lists.jboss.org > >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev(a)lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > >

Can you explain in more details what is going on.Aside from that Workspace has been Sanne's baby lately so he will be the best to see what design will work in HSearch. That being said, I don't like the idea of subclassing / overriding very much. In my experience, it has lead to more bad and unmaintainable code than anything else. On 22 sept. 09, at 02:16, Łukasz Moreń wrote: Hi, Thanks for explanation. Maybe better I will concentrate on the first release and postpone distributed writing. There is already LockStrategy that uses Infinispan. With using it I was wrapping changes made by IndexWriter in Infinispan transaction, because of performance reasons - on lock obtaining transaction was started, on lock release transaction was commited. Hovewer Ispn transaction commit on lock release is not good idea since IndexWriter calls index commit before lock is released(and ispn transaction is committed). I was thinking to override Workspace class and getIndexWriter(start infinispan tx), commitIndexWriter (commit tx) methods to wrap IndexWrite lifecycle, but this needs few other changes. Some other ideas? Cheers, Lukasz 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; > Hi Łukasz, > you've rightful concerns, because the way the IndexWriter tries to > achieve the lock > that will bring some trouble; As far as I remember we decided in this > first release > to avoid multiple writer nodes because of this reasons > (that's written in your docs?) > > Actually it shouldn't be very hard to do, as the LockStrategy is > pluggable (see changes from HSEARCH-345) > and you could implement one delegating to an Infinispan eager lock on some > key, > like the default LockStrategy takes a file lock in the index directory. > > Maybe it's simpler to support this distributed writing instead of > sending the queue to some single > (elected) node? Would be cool, as the Document Analysis effort would > be distributed, > but I have no idea if this would be more or less efficient than a > single node writing; it could > bring some huge data transfers along the wire during segments merging > (basically fetching > the whole index data at each node performing a segment merge); maybe > you'll need to > play with IndexWriter settings ( > > http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... > ) > probably need to find the sweet spot for "merge_factor". > I just saw now that MergePolicy is now re-implementable, but I hope > that won't be needed. > > Sanne > > 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: > > Hi, > > > > I'm wondering if it is reasonable to have multiple threads/nodes that > > modifies indexes in Lucene Directory based on Infinispan? Let's assume > that > > two nodes try to update index in this same time. First one creates > > IndexWriter and obtains > > write lock. There is high propability that second node throws > > LockObtainFailedException (as one IndexWriter is allowed on single > index) > > and index is not modified. How is that? Should be always only one node > that > > makes changes in > > the index? > > > > Cheers, > > Lukasz > > > > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń > > <lukasz.moren(a)gmail.com&gt; napisał: > >> > >> Hi, > >> > >> With using JMeter I wanted to check if Infinispan dir does not crash > under > >> heavy load in "real" use and check performance in comparison with > none/other > >> directories. > >> However appeared problem when multiple IndexWriters tries to modify > index > >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene > exceptions. > >> IndexWriter tries to access files in index that were removed before. > I'm > >> looking into it, but not having good idea. > >> > >> Concerning the last part, I think similar thing is done in > >> InfinispanDirectoryProviderTest. Many threads are making changes and > >> searching (not checking if db is in sync with index). > >> If threads finish their work, with Lucene query I'm checking if index > >> contains as many results as expected. Maybe you meant something else? > >> Would be good to run each node in different VM. > >> > >>> Great ! Looking forward to it. What state are things in at the moment > >>> if I want to play around with it ? > >> > >> Should work with with one master(updates index) and one many slave > nodes > >> (sends changes to master). I tried with one master and one slave (both > with > >> jms and jgroups backend) and worked ok. Still fails if multiple nodes > want > >> to modify index. > >> > >> I've attached patch with current version. > >> > >> Cheers, > >> Łukasz > >> > >> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; > >>> > >>> Great ! Looking forward to it. What state are things in at the moment > >>> if I want to play around with it ? > >>> > >>> Sent from my phone. > >>> > >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero <sanne.grinovero(a)gmail.com > > > >>> wrote: > >>> > >>> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: > >>> >> That does sounds pretty cool. Would be nice if the lucene indexes > >>> >> could scale along with how people will want to use infinispan. > >>> >> Probably worth playing with. > >>> > > >>> > Sure, this is the goal of Łukasz's work; We know compass has > >>> > some good Directories, but we're building our own as one based > >>> > on Infinispan is not yet available. > >>> > > >>> >> > >>> >> Sent from my phone. > >>> >> > >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale <jeff.ramsdale(a)gmail.com&gt; > >>> >> wrote: > >>> >> > >>> >>> I'm afraid I haven't followed the Infinispan-Lucene implementation > >>> >>> closely, but have you looked at the Compass Project? > >>> >>> (http://www.compass-project.org/overview.html) It provides a > >>> >>> simplified interface to Lucene (optional) as well as Directory > >>> >>> implementations built on Terracotta, Gigaspaces and Coherence. The > >>> >>> latter, in particular, might be a useful guide for the Infinispan > >>> >>> implementation. I believe it's mature enough to have solved many > of > >>> >>> the most difficult problems of implementing Directory on a > >>> >>> distributed > >>> >>> Map. > >>> >>> > >>> >>> If someone has any experience with Compass (particularly it's > >>> >>> Directory implementations) I'd be interested in hearing about > it... > >>> >>> It's Apache 2.0 licensed, btw. > >>> >>> > >>> >>> -jeff > >>> >>> _______________________________________________ > >>> >>> infinispan-dev mailing list > >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >> _______________________________________________ > >>> >> infinispan-dev mailing list > >>> >> infinispan-dev(a)lists.jboss.org > >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >> > >>> > > >>> > _______________________________________________ > >>> > infinispan-dev mailing list > >>> > infinispan-dev(a)lists.jboss.org > >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev(a)lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > >

I need to provide this same lifecycle for IndexWriter as for Infinispan tx - IW is created: tx is started, IW is commited: tx is commited. It assures that IndexReader doesn't read old data from directory. Infinispan transaction can be started when IW acquires the lock, but its commit on IW lock release, as it is done so far, causes a problem: index writer close { index writer commit(); //changes are visible for IndexReaders //Index reader starts reading here, i.e. tries to access file "A" index writer lockRelease(); //changes in Infinispan directory are commited, file "A" was removed, IndexReader cannot find it and crashes } I think Infinispan tx have to be commited just before IW commit, and the problem is where to put in code. W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard <emmanuel(a)hibernate.org > napisał: Can you explain in more details what is going on. Aside from that Workspace has been Sanne's baby lately so he will be the best to see what design will work in HSearch. That being said, I don't like the idea of subclassing / overriding very much. In my experience, it has lead to more bad and unmaintainable code than anything else. On 22 sept. 09, at 02:16, Łukasz Moreń wrote: > Hi, > > Thanks for explanation. > Maybe better I will concentrate on the first release and postpone > distributed writing. > > There is already LockStrategy that uses Infinispan. With using it I > was wrapping changes made by IndexWriter in Infinispan transaction, > because of performance reasons - > on lock obtaining transaction was started, on lock release > transaction was commited. Hovewer Ispn transaction commit on lock > release is not good idea since IndexWriter calls index commit > before lock is released(and ispn transaction is committed). > I was thinking to override Workspace class and getIndexWriter(start > infinispan tx), commitIndexWriter (commit tx) methods to wrap > IndexWrite lifecycle, but this needs few other changes. Some other > ideas? > > Cheers, > Lukasz > > 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; > Hi Łukasz, > you've rightful concerns, because the way the IndexWriter tries to > achieve the lock > that will bring some trouble; As far as I remember we decided in this > first release > to avoid multiple writer nodes because of this reasons > (that's written in your docs?) > > Actually it shouldn't be very hard to do, as the LockStrategy is > pluggable (see changes from HSEARCH-345) > and you could implement one delegating to an Infinispan eager lock > on some key, > like the default LockStrategy takes a file lock in the index > directory. > > Maybe it's simpler to support this distributed writing instead of > sending the queue to some single > (elected) node? Would be cool, as the Document Analysis effort would > be distributed, > but I have no idea if this would be more or less efficient than a > single node writing; it could > bring some huge data transfers along the wire during segments merging > (basically fetching > the whole index data at each node performing a segment merge); maybe > you'll need to > play with IndexWriter settings ( > http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... > ) > probably need to find the sweet spot for "merge_factor". > I just saw now that MergePolicy is now re-implementable, but I hope > that won't be needed. > > Sanne > > 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: > > Hi, > > > > I'm wondering if it is reasonable to have multiple threads/nodes > that > > modifies indexes in Lucene Directory based on Infinispan? Let's > assume that > > two nodes try to update index in this same time. First one creates > > IndexWriter and obtains > > write lock. There is high propability that second node throws > > LockObtainFailedException (as one IndexWriter is allowed on > single index) > > and index is not modified. How is that? Should be always only one > node that > > makes changes in > > the index? > > > > Cheers, > > Lukasz > > > > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń > > <lukasz.moren(a)gmail.com&gt; napisał: > >> > >> Hi, > >> > >> With using JMeter I wanted to check if Infinispan dir does not > crash under > >> heavy load in "real" use and check performance in comparison > with none/other > >> directories. > >> However appeared problem when multiple IndexWriters tries to > modify index > >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene > exceptions. > >> IndexWriter tries to access files in index that were removed > before. I'm > >> looking into it, but not having good idea. > >> > >> Concerning the last part, I think similar thing is done in > >> InfinispanDirectoryProviderTest. Many threads are making changes > and > >> searching (not checking if db is in sync with index). > >> If threads finish their work, with Lucene query I'm checking if > index > >> contains as many results as expected. Maybe you meant something > else? > >> Would be good to run each node in different VM. > >> > >>> Great ! Looking forward to it. What state are things in at the > moment > >>> if I want to play around with it ? > >> > >> Should work with with one master(updates index) and one many > slave nodes > >> (sends changes to master). I tried with one master and one slave > (both with > >> jms and jgroups backend) and worked ok. Still fails if multiple > nodes want > >> to modify index. > >> > >> I've attached patch with current version. > >> > >> Cheers, > >> Łukasz > >> > >> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; > >>> > >>> Great ! Looking forward to it. What state are things in at the > moment > >>> if I want to play around with it ? > >>> > >>> Sent from my phone. > >>> > >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero <sanne.grinovero(a)gmail.com > > > >>> wrote: > >>> > >>> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: > >>> >> That does sounds pretty cool. Would be nice if the lucene > indexes > >>> >> could scale along with how people will want to use infinispan. > >>> >> Probably worth playing with. > >>> > > >>> > Sure, this is the goal of Łukasz's work; We know compass has > >>> > some good Directories, but we're building our own as one based > >>> > on Infinispan is not yet available. > >>> > > >>> >> > >>> >> Sent from my phone. > >>> >> > >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale <jeff.ramsdale(a)gmail.com > > > >>> >> wrote: > >>> >> > >>> >>> I'm afraid I haven't followed the Infinispan-Lucene > implementation > >>> >>> closely, but have you looked at the Compass Project? > >>> >>> (http://www.compass-project.org/overview.html) It provides a > >>> >>> simplified interface to Lucene (optional) as well as > Directory > >>> >>> implementations built on Terracotta, Gigaspaces and > Coherence. The > >>> >>> latter, in particular, might be a useful guide for the > Infinispan > >>> >>> implementation. I believe it's mature enough to have solved > many of > >>> >>> the most difficult problems of implementing Directory on a > >>> >>> distributed > >>> >>> Map. > >>> >>> > >>> >>> If someone has any experience with Compass (particularly it's > >>> >>> Directory implementations) I'd be interested in hearing > about it... > >>> >>> It's Apache 2.0 licensed, btw. > >>> >>> > >>> >>> -jeff > >>> >>> _______________________________________________ > >>> >>> infinispan-dev mailing list > >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >> _______________________________________________ > >>> >> infinispan-dev mailing list > >>> >> infinispan-dev(a)lists.jboss.org > >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >> > >>> > > >>> > _______________________________________________ > >>> > infinispan-dev mailing list > >>> > infinispan-dev(a)lists.jboss.org > >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > >>> _______________________________________________ > >>> infinispan-dev mailing list > >>> infinispan-dev(a)lists.jboss.org > >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > > > >

reading them (how would that be detected)?

I agree It should work the same way; The IndexWriter cleans files whenever it likes to, it doesn't try to detect readers, and this shouldn't have any effect on the working of readers. The IndexReader opens the "SegmentsInfo" first, and immediately after** gets a reference to the segments listed in this SegmentsInfo. No IndexWriter will ever change an existing segment, only add new files or eventually delete old ones (segments merge,optimize). The deletion of segments is the interesting subject: when using Files it uses "delete at last close", which works because the IR needing it have it opened already**; when using the RAMDirectory they have a reference preventing garbage collection. ( the two "**" are assuming the same event occurred correctly, otherwise an exception is thrown at opening) When using Infinispan it shouldn't be much different than the RAMDirectory? so even if the needed segment is deleted, the IR holds a reference to the Java object locally since it was opened. Łukcasz, do you have some failing test? Sanne 2009/9/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: > Conceptually I don't understand why it does work in a pure file system > directory (ie IndexReader can go and process queries with the IndexWriter > goes about its business) and not when using Infinispan. > Is it because, the IndexWriter only clean files if no indexReaders are > reading them (how would that be detected)? > On 22 sept. 09, at 20:46, Łukasz Moreń wrote: > > I need to provide this same lifecycle for IndexWriter as for Infinispan tx - > IW is created: tx is started, IW is commited: tx is commited. It assures > that IndexReader doesn't read old data from directory. > Infinispan transaction can be started when IW acquires the lock, but its > commit on IW lock release, as it is done so far, causes a problem: > > index writer close { > index writer commit(); //changes are visible for IndexReaders > > //Index reader starts reading here, i.e. tries to access file "A" > > index writer lockRelease(); //changes in Infinispan directory are > commited, file "A" was removed, IndexReader cannot find it and crashes > } > > I think Infinispan tx have to be commited just before IW commit, and the > problem is where to put in code. > > W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard > <emmanuel(a)hibernate.org&gt; napisał: >> >> Can you explain in more details what is going on. >> Aside from that Workspace has been Sanne's baby lately so he will be the >> best to see what design will work in HSearch. That being said, I don't like >> the idea of subclassing / overriding very much. In my experience, it has >> lead to more bad and unmaintainable code than anything else. >> On 22 sept. 09, at 02:16, Łukasz Moreń wrote: >> >> Hi, >> >> Thanks for explanation. >> Maybe better I will concentrate on the first release and postpone >> distributed writing. >> >> There is already LockStrategy that uses Infinispan. With using it I was >> wrapping changes made by IndexWriter in Infinispan transaction, because of >> performance reasons - >> on lock obtaining transaction was started, on lock release transaction was >> commited. Hovewer Ispn transaction commit on lock release is not good idea >> since IndexWriter calls index commit before lock is released(and ispn >> transaction is committed). >> I was thinking to override Workspace class and getIndexWriter(start >> infinispan tx), commitIndexWriter (commit tx) methods to wrap IndexWrite >> lifecycle, but this needs few other changes. Some other ideas? >> >> Cheers, >> Lukasz >> >> 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >>> >>> Hi Łukasz, >>> you've rightful concerns, because the way the IndexWriter tries to >>> achieve the lock >>> that will bring some trouble; As far as I remember we decided in this >>> first release >>> to avoid multiple writer nodes because of this reasons >>> (that's written in your docs?) >>> >>> Actually it shouldn't be very hard to do, as the LockStrategy is >>> pluggable (see changes from HSEARCH-345) >>> and you could implement one delegating to an Infinispan eager lock on >>> some key, >>> like the default LockStrategy takes a file lock in the index directory. >>> >>> Maybe it's simpler to support this distributed writing instead of >>> sending the queue to some single >>> (elected) node? Would be cool, as the Document Analysis effort would >>> be distributed, >>> but I have no idea if this would be more or less efficient than a >>> single node writing; it could >>> bring some huge data transfers along the wire during segments merging >>> (basically fetching >>> the whole index data at each node performing a segment merge); maybe >>> you'll need to >>> play with IndexWriter settings ( >>> >>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... >>> ) >>> probably need to find the sweet spot for "merge_factor". >>> I just saw now that MergePolicy is now re-implementable, but I hope >>> that won't be needed. >>> >>> Sanne >>> >>> 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: >>> > Hi, >>> > >>> > I'm wondering if it is reasonable to have multiple threads/nodes that >>> > modifies indexes in Lucene Directory based on Infinispan? Let's assume >>> > that >>> > two nodes try to update index in this same time. First one creates >>> > IndexWriter and obtains >>> > write lock. There is high propability that second node throws >>> > LockObtainFailedException (as one IndexWriter is allowed on single >>> > index) >>> > and index is not modified. How is that? Should be always only one node >>> > that >>> > makes changes in >>> > the index? >>> > >>> > Cheers, >>> > Lukasz >>> > >>> > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń >>> > <lukasz.moren(a)gmail.com&gt; napisał: >>> >> >>> >> Hi, >>> >> >>> >> With using JMeter I wanted to check if Infinispan dir does not crash >>> >> under >>> >> heavy load in "real" use and check performance in comparison with >>> >> none/other >>> >> directories. >>> >> However appeared problem when multiple IndexWriters tries to modify >>> >> index >>> >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene >>> >> exceptions. >>> >> IndexWriter tries to access files in index that were removed before. >>> >> I'm >>> >> looking into it, but not having good idea. >>> >> >>> >> Concerning the last part, I think similar thing is done in >>> >> InfinispanDirectoryProviderTest. Many threads are making changes and >>> >> searching (not checking if db is in sync with index). >>> >> If threads finish their work, with Lucene query I'm checking if index >>> >> contains as many results as expected. Maybe you meant something else? >>> >> Would be good to run each node in different VM. >>> >> >>> >>> Great ! Looking forward to it. What state are things in at the moment >>> >>> if I want to play around with it ? >>> >> >>> >> Should work with with one master(updates index) and one many slave >>> >> nodes >>> >> (sends changes to master). I tried with one master and one slave (both >>> >> with >>> >> jms and jgroups backend) and worked ok. Still fails if multiple nodes >>> >> want >>> >> to modify index. >>> >> >>> >> I've attached patch with current version. >>> >> >>> >> Cheers, >>> >> Łukasz >>> >> >>> >> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; >>> >>> >>> >>> Great ! Looking forward to it. What state are things in at the moment >>> >>> if I want to play around with it ? >>> >>> >>> >>> Sent from my phone. >>> >>> >>> >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero >>> >>> <sanne.grinovero(a)gmail.com&gt; >>> >>> wrote: >>> >>> >>> >>> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: >>> >>> >> That does sounds pretty cool. Would be nice if the lucene indexes >>> >>> >> could scale along with how people will want to use infinispan. >>> >>> >> Probably worth playing with. >>> >>> > >>> >>> > Sure, this is the goal of Łukasz's work; We know compass has >>> >>> > some good Directories, but we're building our own as one based >>> >>> > on Infinispan is not yet available. >>> >>> > >>> >>> >> >>> >>> >> Sent from my phone. >>> >>> >> >>> >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale < jeff.ramsdale(a)gmail.com&gt; >>> >>> >> wrote: >>> >>> >> >>> >>> >>> I'm afraid I haven't followed the Infinispan-Lucene >>> >>> >>> implementation >>> >>> >>> closely, but have you looked at the Compass Project? >>> >>> >>> (http://www.compass-project.org/overview.html) It provides a >>> >>> >>> simplified interface to Lucene (optional) as well as Directory >>> >>> >>> implementations built on Terracotta, Gigaspaces and Coherence. >>> >>> >>> The >>> >>> >>> latter, in particular, might be a useful guide for the Infinispan >>> >>> >>> implementation. I believe it's mature enough to have solved many >>> >>> >>> of >>> >>> >>> the most difficult problems of implementing Directory on a >>> >>> >>> distributed >>> >>> >>> Map. >>> >>> >>> >>> >>> >>> If someone has any experience with Compass (particularly it's >>> >>> >>> Directory implementations) I'd be interested in hearing about >>> >>> >>> it... >>> >>> >>> It's Apache 2.0 licensed, btw. >>> >>> >>> >>> >>> >>> -jeff >>> >>> >>> _______________________________________________ >>> >>> >>> infinispan-dev mailing list >>> >>> >>> infinispan-dev(a)lists.jboss.org >>> >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >> _______________________________________________ >>> >>> >> infinispan-dev mailing list >>> >>> >> infinispan-dev(a)lists.jboss.org >>> >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >> >>> >>> > >>> >>> > _______________________________________________ >>> >>> > infinispan-dev mailing list >>> >>> > infinispan-dev(a)lists.jboss.org >>> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> >>> >>> _______________________________________________ >>> >>> infinispan-dev mailing list >>> >>> infinispan-dev(a)lists.jboss.org >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> > >>> > >> >> > > >

On 23 sept. 09, at 14:58, Łukasz Moreń wrote: > Is it because, the IndexWriter only clean files if no indexReaders are > reading them (how would that be detected)? > It can happen if IndexWriter clean file, and IndexReader try to access > that cleaned file. IndexWriter doe snot clean files AFAIK (unless you optimize I think but that's another story).

I agree that Infinispan case is not much different from RamDirectory. The major difference is that in RD (also FileDirectory) changes are not batched like in ID. If I do not wrap changes in InfinispanDirectory(simple remove tx.begin() from obtain() method and tx.commit() from release() in InfinispanLock), and immediately commit every change made by IW it works well. Hovewer it makes indexing really slower, because of frequent replication to other nodes. Sanne it's good remark that IW commit is kind of flush. I've attached patch with InfinispanDirectory, failing test is testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. It fails randomly. I think problem is Infinispan commit on lockRelease() in org.apache.lucene.index.IndexWriter (line 1658) is after IW commit() (line 1654).

Minorly off topic, but rather than working with patches, do we want this Directory impl in source control somewhere? Being dependent on LGPL, it won't be accepted into Lucene's contribs. If it doesn't depend on any Hibernate Search code, I could host it in Infinispan's SVN repo... On 23 Sep 2009, at 13:58, Łukasz Moreń wrote: > I agree that Infinispan case is not much different from > RamDirectory. The major difference is that in RD (also > FileDirectory) changes are not batched like in ID. If I do not wrap > changes in InfinispanDirectory(simple remove tx.begin() from > obtain() method and tx.commit() from release() in InfinispanLock), > and immediately commit every change made by IW it works well. > Hovewer it makes indexing really slower, because of frequent > replication to other nodes. > Sanne it's good remark that IW commit is kind of flush. > > I've attached patch with InfinispanDirectory, failing test is > testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. > It fails randomly. I think problem is Infinispan commit on > lockRelease() in org.apache.lucene.index.IndexWriter (line 1658) is > after IW commit() (line 1654). > > Is it because, the IndexWriter only clean files if no indexReaders > are reading them (how would that be detected)? > It can happen if IndexWriter clean file, and IndexReader try to > access that cleaned file. > > 2009/9/23 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; > I agree It should work the same way; The IndexWriter cleans files > whenever it likes to, it doesn't try to detect readers, and this > shouldn't have any effect on the working of readers. > The IndexReader opens the "SegmentsInfo" first, and immediately > after** gets a reference to the segments listed in this SegmentsInfo. > No IndexWriter will ever change an existing segment, only add new > files or eventually delete old ones (segments merge,optimize). > The deletion of segments is the interesting subject: when using Files > it uses "delete at last close", which works because the IR needing it > have it opened already**; when using the RAMDirectory they have a > reference preventing garbage collection. > > ( the two "**" are assuming the same event occurred correctly, > otherwise an exception is thrown at opening) > > When using Infinispan it shouldn't be much different than the > RAMDirectory? so even if the needed segment is deleted, the IR > holds a > reference to the Java object locally since it was opened. > > Łukcasz, do you have some failing test? > > Sanne > > 2009/9/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: > > Conceptually I don't understand why it does work in a pure file > system > > directory (ie IndexReader can go and process queries with the > IndexWriter > > goes about its business) and not when using Infinispan. > > Is it because, the IndexWriter only clean files if no > indexReaders are > > reading them (how would that be detected)? > > On 22 sept. 09, at 20:46, Łukasz Moreń wrote: > > > > I need to provide this same lifecycle for IndexWriter as for > Infinispan tx - > > IW is created: tx is started, IW is commited: tx is commited. It > assures > > that IndexReader doesn't read old data from directory. > > Infinispan transaction can be started when IW acquires the lock, > but its > > commit on IW lock release, as it is done so far, causes a problem: > > > > index writer close { > > index writer commit(); //changes are visible for IndexReaders > > > > //Index reader starts reading here, i.e. tries to access > file "A" > > > > index writer lockRelease(); //changes in Infinispan directory are > > commited, file "A" was removed, IndexReader cannot find it and > crashes > > } > > > > I think Infinispan tx have to be commited just before IW commit, > and the > > problem is where to put in code. > > > > W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard > > <emmanuel(a)hibernate.org&gt; napisał: > >> > >> Can you explain in more details what is going on. > >> Aside from that Workspace has been Sanne's baby lately so he > will be the > >> best to see what design will work in HSearch. That being said, I > don't like > >> the idea of subclassing / overriding very much. In my > experience, it has > >> lead to more bad and unmaintainable code than anything else. > >> On 22 sept. 09, at 02:16, Łukasz Moreń wrote: > >> > >> Hi, > >> > >> Thanks for explanation. > >> Maybe better I will concentrate on the first release and postpone > >> distributed writing. > >> > >> There is already LockStrategy that uses Infinispan. With using > it I was > >> wrapping changes made by IndexWriter in Infinispan transaction, > because of > >> performance reasons - > >> on lock obtaining transaction was started, on lock release > transaction was > >> commited. Hovewer Ispn transaction commit on lock release is not > good idea > >> since IndexWriter calls index commit before lock is released(and > ispn > >> transaction is committed). > >> I was thinking to override Workspace class and > getIndexWriter(start > >> infinispan tx), commitIndexWriter (commit tx) methods to wrap > IndexWrite > >> lifecycle, but this needs few other changes. Some other ideas? > >> > >> Cheers, > >> Lukasz > >> > >> 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; > >>> > >>> Hi Łukasz, > >>> you've rightful concerns, because the way the IndexWriter tries > to > >>> achieve the lock > >>> that will bring some trouble; As far as I remember we decided > in this > >>> first release > >>> to avoid multiple writer nodes because of this reasons > >>> (that's written in your docs?) > >>> > >>> Actually it shouldn't be very hard to do, as the LockStrategy is > >>> pluggable (see changes from HSEARCH-345) > >>> and you could implement one delegating to an Infinispan eager > lock on > >>> some key, > >>> like the default LockStrategy takes a file lock in the index > directory. > >>> > >>> Maybe it's simpler to support this distributed writing instead of > >>> sending the queue to some single > >>> (elected) node? Would be cool, as the Document Analysis effort > would > >>> be distributed, > >>> but I have no idea if this would be more or less efficient than a > >>> single node writing; it could > >>> bring some huge data transfers along the wire during segments > merging > >>> (basically fetching > >>> the whole index data at each node performing a segment merge); > maybe > >>> you'll need to > >>> play with IndexWriter settings ( > >>> > >>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... > >>> ) > >>> probably need to find the sweet spot for "merge_factor". > >>> I just saw now that MergePolicy is now re-implementable, but I > hope > >>> that won't be needed. > >>> > >>> Sanne > >>> > >>> 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: > >>> > Hi, > >>> > > >>> > I'm wondering if it is reasonable to have multiple threads/ > nodes that > >>> > modifies indexes in Lucene Directory based on Infinispan? > Let's assume > >>> > that > >>> > two nodes try to update index in this same time. First one > creates > >>> > IndexWriter and obtains > >>> > write lock. There is high propability that second node throws > >>> > LockObtainFailedException (as one IndexWriter is allowed on > single > >>> > index) > >>> > and index is not modified. How is that? Should be always only > one node > >>> > that > >>> > makes changes in > >>> > the index? > >>> > > >>> > Cheers, > >>> > Lukasz > >>> > > >>> > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń > >>> > <lukasz.moren(a)gmail.com&gt; napisał: > >>> >> > >>> >> Hi, > >>> >> > >>> >> With using JMeter I wanted to check if Infinispan dir does > not crash > >>> >> under > >>> >> heavy load in "real" use and check performance in comparison > with > >>> >> none/other > >>> >> directories. > >>> >> However appeared problem when multiple IndexWriters tries to > modify > >>> >> index > >>> >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene > >>> >> exceptions. > >>> >> IndexWriter tries to access files in index that were removed > before. > >>> >> I'm > >>> >> looking into it, but not having good idea. > >>> >> > >>> >> Concerning the last part, I think similar thing is done in > >>> >> InfinispanDirectoryProviderTest. Many threads are making > changes and > >>> >> searching (not checking if db is in sync with index). > >>> >> If threads finish their work, with Lucene query I'm checking > if index > >>> >> contains as many results as expected. Maybe you meant > something else? > >>> >> Would be good to run each node in different VM. > >>> >> > >>> >>> Great ! Looking forward to it. What state are things in at > the moment > >>> >>> if I want to play around with it ? > >>> >> > >>> >> Should work with with one master(updates index) and one many > slave > >>> >> nodes > >>> >> (sends changes to master). I tried with one master and one > slave (both > >>> >> with > >>> >> jms and jgroups backend) and worked ok. Still fails if > multiple nodes > >>> >> want > >>> >> to modify index. > >>> >> > >>> >> I've attached patch with current version. > >>> >> > >>> >> Cheers, > >>> >> Łukasz > >>> >> > >>> >> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; > >>> >>> > >>> >>> Great ! Looking forward to it. What state are things in at > the moment > >>> >>> if I want to play around with it ? > >>> >>> > >>> >>> Sent from my phone. > >>> >>> > >>> >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero > >>> >>> <sanne.grinovero(a)gmail.com&gt; > >>> >>> wrote: > >>> >>> > >>> >>> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: > >>> >>> >> That does sounds pretty cool. Would be nice if the > lucene indexes > >>> >>> >> could scale along with how people will want to use > infinispan. > >>> >>> >> Probably worth playing with. > >>> >>> > > >>> >>> > Sure, this is the goal of Łukasz's work; We know compass > has > >>> >>> > some good Directories, but we're building our own as one > based > >>> >>> > on Infinispan is not yet available. > >>> >>> > > >>> >>> >> > >>> >>> >> Sent from my phone. > >>> >>> >> > >>> >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale <jeff.ramsdale(a)gmail.com > > > >>> >>> >> wrote: > >>> >>> >> > >>> >>> >>> I'm afraid I haven't followed the Infinispan-Lucene > >>> >>> >>> implementation > >>> >>> >>> closely, but have you looked at the Compass Project? > >>> >>> >>> (http://www.compass-project.org/overview.html) It > provides a > >>> >>> >>> simplified interface to Lucene (optional) as well as > Directory > >>> >>> >>> implementations built on Terracotta, Gigaspaces and > Coherence. > >>> >>> >>> The > >>> >>> >>> latter, in particular, might be a useful guide for the > Infinispan > >>> >>> >>> implementation. I believe it's mature enough to have > solved many > >>> >>> >>> of > >>> >>> >>> the most difficult problems of implementing Directory > on a > >>> >>> >>> distributed > >>> >>> >>> Map. > >>> >>> >>> > >>> >>> >>> If someone has any experience with Compass > (particularly it's > >>> >>> >>> Directory implementations) I'd be interested in hearing > about > >>> >>> >>> it... > >>> >>> >>> It's Apache 2.0 licensed, btw. > >>> >>> >>> > >>> >>> >>> -jeff > >>> >>> >>> _______________________________________________ > >>> >>> >>> infinispan-dev mailing list > >>> >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >>> >> _______________________________________________ > >>> >>> >> infinispan-dev mailing list > >>> >>> >> infinispan-dev(a)lists.jboss.org > >>> >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >>> >> > >>> >>> > > >>> >>> > _______________________________________________ > >>> >>> > infinispan-dev mailing list > >>> >>> > infinispan-dev(a)lists.jboss.org > >>> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >>> > >>> >>> _______________________________________________ > >>> >>> infinispan-dev mailing list > >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > > >>> > > >> > >> > > > > > > > > < > InfinispanDirectoryProvider_22_09_2009 > .patch>_______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Manik Surtani manik(a)jboss.org Lead, Infinispan Lead, JBoss Cache http://www.infinispan.org http://www.jbosscache.org _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Another alternative would be to see if the Compass Project would be interested in hosting it: http://www.compass-project.org/

Even if Infinispan ends up hosting it there might be value in doing some cross-pollination with the Compass folks since this aligns directly with what they are working on. -jeff 2009/9/24 Manik Surtani <manik(a)jboss.org&gt;: > Minorly off topic, but rather than working with patches, do we want > this > Directory impl in source control somewhere? > Being dependent on LGPL, it won't be accepted into Lucene's > contribs. If it > doesn't depend on any Hibernate Search code, I could host it in > Infinispan's > SVN repo... > > On 23 Sep 2009, at 13:58, Łukasz Moreń wrote: > > I agree that Infinispan case is not much different from > RamDirectory. The > major difference is that in RD (also FileDirectory) changes are not > batched > like in ID. If I do not wrap changes in InfinispanDirectory(simple > remove > tx.begin() from obtain() method and tx.commit() from release() in > InfinispanLock), and immediately commit every change made by IW it > works > well. Hovewer it makes indexing really slower, because of frequent > replication to other nodes. > Sanne it's good remark that IW commit is kind of flush. > > I've attached patch with InfinispanDirectory, failing test is > testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. > It fails > randomly. I think problem is Infinispan commit on lockRelease() in > org.apache.lucene.index.IndexWriter (line 1658) is after IW commit > () (line > 1654). > >> Is it because, the IndexWriter only clean files if no indexReaders >> are >> reading them (how would that be detected)? > > It can happen if IndexWriter clean file, and IndexReader try to > access that > cleaned file. > > 2009/9/23 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >> >> I agree It should work the same way; The IndexWriter cleans files >> whenever it likes to, it doesn't try to detect readers, and this >> shouldn't have any effect on the working of readers. >> The IndexReader opens the "SegmentsInfo" first, and immediately >> after** gets a reference to the segments listed in this >> SegmentsInfo. >> No IndexWriter will ever change an existing segment, only add new >> files or eventually delete old ones (segments merge,optimize). >> The deletion of segments is the interesting subject: when using >> Files >> it uses "delete at last close", which works because the IR needing >> it >> have it opened already**; when using the RAMDirectory they have a >> reference preventing garbage collection. >> >> ( the two "**" are assuming the same event occurred correctly, >> otherwise an exception is thrown at opening) >> >> When using Infinispan it shouldn't be much different than the >> RAMDirectory? so even if the needed segment is deleted, the IR >> holds a >> reference to the Java object locally since it was opened. >> >> Łukcasz, do you have some failing test? >> >> Sanne >> >> 2009/9/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: >>> Conceptually I don't understand why it does work in a pure file >>> system >>> directory (ie IndexReader can go and process queries with the >>> IndexWriter >>> goes about its business) and not when using Infinispan. >>> Is it because, the IndexWriter only clean files if no >>> indexReaders are >>> reading them (how would that be detected)? >>> On 22 sept. 09, at 20:46, Łukasz Moreń wrote: >>> >>> I need to provide this same lifecycle for IndexWriter as for >>> Infinispan >>> tx - >>> IW is created: tx is started, IW is commited: tx is commited. It >>> assures >>> that IndexReader doesn't read old data from directory. >>> Infinispan transaction can be started when IW acquires the lock, >>> but its >>> commit on IW lock release, as it is done so far, causes a problem: >>> >>> index writer close { >>> index writer commit(); //changes are visible for IndexReaders >>> >>> //Index reader starts reading here, i.e. tries to access >>> file "A" >>> >>> index writer lockRelease(); //changes in Infinispan directory are >>> commited, file "A" was removed, IndexReader cannot find it and >>> crashes >>> } >>> >>> I think Infinispan tx have to be commited just before IW commit, >>> and the >>> problem is where to put in code. >>> >>> W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard >>> <emmanuel(a)hibernate.org&gt; napisał: >>>> >>>> Can you explain in more details what is going on. >>>> Aside from that Workspace has been Sanne's baby lately so he >>>> will be >>>> the >>>> best to see what design will work in HSearch. That being said, I >>>> don't >>>> like >>>> the idea of subclassing / overriding very much. In my >>>> experience, it >>>> has >>>> lead to more bad and unmaintainable code than anything else. >>>> On 22 sept. 09, at 02:16, Łukasz Moreń wrote: >>>> >>>> Hi, >>>> >>>> Thanks for explanation. >>>> Maybe better I will concentrate on the first release and postpone >>>> distributed writing. >>>> >>>> There is already LockStrategy that uses Infinispan. With using >>>> it I was >>>> wrapping changes made by IndexWriter in Infinispan transaction, >>>> because >>>> of >>>> performance reasons - >>>> on lock obtaining transaction was started, on lock release >>>> transaction >>>> was >>>> commited. Hovewer Ispn transaction commit on lock release is not >>>> good >>>> idea >>>> since IndexWriter calls index commit before lock is released(and >>>> ispn >>>> transaction is committed). >>>> I was thinking to override Workspace class and getIndexWriter >>>> (start >>>> infinispan tx), commitIndexWriter (commit tx) methods to wrap >>>> IndexWrite >>>> lifecycle, but this needs few other changes. Some other ideas? >>>> >>>> Cheers, >>>> Lukasz >>>> >>>> 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >>>>> >>>>> Hi Łukasz, >>>>> you've rightful concerns, because the way the IndexWriter tries >>>>> to >>>>> achieve the lock >>>>> that will bring some trouble; As far as I remember we decided >>>>> in this >>>>> first release >>>>> to avoid multiple writer nodes because of this reasons >>>>> (that's written in your docs?) >>>>> >>>>> Actually it shouldn't be very hard to do, as the LockStrategy is >>>>> pluggable (see changes from HSEARCH-345) >>>>> and you could implement one delegating to an Infinispan eager >>>>> lock on >>>>> some key, >>>>> like the default LockStrategy takes a file lock in the index >>>>> directory. >>>>> >>>>> Maybe it's simpler to support this distributed writing instead of >>>>> sending the queue to some single >>>>> (elected) node? Would be cool, as the Document Analysis effort >>>>> would >>>>> be distributed, >>>>> but I have no idea if this would be more or less efficient than a >>>>> single node writing; it could >>>>> bring some huge data transfers along the wire during segments >>>>> merging >>>>> (basically fetching >>>>> the whole index data at each node performing a segment merge); >>>>> maybe >>>>> you'll need to >>>>> play with IndexWriter settings ( >>>>> >>>>> >>>>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... >>>>> ) >>>>> probably need to find the sweet spot for "merge_factor". >>>>> I just saw now that MergePolicy is now re-implementable, but I >>>>> hope >>>>> that won't be needed. >>>>> >>>>> Sanne >>>>> >>>>> 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: >>>>>> Hi, >>>>>> >>>>>> I'm wondering if it is reasonable to have multiple threads/nodes >>>>>> that >>>>>> modifies indexes in Lucene Directory based on Infinispan? Let's >>>>>> assume >>>>>> that >>>>>> two nodes try to update index in this same time. First one >>>>>> creates >>>>>> IndexWriter and obtains >>>>>> write lock. There is high propability that second node throws >>>>>> LockObtainFailedException (as one IndexWriter is allowed on >>>>>> single >>>>>> index) >>>>>> and index is not modified. How is that? Should be always only >>>>>> one >>>>>> node >>>>>> that >>>>>> makes changes in >>>>>> the index? >>>>>> >>>>>> Cheers, >>>>>> Lukasz >>>>>> >>>>>> W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń >>>>>> <lukasz.moren(a)gmail.com&gt; napisał: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> With using JMeter I wanted to check if Infinispan dir does not >>>>>>> crash >>>>>>> under >>>>>>> heavy load in "real" use and check performance in comparison >>>>>>> with >>>>>>> none/other >>>>>>> directories. >>>>>>> However appeared problem when multiple IndexWriters tries to >>>>>>> modify >>>>>>> index >>>>>>> (test InfinispanDirectoryTest) - random deadlocks, and Lucene >>>>>>> exceptions. >>>>>>> IndexWriter tries to access files in index that were removed >>>>>>> before. >>>>>>> I'm >>>>>>> looking into it, but not having good idea. >>>>>>> >>>>>>> Concerning the last part, I think similar thing is done in >>>>>>> InfinispanDirectoryProviderTest. Many threads are making >>>>>>> changes >>>>>>> and >>>>>>> searching (not checking if db is in sync with index). >>>>>>> If threads finish their work, with Lucene query I'm checking if >>>>>>> index >>>>>>> contains as many results as expected. Maybe you meant something >>>>>>> else? >>>>>>> Would be good to run each node in different VM. >>>>>>> >>>>>>>> Great ! Looking forward to it. What state are things in at the >>>>>>>> moment >>>>>>>> if I want to play around with it ? >>>>>>> >>>>>>> Should work with with one master(updates index) and one many >>>>>>> slave >>>>>>> nodes >>>>>>> (sends changes to master). I tried with one master and one >>>>>>> slave >>>>>>> (both >>>>>>> with >>>>>>> jms and jgroups backend) and worked ok. Still fails if multiple >>>>>>> nodes >>>>>>> want >>>>>>> to modify index. >>>>>>> >>>>>>> I've attached patch with current version. >>>>>>> >>>>>>> Cheers, >>>>>>> Łukasz >>>>>>> >>>>>>> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; >>>>>>>> >>>>>>>> Great ! Looking forward to it. What state are things in at the >>>>>>>> moment >>>>>>>> if I want to play around with it ? >>>>>>>> >>>>>>>> Sent from my phone. >>>>>>>> >>>>>>>> On 13/09/2009, at 7:26 PM, Sanne Grinovero >>>>>>>> <sanne.grinovero(a)gmail.com&gt; >>>>>>>> wrote: >>>>>>>> >>>>>>>>> 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: >>>>>>>>>> That does sounds pretty cool. Would be nice if the lucene >>>>>>>>>> indexes >>>>>>>>>> could scale along with how people will want to use >>>>>>>>>> infinispan. >>>>>>>>>> Probably worth playing with. >>>>>>>>> >>>>>>>>> Sure, this is the goal of Łukasz's work; We know compass has >>>>>>>>> some good Directories, but we're building our own as one >>>>>>>>> based >>>>>>>>> on Infinispan is not yet available. >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Sent from my phone. >>>>>>>>>> >>>>>>>>>> On 13/09/2009, at 8:37 AM, Jeff Ramsdale >>>>>>>>>> <jeff.ramsdale(a)gmail.com&gt; >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I'm afraid I haven't followed the Infinispan-Lucene >>>>>>>>>>> implementation >>>>>>>>>>> closely, but have you looked at the Compass Project? >>>>>>>>>>> (http://www.compass-project.org/overview.html) It >>>>>>>>>>> provides a >>>>>>>>>>> simplified interface to Lucene (optional) as well as >>>>>>>>>>> Directory >>>>>>>>>>> implementations built on Terracotta, Gigaspaces and >>>>>>>>>>> Coherence. >>>>>>>>>>> The >>>>>>>>>>> latter, in particular, might be a useful guide for the >>>>>>>>>>> Infinispan >>>>>>>>>>> implementation. I believe it's mature enough to have solved >>>>>>>>>>> many >>>>>>>>>>> of >>>>>>>>>>> the most difficult problems of implementing Directory on a >>>>>>>>>>> distributed >>>>>>>>>>> Map. >>>>>>>>>>> >>>>>>>>>>> If someone has any experience with Compass (particularly >>>>>>>>>>> it's >>>>>>>>>>> Directory implementations) I'd be interested in hearing >>>>>>>>>>> about >>>>>>>>>>> it... >>>>>>>>>>> It's Apache 2.0 licensed, btw. >>>>>>>>>>> >>>>>>>>>>> -jeff >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>> _______________________________________________ >>>>>>>>>> infinispan-dev mailing list >>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>> >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> infinispan-dev mailing list >>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> infinispan-dev mailing list >>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>> >>>>>> >>>> >>>> >>> >>> >>> > > < > InfinispanDirectoryProvider_22_09_2009 > .patch>_______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

Compass is Apache License, as Lucene is. Some Apache members would love to have this inside Lucene, but it appears to be a no-go because of the dependency on Infinispan (incompatible license); so either Compass can't have it either or it could be included into Lucene. There wouldn't be any problems if the code could depend on an Apache licensed API... if it's not out of question here to split into API/Impl. Sanne 2009/9/24 Manik Surtani <manik(a)jboss.org&gt;: > > On 24 Sep 2009, at 17:36, Jeff Ramsdale wrote: > >> Another alternative would be to see if the Compass Project would be >> interested in hosting it: http://www.compass-project.org/ > > Good idea.  Anybody have traction with the compass folk to propose this? > >> Even if Infinispan ends up hosting it there might be value in doing >> some cross-pollination with the Compass folks since this aligns >> directly with what they are working on. >> >> -jeff >> >> 2009/9/24 Manik Surtani <manik(a)jboss.org&gt;: >>> Minorly off topic, but rather than working with patches, do we want >>> this >>> Directory impl in source control somewhere? >>> Being dependent on LGPL, it won't be accepted into Lucene's >>> contribs.  If it >>> doesn't depend on any Hibernate Search code, I could host it in >>> Infinispan's >>> SVN repo... >>> >>> On 23 Sep 2009, at 13:58, Łukasz Moreń wrote: >>> >>> I agree that Infinispan case is not much different from >>> RamDirectory. The >>> major difference is that in RD (also FileDirectory) changes are not >>> batched >>> like in ID. If I do not wrap changes in InfinispanDirectory(simple >>> remove >>> tx.begin() from obtain() method and tx.commit() from release() in >>> InfinispanLock), and immediately commit every change made by IW it >>> works >>> well. Hovewer it makes indexing really slower, because of frequent >>> replication to other nodes. >>> Sanne it's good remark that IW commit is kind of flush. >>> >>> I've attached patch with InfinispanDirectory, failing test is >>> testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. >>> It fails >>> randomly. I think problem is Infinispan commit on lockRelease() in >>> org.apache.lucene.index.IndexWriter (line 1658) is after IW commit >>> () (line >>> 1654). >>> >>>> Is it because, the IndexWriter only clean files if no indexReaders >>>> are >>>> reading them (how would that be detected)? >>> >>> It can happen if IndexWriter clean file, and IndexReader try to >>> access that >>> cleaned file. >>> >>> 2009/9/23 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >>>> >>>> I agree It should work the same way; The IndexWriter cleans files >>>> whenever it likes to, it doesn't try to detect readers, and this >>>> shouldn't have any effect on the working of readers. >>>> The IndexReader opens the "SegmentsInfo" first, and immediately >>>> after** gets a reference to the segments listed in this >>>> SegmentsInfo. >>>> No IndexWriter will ever change an existing segment, only add new >>>> files or eventually delete old ones (segments merge,optimize). >>>> The deletion of segments is the interesting subject: when using >>>> Files >>>> it uses "delete at last close", which works because the IR needing >>>> it >>>> have it opened already**; when using the RAMDirectory they have a >>>> reference preventing garbage collection. >>>> >>>> ( the two "**" are assuming the same event occurred correctly, >>>> otherwise an exception is thrown at opening) >>>> >>>> When using Infinispan it shouldn't be much different than the >>>> RAMDirectory? so even if the needed segment is deleted, the IR >>>> holds a >>>> reference to the Java object locally since it was opened. >>>> >>>>  Łukcasz, do you have some failing test? >>>> >>>> Sanne >>>> >>>> 2009/9/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: >>>>> Conceptually I don't understand why it does work in a pure file >>>>> system >>>>> directory (ie IndexReader can go and process queries with the >>>>> IndexWriter >>>>> goes about its business) and not when using Infinispan. >>>>> Is it because, the IndexWriter only clean files if no >>>>> indexReaders are >>>>> reading them (how would that be detected)? >>>>> On 22 sept. 09, at 20:46, Łukasz Moreń wrote: >>>>> >>>>> I need to provide this same lifecycle for IndexWriter as for >>>>> Infinispan >>>>> tx - >>>>> IW is created: tx is started, IW is commited: tx is commited. It >>>>> assures >>>>> that IndexReader doesn't read old data from directory. >>>>> Infinispan transaction can be started when IW acquires the lock, >>>>> but its >>>>> commit on IW lock release, as it is done so far, causes a problem: >>>>> >>>>> index writer close { >>>>>   index writer commit(); //changes are visible for IndexReaders >>>>> >>>>>        //Index reader starts reading here, i.e. tries to access >>>>> file "A" >>>>> >>>>>   index writer lockRelease(); //changes in Infinispan directory are >>>>> commited, file "A" was removed, IndexReader cannot find it and >>>>> crashes >>>>> } >>>>> >>>>> I think Infinispan tx have to be commited just before IW commit, >>>>> and the >>>>> problem is where to put in code. >>>>> >>>>> W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard >>>>> <emmanuel(a)hibernate.org&gt; napisał: >>>>>> >>>>>> Can you explain in more details what is going on. >>>>>> Aside from that Workspace has been Sanne's baby lately so he >>>>>> will be >>>>>> the >>>>>> best to see what design will work in HSearch. That being said, I >>>>>> don't >>>>>> like >>>>>> the idea of subclassing / overriding very much. In my >>>>>> experience, it >>>>>> has >>>>>> lead to more bad and unmaintainable code than anything else. >>>>>> On 22 sept. 09, at 02:16, Łukasz Moreń wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> Thanks for explanation. >>>>>> Maybe better I will concentrate on the first release and postpone >>>>>> distributed writing. >>>>>> >>>>>> There is already LockStrategy that uses Infinispan. With using >>>>>> it I was >>>>>> wrapping changes made by IndexWriter in Infinispan transaction, >>>>>> because >>>>>> of >>>>>> performance reasons - >>>>>> on lock obtaining transaction was started, on lock release >>>>>> transaction >>>>>> was >>>>>> commited. Hovewer Ispn transaction commit on lock release is not >>>>>> good >>>>>> idea >>>>>> since IndexWriter calls index commit before lock is released(and >>>>>> ispn >>>>>> transaction is committed). >>>>>> I was thinking to override Workspace class and getIndexWriter >>>>>> (start >>>>>> infinispan tx), commitIndexWriter (commit tx) methods to wrap >>>>>> IndexWrite >>>>>> lifecycle, but this needs few other changes. Some other ideas? >>>>>> >>>>>> Cheers, >>>>>> Lukasz >>>>>> >>>>>> 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >>>>>>> >>>>>>> Hi Łukasz, >>>>>>> you've rightful concerns, because the way the IndexWriter tries >>>>>>> to >>>>>>> achieve the lock >>>>>>> that will bring some trouble; As far as I remember we decided >>>>>>> in this >>>>>>> first release >>>>>>> to avoid multiple writer nodes because of this reasons >>>>>>> (that's written in your docs?) >>>>>>> >>>>>>> Actually it shouldn't be very hard to do, as the LockStrategy is >>>>>>> pluggable (see changes from HSEARCH-345) >>>>>>> and you could implement one delegating to an Infinispan eager >>>>>>> lock on >>>>>>> some key, >>>>>>> like the default LockStrategy takes a file lock in the index >>>>>>> directory. >>>>>>> >>>>>>> Maybe it's simpler to support this distributed writing instead of >>>>>>> sending the queue to some single >>>>>>> (elected) node? Would be cool, as the Document Analysis effort >>>>>>> would >>>>>>> be distributed, >>>>>>> but I have no idea if this would be more or less efficient than a >>>>>>> single node writing; it could >>>>>>> bring some huge data transfers along the wire during segments >>>>>>> merging >>>>>>> (basically fetching >>>>>>> the whole index data at each node performing a segment merge); >>>>>>> maybe >>>>>>> you'll need to >>>>>>> play with IndexWriter settings ( >>>>>>> >>>>>>> >>>>>>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... >>>>>>> ) >>>>>>> probably need to find the sweet spot for "merge_factor". >>>>>>> I just saw now that MergePolicy is now re-implementable, but I >>>>>>> hope >>>>>>> that won't be needed. >>>>>>> >>>>>>> Sanne >>>>>>> >>>>>>> 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm wondering if it is reasonable to have multiple threads/nodes >>>>>>>> that >>>>>>>> modifies indexes in Lucene Directory based on Infinispan? Let's >>>>>>>> assume >>>>>>>> that >>>>>>>> two nodes try to update index in this same time. First one >>>>>>>> creates >>>>>>>> IndexWriter and obtains >>>>>>>> write lock. There is high propability that second node throws >>>>>>>> LockObtainFailedException (as one IndexWriter is allowed on >>>>>>>> single >>>>>>>> index) >>>>>>>> and index is not modified. How is that? Should be always only >>>>>>>> one >>>>>>>> node >>>>>>>> that >>>>>>>> makes changes in >>>>>>>> the index? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Lukasz >>>>>>>> >>>>>>>> W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń >>>>>>>> <lukasz.moren(a)gmail.com&gt; napisał: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> With using JMeter I wanted to check if Infinispan dir does not >>>>>>>>> crash >>>>>>>>> under >>>>>>>>> heavy load in "real" use and check performance in comparison >>>>>>>>> with >>>>>>>>> none/other >>>>>>>>> directories. >>>>>>>>> However appeared problem when multiple IndexWriters tries to >>>>>>>>> modify >>>>>>>>> index >>>>>>>>> (test InfinispanDirectoryTest) - random deadlocks, and Lucene >>>>>>>>> exceptions. >>>>>>>>> IndexWriter tries to access files in index that were removed >>>>>>>>> before. >>>>>>>>> I'm >>>>>>>>> looking into it, but not having good idea. >>>>>>>>> >>>>>>>>> Concerning the last part, I think similar thing is done in >>>>>>>>> InfinispanDirectoryProviderTest. Many threads are making >>>>>>>>> changes >>>>>>>>> and >>>>>>>>> searching (not checking if db is in sync with index). >>>>>>>>> If threads finish their work, with Lucene query I'm checking if >>>>>>>>> index >>>>>>>>> contains as many results as expected. Maybe you meant something >>>>>>>>> else? >>>>>>>>> Would be good to run each node in different VM. >>>>>>>>> >>>>>>>>>> Great ! Looking forward to it. What state are things in at the >>>>>>>>>> moment >>>>>>>>>> if I want to play around with it ? >>>>>>>>> >>>>>>>>> Should work with with one master(updates index) and one many >>>>>>>>> slave >>>>>>>>> nodes >>>>>>>>> (sends changes to master). I tried with one master and one >>>>>>>>> slave >>>>>>>>> (both >>>>>>>>> with >>>>>>>>> jms and jgroups backend) and worked ok. Still fails if multiple >>>>>>>>> nodes >>>>>>>>> want >>>>>>>>> to modify index. >>>>>>>>> >>>>>>>>> I've attached patch with current version. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Łukasz >>>>>>>>> >>>>>>>>> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; >>>>>>>>>> >>>>>>>>>> Great ! Looking forward to it. What state are things in at the >>>>>>>>>> moment >>>>>>>>>> if I want to play around with it ? >>>>>>>>>> >>>>>>>>>> Sent from my phone. >>>>>>>>>> >>>>>>>>>> On 13/09/2009, at 7:26 PM, Sanne Grinovero >>>>>>>>>> <sanne.grinovero(a)gmail.com&gt; >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: >>>>>>>>>>>> That does sounds pretty cool. Would be nice if the lucene >>>>>>>>>>>> indexes >>>>>>>>>>>> could scale along with how people will want to use >>>>>>>>>>>> infinispan. >>>>>>>>>>>> Probably worth playing with. >>>>>>>>>>> >>>>>>>>>>> Sure, this is the goal of Łukasz's work; We know compass has >>>>>>>>>>> some good Directories, but we're building our own as one >>>>>>>>>>> based >>>>>>>>>>> on Infinispan is not yet available. >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Sent from my phone. >>>>>>>>>>>> >>>>>>>>>>>> On 13/09/2009, at 8:37 AM, Jeff Ramsdale >>>>>>>>>>>> <jeff.ramsdale(a)gmail.com&gt; >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I'm afraid I haven't followed the Infinispan-Lucene >>>>>>>>>>>>> implementation >>>>>>>>>>>>> closely, but have you looked at the Compass Project? >>>>>>>>>>>>> (http://www.compass-project.org/overview.html) It >>>>>>>>>>>>> provides a >>>>>>>>>>>>> simplified interface to Lucene (optional) as well as >>>>>>>>>>>>> Directory >>>>>>>>>>>>> implementations built on Terracotta, Gigaspaces and >>>>>>>>>>>>> Coherence. >>>>>>>>>>>>> The >>>>>>>>>>>>> latter, in particular, might be a useful guide for the >>>>>>>>>>>>> Infinispan >>>>>>>>>>>>> implementation. I believe it's mature enough to have solved >>>>>>>>>>>>> many >>>>>>>>>>>>> of >>>>>>>>>>>>> the most difficult problems of implementing Directory on a >>>>>>>>>>>>> distributed >>>>>>>>>>>>> Map. >>>>>>>>>>>>> >>>>>>>>>>>>> If someone has any experience with Compass (particularly >>>>>>>>>>>>> it's >>>>>>>>>>>>> Directory implementations) I'd be interested in hearing >>>>>>>>>>>>> about >>>>>>>>>>>>> it... >>>>>>>>>>>>> It's Apache 2.0 licensed, btw. >>>>>>>>>>>>> >>>>>>>>>>>>> -jeff >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> infinispan-dev mailing list >>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>> >>> < >>> InfinispanDirectoryProvider_22_09_2009 >>> .patch>_______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> -- >>> Manik Surtani >>> manik(a)jboss.org >>> Lead, Infinispan >>> Lead, JBoss Cache >>> http://www.infinispan.org >>> http://www.jbosscache.org >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > -- > Manik Surtani > manik(a)jboss.org > Lead, Infinispan > Lead, JBoss Cache > http://www.infinispan.org > http://www.jbosscache.org > > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I'm Sorry, I'm no legal: this is what I understood in a chat but I could be wrong; I'll ask again for clarifications. Whatever the real reason to deny it as-is, it appears to be possible to include it in Lucene if this API/Impl split could happen, I don't mean to push anyone for this, just make it clear that the option was offered; as a personal opinion I would favour it. Sanne 2009/9/24 Jeff Ramsdale <jeff.ramsdale(a)gmail.com&gt;: > Untrue. Apache-licensed projects are certainly able to include LGPL > components. However, the Apache FOUNDATION does not allow LPGL > components in its own projects. This is not a limitation of the Apache > license, this is a matter of Foundation policy. As Compass is not a > product of the Apache Foundation it shouldn't be an issue (unless, of > course, they have their own policy against including LGPL components). > > -jeff > > On Thu, Sep 24, 2009 at 1:24 PM, Sanne Grinovero > <sanne.grinovero(a)gmail.com&gt; wrote: >> Compass is Apache License, as Lucene is. >> Some Apache members would love to have this inside Lucene, but it >> appears to be a no-go because of the dependency on Infinispan >> (incompatible license); so either Compass can't have it either or it >> could be included into Lucene. >> >> There wouldn't be any problems if the code could depend on an Apache >> licensed API... if it's not out of question here to split into >> API/Impl. >> >> Sanne >> >> 2009/9/24 Manik Surtani <manik(a)jboss.org&gt;: >>> >>> On 24 Sep 2009, at 17:36, Jeff Ramsdale wrote: >>> >>>> Another alternative would be to see if the Compass Project would be >>>> interested in hosting it: http://www.compass-project.org/ >>> >>> Good idea.  Anybody have traction with the compass folk to propose this? >>> >>>> Even if Infinispan ends up hosting it there might be value in doing >>>> some cross-pollination with the Compass folks since this aligns >>>> directly with what they are working on. >>>> >>>> -jeff >>>> >>>> 2009/9/24 Manik Surtani <manik(a)jboss.org&gt;: >>>>> Minorly off topic, but rather than working with patches, do we want >>>>> this >>>>> Directory impl in source control somewhere? >>>>> Being dependent on LGPL, it won't be accepted into Lucene's >>>>> contribs.  If it >>>>> doesn't depend on any Hibernate Search code, I could host it in >>>>> Infinispan's >>>>> SVN repo... >>>>> >>>>> On 23 Sep 2009, at 13:58, Łukasz Moreń wrote: >>>>> >>>>> I agree that Infinispan case is not much different from >>>>> RamDirectory. The >>>>> major difference is that in RD (also FileDirectory) changes are not >>>>> batched >>>>> like in ID. If I do not wrap changes in InfinispanDirectory(simple >>>>> remove >>>>> tx.begin() from obtain() method and tx.commit() from release() in >>>>> InfinispanLock), and immediately commit every change made by IW it >>>>> works >>>>> well. Hovewer it makes indexing really slower, because of frequent >>>>> replication to other nodes. >>>>> Sanne it's good remark that IW commit is kind of flush. >>>>> >>>>> I've attached patch with InfinispanDirectory, failing test is >>>>> testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. >>>>> It fails >>>>> randomly. I think problem is Infinispan commit on lockRelease() in >>>>> org.apache.lucene.index.IndexWriter (line 1658) is after IW commit >>>>> () (line >>>>> 1654). >>>>> >>>>>> Is it because, the IndexWriter only clean files if no indexReaders >>>>>> are >>>>>> reading them (how would that be detected)? >>>>> >>>>> It can happen if IndexWriter clean file, and IndexReader try to >>>>> access that >>>>> cleaned file. >>>>> >>>>> 2009/9/23 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >>>>>> >>>>>> I agree It should work the same way; The IndexWriter cleans files >>>>>> whenever it likes to, it doesn't try to detect readers, and this >>>>>> shouldn't have any effect on the working of readers. >>>>>> The IndexReader opens the "SegmentsInfo" first, and immediately >>>>>> after** gets a reference to the segments listed in this >>>>>> SegmentsInfo. >>>>>> No IndexWriter will ever change an existing segment, only add new >>>>>> files or eventually delete old ones (segments merge,optimize). >>>>>> The deletion of segments is the interesting subject: when using >>>>>> Files >>>>>> it uses "delete at last close", which works because the IR needing >>>>>> it >>>>>> have it opened already**; when using the RAMDirectory they have a >>>>>> reference preventing garbage collection. >>>>>> >>>>>> ( the two "**" are assuming the same event occurred correctly, >>>>>> otherwise an exception is thrown at opening) >>>>>> >>>>>> When using Infinispan it shouldn't be much different than the >>>>>> RAMDirectory? so even if the needed segment is deleted, the IR >>>>>> holds a >>>>>> reference to the Java object locally since it was opened. >>>>>> >>>>>>  Łukcasz, do you have some failing test? >>>>>> >>>>>> Sanne >>>>>> >>>>>> 2009/9/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: >>>>>>> Conceptually I don't understand why it does work in a pure file >>>>>>> system >>>>>>> directory (ie IndexReader can go and process queries with the >>>>>>> IndexWriter >>>>>>> goes about its business) and not when using Infinispan. >>>>>>> Is it because, the IndexWriter only clean files if no >>>>>>> indexReaders are >>>>>>> reading them (how would that be detected)? >>>>>>> On 22 sept. 09, at 20:46, Łukasz Moreń wrote: >>>>>>> >>>>>>> I need to provide this same lifecycle for IndexWriter as for >>>>>>> Infinispan >>>>>>> tx - >>>>>>> IW is created: tx is started, IW is commited: tx is commited. It >>>>>>> assures >>>>>>> that IndexReader doesn't read old data from directory. >>>>>>> Infinispan transaction can be started when IW acquires the lock, >>>>>>> but its >>>>>>> commit on IW lock release, as it is done so far, causes a problem: >>>>>>> >>>>>>> index writer close { >>>>>>>   index writer commit(); //changes are visible for IndexReaders >>>>>>> >>>>>>>        //Index reader starts reading here, i.e. tries to access >>>>>>> file "A" >>>>>>> >>>>>>>   index writer lockRelease(); //changes in Infinispan directory are >>>>>>> commited, file "A" was removed, IndexReader cannot find it and >>>>>>> crashes >>>>>>> } >>>>>>> >>>>>>> I think Infinispan tx have to be commited just before IW commit, >>>>>>> and the >>>>>>> problem is where to put in code. >>>>>>> >>>>>>> W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard >>>>>>> <emmanuel(a)hibernate.org&gt; napisał: >>>>>>>> >>>>>>>> Can you explain in more details what is going on. >>>>>>>> Aside from that Workspace has been Sanne's baby lately so he >>>>>>>> will be >>>>>>>> the >>>>>>>> best to see what design will work in HSearch. That being said, I >>>>>>>> don't >>>>>>>> like >>>>>>>> the idea of subclassing / overriding very much. In my >>>>>>>> experience, it >>>>>>>> has >>>>>>>> lead to more bad and unmaintainable code than anything else. >>>>>>>> On 22 sept. 09, at 02:16, Łukasz Moreń wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> Thanks for explanation. >>>>>>>> Maybe better I will concentrate on the first release and postpone >>>>>>>> distributed writing. >>>>>>>> >>>>>>>> There is already LockStrategy that uses Infinispan. With using >>>>>>>> it I was >>>>>>>> wrapping changes made by IndexWriter in Infinispan transaction, >>>>>>>> because >>>>>>>> of >>>>>>>> performance reasons - >>>>>>>> on lock obtaining transaction was started, on lock release >>>>>>>> transaction >>>>>>>> was >>>>>>>> commited. Hovewer Ispn transaction commit on lock release is not >>>>>>>> good >>>>>>>> idea >>>>>>>> since IndexWriter calls index commit before lock is released(and >>>>>>>> ispn >>>>>>>> transaction is committed). >>>>>>>> I was thinking to override Workspace class and getIndexWriter >>>>>>>> (start >>>>>>>> infinispan tx), commitIndexWriter (commit tx) methods to wrap >>>>>>>> IndexWrite >>>>>>>> lifecycle, but this needs few other changes. Some other ideas? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Lukasz >>>>>>>> >>>>>>>> 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; >>>>>>>>> >>>>>>>>> Hi Łukasz, >>>>>>>>> you've rightful concerns, because the way the IndexWriter tries >>>>>>>>> to >>>>>>>>> achieve the lock >>>>>>>>> that will bring some trouble; As far as I remember we decided >>>>>>>>> in this >>>>>>>>> first release >>>>>>>>> to avoid multiple writer nodes because of this reasons >>>>>>>>> (that's written in your docs?) >>>>>>>>> >>>>>>>>> Actually it shouldn't be very hard to do, as the LockStrategy is >>>>>>>>> pluggable (see changes from HSEARCH-345) >>>>>>>>> and you could implement one delegating to an Infinispan eager >>>>>>>>> lock on >>>>>>>>> some key, >>>>>>>>> like the default LockStrategy takes a file lock in the index >>>>>>>>> directory. >>>>>>>>> >>>>>>>>> Maybe it's simpler to support this distributed writing instead of >>>>>>>>> sending the queue to some single >>>>>>>>> (elected) node? Would be cool, as the Document Analysis effort >>>>>>>>> would >>>>>>>>> be distributed, >>>>>>>>> but I have no idea if this would be more or less efficient than a >>>>>>>>> single node writing; it could >>>>>>>>> bring some huge data transfers along the wire during segments >>>>>>>>> merging >>>>>>>>> (basically fetching >>>>>>>>> the whole index data at each node performing a segment merge); >>>>>>>>> maybe >>>>>>>>> you'll need to >>>>>>>>> play with IndexWriter settings ( >>>>>>>>> >>>>>>>>> >>>>>>>>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... >>>>>>>>> ) >>>>>>>>> probably need to find the sweet spot for "merge_factor". >>>>>>>>> I just saw now that MergePolicy is now re-implementable, but I >>>>>>>>> hope >>>>>>>>> that won't be needed. >>>>>>>>> >>>>>>>>> Sanne >>>>>>>>> >>>>>>>>> 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I'm wondering if it is reasonable to have multiple threads/nodes >>>>>>>>>> that >>>>>>>>>> modifies indexes in Lucene Directory based on Infinispan? Let's >>>>>>>>>> assume >>>>>>>>>> that >>>>>>>>>> two nodes try to update index in this same time. First one >>>>>>>>>> creates >>>>>>>>>> IndexWriter and obtains >>>>>>>>>> write lock. There is high propability that second node throws >>>>>>>>>> LockObtainFailedException (as one IndexWriter is allowed on >>>>>>>>>> single >>>>>>>>>> index) >>>>>>>>>> and index is not modified. How is that? Should be always only >>>>>>>>>> one >>>>>>>>>> node >>>>>>>>>> that >>>>>>>>>> makes changes in >>>>>>>>>> the index? >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Lukasz >>>>>>>>>> >>>>>>>>>> W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń >>>>>>>>>> <lukasz.moren(a)gmail.com&gt; napisał: >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> With using JMeter I wanted to check if Infinispan dir does not >>>>>>>>>>> crash >>>>>>>>>>> under >>>>>>>>>>> heavy load in "real" use and check performance in comparison >>>>>>>>>>> with >>>>>>>>>>> none/other >>>>>>>>>>> directories. >>>>>>>>>>> However appeared problem when multiple IndexWriters tries to >>>>>>>>>>> modify >>>>>>>>>>> index >>>>>>>>>>> (test InfinispanDirectoryTest) - random deadlocks, and Lucene >>>>>>>>>>> exceptions. >>>>>>>>>>> IndexWriter tries to access files in index that were removed >>>>>>>>>>> before. >>>>>>>>>>> I'm >>>>>>>>>>> looking into it, but not having good idea. >>>>>>>>>>> >>>>>>>>>>> Concerning the last part, I think similar thing is done in >>>>>>>>>>> InfinispanDirectoryProviderTest. Many threads are making >>>>>>>>>>> changes >>>>>>>>>>> and >>>>>>>>>>> searching (not checking if db is in sync with index). >>>>>>>>>>> If threads finish their work, with Lucene query I'm checking if >>>>>>>>>>> index >>>>>>>>>>> contains as many results as expected. Maybe you meant something >>>>>>>>>>> else? >>>>>>>>>>> Would be good to run each node in different VM. >>>>>>>>>>> >>>>>>>>>>>> Great ! Looking forward to it. What state are things in at the >>>>>>>>>>>> moment >>>>>>>>>>>> if I want to play around with it ? >>>>>>>>>>> >>>>>>>>>>> Should work with with one master(updates index) and one many >>>>>>>>>>> slave >>>>>>>>>>> nodes >>>>>>>>>>> (sends changes to master). I tried with one master and one >>>>>>>>>>> slave >>>>>>>>>>> (both >>>>>>>>>>> with >>>>>>>>>>> jms and jgroups backend) and worked ok. Still fails if multiple >>>>>>>>>>> nodes >>>>>>>>>>> want >>>>>>>>>>> to modify index. >>>>>>>>>>> >>>>>>>>>>> I've attached patch with current version. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Łukasz >>>>>>>>>>> >>>>>>>>>>> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; >>>>>>>>>>>> >>>>>>>>>>>> Great ! Looking forward to it. What state are things in at the >>>>>>>>>>>> moment >>>>>>>>>>>> if I want to play around with it ? >>>>>>>>>>>> >>>>>>>>>>>> Sent from my phone. >>>>>>>>>>>> >>>>>>>>>>>> On 13/09/2009, at 7:26 PM, Sanne Grinovero >>>>>>>>>>>> <sanne.grinovero(a)gmail.com&gt; >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: >>>>>>>>>>>>>> That does sounds pretty cool. Would be nice if the lucene >>>>>>>>>>>>>> indexes >>>>>>>>>>>>>> could scale along with how people will want to use >>>>>>>>>>>>>> infinispan. >>>>>>>>>>>>>> Probably worth playing with. >>>>>>>>>>>>> >>>>>>>>>>>>> Sure, this is the goal of Łukasz's work; We know compass has >>>>>>>>>>>>> some good Directories, but we're building our own as one >>>>>>>>>>>>> based >>>>>>>>>>>>> on Infinispan is not yet available. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sent from my phone. >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 13/09/2009, at 8:37 AM, Jeff Ramsdale >>>>>>>>>>>>>> <jeff.ramsdale(a)gmail.com&gt; >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm afraid I haven't followed the Infinispan-Lucene >>>>>>>>>>>>>>> implementation >>>>>>>>>>>>>>> closely, but have you looked at the Compass Project? >>>>>>>>>>>>>>> (http://www.compass-project.org/overview.html) It >>>>>>>>>>>>>>> provides a >>>>>>>>>>>>>>> simplified interface to Lucene (optional) as well as >>>>>>>>>>>>>>> Directory >>>>>>>>>>>>>>> implementations built on Terracotta, Gigaspaces and >>>>>>>>>>>>>>> Coherence. >>>>>>>>>>>>>>> The >>>>>>>>>>>>>>> latter, in particular, might be a useful guide for the >>>>>>>>>>>>>>> Infinispan >>>>>>>>>>>>>>> implementation. I believe it's mature enough to have solved >>>>>>>>>>>>>>> many >>>>>>>>>>>>>>> of >>>>>>>>>>>>>>> the most difficult problems of implementing Directory on a >>>>>>>>>>>>>>> distributed >>>>>>>>>>>>>>> Map. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If someone has any experience with Compass (particularly >>>>>>>>>>>>>>> it's >>>>>>>>>>>>>>> Directory implementations) I'd be interested in hearing >>>>>>>>>>>>>>> about >>>>>>>>>>>>>>> it... >>>>>>>>>>>>>>> It's Apache 2.0 licensed, btw. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -jeff >>>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> infinispan-dev mailing list >>>>>>>>>>>> infinispan-dev(a)lists.jboss.org >>>>>>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>> >>>>> < >>>>> InfinispanDirectoryProvider_22_09_2009 >>>>> .patch>_______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>>> -- >>>>> Manik Surtani >>>>> manik(a)jboss.org >>>>> Lead, Infinispan >>>>> Lead, JBoss Cache >>>>> http://www.infinispan.org >>>>> http://www.jbosscache.org >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> infinispan-dev mailing list >>>>> infinispan-dev(a)lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>>>> >>>> >>>> _______________________________________________ >>>> infinispan-dev mailing list >>>> infinispan-dev(a)lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >>> -- >>> Manik Surtani >>> manik(a)jboss.org >>> Lead, Infinispan >>> Lead, JBoss Cache >>> http://www.infinispan.org >>> http://www.jbosscache.org >>> >>> >>> >>> >>> >>> _______________________________________________ >>> infinispan-dev mailing list >>> infinispan-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev >>> >> >> _______________________________________________ >> infinispan-dev mailing list >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev > _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

I agree that Infinispan case is not much different from RamDirectory. The major difference is that in RD (also FileDirectory) changes are not batched like in ID. If I do not wrap changes in InfinispanDirectory(simple remove tx.begin() from obtain() method and tx.commit() from release() in InfinispanLock), and immediately commit every change made by IW it works well. Hovewer it makes indexing really slower, because of frequent replication to other nodes. Sanne it's good remark that IW commit is kind of flush. I've attached patch with InfinispanDirectory, failing test is testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. It fails randomly. I think problem is Infinispan commit on lockRelease() in org.apache.lucene.index.IndexWriter (line 1658) is after IW commit() (line 1654). > Is it because, the IndexWriter only clean files if no indexReaders are > reading them (how would that be detected)? It can happen if IndexWriter clean file, and IndexReader try to access that cleaned file. 2009/9/23 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; > > I agree It should work the same way; The IndexWriter cleans files > whenever it likes to, it doesn't try to detect readers, and this > shouldn't have any effect on the working of readers. > The IndexReader opens the "SegmentsInfo" first, and immediately > after** gets a reference to the segments listed in this SegmentsInfo. > No IndexWriter will ever change an existing segment, only add new > files or eventually delete old ones (segments merge,optimize). > The deletion of segments is the interesting subject: when using Files > it uses "delete at last close", which works because the IR needing it > have it opened already**; when using the RAMDirectory they have a > reference preventing garbage collection. > > ( the two "**" are assuming the same event occurred correctly, > otherwise an exception is thrown at opening) > > When using Infinispan it shouldn't be much different than the > RAMDirectory? so even if the needed segment is deleted, the IR holds a > reference to the Java object locally since it was opened. > >  Łukcasz, do you have some failing test? > > Sanne > > 2009/9/23 Emmanuel Bernard <emmanuel(a)hibernate.org&gt;: > > Conceptually I don't understand why it does work in a pure file system > > directory (ie IndexReader can go and process queries with the > > IndexWriter > > goes about its business) and not when using Infinispan. > > Is it because, the IndexWriter only clean files if no indexReaders are > > reading them (how would that be detected)? > > On 22 sept. 09, at 20:46, Łukasz Moreń wrote: > > > > I need to provide this same lifecycle for IndexWriter as for Infinispan > > tx - > > IW is created: tx is started, IW is commited: tx is commited. It assures > > that IndexReader doesn't read old data from directory. > > Infinispan transaction can be started when IW acquires the lock, but its > > commit on IW lock release, as it is done so far, causes a problem: > > > > index writer close { > >   index writer commit(); //changes are visible for IndexReaders > > > >        //Index reader starts reading here, i.e. tries to access file "A" > > > >   index writer lockRelease(); //changes in Infinispan directory are > > commited, file "A" was removed, IndexReader cannot find it and crashes > > } > > > > I think Infinispan tx have to be commited just before IW commit, and the > > problem is where to put in code. > > > > W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard > > <emmanuel(a)hibernate.org&gt; napisał: > >> > >> Can you explain in more details what is going on. > >> Aside from that Workspace has been Sanne's baby lately so he will be > >> the > >> best to see what design will work in HSearch. That being said, I don't > >> like > >> the idea of subclassing / overriding very much. In my experience, it > >> has > >> lead to more bad and unmaintainable code than anything else. > >> On 22 sept. 09, at 02:16, Łukasz Moreń wrote: > >> > >> Hi, > >> > >> Thanks for explanation. > >> Maybe better I will concentrate on the first release and postpone > >> distributed writing. > >> > >> There is already LockStrategy that uses Infinispan. With using it I was > >> wrapping changes made by IndexWriter in Infinispan transaction, because > >> of > >> performance reasons - > >> on lock obtaining transaction was started, on lock release transaction > >> was > >> commited. Hovewer Ispn transaction commit on lock release is not good > >> idea > >> since IndexWriter calls index commit before lock is released(and ispn > >> transaction is committed). > >> I was thinking to override Workspace class and getIndexWriter(start > >> infinispan tx), commitIndexWriter (commit tx) methods to wrap > >> IndexWrite > >> lifecycle, but this needs few other changes. Some other ideas? > >> > >> Cheers, > >> Lukasz > >> > >> 2009/9/21 Sanne Grinovero <sanne.grinovero(a)gmail.com&gt; > >>> > >>> Hi Łukasz, > >>> you've rightful concerns, because the way the IndexWriter tries to > >>> achieve the lock > >>> that will bring some trouble; As far as I remember we decided in this > >>> first release > >>> to avoid multiple writer nodes because of this reasons > >>> (that's written in your docs?) > >>> > >>> Actually it shouldn't be very hard to do, as the LockStrategy is > >>> pluggable (see changes from HSEARCH-345) > >>> and you could implement one delegating to an Infinispan eager lock on > >>> some key, > >>> like the default LockStrategy takes a file lock in the index > >>> directory. > >>> > >>> Maybe it's simpler to support this distributed writing instead of > >>> sending the queue to some single > >>> (elected) node? Would be cool, as the Document Analysis effort would > >>> be distributed, > >>> but I have no idea if this would be more or less efficient than a > >>> single node writing; it could > >>> bring some huge data transfers along the wire during segments merging > >>> (basically fetching > >>> the whole index data at each node performing a segment merge); maybe > >>> you'll need to > >>> play with IndexWriter settings ( > >>> > >>> > >>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l... > >>> ) > >>> probably need to find the sweet spot for "merge_factor". > >>> I just saw now that MergePolicy is now re-implementable, but I hope > >>> that won't be needed. > >>> > >>> Sanne > >>> > >>> 2009/9/21 Łukasz Moreń <lukasz.moren(a)gmail.com&gt;: > >>> > Hi, > >>> > > >>> > I'm wondering if it is reasonable to have multiple threads/nodes > >>> > that > >>> > modifies indexes in Lucene Directory based on Infinispan? Let's > >>> > assume > >>> > that > >>> > two nodes try to update index in this same time. First one creates > >>> > IndexWriter and obtains > >>> > write lock. There is high propability that second node throws > >>> > LockObtainFailedException (as one IndexWriter is allowed on single > >>> > index) > >>> > and index is not modified. How is that? Should be always only one > >>> > node > >>> > that > >>> > makes changes in > >>> > the index? > >>> > > >>> > Cheers, > >>> > Lukasz > >>> > > >>> > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń > >>> > <lukasz.moren(a)gmail.com&gt; napisał: > >>> >> > >>> >> Hi, > >>> >> > >>> >> With using JMeter I wanted to check if Infinispan dir does not > >>> >> crash > >>> >> under > >>> >> heavy load in "real" use and check performance in comparison with > >>> >> none/other > >>> >> directories. > >>> >> However appeared problem when multiple IndexWriters tries to modify > >>> >> index > >>> >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene > >>> >> exceptions. > >>> >> IndexWriter tries to access files in index that were removed > >>> >> before. > >>> >> I'm > >>> >> looking into it, but not having good idea. > >>> >> > >>> >> Concerning the last part, I think similar thing is done in > >>> >> InfinispanDirectoryProviderTest. Many threads are making changes > >>> >> and > >>> >> searching (not checking if db is in sync with index). > >>> >> If threads finish their work, with Lucene query I'm checking if > >>> >> index > >>> >> contains as many results as expected. Maybe you meant something > >>> >> else? > >>> >> Would be good to run each node in different VM. > >>> >> > >>> >>> Great ! Looking forward to it. What state are things in at the > >>> >>> moment > >>> >>> if I want to play around with it ? > >>> >> > >>> >> Should work with with one master(updates index) and one many slave > >>> >> nodes > >>> >> (sends changes to master). I tried with one master and one slave > >>> >> (both > >>> >> with > >>> >> jms and jgroups backend) and worked ok. Still fails if multiple > >>> >> nodes > >>> >> want > >>> >> to modify index. > >>> >> > >>> >> I've attached patch with current version. > >>> >> > >>> >> Cheers, > >>> >> Łukasz > >>> >> > >>> >> 2009/9/13 Michael Neale <michael.neale(a)gmail.com&gt; > >>> >>> > >>> >>> Great ! Looking forward to it. What state are things in at the > >>> >>> moment > >>> >>> if I want to play around with it ? > >>> >>> > >>> >>> Sent from my phone. > >>> >>> > >>> >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero > >>> >>> <sanne.grinovero(a)gmail.com&gt; > >>> >>> wrote: > >>> >>> > >>> >>> > 2009/9/12 Michael Neale <michael.neale(a)gmail.com&gt;: > >>> >>> >> That does sounds pretty cool. Would be nice if the lucene > >>> >>> >> indexes > >>> >>> >> could scale along with how people will want to use infinispan. > >>> >>> >> Probably worth playing with. > >>> >>> > > >>> >>> > Sure, this is the goal of Łukasz's work; We know compass has > >>> >>> > some good Directories, but we're building our own as one based > >>> >>> > on Infinispan is not yet available. > >>> >>> > > >>> >>> >> > >>> >>> >> Sent from my phone. > >>> >>> >> > >>> >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale > >>> >>> >> <jeff.ramsdale(a)gmail.com&gt; > >>> >>> >> wrote: > >>> >>> >> > >>> >>> >>> I'm afraid I haven't followed the Infinispan-Lucene > >>> >>> >>> implementation > >>> >>> >>> closely, but have you looked at the Compass Project? > >>> >>> >>> (http://www.compass-project.org/overview.html) It provides a > >>> >>> >>> simplified interface to Lucene (optional) as well as Directory > >>> >>> >>> implementations built on Terracotta, Gigaspaces and Coherence. > >>> >>> >>> The > >>> >>> >>> latter, in particular, might be a useful guide for the > >>> >>> >>> Infinispan > >>> >>> >>> implementation. I believe it's mature enough to have solved > >>> >>> >>> many > >>> >>> >>> of > >>> >>> >>> the most difficult problems of implementing Directory on a > >>> >>> >>> distributed > >>> >>> >>> Map. > >>> >>> >>> > >>> >>> >>> If someone has any experience with Compass (particularly it's > >>> >>> >>> Directory implementations) I'd be interested in hearing about > >>> >>> >>> it... > >>> >>> >>> It's Apache 2.0 licensed, btw. > >>> >>> >>> > >>> >>> >>> -jeff > >>> >>> >>> _______________________________________________ > >>> >>> >>> infinispan-dev mailing list > >>> >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >>> >> _______________________________________________ > >>> >>> >> infinispan-dev mailing list > >>> >>> >> infinispan-dev(a)lists.jboss.org > >>> >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >>> >> > >>> >>> > > >>> >>> > _______________________________________________ > >>> >>> > infinispan-dev mailing list > >>> >>> > infinispan-dev(a)lists.jboss.org > >>> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> >>> > >>> >>> _______________________________________________ > >>> >>> infinispan-dev mailing list > >>> >>> infinispan-dev(a)lists.jboss.org > >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev > >>> > > >>> > > >> > >> > > > > > >

You can try to incease TURNS_NUM (I've tried with 1000) and THREADS_NUM (200) fields in InfinispanDirectoryTest to make it more propable. Same problem appears also in InfinispanDirectoryProviderTest

An example stacktrace is:

21:22:44,441 ERROR InfinispanDirectoryTest:142 - Error java.io.IOException: File [ segments_nl ] for index [ indexName ] was not found at org.hibernate.search.store.infinispan.InfinispanIndexIO$InfinispanIndexInput.(InfinispanIndexIO.java:79)

at org.hibernate.search.store.infinispan.InfinispanDirectory.openInput(InfinispanDirectory.java:201) at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:214) at org.apache.lucene.index.DirectoryIndexReader$1.doBody(DirectoryIndexReader.java:95)

at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:653) at org.apache.lucene.index.DirectoryIndexReader.open(DirectoryIndexReader.java:115) at org.apache.lucene.index.IndexReader.open(IndexReader.java:316)

at org.apache.lucene.index.IndexReader.open(IndexReader.java:227) at org.apache.lucene.search.IndexSearcher.(IndexSearcher.java:55) at org.hibernate.search.test.directoryProvider.infinispan.CacheTestSupport.doReadOperation(CacheTestSupport.java:106)

at org.hibernate.search.test.directoryProvider.infinispan.InfinispanDirectoryTest$InfinispanDirectoryThread.run(InfinispanDirectoryTest.java:130)

Cheers, Lukasz

2009/9/27 Sanne Grinovero sanne.grinovero(a)gmail.com&gt;

Hi Łukasz,

I'm unable to reproduce the problem, you said it happens randomly:

I've tried several times

and I'm not getting errors. Do you know something I could do to make it happen?

Could you share a stacktrace?

Anyway if you are confident it's about the segments getting lost when

they are still being read,

you could introduce a per-segment counter of usage; like it starts at

value 1 to mark the segment

as "most current", gets a +1 vote at each reader opening it, -1

closing, and -1 deleting.

Each decrement method should check for the value reaching 0 to really delete it,

and this counting method would be easy to add inside the Directory.

When opening a new indexReader, you

1) get the SegmentsInfo

2) increment all counters (eager-lock, verify>0 or retry : set changed

counters back and get a new SegmentsInfo-->1)

3) get the needed segments

Getting a counter should be much faster than getting a segment in case

the data is downloaded

from another node, so we can use a different key while still relating

to the segment.

Sanne

2009/9/23 Łukasz Moreń lukasz.moren(a)gmail.com&gt;:

> I agree that Infinispan case is not much different from RamDirectory. The

> major difference is that in RD (also FileDirectory) changes are not batched

> like in ID. If I do not wrap changes in InfinispanDirectory(simple remove

> tx.begin() from obtain() method and tx.commit() from release() in

> InfinispanLock), and immediately commit every change made by IW it works

> well. Hovewer it makes indexing really slower, because of frequent

> replication to other nodes.

> Sanne it's good remark that IW commit is kind of flush.

>

> I've attached patch with InfinispanDirectory, failing test is

> testDirectoryWithMultipleThreads in InfinispanDirectoryTest class. It fails

> randomly. I think problem is Infinispan commit on lockRelease() in

> org.apache.lucene.index.IndexWriter (line 1658) is after IW commit() (line

> 1654).

>

>> Is it because, the IndexWriter only clean files if no indexReaders are

>> reading them (how would that be detected)?

>

> It can happen if IndexWriter clean file, and IndexReader try to access that

> cleaned file.

>

> 2009/9/23 Sanne Grinovero sanne.grinovero(a)gmail.com&gt;

>>

>> I agree It should work the same way; The IndexWriter cleans files

>> whenever it likes to, it doesn't try to detect readers, and this

>> shouldn't have any effect on the working of readers.

>> The IndexReader opens the "SegmentsInfo" first, and immediately

>> after** gets a reference to the segments listed in this SegmentsInfo.

>> No IndexWriter will ever change an existing segment, only add new

>> files or eventually delete old ones (segments merge,optimize).

>> The deletion of segments is the interesting subject: when using Files

>> it uses "delete at last close", which works because the IR needing it

>> have it opened already**; when using the RAMDirectory they have a

>> reference preventing garbage collection.

>>

>> ( the two "**" are assuming the same event occurred correctly,

>> otherwise an exception is thrown at opening)

>>

>> When using Infinispan it shouldn't be much different than the

>> RAMDirectory? so even if the needed segment is deleted, the IR holds a

>> reference to the Java object locally since it was opened.

>>

>> Łukcasz, do you have some failing test?

>>

Sanne

>>

>> 2009/9/23 Emmanuel Bernard emmanuel(a)hibernate.org&gt;:

>> > Conceptually I don't understand why it does work in a pure file system

>> > directory (ie IndexReader can go and process queries with the

>> > IndexWriter

>> > goes about its business) and not when using Infinispan.

>> > Is it because, the IndexWriter only clean files if no indexReaders are

>> > reading them (how would that be detected)?

>> > On 22 sept. 09, at 20:46, Łukasz Moreń wrote:

>

>> > I need to provide this same lifecycle for IndexWriter as for Infinispan

>> > tx -

>> > IW is created: tx is started, IW is commited: tx is commited. It assures

>> > that IndexReader doesn't read old data from directory.

>> > Infinispan transaction can be started when IW acquires the lock, but its

>> > commit on IW lock release, as it is done so far, causes a problem:

>

>> > index writer close {

>> > index writer commit(); //changes are visible for IndexReaders

>

>> > //Index reader starts reading here, ie tries to access file "A"

>

>> > index writer lockRelease(); //changes in Infinispan directory are

>> > commited, file "A" was removed, IndexReader cannot find it and crashes

>> > }

>

>> > I think Infinispan tx have to be commited just before IW commit, and the

>> > problem is where to put in code.

>

>> > W dniu 22 września 2009 18:24 użytkownik Emmanuel Bernard

>> > emmanuel(a)hibernate.org&gt; napisał:

>>

>> >> Can you explain in more details what is going on.

>> >> Aside from that Workspace has been Sanne's baby lately so he will be

>> >> the

>> >> best to see what design will work in HSearch. That being said, I don't

>> >> like

>> >> the idea of subclassing / overriding very much. In my experience, it

>> >> has

>> >> lead to more bad and unmaintainable code than anything else.

>> >> On 22 sept. 09, at 02:16, Łukasz Moreń wrote:

>>

>> >> Hi,

>>

>> >> Thanks for explanation.

>> >> Maybe better I will concentrate on the first release and postpone

>> >> distributed writing.

>>

>> >> There is already LockStrategy that uses Infinispan. With using it I was

>> >> wrapping changes made by IndexWriter in Infinispan transaction, because

>> >> of

>> >> performance reasons -

>> >> on lock obtaining transaction was started, on lock release transaction

>> >> was

>> >> commited. Hovewer Ispn transaction commit on lock release is not good

>> >> idea

>> >> since IndexWriter calls index commit before lock is released(and ispn

>> >> transaction is committed).

>> >> I was thinking to override Workspace class and getIndexWriter(start

>> >> infinispan tx), commitIndexWriter (commit tx) methods to wrap

>> >> IndexWrite

>> >> lifecycle, but this needs few other changes. Some other ideas?

>>

>> >> Cheers,

>> >> Lukasz

>>

>> >> 2009/9/21 Sanne Grinovero sanne.grinovero(a)gmail.com&gt;

>> >>>

Hi Łukasz,

>> >>> you've rightful concerns, because the way the IndexWriter tries to

>> >>> achieve the lock

>> >>> that will bring some trouble; As far as I remember we decided in this

>> >>> first release

>> >>> to avoid multiple writer nodes because of this reasons

>> >>> (that's written in your docs?)

>> >>>

>> >>> Actually it shouldn't be very hard to do, as the LockStrategy is

>> >>> pluggable (see changes from HSEARCH-345)

>> >>> and you could implement one delegating to an Infinispan eager lock on

>> >>> some key,

>> >>> like the default LockStrategy takes a file lock in the index

>> >>> directory.

>> >>>

>> >>> Maybe it's simpler to support this distributed writing instead of

>> >>> sending the queue to some single

>> >>> (elected) node? Would be cool, as the Document Analysis effort would

>> >>> be distributed,

>> >>> but I have no idea if this would be more or less efficient than a

>> >>> single node writing; it could

>> >>> bring some huge data transfers along the wire during segments merging

>> >>> (basically fetching

>> >>> the whole index data at each node performing a segment merge); maybe

>> >>> you'll need to

>> >>> play with IndexWriter settings (

>> >>>

>> >>>

>> >>> http://docs.jboss.org/hibernate/stable/search/reference/en/html_single/#l...

>> >>> )

>> >>> probably need to find the sweet spot for "merge_factor".

>> >>> I just saw now that MergePolicy is now re-implementable, but I hope

>> >>> that won't be needed.

>> >>>

Sanne

>> >>>

>> >>> 2009/9/21 Łukasz Moreń lukasz.moren(a)gmail.com&gt;:

>> >>> > Hi,

>

>> >>> > I'm wondering if it is reasonable to have multiple threads/nodes

>> >>> > that

>> >>> > modifies indexes in Lucene Directory based on Infinispan? Let's

>> >>> > assume

>> >>> > that

>> >>> > two nodes try to update index in this same time. First one creates

>> >>> > IndexWriter and obtains

>> >>> > write lock. There is high propability that second node throws

>> >>> > LockObtainFailedException (as one IndexWriter is allowed on single

>> >>> > index)

>> >>> > and index is not modified. How is that? Should be always only one

>> >>> > node

>> >>> > that

>> >>> > makes changes in

>> >>> > the index?

>

>> >>> > Cheers,

>> >>> > Lukasz

>

>> >>> > W dniu 15 września 2009 01:39 użytkownik Łukasz Moreń

>> >>> > lukasz.moren(a)gmail.com&gt; napisał:

>>

>> >>> >> Hi,

>>

>> >>> >> With using JMeter I wanted to check if Infinispan dir does not

>> >>> >> crash

>> >>> >> under

>> >>> >> heavy load in "real" use and check performance in comparison with

>> >>> >> none/other

>> >>> >> directories.

>> >>> >> However appeared problem when multiple IndexWriters tries to modify

>> >>> >> index

>> >>> >> (test InfinispanDirectoryTest) - random deadlocks, and Lucene

>> >>> >> exceptions.

>> >>> >> IndexWriter tries to access files in index that were removed

>> >>> >> before.

>> >>> >> I'm

>> >>> >> looking into it, but not having good idea.

>>

>> >>> >> Concerning the last part, I think similar thing is done in

>> >>> >> InfinispanDirectoryProviderTest. Many threads are making changes

>> >>> >> and

>> >>> >> searching (not checking if db is in sync with index).

>> >>> >> If threads finish their work, with Lucene query I'm checking if

>> >>> >> index

>> >>> >> contains as many results as expected. Maybe you meant something

>> >>> >> else?

>> >>> >> Would be good to run each node in different VM.

>>

>> >>> >>> Great ! Looking forward to it. What state are things in at the

>> >>> >>> moment

>> >>> >>> if I want to play around with it ?

>>

>> >>> >> Should work with with one master(updates index) and one many slave

>> >>> >> nodes

>> >>> >> (sends changes to master). I tried with one master and one slave

>> >>> >> (both

>> >>> >> with

>> >>> >> jms and jgroups backend) and worked ok. Still fails if multiple

>> >>> >> nodes

>> >>> >> want

>> >>> >> to modify index.

>>

>> >>> >> I've attached patch with current version.

>>

>> >>> >> Cheers,

>> >>> >> Łukasz

>>

>> >>> >> 2009/9/13 Michael Neale michael.neale(a)gmail.com&gt;

>> >>> >>>

>> >>> >>> Great ! Looking forward to it. What state are things in at the

>> >>> >>> moment

>> >>> >>> if I want to play around with it ?

>> >>> >>>

>> >>> >>> Sent from my phone.

>> >>> >>>

>> >>> >>> On 13/09/2009, at 7:26 PM, Sanne Grinovero

>> >>> >>> sanne.grinovero(a)gmail.com&gt;

>> >>> >>> wrote:

>> >>> >>>

>> >>> >>> > 2009/9/12 Michael Neale michael.neale(a)gmail.com&gt;:

>> >>> >>> >> That does sounds pretty cool. Would be nice if the lucene

>> >>> >>> >> indexes

>> >>> >>> >> could scale along with how people will want to use infinispan.

>> >>> >>> >> Probably worth playing with.

>

>> >>> >>> > Sure, this is the goal of Łukasz's work; We know compass has

>> >>> >>> > some good Directories, but we're building our own as one based

>> >>> >>> > on Infinispan is not yet available.

>

>>

>> >>> >>> >> Sent from my phone.

>>

>> >>> >>> >> On 13/09/2009, at 8:37 AM, Jeff Ramsdale

>> >>> >>> >> jeff.ramsdale(a)gmail.com&gt;

>> >>> >>> >> wrote:

>>

>> >>> >>> >>> I'm afraid I haven't followed the Infinispan-Lucene

>> >>> >>> >>> implementation

>> >>> >>> >>> closely, but have you looked at the Compass Project?

>> >>> >>> >>> (http://www.compass-project.org/overview.html) It provides a

>> >>> >>> >>> simplified interface to Lucene (optional) as well as Directory

>> >>> >>> >>> implementations built on Terracotta, Gigaspaces and Coherence.

>> >>> >>> >>> The

>> >>> >>> >>> latter, in particular, might be a useful guide for the

>> >>> >>> >>> Infinispan

>> >>> >>> >>> implementation. I believe it's mature enough to have solved

>> >>> >>> >>> many

>> >>> >>> >>> of

>> >>> >>> >>> the most difficult problems of implementing Directory on a

>> >>> >>> >>> distributed

>> >>> >>> >>> Map.

>> >>> >>> >>>

>> >>> >>> >>> If someone has any experience with Compass (particularly it's

>> >>> >>> >>> Directory implementations) I'd be interested in hearing about

>> >>> >>> >>> it...

>> >>> >>> >>> It's Apache 2.0 licensed, btw.

>> >>> >>> >>>

>> >>> >>> >>> -jeff

>> >>> >>> >>> _______________________________________________

>> >>> >>> >>> infinispan-dev mailing list

>> >>> >>> >>> infinispan-dev(a)lists.jboss.org

>> >>> >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev

>> >>> >>> >> _______________________________________________

>> >>> >>> >> infinispan-dev mailing list

>> >>> >>> >> infinispan-dev(a)lists.jboss.org

>> >>> >>> >> https://lists.jboss.org/mailman/listinfo/infinispan-dev

>>

>

>> >>> >>> > _______________________________________________

>> >>> >>> > infinispan-dev mailing list

>> >>> >>> > infinispan-dev(a)lists.jboss.org

>> >>> >>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev

>> >>> >>>

>> >>> >>> _______________________________________________

>> >>> >>> infinispan-dev mailing list

>> >>> >>> infinispan-dev(a)lists.jboss.org

>> >>> >>> https://lists.jboss.org/mailman/listinfo/infinispan-dev

>

>

>>

>>

>

>

>

>

>