Hi Sanne,<br><br>I think Lucene dir works ok. I improved it following suggestions and fixed bugs that came up.<br>Now I'm doing some performance tests, and would like to compare it with other Directories.<br>To do it, I've created web app, that inserts and queries data (one insert and one query per request).<br>
With JMeter I'm simulating dozens of request inserting and quering data simultaneously.<br>I don't have experience with performance test. if you have some suggestions how to better do it plese send me.<br><br>I hope on Monday I will send patch.<br>
<br>Cheers,<br>Lukasz<br><br><br><div class="gmail_quote">2009/9/12 Sanne Grinovero <span dir="ltr"><<a href="mailto:sanne.grinovero@gmail.com">sanne.grinovero@gmail.com</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi Łukasz,<br>
what are the news about the Lucene Directory?<br>
I'm very eager to test it, and have some time to help you if needed.<br>
<br>
Is there an updated patch to see?<br>
<br>
How are you testing it? Maybe I could help on that?<br>
<br>
Also we're going to need some "glue" to integrate the first part of your work<br>
(the jgroups backend) with the second part (the directory), as the jgroups<br>
backend will need to choose a single node to be used for indexwrites; if this<br>
node is removed from the cluster a new one should be elected.<br>
Manik had commented about this:<br>
"One way to do this is to use the JGroups coordinator as the master..."<br>
but since then the discussion on this was discontinued.<br>
<br>
Sanne<br>
<br>
<br>
2009/8/26 Łukasz Moreń <<a href="mailto:lukasz.moren@gmail.com">lukasz.moren@gmail.com</a>>:<br>
<div><div></div><div class="h5">> Hi, thanks for comments ant tips. I'm improving it.<br>
> Yes, I was checking with profiler tool and hashcode - even not so heavy -<br>
> was called often summary took some time.<br>
> There is one test where multiple threads can read or write from/to different<br>
> cache instances. However I think would be good to do some real test e.g.<br>
> with JMeter on sample app.<br>
><br>
><br>
> 2009/8/26 Manik Surtani <<a href="mailto:manik@jboss.org">manik@jboss.org</a>><br>
>><br>
>> Hi there - all looks good. Some comments:<br>
>><br>
>> Summary documentation - is this going to be published on a wiki page or<br>
>> something somewhere? Especially the Infinispan bit? I think people will<br>
>> find this info very useful...<br>
>> CacheKey - if this class is what everything is going to be used in the<br>
>> cache, for performance you should cache the hashcode. Calculate it once and<br>
>> then cache it as an instance variable. If this class is immutable it can be<br>
>> done on construction, even. Infinispan uses hashcode() a lot. :-) But<br>
>> then again, depending on how many entries live in the cache, the overhead of<br>
>> an extra int for every entry may be heavy ...<br>
>> LockCacheKey - is probably more performant if this is implemented as a<br>
>> boolean flag on CacheKey. Then you won't need to look at the class type<br>
>> when working out hashcodes<br>
>> Have you written any stress or performance tests?<br>
>><br>
>> Cheers<br>
>> Manik<br>
>> On 23 Aug 2009, at 22:53, Łukasz Moreń wrote:<br>
>><br>
>> Hi,<br>
>> Yes, I can adjust the patch next days. I've just noticed that I send<br>
>> summary in not friendly format :), better one is now attached.<br>
>> There is explanation for yours questions below.<br>
>><br>
>><br>
>> 2009/8/23 Emmanuel Bernard <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>><br>
>>><br>
>>> Hey Lukasz,<br>
>>> Your patch looks quite good and pass tests on my side.<br>
>>> I encourage others to check out the patch before we apply it (ideally<br>
>>> another person form HSearch and one person from infinispan.<br>
>>><br>
>>> Lukasz, I have a few questions/remarks though before applying it. Can you<br>
>>> answer / adjust the patch?<br>
>>> IndexWriterSetting<br>
>>> Why move to return Object in parsing from the initial int?<br>
>><br>
>> IndexWriterSetting has to set up MergeScheduler in IndexWriter. Before,<br>
>> parsing was responsible for number conversion from String to int. Now I have<br>
>> to parse class name, and build/return MergeScheduler from it.<br>
>><br>
>>><br>
>>> Move DPHelper#createInfinispanCacheManager to IDP<br>
>>> this is not something that can be shared as it creates a hard dependency<br>
>>> on infinispan otherwise.<br>
>>> in createInfinispanCacheManager<br>
>>> Don't log in error the fact that xml is not used if a default config is<br>
>>> used. Just log in trace at best.<br>
>>> Rename InfinispanCacheManagerConfigurationImpl to<br>
>>> DefaultInfinispanCacheManagerConfiguration or even better with a name<br>
>>> describing nicely the behavior of the infinispan config.<br>
>>> in InfinispanIndexOutput, is it possible to get writeBytes bigger than<br>
>>> buffer size? If yes, does newCheck creates the appropriate numbers of<br>
>>> chunks?<br>
>><br>
>> Yes it is possible. Writing process is divided into stages, during every<br>
>> stage can be written max: buffer_size bytes. At the end of the stage its<br>
>> checked if necessary is new chunk, if so new chunk is created.<br>
>><br>
>>><br>
>>> InfinispanDirectoryProvider<br>
>>> put the configuration proeprties available in the<br>
>>> InfinispanDirectoryProvider javadoc.<br>
>>> I think the default cache name should be "Hibernate Search" instead of<br>
>>> "HSInfinispanCache". We know it's in infinispan :)<br>
>>> what's the try catch opening and closing an IW about? It looks weird.<br>
>><br>
>> IW is opened with create=true parameter, first index have to be<br>
>> initialized/created. Always next IW is opened with create=false parameter,<br>
>> then data is appended to exisitng index. Similar things are done in other<br>
>> DP's.<br>
>>><br>
>>> in stop()<br>
>>> you don't close the CacheManager? How is that?<br>
>><br>
>> Yes. Should be closed.<br>
>><br>
>>><br>
>>> InfinispanCacheManagerConfigurationImpl<br>
>>> What does "Infinispan-Cluster" correspond to? Why this name? Shouldn't it<br>
>>> be "Hibernate Search cluster"?<br>
>>> Is it safe to override the GlobalConfiguration? What if JBoss AS use<br>
>>> infinispan to run?<br>
>><br>
>> This name is used to distinguish cluster used by HSearch - All nodes with<br>
>> the same name form a group. Yes, rather "Hibernate Search cluster" is better<br>
>> name. It is safe to modify GlobalConfiguration, there can be set up<br>
>> configuration for CacheManager like communication way (JGroups or something<br>
>> else), stack configuration for JGroups, etc.; where Configuration is used to<br>
>> configure specific cache. I think just the infinispan cluster name on JBoss<br>
>> AS have to be different from HSearch, then they will be independent.<br>
>><br>
>>><br>
>>> Why the use of DummyTransactionManagerLookup. Doesn't Infinispan guess<br>
>>> the right TM depending on the environment? e in JBoss As use the JBoss one<br>
>>> etc? I think GenericTransactionManagerLookup does that.<br>
>><br>
>> Yes right, I was testing it with DummyTM and forgot to change it later.<br>
>><br>
>>><br>
>>> InfinispanCacheManagerConfiguration<br>
>>> some javadoc on the methods would be useful. I don't know what do<br>
>>> implement here.<br>
>>><br>
>>> Is there a better name for Metadata? Like FileMetadata maybe?<br>
>><br>
>> Better FileMetadata or maybe FileHeader.<br>
>>><br>
>>> Where is ispn-cache-default-conf.xml used? For tests only? If not: is it<br>
>>> possible to use a programmatic version instead and what is "It's a movie<br>
>>> cache"?<br>
>><br>
>> Yes in tests only so far. However it can be used as a provided default<br>
>> configuration. I will send maybe question to infinispan group about best<br>
>> configuration parameters. "It's a movie cache" it's the name for cache<br>
>> configured in ispn-cache-default-conf.xml. In tests in this cache are stored<br>
>> indexes for entity Movie. Indexes for all other entities are stored in<br>
>> default HSInfinispanCache.<br>
>>><br>
>>> Emmanuel<br>
>>><br>
>>><br>
>>> Begin forwarded message:<br>
>>><br>
>>> From: Łukasz Moreń <<a href="mailto:lukasz.moren@gmail.com">lukasz.moren@gmail.com</a>><br>
>>> Date: 21 août 2009 02:11:03 HAEC<br>
>>> To: Emmanuel Bernard <<a href="mailto:emmanuel@hibernate.org">emmanuel@hibernate.org</a>><br>
>>> Subject: GSoC patch with Infinispan Directory Provider<br>
>>> I'm sending patch and piece of documentation - not much but necessary<br>
>>> information are included.<br>
>>> There are some todos but I didn't manage to finish it yet.<br>
>>> I changed maven jgroups dependency to 2.8.beta2, before version was<br>
>>> clashed with used by infinispan.<br>
>>> In pom file there was dependency on hibernate common annotations<br>
>>> 3.2.shapshot. It should't be 3.5?<br>
>>> Cheers,<br>
>>> Lukasz<br>
>>><br>
>>><br>
>>><br>
>><br>
>> <GSoC2009_summary.pdf>_______________________________________________<br>
>> infinispan-dev mailing list<br>
>> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
>> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
>><br>
>> --<br>
>> Manik Surtani<br>
>> <a href="mailto:manik@jboss.org">manik@jboss.org</a><br>
>> Lead, Infinispan<br>
>> Lead, JBoss Cache<br>
>> <a href="http://www.infinispan.org" target="_blank">http://www.infinispan.org</a><br>
>> <a href="http://www.jbosscache.org" target="_blank">http://www.jbosscache.org</a><br>
>><br>
>><br>
>><br>
><br>
><br>
> _______________________________________________<br>
> infinispan-dev mailing list<br>
> <a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
> <a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
><br>
</div></div></blockquote></div><br>