[infinispan-issues] [JBoss JIRA] (ISPN-5695) Number of entities indexes in Multi node Distributed cache not correct

[Prashant Thakur (JIRA)]

     [ https://issues.jboss.org/browse/ISPN-5695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prashant Thakur updated ISPN-5695:
----------------------------------
    Steps to Reproduce: 
1. Create a distributed Cache with 1 million entries 
2. Provide the configuration as given below in xml file
    <distributed-cache name="SUBSCRIBER" mode="SYNC" owners="2" segments="100" capacity="1" l1-lifespan="0" remote-timeout="864000000">
    <locking acquire-timeout="20000" write-skew="false" concurrency-level="500" striping="false" />
      <custom-interceptors>
        <interceptor after="org.infinispan.interceptors.compat.TypeConverterInterceptor" class="com.subex.spark.common.distributedcaching.server.interceptors.LastModifiedDateInterceptor"/>
      </custom-interceptors>
      <transaction transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup" mode="NON_XA" locking="OPTIMISTIC"/>
      <state-transfer timeout="480000" enabled="true" />
    </distributed-cache>
         <!-- *************************************** -->
      <!--  Cache to store Lucene's file metadata  -->
      <!-- *************************************** -->
      <replicated-cache name="LuceneIndexesMetadata" mode="SYNC" remote-timeout="864000000">
          <state-transfer enabled="true" />
          <indexing index="NONE" />
      </replicated-cache>
   <!-- **************************** -->
      <!--  Cache to store Lucene data  -->
      <!-- **************************** -->
      <distributed-cache name="LuceneIndexesData" mode="SYNC" remote-timeout="864000000">
          <state-transfer enabled="true" />
          <indexing index="NONE" />
      </distributed-cache>
      <!-- ***************************** -->
      <!--  Cache to store Lucene locks  -->
      <!-- ***************************** -->
      <replicated-cache name="LuceneIndexesLocking" mode="SYNC" remote-timeout="864000000">
          <state-transfer enabled="true" />
          <indexing index="NONE" />
     </replicated-cache>
3. Provide following configuration in program

			SearchMapping mapping = new SearchMapping();
						mapping.entity(classType).indexed().providedId() ;
						Properties props = new Properties();
						props.put(org.hibernate.search.cfg.Environment.MODEL_MAPPING, mapping);

						props.put("hibernate.search.default.directory_provider", "infinispan");
						props.put("hibernate.search.default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager");
						props.put("hibernate.search.default.exclusive_index_use", "false");
						props.put("hibernate.search.default.indexwriter.merge_factor","30");
						props.put("hibernate.search.default.indexwriter.merge_max_size", "50000");
						props.put("hibernate.search.default.indexwriter.max_merge_docs", "6000000");
						props.put("hibernate.search.default.indexwriter.ram_buffer_size", "512");
6);
						props.put("hibernate.search.default.max_queue_length", "1000000");
						props.put("hibernate.search.default.worker.execution", "async");
						props.put("hibernate.search.default.worker.thread_pool.size", "6");
						props.put("hibernate.search.default.chunk_size", 128000);
						configurationBuilder.indexing().index(Index.LOCAL).autoConfig(true).withProperties(props);				

4. Run INdex Writer from JConsole and check the count of entities indexed
5. Run Index Writer again from JConsole and take another screen shot for entities indexed. 
6. The numbers taken are inconsistent each time we run index manager
7. Expected behaviour is numbers should remain change with each run of index manager

  was:
1. Create a distributed Cache with 1 million entries 
2. Provide the configuration as given below in xml file
    <distributed-cache name="SUBSCRIBER" mode="SYNC" owners="2" segments="100" capacity="1" l1-lifespan="0" remote-timeout="864000000">
    <locking acquire-timeout="20000" write-skew="false" concurrency-level="500" striping="false" />
      <custom-interceptors>
        <interceptor after="org.infinispan.interceptors.compat.TypeConverterInterceptor" class="com.subex.spark.common.distributedcaching.server.interceptors.LastModifiedDateInterceptor"/>
      </custom-interceptors>
      <transaction transaction-manager-lookup="org.infinispan.transaction.lookup.GenericTransactionManagerLookup" mode="NON_XA" locking="OPTIMISTIC"/>
      <state-transfer timeout="480000" enabled="true" />
    </distributed-cache>
         <!-- *************************************** -->
      <!--  Cache to store Lucene's file metadata  -->
      <!-- *************************************** -->
      <replicated-cache name="LuceneIndexesMetadata" mode="SYNC" remote-timeout="864000000">
          <state-transfer enabled="true" />
          <indexing index="NONE" />
      </replicated-cache>
   <!-- **************************** -->
      <!--  Cache to store Lucene data  -->
      <!-- **************************** -->
      <distributed-cache name="LuceneIndexesData" mode="SYNC" remote-timeout="864000000">
          <state-transfer enabled="true" />
          <indexing index="NONE" />
      </distributed-cache>
      <!-- ***************************** -->
      <!--  Cache to store Lucene locks  -->
      <!-- ***************************** -->
      <replicated-cache name="LuceneIndexesLocking" mode="SYNC" remote-timeout="864000000">
          <state-transfer enabled="true" />
          <indexing index="NONE" />
     </replicated-cache>
3. Provide following configuration in program

			SearchMapping mapping = new SearchMapping();
						mapping.entity(classType).indexed().providedId() ;
						Properties props = new Properties();
						props.put(org.hibernate.search.cfg.Environment.MODEL_MAPPING, mapping);

						props.put("hibernate.search.default.directory_provider", "infinispan");
						props.put("hibernate.search.default.indexmanager", "org.infinispan.query.indexmanager.InfinispanIndexManager");
						props.put("hibernate.search.default.exclusive_index_use", "false");
//						props.put("hibernate.search.default.locking_strategy", "none");
						props.put("hibernate.search.default.indexwriter.merge_factor","30");
						props.put("hibernate.search.default.indexwriter.merge_max_size", "50000");
						props.put("hibernate.search.default.indexwriter.max_merge_docs", "6000000");
//                        props.put("hibernate.search.default.worker.backend", "jgroups");
                        //props.put("hibernate.search.default.indexwriter.merge_max_size", 50000);
						props.put("hibernate.search.default.indexwriter.ram_buffer_size", "512");
//						props.put("hibernate.search.default.sharding_strategy.nbr_of_shards", 6);
						props.put("hibernate.search.default.max_queue_length", "1000000");
						props.put("hibernate.search.default.worker.execution", "async");
						props.put("hibernate.search.default.worker.thread_pool.size", "6");
						props.put("hibernate.search.default.chunk_size", 128000);
//						props.put("hibernate.search.default.reader.strategy", "shared");
//                        props.put("hibernate.search.default.locking_strategy", "native");
//                        props.put("hibernate.search.default.exclusive_index_use", "false");
                                                
//                        org.apache.lucene.index.IndexWriterConfig.WRITE_LOCK_TIMEOUT=5000;
//                        org.apache.lucene.store.Lock.LOCK_POLL_INTERVAL=10;
						configurationBuilder.indexing().index(Index.LOCAL).autoConfig(true).withProperties(props);				

4. Run INdex Writer from JConsole and check the count of entities indexed
5. Run Index Writer again from JConsole and take another screen shot for entities indexed. 
6. The numbers taken are inconsistent each time we run index manager
7. Expected behaviour is numbers should remain change with each run of index manager



> Number of entities indexes in Multi node Distributed cache not correct
> ----------------------------------------------------------------------
>
>                 Key: ISPN-5695
>                 URL: https://issues.jboss.org/browse/ISPN-5695
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Embedded Querying, Remote Querying
>    Affects Versions: 8.0.0.Beta3
>         Environment: CentOS
>            Reporter: Prashant Thakur
>         Attachments: instance1.png, instance2.png
>
>
> When we configure 3 node cluster we are observing a different number of entities being indexed each time we run IndexWriter Instance with cache data remaining the same.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)