August 2014 - infinispan-issues - Jboss List Archives

[JBoss JIRA] (ISPN-4650) MassIndexer should not use UpdateDocument when adding to Lucene

by Gustavo Fernandes (JIRA)

[ https://issues.jboss.org/browse/ISPN-4650?page=com.atlassian.jira.plugin.... ] Gustavo Fernandes updated ISPN-4650: ------------------------------------ Description: The MassIndexer currently issues an Update operation to hibernate search backend, which in turn becomes a delete plus and add in the index. Lucene buffers those deletes queries and during merge it tries to 'apply' those deletes wasting a massive amount of time doing seeks and queries unnecessarily. Since the mass indexer wipes the index at the beginning, it should simply issue an add operation. Performance wise this make a huge difference: * indexing 50k documents brings down the indexing time from 195s to 33s * indexing 200k documents brings down the indexing time from 600s to 55s was: The MassIndexer currently issues an Update operation to hibernate search backend, which in turn becomes a delete plus and add in the index. Lucene buffers those deletes queries and during merge it tries to 'apply' those deletes wasting a massive amount of time doing seeks and queries unnecessarily. Since the mass indexer wipes the index at the beginning, it should simply issue an add operation. Performance wise this make a huge difference: * indexing 50k documents brings the final merge time from 195s to 33s * indexing 200k documents brings the final merge time from 600s to 55s > MassIndexer should not use UpdateDocument when adding to Lucene > --------------------------------------------------------------- > > Key: ISPN-4650 > URL: https://issues.jboss.org/browse/ISPN-4650 > Project: Infinispan > Issue Type: Feature Request > Security Level: Public(Everyone can see) > Components: Embedded Querying > Affects Versions: 7.0.0.Beta1 > Reporter: Gustavo Fernandes > Assignee: Gustavo Fernandes > Fix For: 7.0.0.Beta2 > > > The MassIndexer currently issues an Update operation to hibernate search backend, which in turn becomes a delete plus and add in the index. > Lucene buffers those deletes queries and during merge it tries to 'apply' those deletes wasting a massive amount of time doing seeks and queries unnecessarily. > Since the mass indexer wipes the index at the beginning, it should simply issue an add operation. Performance wise this make a huge difference: > * indexing 50k documents brings down the indexing time from 195s to 33s > * indexing 200k documents brings down the indexing time from 600s to 55s -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4650) MassIndexer should not use UpdateDocument when adding to Lucene

by Gustavo Fernandes (JIRA)

Gustavo Fernandes created ISPN-4650: --------------------------------------- Summary: MassIndexer should not use UpdateDocument when adding to Lucene Key: ISPN-4650 URL: https://issues.jboss.org/browse/ISPN-4650 Project: Infinispan Issue Type: Feature Request Security Level: Public (Everyone can see) Components: Embedded Querying Affects Versions: 7.0.0.Beta1 Reporter: Gustavo Fernandes Assignee: Gustavo Fernandes Fix For: 7.0.0.Beta2 The MassIndexer currently issues an Update operation to hibernate search backend, which in turn becomes a delete plus and add in the index. Lucene buffers those deletes queries and during merge it tries to 'apply' those deletes wasting a massive amount of time doing seeks and queries unnecessarily. Since the mass indexer wipes the index at the beginning, it should simply issue an add operation. Performance wise this make a huge difference: * indexing 50k documents brings the final merge time from 195s to 33s * indexing 200k documents brings the final merge time from 600s to 55s -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4649) RemoveCommand does not activate the key in Passivation

by Pedro Ruivo (JIRA)

Pedro Ruivo created ISPN-4649: --------------------------------- Summary: RemoveCommand does not activate the key in Passivation Key: ISPN-4649 URL: https://issues.jboss.org/browse/ISPN-4649 Project: Infinispan Issue Type: Bug Security Level: Public (Everyone can see) Affects Versions: 7.0.0.Beta1 Reporter: Pedro Ruivo Assignee: Pedro Ruivo Fix For: 7.0.0.Beta2 The RemoveCommand does not activate the key in CacheStore. This creates an inconsistency because the key is never really removed. More precisely, the key is removed from DataContainer but not from CacheStore and during any operation, the old value is loaded from the CacheStore. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4649) RemoveCommand does not activate the key in Passivation

by Pedro Ruivo (JIRA)

[ https://issues.jboss.org/browse/ISPN-4649?page=com.atlassian.jira.plugin.... ] Work on ISPN-4649 started by Pedro Ruivo. > RemoveCommand does not activate the key in Passivation > ------------------------------------------------------ > > Key: ISPN-4649 > URL: https://issues.jboss.org/browse/ISPN-4649 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Affects Versions: 7.0.0.Beta1 > Reporter: Pedro Ruivo > Assignee: Pedro Ruivo > Fix For: 7.0.0.Beta2 > > > The RemoveCommand does not activate the key in CacheStore. This creates an inconsistency because the key is never really removed. More precisely, the key is removed from DataContainer but not from CacheStore and during any operation, the old value is loaded from the CacheStore. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4649) RemoveCommand does not activate the key in Passivation

by Pedro Ruivo (JIRA)

[ https://issues.jboss.org/browse/ISPN-4649?page=com.atlassian.jira.plugin.... ] Pedro Ruivo commented on ISPN-4649: ----------------------------------- Note: no tests will be added since the tests from https://issues.jboss.org/browse/ISPN-4559 will cover this scenario. > RemoveCommand does not activate the key in Passivation > ------------------------------------------------------ > > Key: ISPN-4649 > URL: https://issues.jboss.org/browse/ISPN-4649 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Affects Versions: 7.0.0.Beta1 > Reporter: Pedro Ruivo > Assignee: Pedro Ruivo > Fix For: 7.0.0.Beta2 > > > The RemoveCommand does not activate the key in CacheStore. This creates an inconsistency because the key is never really removed. More precisely, the key is removed from DataContainer but not from CacheStore and during any operation, the old value is loaded from the CacheStore. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4638) Add capability to remove filter/converter factories from Hot Rod server

by Galder Zamarreño (JIRA)

[ https://issues.jboss.org/browse/ISPN-4638?page=com.atlassian.jira.plugin.... ] Galder Zamarreño updated ISPN-4638: ----------------------------------- Status: Pull Request Sent (was: Open) Git Pull Request: https://github.com/infinispan/infinispan/pull/2805 > Add capability to remove filter/converter factories from Hot Rod server > ----------------------------------------------------------------------- > > Key: ISPN-4638 > URL: https://issues.jboss.org/browse/ISPN-4638 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Remote Protocols, Server > Affects Versions: 7.0.0.Beta1 > Reporter: Galder Zamarreño > Assignee: Galder Zamarreño > Fix For: 7.0.0.Beta2 > > > As a result of ISPN-3950, we've added ability to add filter/converter factories at runtime to Hot Rod servers. These however need to be removed when they're not needed any more, e.g. when filter/converter factory jars undeployed. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3076) Infinispan should enable use of log4j2

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-3076?page=com.atlassian.jira.plugin.... ] Dan Berindei commented on ISPN-3076: ------------------------------------ Thanks Bela, for now the blocker is in jboss-logging. I may try to switch the jboss-logging backend to JDK logging and/or jboss-logmanager (in which case your CustomLogFactory will come in handy). > Infinispan should enable use of log4j2 > -------------------------------------- > > Key: ISPN-3076 > URL: https://issues.jboss.org/browse/ISPN-3076 > Project: Infinispan > Issue Type: Enhancement > Security Level: Public(Everyone can see) > Components: Core > Affects Versions: 5.2.5.Final, 6.0.0.Final > Environment: Any > Reporter: Divya Mehra > Assignee: Dan Berindei > > log4j locks up on a certain amount of load, especially under concurrent access. The recommendation was to move up to log4j2. This can be done by removing > 1 JAR (log4j.jar) and adding 2 JARs (log4j-bridge and log4j-core.jar). > Also refer: https://issues.jboss.org/browse/ISPN-2976 -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-3979) JdbcStringBasedStore loading of Key2StringMapper class is too restrictive

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-3979?page=com.atlassian.jira.plugin.... ] Dan Berindei reassigned ISPN-3979: ---------------------------------- Assignee: William Burns (was: Dan Berindei) > JdbcStringBasedStore loading of Key2StringMapper class is too restrictive > ------------------------------------------------------------------------- > > Key: ISPN-3979 > URL: https://issues.jboss.org/browse/ISPN-3979 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Loaders and Stores > Affects Versions: 6.0.1.Final > Reporter: Paul Ferraro > Assignee: William Burns > Priority: Critical > Fix For: 7.0.0.Final > > > Currently the Key2StringMapper of a JdbcStringBasedStore is specified to the JdbcStringBasedStoreConfigurationBuilder as a class name. Yes, there is a method that accepts a Class<? extends Key2StringMapper>, but that simply stores the getName() of the Class! The JdbcStringBasedStore loads the class using the class loader of the JdbcStringBasedStore class (via Class.forName(...). This is too restrictive. At the very least, JdbcStringBasedStore should use the classLoader defined in the cache Configuration (i.e. via ConfigurationBuilder.classLoader()) to load the class. Why not also allow the JdbcStringBasedStoreConfigurationBuilder to specify a Key2StringMapper instance? > I would really like to make use of Key2StringMapper in WildFly to allow users the option to persist web sessions and SFSBs via the JdbcStringBasedStore (instead of the binary bucket-based store), but the current mechanism is incompatible with use cases where the Key2StringMapper is not known to class loader of the infinispan module. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4639) IndexingConfigurationBuilder query module validation is wrong

by Dan Berindei (JIRA)

[ https://issues.jboss.org/browse/ISPN-4639?page=com.atlassian.jira.plugin.... ] Dan Berindei reassigned ISPN-4639: ---------------------------------- Assignee: William Burns (was: Dan Berindei) > IndexingConfigurationBuilder query module validation is wrong > ------------------------------------------------------------- > > Key: ISPN-4639 > URL: https://issues.jboss.org/browse/ISPN-4639 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Configuration > Affects Versions: 7.0.0.Beta1 > Reporter: Paul Ferraro > Assignee: William Burns > Priority: Critical > > The query module is ultimately loaded by the ComponentRegistry, which will load its modules via the classloader returned by GlobalConfiguration.classLoader(). However, the logic in IndexingConfigurationBuilder.validate(...) tests that the query module classes are accessible from the classloader that loaded the IndexingConfigurationBuilder itself. > In WildFly, to use querying, one would use the following configuration: > <cache-container module="org.infinispan.query"/> > Internally, this configures Infinispan's GlobalConfiguration with the classloader of the "org.infinispan.query" module. > However, the IndexingConfigurationBuilder class is contained in the "org.infinispan" - which does not depend on the "org.infinispan.query" module. Consequently, IndexingConfigurationBuilder validation fails, even though the module would have been successfully loaded at runtime. -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

[JBoss JIRA] (ISPN-4323) ConditionalOperationsConcurrentWriteSkewTest fails randomly

by William Burns (JIRA)

[ https://issues.jboss.org/browse/ISPN-4323?page=com.atlassian.jira.plugin.... ] William Burns updated ISPN-4323: -------------------------------- Status: Pull Request Sent (was: Coding In Progress) Git Pull Request: https://github.com/infinispan/infinispan/pull/2804 > ConditionalOperationsConcurrentWriteSkewTest fails randomly > ----------------------------------------------------------- > > Key: ISPN-4323 > URL: https://issues.jboss.org/browse/ISPN-4323 > Project: Infinispan > Issue Type: Bug > Security Level: Public(Everyone can see) > Components: Core > Affects Versions: 7.0.0.Alpha1, 7.0.0.Alpha2, 7.0.0.Alpha3, 7.0.0.Alpha4 > Environment: RHEL and Open JDK > Reporter: Vitalii Chepeliuk > Assignee: William Burns > Priority: Blocker > Labels: testsuite_stability > Attachments: ConditionalOperationsConcurrentWriteSkewTest.log.zip, ConditionalOperationsConcurrentWriteSkewTest.tar.bz2 > > > Following tests fail with TImeout exception > org.infinispan.api.ConditionalOperationsConcurrentWriteSkewTest.testConditionalRemove > org.infinispan.api.ConditionalOperationsConcurrentWriteSkewTest.testReplace > org.infinispan.api.ConditionalOperationsConcurrentWriteSkewTest.testPutIfAbsent > {noformat} > java.lang.AssertionError: java.lang.RuntimeException: java.util.concurrent.TimeoutException > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertFalse(Assert.java:64) > at org.infinispan.api.ConditionalOperationsConcurrentTest.testOnCaches(ConditionalOperationsConcurrentTest.java:129) > at org.infinispan.api.ConditionalOperationsConcurrentWriteSkewTest.testConditionalRemove(ConditionalOperationsConcurrentWriteSkewTest.java:56) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80) > at org.testng.internal.Invoker.invokeMethod(Invoker.java:714) > at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901) > at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231) > at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127) > at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111) > at org.testng.TestRunner.privateRun(TestRunner.java:767) > at org.testng.TestRunner.run(TestRunner.java:617) > at org.testng.SuiteRunner.runTest(SuiteRunner.java:334) > at org.testng.SuiteRunner.access$000(SuiteRunner.java:37) > at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368) > at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:744) > {noformat} > Failing tests in Jenkins > https://jenkins.mw.lab.eng.bos.redhat.com/hudson/view/JDG/view/FUNC/job/e... -- This message was sent by Atlassian JIRA (v6.2.6#6264)

11 years, 7 months

1
0
0 / 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

infinispan-issues August 2014