[JBoss JIRA] (ISPN-4678) Trim down ExpandableMarshalledValueByteStream to the actual data length before storing in cache
by Martin Gencur (JIRA)
[ https://issues.jboss.org/browse/ISPN-4678?page=com.atlassian.jira.plugin.... ]
Martin Gencur commented on ISPN-4678:
-------------------------------------
Hmm, growing by 1.5 factor instead of 2 is possible. This would require a perf. test to actually see what is better, and for different types of data. I wouldn't like to over-engineer this, though. Maybe growing by factor of 2 is still better than 1.5.
Also, I still think it makes sense to store only the actual data, not more. The time it takes to copy the byte array seems very small compared to the whole chain of invocations that happen in Infinispan. And it also appears that storing smaller data has a very positive effect on performance.
> Trim down ExpandableMarshalledValueByteStream to the actual data length before storing in cache
> -----------------------------------------------------------------------------------------------
>
> Key: ISPN-4678
> URL: https://issues.jboss.org/browse/ISPN-4678
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core
> Affects Versions: 7.0.0.Beta1
> Reporter: Martin Gencur
> Assignee: Martin Gencur
>
> When storeAsBinary configuration flag is used, the ExpandableMarshalledValueByteStream is used to serialize objects to byte arrays.
> However, for value size up to 4kB, almost half of the space for each cache entry can be wasted because the ByteStream is stored as is - with the additional space for future expansion.
> For value size greater than 4kB, only additional 25% of actual value size can be wasted. But as these values can be quite large, the wasted space gets bigger too.
> It is not necessary to store ExpandableMarshalledValueByteStream as is and can be shrinked to actual data size before storing in cache.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4667) Query DSL fails to load entity class in Wildfly
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4667?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4667:
-----------------------------------------------
Vojtech Juranek <vjuranek(a)redhat.com> changed the Status of [bug 1134112|https://bugzilla.redhat.com/show_bug.cgi?id=1134112] from ON_QA to VERIFIED
> Query DSL fails to load entity class in Wildfly
> -----------------------------------------------
>
> Key: ISPN-4667
> URL: https://issues.jboss.org/browse/ISPN-4667
> Project: Infinispan
> Issue Type: Bug
> Components: Embedded Querying
> Affects Versions: 7.0.0.Beta1
> Reporter: Gustavo Fernandes
> Assignee: Gustavo Fernandes
> Fix For: 7.0.0.Beta2
>
>
> When using the as modules and an application that depends on those modules, the following code:
> {code}
> Cache cache = ...
> Query query = Search.getQueryFactory(cache).from(Book.class).having("field").contains("value").toBuilder().build();
> List<Object> results = query.list();
> {code}
> fails with
> {code}
> java.lang.IllegalStateException: Unknown entity name org.infinispan.test.integration.as.query.Book
> at org.hibernate.hql.lucene.internal.ClassBasedLuceneQueryResolverDelegate.registerPersisterSpace(ClassBasedLuceneQueryResolverDelegate.java:104)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.entityName(GeneratedHQLResolver.java:12729)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.persisterSpaceRoot(GeneratedHQLResolver.java:3064)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.persisterSpace(GeneratedHQLResolver.java:2956)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.persisterSpaces(GeneratedHQLResolver.java:2893)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.fromClause(GeneratedHQLResolver.java:2803)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.selectFrom(GeneratedHQLResolver.java:2704)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.querySpec(GeneratedHQLResolver.java:2182)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.queryExpression(GeneratedHQLResolver.java:2106)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.queryStatement(GeneratedHQLResolver.java:1745)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.queryStatementSet(GeneratedHQLResolver.java:1658)
> at org.hibernate.hql.ast.origin.hql.resolve.GeneratedHQLResolver.statement(GeneratedHQLResolver.java:654)
> at org.hibernate.hql.ast.spi.QueryResolverProcessor.process(QueryResolverProcessor.java:52)
> at org.hibernate.hql.QueryParser.parseQuery(QueryParser.java:82)
> at org.infinispan.query.dsl.embedded.impl.EmbeddedLuceneQueryBuilder.build(EmbeddedLuceneQueryBuilder.java:43)
> at org.infinispan.query.dsl.embedded.impl.EmbeddedLuceneQueryBuilder.build(EmbeddedLuceneQueryBuilder.java:19)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4237) Security manager test QueryAuthorizationTest.testQuery fails
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4237?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant updated ISPN-4237:
----------------------------------
Status: Pull Request Sent (was: Reopened)
Git Pull Request: https://github.com/infinispan/infinispan/pull/2849
> Security manager test QueryAuthorizationTest.testQuery fails
> ------------------------------------------------------------
>
> Key: ISPN-4237
> URL: https://issues.jboss.org/browse/ISPN-4237
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.0.Alpha3
> Reporter: Vojtech Juranek
> Assignee: Tristan Tarrant
> Labels: testsuite_stability
> Fix For: 7.0.0.Alpha5
>
> Attachments: QueryAuthorizationTest.log.zip
>
>
> {{org.infinispan.security.QueryAuthorizationTest.testQuery}} fails (no matter which JDK is used) with
> {noformat}
> java.lang.IllegalArgumentException: Indexing was not enabled on this cache. interface org.hibernate.search.spi.SearchFactoryIntegrator not found in registry
> at org.infinispan.query.impl.ComponentRegistryUtils.getComponent(ComponentRegistryUtils.java:27)
> at org.infinispan.query.impl.ComponentRegistryUtils.getComponent(ComponentRegistryUtils.java:20)
> at org.infinispan.query.impl.SearchManagerImpl.<init>(SearchManagerImpl.java:42)
> at org.infinispan.query.Search.getSearchManager(Search.java:21)
> at org.infinispan.security.QueryAuthorizationTest.queryTest(QueryAuthorizationTest.java:97)
> at org.infinispan.security.QueryAuthorizationTest.access$300(QueryAuthorizationTest.java:32)
> at org.infinispan.security.QueryAuthorizationTest$4.run(QueryAuthorizationTest.java:113)
> at org.infinispan.security.QueryAuthorizationTest$4.run(QueryAuthorizationTest.java:109)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.infinispan.security.QueryAuthorizationTest.testQuery(QueryAuthorizationTest.java:109)
> {noformat}
> Example of test failure in Jenkins on RHEL6 and Oracle JDK 7 is e.g. [here|https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-tes...].
> Maybe it's some setup issue, as on Jenkins it fails on all platforms, while I;m not able to reproduce it on my machine.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4237) Security manager test QueryAuthorizationTest.testQuery fails
by Tristan Tarrant (JIRA)
[ https://issues.jboss.org/browse/ISPN-4237?page=com.atlassian.jira.plugin.... ]
Tristan Tarrant reopened ISPN-4237:
-----------------------------------
Assignee: Tristan Tarrant (was: Dan Berindei)
> Security manager test QueryAuthorizationTest.testQuery fails
> ------------------------------------------------------------
>
> Key: ISPN-4237
> URL: https://issues.jboss.org/browse/ISPN-4237
> Project: Infinispan
> Issue Type: Bug
> Components: Test Suite - Core
> Affects Versions: 7.0.0.Alpha3
> Reporter: Vojtech Juranek
> Assignee: Tristan Tarrant
> Labels: testsuite_stability
> Fix For: 7.0.0.Alpha5
>
> Attachments: QueryAuthorizationTest.log.zip
>
>
> {{org.infinispan.security.QueryAuthorizationTest.testQuery}} fails (no matter which JDK is used) with
> {noformat}
> java.lang.IllegalArgumentException: Indexing was not enabled on this cache. interface org.hibernate.search.spi.SearchFactoryIntegrator not found in registry
> at org.infinispan.query.impl.ComponentRegistryUtils.getComponent(ComponentRegistryUtils.java:27)
> at org.infinispan.query.impl.ComponentRegistryUtils.getComponent(ComponentRegistryUtils.java:20)
> at org.infinispan.query.impl.SearchManagerImpl.<init>(SearchManagerImpl.java:42)
> at org.infinispan.query.Search.getSearchManager(Search.java:21)
> at org.infinispan.security.QueryAuthorizationTest.queryTest(QueryAuthorizationTest.java:97)
> at org.infinispan.security.QueryAuthorizationTest.access$300(QueryAuthorizationTest.java:32)
> at org.infinispan.security.QueryAuthorizationTest$4.run(QueryAuthorizationTest.java:113)
> at org.infinispan.security.QueryAuthorizationTest$4.run(QueryAuthorizationTest.java:109)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.infinispan.security.QueryAuthorizationTest.testQuery(QueryAuthorizationTest.java:109)
> {noformat}
> Example of test failure in Jenkins on RHEL6 and Oracle JDK 7 is e.g. [here|https://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/edg-60-ispn-tes...].
> Maybe it's some setup issue, as on Jenkins it fails on all platforms, while I;m not able to reproduce it on my machine.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4575) Map/Reduce incorrect results with a non-shared non-tx intermediate cache
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4575?page=com.atlassian.jira.plugin.... ]
Dan Berindei commented on ISPN-4575:
------------------------------------
Whichever you prefer, I would go for CreateCacheCommand.perform.
Since you send the create cache command to all the nodes in the cluster, the expected size should be the number of nodes in the cluster (i.e. {{transport.getMembers().size()}}). Of course, reality creeps in, nodes can join or leave the cluster while you're waiting, or even while the command is in flight. So you actually need to take the initial list of members on the originator, send it to every node as a command parameter, and remove leavers on every iteration. Something like this should work, though the {{sleep}} still isn't very nice:
{code}
int expectedSize = cacheMembers.size();
while (stm.getCacheTopology().getMembers() != expectedSize && stm.getCacheTopology().getPendingCH() != null) {
Thread.sleep(50);
cacheMembers.retain(rpcManager.getTransport().getMembers());
expectedSize = cacheMembers.size();
}
{code}
To be fair, I don't think the current M/R code works very well with nodes joining or leaving the cluster anyway, so perhaps it would be simpler to get the cluster size once at the beginning of the loop and add a timeout.
> Map/Reduce incorrect results with a non-shared non-tx intermediate cache
> ------------------------------------------------------------------------
>
> Key: ISPN-4575
> URL: https://issues.jboss.org/browse/ISPN-4575
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Distributed Execution and Map/Reduce
> Affects Versions: 7.0.0.Alpha5
> Reporter: Dan Berindei
> Assignee: Vladimir Blagojevic
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 7.0.0.Beta2
>
>
> In a non-tx cache, if a command is started with topology id {{T}}, and when it is replicated on another node the distribution interceptor sees topology {{T+1}}, it throws an {{OutdatedTopologyException}}. The originator of the command will then retry the command, setting topology {{T+1}}.
> When this happens with a {{PutKeyValueCommand(k, MapReduceManagerImpl.DeltaAwareList)}}, it can lead to duplicate intermediate values.
> Say _A_ is the primary owner of {{k}} in {{T}}, _B_ is a backup owner both in {{T}} and {{T+1}}, and _C_ is the backup owner in {{T}} and the primary owner in {{T+1}} (i.e. _C_ just joined and a rebalance is in progress during {{T}} - see {{NonTxBackupOwnerBecomingPrimaryOwnerTest}}).
> _A_ starts the {{PutKeyValueCommand}} and replicates it to _B_ and _C_. _C_ applies the command, but _B_ already has topology {{T+1}} and throws an {{OutdatedTopologyException}}. _A_ installs topology {{T+1}}, sends the command to _C_ (as the new primary owner), which replicates it to _B_ and then applies it locally a second time.
> This scenario can happen during a M/R task even without nodes joining or leaving. That's because {{CreateCacheCommand}} only calls {{getCache()}} on each member, it doesn't wait for the cache to have a certain number of members or for state transfer to be complete for all the members. The last member to join the intermediate cache is guaranteed to have topology {{T+1}}, but the others may have topology {{T}} by the time the combine phase starts inserting values in the intermediate cache.
> I have seen the {{OutdatedTopologyException}} happen pretty often during the test suite, especially after I removed the duplicate {{invokeRemotely}} call in {{MapReduceTask.executeTaskInit()}}. Most of them were harmless, but there was one failure in CI: http://ci.infinispan.org/viewLog.html?buildId=9811&tab=buildResultsDiv&bu...
> A short-term fix would be to wait for all the members to finish joining in {{CreateCacheCommand}}. Long-term, M/R tasks should be resilient to topology changes, so we should investigate making {{PutKeyValue(k, DeltaAwareList)}} handle {{OutdatedTopologyException}} s.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4693) Locks acquired before a partition will never be released
by Erik Salter (JIRA)
[ https://issues.jboss.org/browse/ISPN-4693?page=com.atlassian.jira.plugin.... ]
Erik Salter commented on ISPN-4693:
-----------------------------------
Yes, in my prototype (using the aforementioned implementation), I use the completedTxTimeout value. I defaulted it to 60 seconds in the 6.0.x branch to keep the default greater than the transaction manager timeout.
> Locks acquired before a partition will never be released
> --------------------------------------------------------
>
> Key: ISPN-4693
> URL: https://issues.jboss.org/browse/ISPN-4693
> Project: Infinispan
> Issue Type: Bug
> Components: Core
> Affects Versions: 6.0.2.Final
> Reporter: Erik Salter
> Assignee: Dan Berindei
> Fix For: 7.0.0.Final
>
>
> Consider the following case:
> 1. A prepares a transaction for key K on nodes B and C.
> 2. The primary owner B acquires and registers this lock.
> 3. The cluster partitions and A,C remain in partition one, and B in a separate partition.
> 4. The transaction rolls back. However, since B left the cluster, the rollback will not be sent to B.
> 5. B rejoins the cluster. It still has a lock acquired for key K.
> This can lead to a lock being acquired indefinitely.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4683) DefaultConsistentHash segment size is not not computed correctly
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4683?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-4683:
-----------------------------------
Fix Version/s: 7.0.0.Final
> DefaultConsistentHash segment size is not not computed correctly
> ----------------------------------------------------------------
>
> Key: ISPN-4683
> URL: https://issues.jboss.org/browse/ISPN-4683
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Remote Protocols
> Affects Versions: 6.0.2.Final, 7.0.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Priority: Critical
> Fix For: 7.0.0.Beta2, 7.0.0.Final
>
>
> The segment size is computed using {{o.i.commons.util.Util.getSegmentSize()}} both on the server and in the Java client. The computation uses {{float}} division, but {{float}} precision is not good enough and sometimes the segments are too small (e.g. with 200 segments, the segment size is 10737418 instead of 10737419). This means the keys with a hash of {{Integer.MAX_VALUE}} or {{-1}} are not properly mapped to a segment.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months
[JBoss JIRA] (ISPN-4682) Improve SyncConsistentHashFactory key distribution
by Galder Zamarreño (JIRA)
[ https://issues.jboss.org/browse/ISPN-4682?page=com.atlassian.jira.plugin.... ]
Galder Zamarreño updated ISPN-4682:
-----------------------------------
Fix Version/s: 7.0.0.Final
> Improve SyncConsistentHashFactory key distribution
> --------------------------------------------------
>
> Key: ISPN-4682
> URL: https://issues.jboss.org/browse/ISPN-4682
> Project: Infinispan
> Issue Type: Enhancement
> Components: Core
> Affects Versions: 6.0.2.Final, 7.0.0.Beta1
> Reporter: Dan Berindei
> Assignee: Dan Berindei
> Labels: performance
> Fix For: 7.0.0.Beta2, 7.0.0.Final
>
>
> {{SyncConsistentHashFactory}} and {{TopologyAwareSyncConsistentHashFactory}} distribute the segments between nodes randomly, and the number of segments distributed to different nodes is not very even.
> They should have a fail-safe mechanism to limit the variation in segment number between nodes.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 7 months