[JBoss JIRA] (ISPN-4751) Hibernate search, infinispan and Amazon S3 - IllegalArgumentException: bucketId: A96137216.bz2 (expected: integer)
by Bastien Billey (JIRA)
[ https://issues.jboss.org/browse/ISPN-4751?page=com.atlassian.jira.plugin.... ]
Bastien Billey commented on ISPN-4751:
--------------------------------------
I need it to. It's not really cool to put in the doc that it's compatible if not. Hope a solution will be find soon...
> Hibernate search, infinispan and Amazon S3 - IllegalArgumentException: bucketId: A96137216.bz2 (expected: integer)
> ------------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-4751
> URL: https://issues.jboss.org/browse/ISPN-4751
> Project: Infinispan
> Issue Type: Bug
> Components: Loaders and Stores
> Reporter: Lance Ess
> Assignee: William Burns
>
> I'm trying to use hibernate-search to host a Lucene index on Amazon S3 but I'm getting the following exception:
> {code}
> Exception in thread "LuceneIndexesData-CloudCacheStore-0" java.lang.IllegalArgumentException: bucketId: A96137216.bz2 (expected: integer)
> at org.infinispan.loaders.bucket.Bucket.setBucketId(Bucket.java:84)
> at org.infinispan.loaders.cloud.CloudCacheStore.readFromBlob(CloudCacheStore.java:450)
> at org.infinispan.loaders.cloud.CloudCacheStore.scanBlobForExpiredEntries(CloudCacheStore.java:292)
> at org.infinispan.loaders.cloud.CloudCacheStore.purge(CloudCacheStore.java:284)
> at org.infinispan.loaders.cloud.CloudCacheStore.purgeInternal(CloudCacheStore.java:336)
> at org.infinispan.loaders.AbstractCacheStore$2.run(AbstractCacheStore.java:111)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> {code}
> The documentation for persisting Lucene indexes on Amazon-S3 is a little sparse but I think I'm on the right track. I'm trying to start infinispan embedded within my application so I've specified a path to the infinispan XML as follows in my hibernate.cfg.xml
> {code:xml}
> <property name="hibernate.search.default.directory_provider">infinispan</property>
> <property name="hibernate.search.infinispan.configuration_resourcename">infinispan-amazons3.xml</property>
> <property name="hibernate.search.infinispan.chunk_size">300000000</property>
> {code}
> And my infinispan-amazons3.xml is:
> {code:xml}
> <infinispan>
> <default>
> <loaders>
> <cloudStore xmlns="urn:infinispan:config:cloud:5.3"
> cloudService="aws-s3"
> identity="user"
> password="password"
> bucketPrefix="bucket">
> </cloudStore>
> </loaders>
> </default>
> </infinispan>
> {code}
> I'm using the following versions (maven pom.xml)
> {code}
> <dependency>
> <groupId>org.hibernate</groupId>
> <artifactId>hibernate-search</artifactId>
> <version>4.4.4.Final</version>
> </dependency>
> <dependency>
> <groupId>org.hibernate</groupId>
> <artifactId>hibernate-search-infinispan</artifactId>
> <version>4.4.4.Final</version>
> </dependency>
> <dependency>
> <groupId>org.infinispan</groupId>
> <artifactId>infinispan-cachestore-cloud</artifactId>
> <version>5.3.0.Final</version>
> </dependency>
> <dependency>
> <groupId>org.jclouds.provider</groupId>
> <artifactId>aws-s3</artifactId>
> <version>1.4.1</version>
> </dependency>
> {code}
> I initially thought this was related to ISPN-1909 but my version is after the fix for that issue (5.1.3.CR1, 5.1.3.FINAL)
> FYI here's my maven dependency tree (grepped for infinispan)
> {code}
> $ mvn dependency:tree | grep infinispan
> [INFO] +- org.hibernate:hibernate-search-infinispan:jar:4.4.4.Final:compile
> [INFO] | \- org.infinispan:infinispan-lucene-directory:jar:5.3.0.Final:compile
> [INFO] +- org.infinispan:infinispan-cachestore-cloud:jar:5.3.0.Final:compile
> [INFO] | \- org.infinispan:infinispan-core:jar:5.3.0.Final:compile
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4828) Increasing default internal thread pool size
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4828?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4828:
-------------------------------
Status: Pull Request Sent (was: Open)
Git Pull Request: https://github.com/infinispan/infinispan/pull/3019
> Increasing default internal thread pool size
> --------------------------------------------
>
> Key: ISPN-4828
> URL: https://issues.jboss.org/browse/ISPN-4828
> Project: Infinispan
> Issue Type: Enhancement
> Components: Configuration, Core
> Affects Versions: 7.0.0.CR1
> Reporter: Matej Čimbora
> Assignee: Dan Berindei
> Priority: Critical
>
> Using synchronous replication with high number of concurrent clients doing put() operations over a shared set of keys, lock-acquisition timeouts occur when various thread pools (internal, jgroups oob) do not have appropriate size.
> org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [3 seconds] on key [key_00000000000003B4] for requestor [Thread[OOB-66,default,node03-12795,5,main]]! Lock held by [Thread[OOB-314,default,node03-12795,5,main]]
> [org.infinispan.interceptors.InvocationContextInterceptor] (Stressor-1) ISPN000136: Execution error
> org.infinispan.util.concurrent.TimeoutException: org.infinispan.util.concurrent.TimeoutException: Node node04-24454 timed out
> This applies to both transactional and non-transactional configuration. The problem can be mitigated by increasing Infinispan's internal thread pool size (defined for remoteCommandsExecutor, blockingBoundedQueueThreadPool). In order to improve user experience either:
> a) When needed, the size of the thread pool should be increased as the load increases
> b) The default values should be high enough to handle even significant load (in terms of number of concurrent clients per node)
> c) The documentation should describe how the end user should size the thread pools based on expected load on the system
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4890) Entry iterator should throw AvailabilityException in degraded mode
by Dan Berindei (JIRA)
[ https://issues.jboss.org/browse/ISPN-4890?page=com.atlassian.jira.plugin.... ]
Dan Berindei updated ISPN-4890:
-------------------------------
Status: Resolved (was: Pull Request Sent)
Resolution: Done
> Entry iterator should throw AvailabilityException in degraded mode
> ------------------------------------------------------------------
>
> Key: ISPN-4890
> URL: https://issues.jboss.org/browse/ISPN-4890
> Project: Infinispan
> Issue Type: Bug
> Components: Core, Test Suite - Core
> Affects Versions: 7.0.0.CR2
> Reporter: Dan Berindei
> Assignee: William Burns
> Priority: Blocker
> Labels: testsuite_stability
> Fix For: 7.0.0.Final
>
>
> When the distributed entry iterator gets a suspect exception, it assumes that the CH will be soon updated to remove the leaver, and retries to contact the primary owner of the segment.
> But if the cache is in degraded mode, the consistent hash is no longer updated, and the entry iterator enters an infinite loop.
> {noformat}
> 11:58:35,622 TRACE (testng-DelayedAvailabilityUpdateTest:) [DistributedEntryRetriever] Request to NodeC-51013=[50, 51, 20, 52, 21, 53, 22, 54, 23, 24, 25, 26, 27, 28, 29] for cd886604-960b-431a-81a0-efbb51eed57f was suspect, must resend to a new node!
> 11:58:35,622 TRACE (testng-DelayedAvailabilityUpdateTest:) [DistributedEntryRetriever] Request to NodeC-51013=[50, 51, 20, 52, 21, 53, 22, 54, 23, 24, 25, 26, 27, 28, 29] for cd886604-960b-431a-81a0-efbb51eed57f was suspect, must resend to a new node!
> 11:58:35,622 TRACE (testng-DelayedAvailabilityUpdateTest:) [DistributedEntryRetriever] Request to NodeC-51013=[50, 51, 20, 52, 21, 53, 22, 54, 23, 24, 25, 26, 27, 28, 29] for cd886604-960b-431a-81a0-efbb51eed57f was suspect, must resend to a new node!
> ...
> {noformat}
> This is visible when running the partition handling tests (e.g. DelayedAvailabilityUpdateTest) with trace logging enabled, as {{TestingUtil.killCaches()}} logs the cache contents at trace level. As an aside, {{TestingUtil.killCaches()}} should probably use {{Flag.CACHE_MODE_LOCAL}} to print only the entries on each node.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months
[JBoss JIRA] (ISPN-4027) TransactionTable.start() initialize the TxCleanupService thread pool even when the cache is NON_TRANSACTIONAL
by RH Bugzilla Integration (JIRA)
[ https://issues.jboss.org/browse/ISPN-4027?page=com.atlassian.jira.plugin.... ]
RH Bugzilla Integration commented on ISPN-4027:
-----------------------------------------------
Martin Gencur <mgencur(a)redhat.com> changed the Status of [bug 1152942|https://bugzilla.redhat.com/show_bug.cgi?id=1152942] from ON_QA to ASSIGNED
> TransactionTable.start() initialize the TxCleanupService thread pool even when the cache is NON_TRANSACTIONAL
> -------------------------------------------------------------------------------------------------------------
>
> Key: ISPN-4027
> URL: https://issues.jboss.org/browse/ISPN-4027
> Project: Infinispan
> Issue Type: Bug
> Affects Versions: 6.0.1.Final
> Reporter: Guillermo GARCIA OCHOA
> Assignee: Takayoshi Kimura
> Labels: 630
> Fix For: 7.0.0.CR2
>
>
> In the {{TransactionTable.start()}} each cache creates a thread pool and a job is scheduled to clean up completed transactions.
> {code:java}
> private void start() {
> ...
> totalOrder = configuration.transaction().transactionProtocol().isTotalOrder();
> if (!totalOrder) {
> // Periodically run a task to cleanup the transaction table from completed transactions.
> ThreadFactory tf = new ThreadFactory() {
> @Override
> public Thread newThread(Runnable r) {
> String address = rpcManager != null ? rpcManager.getTransport().getAddress().toString() : "local";
> Thread th = new Thread(r, "TxCleanupService," + cacheName + "," + address);
> th.setDaemon(true);
> return th;
> }
> };
> executorService = Executors.newSingleThreadScheduledExecutor(tf);
> long interval = configuration.transaction().reaperWakeUpInterval();
> executorService.scheduleAtFixedRate(new Runnable() {
> @Override
> public void run() {
> cleanupCompletedTransactions();
> }
> }, interval, interval, TimeUnit.MILLISECONDS);
> }
> }
> {code}
> As you can see in the code, even is the cache is {{NON_TRANSACTIONAL}} the job is scheduled, consuming resources to do nothing (the {{completedTransactions}} map is always empty)
> Maybe I'm missing something, but our application profiling is showing us that these threads do nothing but they are consuming precious resources because we have more than 1000 {{NON_TRANSACTIONAL}} caches.
> (i) This can be considered when solving ISPN-3702 too.
--
This message was sent by Atlassian JIRA
(v6.3.1#6329)
11 years, 5 months