[infinispan-dev] state transfer timed out, where to configure?
Radim Vansa
rvansa at redhat.com
Thu Mar 5 11:51:53 EST 2015
It seems this discussion is getting long and should be moved to forums
instead of mailing list.
Anyway, make sure that you're including soft-index file store in your
dependencies and that it's on the classpath.
Radim
On 03/05/2015 04:32 PM, Andreas Kruthoff wrote:
> Ok, I've corrected it, but I don't know what else I'm doing wrong, the
> error is similar:
>
> Exception in thread "main"
> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
> find a parser for element 'soft-index-file-store' in namespace
> 'urn:infinispan:config:store:soft-index:7.1'. Check that your
> configuration is up-to date for this version of Infinispan.
>
> Do I need to change the <infinispan xmlns:... > part, too? How? I'm not
> familiar with those schema namespace references, unfortunately.
>
> <infinispan
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="urn:infinispan:config:7.1
> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
> xmlns="urn:infinispan:config:7.1"
> >
>
>
> On 03/05/2015 04:17 PM, Dan Berindei wrote:
>> I think the namespace should be
>> "urn:infinispan:config:store:soft-index:7.1" (the extra "store" was
>> added in 7.1).
>>
>> On Thu, Mar 5, 2015 at 3:19 PM, Andreas Kruthoff
>> <andreas.kruthoff at nexustelecom.com> wrote:
>>> Thanks for the information. I'd like to try the SoftIndexFileStore, but
>>> no luck with the configuration so far... I'm using version 7.1.1.
>>>
>>> Exception in thread "main"
>>> org.infinispan.commons.CacheConfigurationException: ISPN000327: Cannot
>>> find a parser for element 'soft-index-file-store' in namespace
>>> 'urn:infinispan:config:soft-index:7.1'. Check that your configuration is
>>> up-to date for this version of Infinispan.
>>>
>>> I was trying this:
>>> http://infinispan.org/docs/7.1.x/user_guide/user_guide.html#_soft_index_file_store
>>>
>>> <infinispan
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="urn:infinispan:config:7.1
>>> http://www.infinispan.org/schemas/infinispan-config-7.1.xsd"
>>> xmlns="urn:infinispan:config:7.1">
>>>
>>> <jgroups>
>>> <stack-file name="fast" path="jgroups-fast.xml"/>
>>> </jgroups>
>>>
>>> <cache-container name="DataCacheManager" statistics="false"
>>> default-cache="data-cache" shutdown-hook="DEFAULT">
>>>
>>> <transport stack="fast" cluster="data"/>
>>>
>>> <local-cache name="with_sifs">
>>> <persistence>
>>> <soft-index-file-store
>>> xmlns="urn:infinispan:config:soft-index:7.1">
>>> <index path="/tmp/sifs/testCache/index" />
>>> <data path="/tmp/sifs/testCache/data" />
>>> </soft-index-file-store>
>>> </persistence>
>>> </local-cache>
>>>
>>> </cache-container>
>>>
>>> </infinispan>
>>>
>>>
>>> On 03/05/2015 12:04 PM, Dan Berindei wrote:
>>>> The fix for ISPN-5141 is in master now, if you want to give it a try.
>>>>
>>>> You may also want to try using SoftIndexFileStore instead of
>>>> SingleFileStore. SingleFileStore needs to keep all the keys in memory,
>>>> so the memory overhead is huge when the values are small.
>>>> SoftIndexFileStore doesn't have this problem, and state transfer might
>>>> be even faster than with the ISPN-5141 fix for SingleFileStore.
>>>>
>>>> Cheers
>>>> Dan
>>>>
>>>>
>>>> On Mon, Feb 9, 2015 at 4:52 PM, Andreas Kruthoff
>>>> <andreas.kruthoff at nexustelecom.com> wrote:
>>>>> I've added my vote.
>>>>>
>>>>> I've increased the chunk size-from 512 entries (default) to 10'000
>>>>> entries, as from the xsd: "The number of cache entries to batch in each
>>>>> transfer." I thought that an increased chunk-size helps to populate the
>>>>> cache faster? Or is it the opposite? One entry is very small, just a
>>>>> Long for key and value.
>>>>>
>>>>>
>>>>> On 02/09/2015 02:31 PM, Dan Berindei wrote:
>>>>>> I'm not 100% sure, but this sounds like you're hitting ISPN-5141 [1].
>>>>>> Please add your vote, it might make it into 7.2.0.
>>>>>>
>>>>>> Lowering the state transfer chunkSize may also help, especially if your
>>>>>> values are large. The chunkSize is the number of entries in a single
>>>>>> chunk, you should aim at making the average chunk <= 500KB.
>>>>>>
>>>>>> Cheers
>>>>>> Dan
>>>>>>
>>>>>> [1] https://issues.jboss.org/browse/ISPN-5141
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 9, 2015 at 11:48 AM, Andreas Kruthoff
>>>>>> <andreas.kruthoff at nexustelecom.com
>>>>>> <mailto:andreas.kruthoff at nexustelecom.com>> wrote:
>>>>>>
>>>>>> Thanks. The cache is a distributed cache with 3 nodes and replication
>>>>>> factor 2. It contains ~40 million entries. As soon as a node has loaded
>>>>>> all entries from disk, new entries are added to this node with high
>>>>>> frequency (several thousand/s).
>>>>>>
>>>>>> The other nodes might not be up yet. It takes 'forever' to bring them up
>>>>>> then...?
>>>>>>
>>>>>> On 02/03/2015 12:32 PM, Dan Berindei wrote:
>>>>>> > Hi Andreas
>>>>>> >
>>>>>> > Yes, that's the one. 1200s sounds like a huge timeout, though, how big
>>>>>> > is your cache?
>>>>>> >
>>>>>> > Cheers
>>>>>> > Dan
>>>>>> >
>>>>>> >
>>>>>> > On Thu, Jan 29, 2015 at 7:25 PM, Andreas Kruthoff
>>>>>> > <andreas.kruthoff at nexustelecom.com
>>>>>> <mailto:andreas.kruthoff at nexustelecom.com>
>>>>>> > <mailto:andreas.kruthoff at nexustelecom.com
>>>>>> <mailto:andreas.kruthoff at nexustelecom.com>>> wrote:
>>>>>> >
>>>>>> > Hi dev
>>>>>> >
>>>>>> >
>>>>>> > I'm running into the following exception on the 3rd node out
>>>>>> of 2.
>>>>>> > Distributed cluster, file store with a few millions of entries.
>>>>>> >
>>>>>> > The 3rd node times out during startup, I think. "Initial
>>>>>> state transfer
>>>>>> > timed out". How can I configure/increase the timeout in my
>>>>>> > infinispan.xml?
>>>>>> >
>>>>>> > Is it <state-transfer timeout="1200000"/> within <distributed
>>>>>> cache/> ?
>>>>>> >
>>>>>> >
>>>>>> > thanks for help
>>>>>> >
>>>>>> > -andreas
>>>>>> >
>>>>>> >
>>>>>> > Exception in thread "main"
>>>>>> org.infinispan.commons.CacheException: Unable
>>>>>> > to invoke method public void
>>>>>> >
>>>>>> org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete()
>>>>>> > th
>>>>>> > rows java.lang.InterruptedException on object of type
>>>>>> > StateTransferManagerImpl
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:170)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:869)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.factories.AbstractComponentRegistry.invokeStartMethods(AbstractComponentRegistry.java:638)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:627)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:530)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:216)
>>>>>> > at
>>>>>> > org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:813)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:584)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:539)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:416)
>>>>>> > at
>>>>>> > ch.nexustelecom.lbd.engine.ImeiCache.init(ImeiCache.java:49)
>>>>>> > at
>>>>>> >
>>>>>> ch.nexustelecom.dexclient.engine.DefaultDexClientEngine.init(DefaultDexClientEngine.java:120)
>>>>>> > at
>>>>>> >
>>>>>> ch.nexustelecom.dexclient.DexClient.initClient(DexClient.java:169)
>>>>>> > at
>>>>>> >
>>>>>> ch.nexustelecom.dexclient.tool.DexClientManager.startup(DexClientManager.java:196)
>>>>>> > at
>>>>>> >
>>>>>> ch.nexustelecom.dexclient.tool.DexClientManager.main(DexClientManager.java:83)
>>>>>> > Caused by: org.infinispan.commons.CacheException: Initial
>>>>>> state transfer
>>>>>> > timed out for cache infinicache-lbd-imei on m4sxhpsrm672-11986
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:216)
>>>>>> > at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>>>> > at
>>>>>> >
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>> > at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>> > at
>>>>>> >
>>>>>> org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:168)
>>>>>> > ... 14 more
>>>>>> >
>>>>>> > This email and any attachment may contain confidential
>>>>>> information
>>>>>> > which is intended for use only by the addressee(s) named
>>>>>> above. If
>>>>>> > you received this email by mistake, please notify the sender
>>>>>> > immediately, and delete the email from your system. You are
>>>>>> > prohibited from copying, disseminating or otherwise using the
>>>>>> email
>>>>>> > or any attachment.
>>>>>> >
>>>>>> > _______________________________________________
>>>>>> > infinispan-dev mailing list
>>>>>> > infinispan-dev at lists.jboss.org
>>>>>> <mailto:infinispan-dev at lists.jboss.org>
>>>>>> <mailto:infinispan-dev at lists.jboss.org
>>>>>> <mailto:infinispan-dev at lists.jboss.org>>
>>>>>> > https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>> >
>>>>>> >
>>>>>>
>>>>>> This email and any attachment may contain confidential information
>>>>>> which is intended for use only by the addressee(s) named above. If
>>>>>> you received this email by mistake, please notify the sender
>>>>>> immediately, and delete the email from your system. You are
>>>>>> prohibited from copying, disseminating or otherwise using the email
>>>>>> or any attachment.
>>>>>>
>>>>>> _______________________________________________
>>>>>> infinispan-dev mailing list
>>>>>> infinispan-dev at lists.jboss.org <mailto:infinispan-dev at lists.jboss.org>
>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>
>>>>>>
>>>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>>>
>>>>> _______________________________________________
>>>>> infinispan-dev mailing list
>>>>> infinispan-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
> This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev
--
Radim Vansa <rvansa at redhat.com>
JBoss Performance Team
More information about the infinispan-dev
mailing list