[infinispan-issues] [JBoss JIRA] (ISPN-8912) The SoftIndexFileStore runs into a deadlock at startup if more than ~20000 values have been stored and max node size is > 32777 bytes

[Radim Vansa (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13541292#comment-13541292 ] 

Radim Vansa commented on ISPN-8912:
-----------------------------------

The node size is limited to 32767 as its length is stored in 2 bytes (OK, we could make that 64k or maybe a bit more, but ByteBuffer does not have unsigned integers methods), so the actual bug is that this is not failing gracefully and that we don't document that (documentation mentions only the limitation for single key).

May I ask why are you increasing the node size? Have you found increasing performance with larger nodes? (what disk do you use?)

> The SoftIndexFileStore runs into a deadlock at startup if more than ~20000 values have been stored and max node size is > 32777 bytes
> -------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: ISPN-8912
>                 URL: https://issues.jboss.org/browse/ISPN-8912
>             Project: Infinispan
>          Issue Type: Bug
>          Components: Loaders and Stores
>    Affects Versions: 9.1.6.Final
>            Reporter: Andreas Pabst
>            Assignee: Radim Vansa
>
> The following unit test just puts lots of entries into the cache and restarts the cache repeatedly.
> It succeeds if maxNodeSize is set to anything up to 32777, but runs into a deadlock from 32778 upwards. If going even higher it doesn't run into a deadlock anymore, but throws an IllegalArgumentException instead. 
> As a workaround one can delete the index. But the problem resurfaces after the next restart. 
> {code:java}
> import java.util.concurrent.TimeUnit;
> import java.util.stream.IntStream;
> import org.infinispan.Cache;
> import org.infinispan.configuration.cache.Configuration;
> import org.infinispan.configuration.cache.ConfigurationBuilder;
> import org.infinispan.manager.DefaultCacheManager;
> import org.infinispan.manager.EmbeddedCacheManager;
> import org.infinispan.persistence.sifs.configuration.SoftIndexFileStoreConfigurationBuilder;
> import org.junit.After;
> import org.junit.Before;
> import org.junit.Rule;
> import org.junit.Test;
> import org.junit.rules.Timeout;
> public class SifsLargeNodeTest {
>     private EmbeddedCacheManager cacheManager;
>     @Rule
>     public Timeout timeout = new Timeout(15, TimeUnit.SECONDS);
>     private static Configuration createConfig(String location, int maxNodeSizeInBytes) {
>         return new ConfigurationBuilder()
>                 .persistence().addStore(SoftIndexFileStoreConfigurationBuilder.class)
>                 .dataLocation(location + "/data").indexLocation(location + "/index")
>                 .maxNodeSize(maxNodeSizeInBytes)
>                 .preload(false).purgeOnStartup(false)
>                 .build();
>     }
>     @Before
>     public void setUp() {
>         cacheManager = new DefaultCacheManager();
>     }
>     @After
>     public void tearDown() {
>         cacheManager.stop();
>     }
>     @Test
>     public void testLoad() {
>         int maxNodeSizeInBytes = 32780; // Note: Anything up to 32777 works fine
>         String cacheName = "sifslargenodetestcache";
>         Configuration cfg = createConfig(cacheName, maxNodeSizeInBytes);
>         int numberOfRuns = 10;
>         int valuesToPutPerRun = 10000;
>         for (int run = 0; run < numberOfRuns; run++) {
>             System.out.println("RUN " + run);
>             cacheManager.defineConfiguration(cacheName, cfg);
>             Cache<Integer, String> testCacheBeforeReduction = cacheManager.getCache(cacheName);
>             IntStream.range(run * valuesToPutPerRun, (run + 1) * valuesToPutPerRun)
>                     .forEach(i -> testCacheBeforeReduction.put(i, Integer.toString(i)));
>             cacheManager.stop();
>             cacheManager = new DefaultCacheManager();
>         }
>     }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.5.0#75005)