[keycloak-user] persistent sessions are lost after restart - problem with infinispant file-base persistence

[Marek Posolda]

We never tested with the custom store (like file-store) with our 
infinispan caches. As you pointed, I think that it won't work in latest 
3.3.0 or 3.4.1 versions as we're doing some optimizations in our 
infinispan caches, which were introduced due the cross-dc support (some 
cache operations are called with the IGNORE_REMOTE_STORE and 
IGNORE_REMOTE_LOAD flag).

Feel free to create JIRA.

I think that the remoteStore is currently the only one, which would 
work. So what should work as workaround is, that you will configure the 
infinispan caches with the remoteStore, which will point to external JDG 
server. Then on the JDG server side, you configure caches to point to 
file-store. Not sure it will work and it's not pretty workaround, but 
maybe better than nothing...

Marek

On 03/12/17 14:49, bauer_marie at gmx.net wrote:
>
> Hi,
>   
> we are using version 3.0.0.Final and configured our sessions to be file based persisted by infinispan with the file store.
> ...
> <subsystem xmlns="urn:jboss:domain:infinispan:4.0">
>     <cache-container name="keycloak" jndi-name="infinispan/Keycloak">
>        <local-cache name="realms">
>             <eviction max-entries="10000" strategy="LRU"/>
>        </local-cache>
>        <local-cache name="users">
>             <eviction max-entries="10000" strategy="LRU"/>
>        </local-cache>
>        <local-cache name="sessions">
>          <file-store passivation="false" fetch-state="true" purge="false"
>                    relative-to="jboss.home.dir"
>                    path="/infinistate/session"/>
>        </local-cache>
>        <local-cache name="offlineSessions">
>          <file-store passivation="false" fetch-state="true" purge="false"
>                    relative-to="jboss.home.dir"
>                    path="/infinistate/offline-sessions"/>
>        </local-cache>
>        <local-cache name="loginFailures"/>
>        <local-cache name="work">
>          <file-store passivation="false" fetch-state="true" purge="false"
>                    relative-to="jboss.home.dir"
>                    path="/infinistate/work"/>
>        </local-cache>
>        <local-cache name="authorization">
>          <eviction max-entries="100" strategy="LRU"/>
>        </local-cache>
>        <local-cache name="keys">
>          <eviction max-entries="1000" strategy="LRU"/>
>          <expiration max-idle="3600000"/>
>        </local-cache>
>   </cache-container>
>          ...
>   </subsystem>
>   ...
> Since we are experiencing problems with that file based persistence we hoped this will be resolved in newer versions.
> There are error messages telling that there is a persistence problem:
>
> ISPN000252: Error executing parallel store task"
> Caused by: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 52"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1285)"
> 	at org.infinispan.marshall.core.MarshalledEntryImpl.getValue(MarshalledEntryImpl.java:61)"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1745)"
> 	at org.infinispan.persistence.file.SingleFileStore.process(SingleFileStore.java:549)"
> 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)"
> 	at org.infinispan.persistence.file.SingleFileStore.process(SingleFileStore.java:531)"
> 	at org.infinispan.persistence.manager.PersistenceManagerImpl.processOnAllStores(PersistenceManagerImpl.java:432)"
> 	at org.infinispan.persistence.file.SingleFileStore.process(SingleFileStore.java:549)"
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)"
> 	at org.infinispan.executors.ExecutorAllCompletionService.pollUntilEmpty(ExecutorAllCompletionService.java:48)"
> 	... 6 more"
> 	at org.infinispan.persistence.file.SingleFileStore$2.call(SingleFileStore.java:537)"
> 	at org.infinispan.persistence.file.SingleFileStore$2.call(SingleFileStore.java:531)"
> 	at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:21)"
> 	... 7 more"
> 	at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)"
> Caused by: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 52"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:224)"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1745)"
> 	at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:134)"
> 	at org.infinispan.persistence.file.SingleFileStore.process(SingleFileStore.java:549)"
> 	at org.infinispan.executors.ExecutorAllCompletionService.pollUntilEmpty(ExecutorAllCompletionService.java:48)"
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)"
> 	... 18 more"
> Caused by: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 52"
> 	at org.infinispan.persistence.manager.PersistenceManagerImpl.processOnAllStores(PersistenceManagerImpl.java:432)"
> 	at java.lang.Thread.run(Thread.java:748)"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1285)"
> 	at org.infinispan.marshall.core.MarshalledEntryImpl.unmarshall(MarshalledEntryImpl.java:116)"
> 	at org.infinispan.marshall.core.MarshalledEntryImpl.unmarshall(MarshalledEntryImpl.java:114)"
> 	at org.infinispan.persistence.PersistenceUtil.convert(PersistenceUtil.java:133)"
> 	at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)"
> 	at org.infinispan.persistence.file.SingleFileStore.process(SingleFileStore.java:549)"
> 	in field loginUsername"
> 	... 6 more"
> 	at org.infinispan.persistence.util.PersistenceManagerCloseableSupplier$SupplierCacheLoaderTask.processEntry(PersistenceManagerCloseableSupplier.java:75)"
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:754)"
> 	at org.infinispan.marshall.core.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:101)"
> 	at org.infinispan.commons.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:134)"
> 	at org.infinispan.marshall.core.MarshalledEntryImpl.unmarshall(MarshalledEntryImpl.java:114)"
> 	at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1285)"
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)"
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)"
> Caused by: org.infinispan.persistence.spi.PersistenceException: java.io.StreamCorruptedException: Unexpected byte found when reading an object: 52"
> 	at org.infinispan.executors.ExecutorAllCompletionService.pollUntilEmpty(ExecutorAllCompletionService.java:48)"
> 	at org.infinispan.util.concurrent.WithinThreadExecutor.execute(WithinThreadExecutor.java:21)"
> 	... 7 more"
>
> This causes a high CPU load.
>
> As we plan to update to 3.3.0.Final I tried the CR2 earlier. Everything seemed to work fine, but the storeed sessions are lost.
>
> So I tried 3.2.0 which has bad errors about persistence and class not found.
>
> So I tried 3.1.1 - could not get the image.
>
> So I tried 3.1.0 - this could read the persisted sessions, I could see them in the administration console.
>   
>
>
> Is anyone experiencing the same problem? Or is there a solution?
>
> Thanks
> Marie
>   
>   
>   
>   
>
> _______________________________________________
> keycloak-user mailing list
> keycloak-user at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/keycloak-user