<div dir="ltr">Hi Andreas<div><br></div><div>Have you tried to see much worse your performance is without write-behind? I wouldn't expect write-behind to help a lot with SingleFileStore (file-store): since it never calls fsync, writes only go to the OS write-behind buffers before returning.</div><div><br></div><div>Regarding the error per se, I'm afraid the error message doesn't say much. You should enable DEBUG logging for org.infinispan.persistence and see if the async writer threads log any "Failed to process async modifications" messages and create a bug in JIRA. It might also help to get a thread dump after you've seen the error.</div><div><br></div><div>Cheers</div><div>Dan</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Nov 21, 2014 at 12:28 PM, Andreas Kruthoff <span dir="ltr"><<a href="mailto:andreas.kruthoff@nexustelecom.com" target="_blank">andreas.kruthoff@nexustelecom.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi dev<br>
<br>
<br>
I'm running infinispan-7.0.1 (will soon upgrade to 7.0.2).<br>
<br>
I've configured 2 distributed caches, both are similar.<br>
<br>
Example cache1:<br>
...<br>
<distributed-cache name="infinicache-lbd-imei" mode="SYNC"<br>
async-marshalling="false" owners="2"<br>
l1-lifespan="600000" l1-cleanup-interval="60000" statistics="false"><br>
<eviction strategy="LIRS" max-entries="20000000"/><br>
<expiration max-idle="86400000" interval="30000"/><br>
<persistence passivation="true"><br>
<file-store<br>
path="/data1/infini-lbd"<br>
preload="true" shared="true" purge="false"><br>
<write-behind/><br>
</file-store><br>
</persistence><br>
</distributed-cache><br>
...<br>
<br>
Today, I've seen that one of the two caches suddenly doesn't exist<br>
anymore somehow (JMX access to numberOfEntries returns nothing?!).<br>
<br>
After looking into the application logfile, I've found this:<br>
<br>
2014-11-19 13:45:47,310 ERROR<br>
[AsyncStoreCoordinator-infinicache-lbd-imei] AsyncCacheWriter.java:267<br>
ISPN000055: Unexpected error in AsyncStoreCoordinator thread.<br>
AsyncCacheWriter is dead!<br>
org.infinispan.util.concurrent.TimeoutException: ISPN000233: Waiting on<br>
work threads latch failed:<br>
java.util.concurrent.CountDownLatch@6e30928d[Count = 1]<br>
at<br>
org.infinispan.persistence.async.AsyncCacheWriter$AsyncStoreCoordinator.workerThreadsAwait(AsyncCacheWriter.java:297)<br>
~[infinispan-embedded.jar:7.0.1.Final]<br>
at<br>
org.infinispan.persistence.async.AsyncCacheWriter$AsyncStoreCoordinator.run(AsyncCacheWriter.java:254)<br>
~[infinispan-embedded.jar:7.0.1.Final]<br>
at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_72]<br>
<br>
<br>
<br>
No error before nor after this entry. It looks like it's related to<br>
<write-behind/> to me, so I'll remove that setting.<br>
<br>
But I'd like to write asynchronously for best performance.<br>
<br>
Any help would be appreciated!<br>
<br>
-andreas<br>
<br>
This email and any attachment may contain confidential information which is intended for use only by the addressee(s) named above. If you received this email by mistake, please notify the sender immediately, and delete the email from your system. You are prohibited from copying, disseminating or otherwise using the email or any attachment.<br>
<br>
_______________________________________________<br>
infinispan-dev mailing list<br>
<a href="mailto:infinispan-dev@lists.jboss.org">infinispan-dev@lists.jboss.org</a><br>
<a href="https://lists.jboss.org/mailman/listinfo/infinispan-dev" target="_blank">https://lists.jboss.org/mailman/listinfo/infinispan-dev</a><br>
</blockquote></div><br></div>