[infinispan-issues] [JBoss JIRA] (ISPN-2892) View installation loop when restarting cache on multiple nodes

[Dan Berindei (JIRA)]

    [ https://issues.jboss.org/browse/ISPN-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12765852#comment-12765852 ] 

Dan Berindei edited comment on ISPN-2892 at 4/9/13 11:59 AM:
-------------------------------------------------------------

I think it's normal to get *one* instance of this error message, if the coordinator died while installing a cache view: 

{noformat}
ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-19,node1/web) ISPN000172: Failed to prepare view CacheView{viewId=21, members=[node2/web]} for cache default-host/test, rolling back to view CacheView{viewId=20, members=[]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.IllegalStateException: Cannot prepare new view CacheView{viewId=21, members=[node2/web]} on cache default-host/test, we are currently preparing view CacheView{viewId=18, members=[node2/web]}
{noformat}

But after that, the coordinator should send a ROLLBACK_VIEW command, which should clear the "pendingView", and the next PREPARE_VIEW should succeed. Did you see any message with failed (or successful) ROLLBACK_VIEW commands in the log?

                
      was (Author: dan.berindei):
    I think it's normal to get *one* instance of this error message: 

{noformat}
ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-19,node1/web) ISPN000172: Failed to prepare view CacheView{viewId=21, members=[node2/web]} for cache default-host/test, rolling back to view CacheView{viewId=20, members=[]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.IllegalStateException: Cannot prepare new view CacheView{viewId=21, members=[node2/web]} on cache default-host/test, we are currently preparing view CacheView{viewId=18, members=[node2/web]}
{noformat}

But after that, the coordinator should send a ROLLBACK_VIEW command, which should clear the "pendingView", and the next PREPARE_VIEW should succeed. Did you see any message with failed (or successful) ROLLBACK_VIEW commands in the log?

                  
> View installation loop when restarting cache on multiple nodes
> --------------------------------------------------------------
>
>                 Key: ISPN-2892
>                 URL: https://issues.jboss.org/browse/ISPN-2892
>             Project: Infinispan
>          Issue Type: Bug
>    Affects Versions: 5.1.7.Final
>            Reporter: Dennis Reed
>            Assignee: Dan Berindei
>
> Restarting a cache on multiple nodes at the same time can cause the following error:
> ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-19,node1/web) ISPN000172: Failed to prepare view CacheView{viewId=18, members=[node2/web]} for cache default-host/test, rolling back to view CacheView{viewId=17, members=[node1/web, node2/web]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.IllegalStateException: default-host/test: Received cache view prepare request after the local node has already shut down
> After the initial error, the following error began repeating every second for a few minutes until BaseStateTransferManagerImpl.waitForJoinToComplete() timed out and the cache failed to start:
> ERROR [org.infinispan.cacheviews.CacheViewsManagerImpl] (CacheViewInstaller-19,node1/web) ISPN000172: Failed to prepare view CacheView{viewId=21, members=[node2/web]} for cache default-host/test, rolling back to view CacheView{viewId=20, members=[]}: java.util.concurrent.ExecutionException: org.infinispan.CacheException: java.lang.IllegalStateException: Cannot prepare new view CacheView{viewId=21, members=[node2/web]} on cache default-host/test, we are currently preparing view CacheView{viewId=18, members=[node2/web]}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira