Hi Hynek When passing the parameter -Djboss.as.management.blocking.timeout=900 to standalone.sh, I still get an transaction timeout after 5 minutes. The JBoss deployment timeout follows 10 minutes later 13:48:58,480 INFO [org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider] (ServerService Thread Pool -- 54) Updating database. Using changelog META-INF/jpa-changelog-master.xml 13:53:56,978 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000101:3bafd8ef:5a5df4b7:d in state RUN 13:53:56,986 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff7f000101:3bafd8ef:5a5df4b7:d 13:53:57,240 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000101:3bafd8ef:5a5df4b7:10 in state RUN 13:53:57,241 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff7f000101:3bafd8ef:5a5df4b7:10 14:03:55,819 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0348: Timeout after [900] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[ ("core-service" => "management"), ("management-interface" => "http-interface") ]' 14:04:00,825 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0190: Step handler org.jboss.as.server.DeployerChainAddHandler$FinalRuntimeStepHandler@3cf14117<mailto:org.jboss.as.server.DeployerChainAddHandler$FinalRuntimeStepHandler@3cf14117> for operation add-deployer-chains at address [] failed handling operation rollback -- java.util.concurrent.TimeoutException: java.util.concurrent.TimeoutException at org.jboss.as.controller.OperationContextImpl.waitForRemovals(OperationContextImpl.java:522) … However, when the previous keycloak instance is stopped, the upgrade runs through within 55 seconds for about 25k users. As far as i could see, keycloak only uses the java:jboss/datasources/KeycloakDS which points to jdbc:h2:${jboss.server.data.dir}/keycloak;AUTO_SERVER=TRUE For persistence I use the standard builtin H2 database connection so I just assumed that each wildfly installation would use its own h2 database and infinispan cache. Apparently this is not the case. I already tried the upgrade using 900, 1200 and 1800 seconds for timeout, without any luck. Do you have any other idea how I can debug this? Brgds Michael Von: Hynek Mlnarik [mailto:hmlnarik@redhat.com] Gesendet: Dienstag, 16. Januar 2018 11:31 An: Knurr, Michael <Michael.Knurr(a)adesso.ch&gt; Cc: keycloak-user <keycloak-user(a)lists.jboss.org&gt; Betreff: Re: [keycloak-user] Running two Keycloak instances in parallel It could be for many reasons. Just guessing - since it is terminating after 300 seconds (5 minutes) while the last log entry is about updating database, maybe the database connection/operation is too slow. Hence you can try increasing the startup period value to e.g. 900 seconds by passing -Djboss.as.management.blocking.timeout=900 parameter to standalone.sh. Also make sure that the database of the new Keycloak instance is distinct from the original one. On Tue, Jan 16, 2018 at 10:30 AM, Knurr, Michael <Michael.Knurr@adesso.ch<mailto:Michael.Knurr@adesso.ch>> wrote: Hi all When trying to upgrade Keycloak to the latest version I ran into problems. Unfortunately It turned out that we ran into different upgrade behaviors on different machines. With the latest version, the upgrade is running locally, on test and on stage. As a last step I now tried to test the upgrade procedure on the production machine with a keycloak instance running on a different port. When running the upgrade on a server with an already running keycloak instance, the new server runs into a timeout because of concurrency. 10:16:24,926 INFO [org.keycloak.connections.jpa.updater.liquibase.LiquibaseJpaUpdaterProvider] (ServerService Thread Pool -- 58) Updating database. Using changelog META-INF/jpa-changelog-master.xml 10:21:22,322 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0348: Timeout after [300] seconds waiting for service container stability. Operation will roll back. Step that first updated the service container was 'add' at address '[ ("core-service" => "management"), ("management-interface" => "http-interface") ]' 10:21:23,397 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000101:73d290d3:5a5dc2e6:d in state RUN 10:21:23,402 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff7f000101:73d290d3:5a5dc2e6:d 10:21:23,668 WARN [com.arjuna.ats.arjuna] (Transaction Reaper) ARJUNA012117: TransactionReaper::check timeout for TX 0:ffff7f000101:73d290d3:5a5dc2e6:10 in state RUN 10:21:23,669 WARN [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ARJUNA012121: TransactionReaper::doCancellations worker Thread[Transaction Reaper Worker 0,5,main] successfully canceled TX 0:ffff7f000101:73d290d3:5a5dc2e6:10 10:21:27,328 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0190: Step handler org.jboss.as.server.DeployerChainAddHandler$FinalRuntimeStepHandler@20b4c4cb<mailto:org.jboss.as.server.DeployerChainAddHandler$FinalRuntimeStepHandler@20b4c4cb> for operation add-deployer-chains at address [] failed handling operation rollback -- java.util.concurrent.TimeoutException: java.util.concurrent.TimeoutException at org.jboss.as.controller.OperationContextImpl.waitForRemovals(OperationContextImpl.java:522) ... What might be the reason for this? Shouldn't it possible to run two keycloak servers (with a different port offset) on a single machine? How can I test the keycloak upgrade on the production machine without stopping the running server instance? Best regards Michael _______________________________________________ keycloak-user mailing list keycloak-user@lists.jboss.org<mailto:keycloak-user@lists.jboss.org> https://lists.jboss.org/mailman/listinfo/keycloak-user -- --Hynek