Re: [aerogear-dev] Upgrading process for the OpenShift AeroGear UPS cartridge

From: "Farah Juma" <fjuma(a)redhat.com&gt; To: aerogear-dev(a)lists.jboss.org Sent: Monday, January 27, 2014 1:07:31 PM Subject: Re: [aerogear-dev] Upgrading process for the OpenShift AeroGear        UPS        cartridge > From: "Andrea Vibelli" <avibelli(a)redhat.com&gt; > To: aerogear-dev(a)lists.jboss.org > Sent: Monday, January 27, 2014 12:27:04 PM > Subject: [aerogear-dev] Upgrading process for the OpenShift AeroGear UPS >         cartridge > > Hello all, > > I just wanted to let you know that Farah is working on AGPUSH-504, a > snapshot/restore approach for upgrading the downloadable cartridge. > The overall idea is to take a snapshot of the database (and other > configuration) for an older existing cartridge instance, delete this old > instance, and then create a new instance based on the latest cartridge > version and restore the new instance's database (and other configuration) > to > the state in the snapshot. The new instance must have the same name as the > old instance. The main thing that she needed to determine was what needed > to > be included in the snapshot that's not already part of the MySQL database. > The SimplePush token key was one of these things (we generate this value > when a cartridge instance is created and it is used by the server for > encryption and decryption of endpoint URLs). The standalone.xml file also > needed to be backed up just in case the user made any changes to it. > (Creating a backup of this file allows users to manually merge back any > changes they may have made.) > > The snapshot/restore action hooks she has created can be found in [1]. > > The steps she has been using to try upgrading from an 0.8.1 instance are > listed below. It would be great if you could try out these steps as well. > There's one important caveat though: because an 0.8.1 instance won't have > the necessary snapshot hook, you'll need to manually perform some commands > to save the necessary data (step 3 below). If we include these > snapshot/restore action hooks in the 0.10.0 cartridge release, users will > be > able to make use of these hooks for future upgrades, e.g., for upgrading > from 0.10.0 to 0.11.0. >   > 1) Create an 0.8.1 cartridge instance > rhc create-app mypush > "https://cartreflect-claytondev.rhcloud.com/reflect?github=aerogear/openshift-origin-cartridge-aerogear-push&commit=8aada257de52157a837db23a9f7b1a8574fdffab" > mysql-5.1 > > 2) Log into the admin console, change password, create push applications, > create variants, send push notifications, etc. > > 3) ssh into your instance and perform these commands (this step manually > captures needed data) > echo "$OPENSHIFT_AEROGEAR_PUSH_TOKEN_KEY" > > $OPENSHIFT_DATA_DIR/aerogear_push_token_key > cp $OPENSHIFT_AEROGEAR_PUSH_DIR/standalone/configuration/standalone.xml > $OPENSHIFT_DATA_DIR/standalone.snapshot.xml > > 4) Create a snapshot of the 0.8.1 instance: rhc snapshot save mypush > > 5) Delete the 0.8.1 instance: rhc app delete mypush > > 6) Delete the local directory on the local machine (from step 1, you should > have a directory named 'mypush') > > 7) Create a new cartridge instance (this uses her fork of the 0.9.0 > cartridge > repository that contains the snapshot/restore action hooks) > rhc create-app mypush > "https://cartreflect-claytondev.rhcloud.com/reflect?github=fjuma/openshift-origin-cartridge-aerogear-push&commit=956a4fc09edc93fe76394b86b265ceb1d238ccaf" > mysql-5.1 > > 8) Restore the instance: rhc snapshot restore mypush -f > /PATH/TO/mypush.tar.gz > > 9) You should now be able to log into the admin console with the same > password you previously set, see existing push applications, variants, send > push notifications, etc. > > One disadvantage of this snapshot/restore approach is the extra work for > the > user. It would be nice to have a single command that can do the snapshot, > delete, create, and restore all at once. She is going to start looking into > this "wrappering" functionality. > > > Why am I writing about this? > > I have made some test about the above steps. At a first glance, it seemed > that something was broken between steps 6 and 7. > Doing various tests, what I saw in the logs (server.log) were different > errors: > > 2014/01/26 10:19:18,061 ERROR [org.jboss.as] (MSC service thread 1-3) > JBAS015875: JBoss AS 7.1.1.Final "Brontes" started (with errors) in > 296088ms > - Started 349 of 524 services (52 services failed or missing dependencies, > 119 services are passive or on-demand) > 2014/01/26 10:19:18,344 INFO  [org.jboss.as.server] > (DeploymentScanner-threads - 2) JBAS015870: Deploy of deployment "ROOT.war" > was rolled back with failure message {"JBAS014771: Services with > missing/unavailable dependencies" => > ["jboss.persistenceunit.\"ROOT.war#unifiedpush-default\"jboss.naming.context.java.jboss.datasources.PushEEDSMissing[jboss.persistenceunit.\"ROOT.war#unifiedpush-default\"jboss.naming.context.java.jboss.datasources.PushEEDS]","jboss.persistenceunit.\"ROOT.war#picketlink-default\"jboss.naming.context.java.jboss.datasources.PushEEDSMissing[jboss.persistenceunit.\"ROOT.war#picketlink-default\"jboss.naming.context.java.jboss.datasources.PushEEDS]"]} > 2014/01/26 10:19:19,390 INFO  [org.jboss.as.server.deployment] (MSC service > thread 1-2) JBAS015877: Stopped deployment ROOT.war in 1125ms > 2014/01/26 10:19:19,393 INFO  [org.jboss.as.controller] > (DeploymentScanner-threads - 2) JBAS014774: Service status report > JBAS014775:    New missing/unsatisfied dependencies: >       service jboss.naming.context.java.jboss.datasources.PushEEDS >       (missing) >       dependents: [service >       jboss.persistenceunit."ROOT.war#unifiedpush-default", service >       jboss.persistenceunit."ROOT.war#picketlink-default"] > > > 2014/01/26 10:19:19,440 ERROR [org.jboss.as.server.deployment.scanner] > (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation failed > and > was rolled back. Steps that failed:" => {"Operation step-2" => > {"JBAS014771: > Services with missing/unavailable dependencies" => > ["jboss.persistenceunit.\"ROOT.war#unifiedpush-default\"jboss.naming.context.java.jboss.datasources.PushEEDSMissing[jboss.persistenceunit.\"ROOT.war#unifiedpush-default\"jboss.naming.context.java.jboss.datasources.PushEEDS]","jboss.persistenceunit.\"ROOT.war#picketlink-default\"jboss.naming.context.java.jboss.datasources.PushEEDSMissing[jboss.persistenceunit.\"ROOT.war#picketlink-default\"jboss.naming.context.java.jboss.datasources.PushEEDS]"]}}} It looks like the MySQL cartridge either wasn't added or wasn't able to start successfully. The following command can be used to check the status of the MySQL cartridge: rhc cartridge status -a APP -c mysql   > OR > > > 2014/01/26 05:41:46,522 INFO  [org.jboss.as.connector] (MSC service thread > 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 1.0.9.Final) > 2014/01/26 05:41:46,922 ERROR > [org.jboss.as.controller.management-operation] > (ServerService Thread Pool -- 62) Operation ("enable") failed - address: ([ >     ("subsystem" => "datasources"), >     ("data-source" => "PostgreSQLDS") > ]) - failure description: "JBAS014802: Cannot resolve expression > 'expression > \"jdbcostgresql://${env.OPENSHIFT_POSTGRESQL_DB_HOST}:${env.OPENSHIFT_POSTGRESQL_DB_PORT}/mypush13\"' > -- java.lang.IllegalStateException: Failed to resolve expression: > ${env.OPENSHIFT_POSTGRESQL_DB_HOST}" > > > OR > > > no errors in the logs, but in the deployments directory I found > 'ROOT.war.dodeploy' and 'ROOT.war.isdeploying' files, but no > 'ROOT.war.deployed' file. > > > Those error seemed to me as random and too strange (by the way, the > creation > of the cartridge is always correct, I never had problems if I created a new > instance with a never used before name). > > To investigate further, I then tried to do the same steps with a completely > different application: 'Kitchensink on OpenShift', and I had the same > error! > After the deletion and the recreation of a cartridge with a same name, I > encountered: 'The requested URL /app was not found on this server.' > The steps I have done are: > 1) rhc app create -a kitchensink -t jbossas-7 --from-code > git://github.com/openshift/kitchensink-example.git > 2) rhc snapshot save kitchensink > 3) rhc app delete kitchensink > 4) rm -rf kitchensinkhtml5/ > 5) rhc app create -a kitchensink -t jbossas-7 --from-code > git://github.com/openshift/kitchensink-example.git > 6) Access to the application. > > So the good news is that the error I encountered is not related to this > specific AeroGear cartridges, but the bad news is that is something more > general about OpenShift. > We need to investigate further and find out if there is a way to "flush" > the > deletion, it seems that there is something appended or cached that prevent > the recreation of a cartridge with a same name. It's good that the issue you're running into isn't specific to the AeroGear cartridge. I've tried quite a few times today but I'm not able to reproduce the issue with the AeroGear cartridge or with the kitchensink application using the above steps. I suspect that there is indeed an OpenShift timing issue here. Let's check with the OpenShift team on this. Farah > We are currently doing some more tests on these process. > > > > _______________________________________________ > aerogear-dev mailing list > aerogear-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/aerogear-dev > _______________________________________________ aerogear-dev mailing list aerogear-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/aerogear-dev