Warning, I may be repeating an existing query, so tell me politely where to go if that is the case ;-) I've been adding Express support to Forge, in the run up to doing demos at JavaOne. I've hit the point where I want to add an entry (an Infinispan cache) to the standalone.xml (or domain.xml), and I want to do it without having to write a parser. If the node is available via the mgmt interface this is easy - I can use the REST interface etc. However, my node isn't, and the recommended way to configure it is to edit .openshift/configuration/standalone.xml. Any thoughts on how I can make use of the management capabilities of AS to define this (in a machine controllable way without me reinventing the wheel) for OpenShift Express?

On Sep 28, 2011, at 12:13 PM, David M. Lloyd wrote: > On 09/28/2011 10:43 AM, Pete Muir wrote: >> Warning, I may be repeating an existing query, so tell me politely where to go if that is the case ;-) >> >> I've been adding Express support to Forge, in the run up to doing demos at JavaOne. I've hit the point where I want to add an entry (an Infinispan cache) to the standalone.xml (or domain.xml), and I want to do it without having to write a parser. If the node is available via the mgmt interface this is easy - I can use the REST interface etc. However, my node isn't, and the recommended way to configure it is to edit .openshift/configuration/standalone.xml. >> >> Any thoughts on how I can make use of the management capabilities of AS to define this (in a machine controllable way without me reinventing the wheel) for OpenShift Express? > > If there is something that is in standalone.xml that can't be affected > with management operations, I'd say we have a bug. You should be able > to get at any configuration item at runtime. I believe the issue is no management interfaces are enabled in Express. Your only interface is standalone.xml. Ben _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

> Scott has a fix in the pipe for this to expose the management interfaces :-) Any info on what the fix is ?

Since just exposing this doesn't really help since users would then have to remember to invoke these operations always from whatever tool they use, wether that is command line, forge, eclipse, netbeans, maven etc. I hope its done with something that doesn't requires N implementations - but something like the suggested feature of being able to deploy management changes.

/max > > On 28 Sep 2011, at 09:14, Benjamin Browning wrote: > >> >> On Sep 28, 2011, at 12:13 PM, David M. Lloyd wrote: >> >>> On 09/28/2011 10:43 AM, Pete Muir wrote: >>>> Warning, I may be repeating an existing query, so tell me politely where to go if that is the case ;-) >>>> >>>> I've been adding Express support to Forge, in the run up to doing demos at JavaOne. I've hit the point where I want to add an entry (an Infinispan cache) to the standalone.xml (or domain.xml), and I want to do it without having to write a parser. If the node is available via the mgmt interface this is easy - I can use the REST interface etc. However, my node isn't, and the recommended way to configure it is to edit .openshift/configuration/standalone.xml. >>>> >>>> Any thoughts on how I can make use of the management capabilities of AS to define this (in a machine controllable way without me reinventing the wheel) for OpenShift Express? >>> >>> If there is something that is in standalone.xml that can't be affected >>> with management operations, I'd say we have a bug. You should be able >>> to get at any configuration item at runtime. >> >> I believe the issue is no management interfaces are enabled in Express. Your only interface is standalone.xml. >> >> Ben >> >> >> _______________________________________________ >> jboss-as7-dev mailing list >> jboss-as7-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev > > > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev /max http://about.me/maxandersen

>> >> Since just exposing this doesn't really help since users would then have to remember to invoke these operations always from whatever tool they use, wether that is command line, forge, eclipse, netbeans, maven etc. >> >> I hope its done with something that doesn't requires N implementations - but something like the suggested feature of being able to deploy management changes. > > AS7 doesn't allow this generally, right? So why would it be different on express? I don't want Express to be different, I want both AS7 locally, on remote hosts and on express to be easy to configure/setup without relying on a running server or that your tool has access to do the right management calls at the right times. i.e. before we could copy a .war and a -ds.xml and it would work/deploy from any tech. Now we need to have forge, cli, jboss tools, netbeans, eclipse, intellij, my eclipse etc. be aware of AS7 management model and not only be able to tell the server to set something up, but also be able to handle the possible error conditions (i.e. datasource is already there, operation not permitted etc.) to do an outofbox deployment without too much fuzz.

>>> I hope its done with something that doesn't requires N implementations - but something like the suggested feature of being able to deploy management changes. >> >> AS7 doesn't allow this generally, right? So why would it be different on express? > > I don't want Express to be different, I want both AS7 locally, on remote hosts and on express to be easy to configure/setup without relying on a running server or that your tool has access to do the right management calls at the right times. > > i.e. before we could copy a .war and a -ds.xml and it would work/deploy from any tech. > > Now we need to have forge, cli, jboss tools, netbeans, eclipse, intellij, my eclipse etc. be aware of AS7 management model and not only be able to tell the server to set something up, but also be able to handle the possible error conditions (i.e. datasource is already there, operation not permitted etc.) to do an outofbox deployment without too much fuzz. We are going to add support for -ds.xml and )probably jms destination deployments if we can find the time), but don't expect everything that is manageable to become a deployment. That will just never happen.

> For the general case Scott suggested us is that we support executing > a list of .cli operations during deploy - but that would then be very > specific for JBoss tools which is why I was having high hopes for the > support for .cli archives that I believe Jesper was working on since > that would handle the deploy/undeploy cases in a way that is sharable > between the various tools - but not sure where that went. Their are major problems with a CLI deployment: 1. Guaranteeing undeploy actually undoes what it did, which may or may not be possible 2. VM crash forcing a relaunch, forcing re-running non-idempotent operations 3. Lack of visibility into what global state each CLI deployment touches

So the plan, and what I'll be working on this week is to have a batch CLI script that exists in the openshift express application repository, when pushed to the express server, is run against the jbossas server CLI interface. One issue that is coming up is the need to refer to environment variables for things like the interfaces/ports for socket services. These values are a property of the execution environment the server is being administered in,

and activation of a mysql datasource for example depends on the user "embedding" the mysql cartridge into the jbossas7 openshift application. To run the same configuration against a local server in jboss tools, we would need to allow for the typical ${env-var:default} type of syntax. Right now the CLI does not do any system property or environment variable replacement. I think this is a feature that needs to be added to allow the scripts to be portable across provisioning environments. On 10/17/11 10:29 AM, Max Rydahl Andersen wrote: >>> For the general case Scott suggested us is that we support executing >>> a list of .cli operations during deploy - but that would then be very >>> specific for JBoss tools which is why I was having high hopes for the >>> support for .cli archives that I believe Jesper was working on since >>> that would handle the deploy/undeploy cases in a way that is sharable >>> between the various tools - but not sure where that went. >> Their are major problems with a CLI deployment: >> >> 1. Guaranteeing undeploy actually undoes what it did, which may or may not be possible >> 2. VM crash forcing a relaunch, forcing re-running non-idempotent operations >> 3. Lack of visibility into what global state each CLI deployment touches > > yeah I understand that. It's the exact same problems users and devs of the various tools that need to setup and run an app on the server today > - maybe large pieces of this just isn't meant to be automated. > > /max > http://about.me/maxandersen > > > > > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

So the plan, and what I'll be working on this week is to have a batch CLI script that exists in the openshift express application repository, when pushed to the express server, is run against the jbossas server CLI interface.

One issue that is coming up is the need to refer to environment variables for things like the interfaces/ports for socket services. These values are a property of the execution environment the server is being administered in, and activation of a mysql datasource for example depends on the user "embedding" the mysql cartridge into the jbossas7 openshift application. To run the same configuration against a local server in jboss tools, we would need to allow for the typical ${env-var:default} type of syntax. Right now the CLI does not do any system property or environment variable replacement. I think this is a feature that needs to be added to allow the scripts to be portable across provisioning environments.

On 10/17/11 10:29 AM, Max Rydahl Andersen wrote: >>> For the general case Scott suggested us is that we support executing >>> a list of .cli operations during deploy - but that would then be very >>> specific for JBoss tools which is why I was having high hopes for the >>> support for .cli archives that I believe Jesper was working on since >>> that would handle the deploy/undeploy cases in a way that is sharable >>> between the various tools - but not sure where that went. >> Their are major problems with a CLI deployment: >> >> 1. Guaranteeing undeploy actually undoes what it did, which may or may not be possible >> 2. VM crash forcing a relaunch, forcing re-running non-idempotent operations >> 3. Lack of visibility into what global state each CLI deployment touches > > yeah I understand that. It's the exact same problems users and devs of the various tools that need to setup and run an app on the server today > - maybe large pieces of this just isn't meant to be automated. > > /max > http://about.me/maxandersen > > > > > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

On 10/18/11 2:43 AM, Max Rydahl Andersen wrote: >> So the plan, and what I'll be working on this week is to have a batch >> CLI script that exists in the openshift express application repository, >> when pushed to the express server, is run against the jbossas server >> CLI >> interface. > > I assume this starts on a pristine new setup every time to avoid > duplicate/overlapping changes ? >

> That's not the case when running against a local server thus always > doing it as part as tool deploy > is then not feasible IMO - having a way to easily run the script > would be relevant though.

> >> One issue that is coming up is the need to refer to environment >> variables for things like the interfaces/ports for socket services. >> These values are a property of the execution environment the server is >> being administered in, and activation of a mysql datasource for example >> depends on the user "embedding" the mysql cartridge into the jbossas7 >> openshift application. To run the same configuration against a local >> server in jboss tools, we would need to allow for the typical >> ${env-var:default} type of syntax. Right now the CLI does not do any >> system property or environment variable replacement. I think this is a >> feature that needs to be added to allow the scripts to be portable >> across provisioning environments. > > Agreed - this is why I was hoping AS7 would support the .cli archive > notion. I think Brian's question was "Why can't we just put the env expressions in the management operation?" Perhaps the CLI doesn't fully use the expression type when seen?

On 10/18/11 1:28 PM, Scott Stark wrote: > On 10/18/11 7:07 AM, Jason T. Greene wrote: >> I think Brian's question was "Why can't we just put the env >> expressions in the management operation?" Perhaps the CLI doesn't >> fully use the expression type when seen? >> > I am talking about having the env expressions in the batch script of the > management operations. The issue was that these are not being replaced > by the command line tools. They are not meant to be replaced by the command line tools; they should be passed through to the server and stored as expressions. I'll put through a change today in how some less-recently changed parts of the model (e.g. interfaces, sockets) deal with expressions; I believe they are not handling expressions that come in via the management APIs correctly; they just deal with them properly in the parser.

See example below of reading a resource and then setting an attribute to an expression value. This will work properly on master when I'm done today. Note that not all attributes support expressions (most don't). For those that do, the :read-resource-description content for the attribute will include: "expressions-allowed" => true [standalone@localhost:9999 /] cd subsystem=ejb3 [standalone@localhost:9999 subsystem=ejb3] :read-resource { "outcome" => "success", "result" => { "default-mdb-instance-pool" => "mdb-strict-max-pool", "default-resource-adapter-name" => "hornetq-ra", "default-singleton-access-timeout" => 5000L, "default-slsb-instance-pool" => "slsb-strict-max-pool", "default-stateful-access-timeout" => 5000, "service" => { "timer-service" => undefined, "remote" => undefined, "async" => undefined }, "strict-max-bean-instance-pool" => { "slsb-strict-max-pool" => undefined, "mdb-strict-max-pool" => undefined }, "thread-pool" => {"default" => undefined} } } [standalone@localhost:9999 subsystem=ejb3] :write-attribute(name=default-stateful-access-timeout,value=${sfsb.timeout:5000}) {"outcome" => "success"} [standalone@localhost:9999 subsystem=ejb3] :read-resource { "outcome" => "success", "result" => { "default-mdb-instance-pool" => "mdb-strict-max-pool", "default-resource-adapter-name" => "hornetq-ra", "default-singleton-access-timeout" => 5000L, "default-slsb-instance-pool" => "slsb-strict-max-pool", "default-stateful-access-timeout" => expression "${sfsb.timeout:5000}", "service" => { "timer-service" => undefined, "remote" => undefined, "async" => undefined }, "strict-max-bean-instance-pool" => { "slsb-strict-max-pool" => undefined, "mdb-strict-max-pool" => undefined }, "thread-pool" => {"default" => undefined} } }

On 10/18/11 5:34 PM, Scott Stark wrote: > Thanks. I'll check against the attributes I'll need and raise issues to > address those I find missing. > Thanks; that will be very helpful. > One issue I see is that we are only looking to System.getProperty(name) > for the value of an expression in the org.jboss.dmr.ExpressionValue > class. The expressions I'm typically using need to be queried using > System.getenv(name). Would it be ok to fallback to a System.getenv(name) > if there is no System.getProperty(name) value found? This would require > an update to the dmr project. > I'm fine with that. > On 10/18/11 1:35 PM, Brian Stansberry wrote: >> See example below of reading a resource and then setting an attribute to >> an expression value. This will work properly on master when I'm done today. >> >> Note that not all attributes support expressions (most don't). For those >> that do, the :read-resource-description content for the attribute will >> include: >> >> "expressions-allowed" => true >> >> [standalone@localhost:9999 /] cd subsystem=ejb3 >> [standalone@localhost:9999 subsystem=ejb3] :read-resource >> >> { >> "outcome" => "success", >> "result" => { >> "default-mdb-instance-pool" => "mdb-strict-max-pool", >> "default-resource-adapter-name" => "hornetq-ra", >> "default-singleton-access-timeout" => 5000L, >> "default-slsb-instance-pool" => "slsb-strict-max-pool", >> "default-stateful-access-timeout" => 5000, >> "service" => { >> "timer-service" => undefined, >> "remote" => undefined, >> "async" => undefined >> }, >> "strict-max-bean-instance-pool" => { >> "slsb-strict-max-pool" => undefined, >> "mdb-strict-max-pool" => undefined >> }, >> "thread-pool" => {"default" => undefined} >> } >> } >> [standalone@localhost:9999 subsystem=ejb3] >> :write-attribute(name=default-stateful-access-timeout,value=${sfsb.timeout:5000}) >> {"outcome" => "success"} >> [standalone@localhost:9999 subsystem=ejb3] :read-resource >> >> { >> "outcome" => "success", >> "result" => { >> "default-mdb-instance-pool" => "mdb-strict-max-pool", >> "default-resource-adapter-name" => "hornetq-ra", >> "default-singleton-access-timeout" => 5000L, >> "default-slsb-instance-pool" => "slsb-strict-max-pool", >> "default-stateful-access-timeout" => expression >> "${sfsb.timeout:5000}", >> "service" => { >> "timer-service" => undefined, >> "remote" => undefined, >> "async" => undefined >> }, >> "strict-max-bean-instance-pool" => { >> "slsb-strict-max-pool" => undefined, >> "mdb-strict-max-pool" => undefined >> }, >> "thread-pool" => {"default" => undefined} >> } >> } >> >> > > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev -- Brian Stansberry Principal Software Engineer JBoss by Red Hat _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Yes, hopefully. We can have a multi-valued reference like the following that would allow for the env-vars openshift uses, system properties, as well as defaults: <datasource jndi-name="java:jboss/datasources/MysqlDS" enabled="false" use-java-context="true" pool-name="MysqlDS"> <connection-url>jdbc:mysql://${env.OPENSHIFT_DB_HOST,mysql.host:localhost}:${env.OPENSHIFT_DB_PORT,mysql.port:3306}/${env.OPENSHIFT_APP_NAME,mysql.dbname:test}</connection-url> <driver>mysql</driver> <security> <user-name>admin</user-name> <password>${env.OPENSHIFT_DB_PASSWORD,mysql.password:admin}</password> </security> </datasource> I'll shoot something your way once I have it working. On 10/18/11 11:07 PM, Max Rydahl Andersen wrote: > hmm - with this we could actually end up having a shareable standalone.xml between openshift and local that doesn't > need any alterations. > > The trick is only to get agreement on what property/environment names it will use. > > Scott - once you got a standalone.xml that has these things in place could you share it and i'll investigate how much > work we would need to do to support launching AS7 locally with all those variables present. > > p.s. system properties would be our recommended approach since messing with environment variables and java launches > have a history of being unreliable cross platform ;) > > /max > > > On Oct 19, 2011, at 01:22, Brian Stansberry wrote: > >> On 10/18/11 5:34 PM, Scott Stark wrote: >>> Thanks. I'll check against the attributes I'll need and raise issues to >>> address those I find missing. >>> >> Thanks; that will be very helpful. >> >>> One issue I see is that we are only looking to System.getProperty(name) >>> for the value of an expression in the org.jboss.dmr.ExpressionValue >>> class. The expressions I'm typically using need to be queried using >>> System.getenv(name). Would it be ok to fallback to a System.getenv(name) >>> if there is no System.getProperty(name) value found? This would require >>> an update to the dmr project. >>> >> I'm fine with that. >> >>> On 10/18/11 1:35 PM, Brian Stansberry wrote: >>>> See example below of reading a resource and then setting an attribute to >>>> an expression value. This will work properly on master when I'm done today. >>>> >>>> Note that not all attributes support expressions (most don't). For those >>>> that do, the :read-resource-description content for the attribute will >>>> include: >>>> >>>> "expressions-allowed" => true >>>> >>>> [standalone@localhost:9999 /] cd subsystem=ejb3 >>>> [standalone@localhost:9999 subsystem=ejb3] :read-resource >>>> >>>> { >>>> "outcome" => "success", >>>> "result" => { >>>> "default-mdb-instance-pool" => "mdb-strict-max-pool", >>>> "default-resource-adapter-name" => "hornetq-ra", >>>> "default-singleton-access-timeout" => 5000L, >>>> "default-slsb-instance-pool" => "slsb-strict-max-pool", >>>> "default-stateful-access-timeout" => 5000, >>>> "service" => { >>>> "timer-service" => undefined, >>>> "remote" => undefined, >>>> "async" => undefined >>>> }, >>>> "strict-max-bean-instance-pool" => { >>>> "slsb-strict-max-pool" => undefined, >>>> "mdb-strict-max-pool" => undefined >>>> }, >>>> "thread-pool" => {"default" => undefined} >>>> } >>>> } >>>> [standalone@localhost:9999 subsystem=ejb3] >>>> :write-attribute(name=default-stateful-access-timeout,value=${sfsb.timeout:5000}) >>>> {"outcome" => "success"} >>>> [standalone@localhost:9999 subsystem=ejb3] :read-resource >>>> >>>> { >>>> "outcome" => "success", >>>> "result" => { >>>> "default-mdb-instance-pool" => "mdb-strict-max-pool", >>>> "default-resource-adapter-name" => "hornetq-ra", >>>> "default-singleton-access-timeout" => 5000L, >>>> "default-slsb-instance-pool" => "slsb-strict-max-pool", >>>> "default-stateful-access-timeout" => expression >>>> "${sfsb.timeout:5000}", >>>> "service" => { >>>> "timer-service" => undefined, >>>> "remote" => undefined, >>>> "async" => undefined >>>> }, >>>> "strict-max-bean-instance-pool" => { >>>> "slsb-strict-max-pool" => undefined, >>>> "mdb-strict-max-pool" => undefined >>>> }, >>>> "thread-pool" => {"default" => undefined} >>>> } >>>> } >>>> >>>> >>> _______________________________________________ >>> jboss-as7-dev mailing list >>> jboss-as7-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev >> >> -- >> Brian Stansberry >> Principal Software Engineer >> JBoss by Red Hat >> _______________________________________________ >> jboss-as7-dev mailing list >> jboss-as7-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev > /max > http://about.me/maxandersen > > > > > _______________________________________________ > jboss-as7-dev mailing list > jboss-as7-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/jboss-as7-dev _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

Actually it does support alternative properties, and I have added tests for that. I'm working on the update as part of https://issues.jboss.org/browse/AS7-2169.

The full syntax for the expression will now be: ${<[env.]name>[,<[env.]name2>[,<[env.]name3>...]][:<default>]} with the following resolution logic: 1. For each [env.]name value, a. First look for System.getProperty([env.]name) and use it if defined b. else, look for System.getenv(name) and use it if defined 2. If there is no matching system property or environment variable, use the default if provided. I have added tests for all of these combinations to the org.jboss.dmr.ExpressionValueTest test. This will be pushed out shortly.

On 10/20/11 7:45 AM, Brian Stansberry wrote: > <trolling-for-patch>The resolve() logic in dmr doesn't support > alternative properties the way the old jboss-common-core stuff > did.</trolling-for-patch> > > Doesn't deal with System.getEnv() either. ;) > > _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

> One issue I see is that we are only looking to System.getProperty(name) > for the value of an expression in the org.jboss.dmr.ExpressionValue > class. The expressions I'm typically using need to be queried using > System.getenv(name). Would it be ok to fallback to a System.getenv(name) > if there is no System.getProperty(name) value found? This would require > an update to the dmr project. > I'm fine with that.

Are there some examples of what the syntax looks like for the expression? On 10/18/11 11:41 AM, Brian Stansberry wrote: > On 10/18/11 1:28 PM, Scott Stark wrote: >> On 10/18/11 7:07 AM, Jason T. Greene wrote: >>> I think Brian's question was "Why can't we just put the env >>> expressions in the management operation?" Perhaps the CLI doesn't >>> fully use the expression type when seen? >>> >> I am talking about having the env expressions in the batch script of the >> management operations. The issue was that these are not being replaced >> by the command line tools. > They are not meant to be replaced by the command line tools; they should > be passed through to the server and stored as expressions. I'll put > through a change today in how some less-recently changed parts of the > model (e.g. interfaces, sockets) deal with expressions; I believe they > are not handling expressions that come in via the management APIs > correctly; they just deal with them properly in the parser. > _______________________________________________ jboss-as7-dev mailing list jboss-as7-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/jboss-as7-dev

>>> That's not the case when running against a local server thus always doing it as part as tool deploy >>> is then not feasible IMO - having a way to easily run the script would be relevant though. > Right, so, can we have a tools "configured server" notion that takes the standalone.xml at a given point, a cli batch file, and overwrites the standalone.xml and reapplies the batch file as a configure operation? I can easily see us having a "Apply CLI batch…" action or similar which I (guess?) would have to use the local AS7 CLI command to execute them against a server.

Not sure I fully follow your "configured server notion" - do you mean that we would bundled a "openshift setup" containing of a standalone.xml + cli file that users "apply" to the server ? Trying to think where that would fit in…

> In general this will be different from the remote/cloud environment even if one can use env/system property references because I'll have my desktop using a different database than in the remote env for example. Maybe the way to handle this is to have multiple scripts, one for the commands common to each environment, another for the configurations that differ more fundamentally based on the services available in the environment. not sure how to handle that without exposing a lot of complexity to the user…;(