[jboss-as7-dev] problems with deployment overlays and linking

Tue Sep 4 10:04:00 EDT 2012

I guess, it's model-wide hash, not specific to a path? It might not be 
effective if there was a totally unrelated change to rollback a request.

On 09/04/2012 04:01 PM, Brian Stansberry wrote:
> As part of our solution for audit logging, we're planning on creating a
> hash of the model, which would go into the audit log. In the domain
> case, hashes, one of the domain wide model and then each host has a hash
> of its host-specific stuff. This *may* prove useful in providing a kind
> of optimistic locking feature similar to the way Hibernate uses a
> version field. But I haven't thought about it carefully.
>
> On 9/4/12 7:27 AM, Alexey Loubyansky wrote:
>> That'll help, thanks.
>>
>> There is one other issue I can mention (which is not as critical as the
>> original) which makes linking a bit more complicated in the domain mode.
>> Standalone works fine.
>>
>> The linking in the domain mode is done as
>>
>> server-group=sg_name/deployment-overlay=overlay_name/deployment=app.war
>>
>> deployment-overlay=overlay_name has to be created explicitly with its
>> own 'add' (since we don't implicitly create non-existing parent nodes).
>> So, every time a link is added, we have to perform a check whether the
>> server-group=sg_name/deployment-overlay=overlay_name part exists. If
>> not, create it. Then proceed with the actual linking.
>>
>> It's doable, of course. But from the tools' side, to compose the final
>> composite op, it will need to perform some intermediary ops. Of course,
>> adding the non existing parent can and should be put into the same final
>> composite op for efficiency.
>>
>> This is not the only command which behaves this way. For these commands,
>> it means:
>> - the final composite request may differ from one execution to another
>> depending on the state of the model (not only argument values but actual
>> requested actions);
>> - the model checks (or other necessary intermediate ops) are performed
>> not at the same time with the actual final request (i.e. kind of in a
>> different transaction) which doesn't guarantee the same state when the
>> actual request is executed;
>> - the previous one may affect run-time batches, since every command
>> added to a batch is translated into a DMR request immediately, not right
>> before the batch is actually run (which would be safer and which I
>> should do actually).
>>
>> Alexey
>>
>>
>> On 09/04/2012 12:40 AM, Stuart Douglas wrote:
>>> I will fix this to make it work like the deploy operation. Not sure if I
>>> will have time to get to it today.
>>>
>>> Stuart
>>>> Alexey Loubyansky <mailto:alexey.loubyansky at redhat.com>
>>>> 3 September 2012 10:39 PM
>>>> Sorry for the late reply, I'll cherry-pick it, ok.
>>>>
>>>> One issue that I mentioned and still there is that you can't add a new
>>>> overlay with content in a single composite operation (first the content
>>>> should be uploaded then its references can be used in the overlay). So,
>>>> this command still can't be executed as part of a batch.
>>>>
>>>> Alexey
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>> jboss-as7-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>> Brian Stansberry <mailto:brian.stansberry at redhat.com>
>>>> 1 September 2012 2:28 AM
>>>> Do you see value in cherry-picking his patch and adjusting the command
>>>> first?  The issues you find implementing the command are what led to
>>>> these changes, so it'd be nice to know your commands work before
>>>> merging his stuff.
>>>>
>>>>
>>>>
>>>>
>>>> Alexey Loubyansky <mailto:olubyans at redhat.com>
>>>> 31 August 2012 8:03 PM
>>>> Great. Once it's merged I'll adjust the deployment-overlay command and
>>>> submit a pull request.
>>>>
>>>> Thanks,
>>>> Alexey
>>>>
>>>>
>>>> Stuart Douglas <mailto:stuart.w.douglas at gmail.com>
>>>> 31 August 2012 1:29 PM
>>>>
>>>> I have done up an initial implementation of this here
>>>> https://github.com/jbossas/jboss-as/pull/2989 .
>>>>
>>>>
>>>> Stuart
>>>>
>>>>
>>>>
>>>> Brian Stansberry <mailto:brian.stansberry at redhat.com>
>>>> 24 August 2012 2:08 AM
>>>> On 8/23/12 3:39 AM, Emanuel Muckenhuber wrote:
>>>>> On 08/23/2012 01:11 AM, Brian Stansberry wrote:
>>>>>> Sorry for the slow responses on this thread.
>>>>>>
>>>>>> I think we should go with Stuart's approach, particularly if
>>>>>> Alexey is
>>>>>> comfortable that the CLI can support high level commands associated
>>>>>> with it.
>>>>>>
>>>>> Yeah, i am fine then as well - nevertheless some comments inline :)
>>>>>
>>>>
>>>> Me too. ;)
>>>>
>>>>>> On 8/22/12 8:22 AM, Emanuel Muckenhuber wrote:
>>>>>>> My main concern is regarding the domain mode, since the additional
>>>>>>> linking does not seem to be necessary to me and just additional
>>>>>>> steps
>>>>>>> mgmt clients have to take care of.
>>>>>>>
>>>>>>> Whereas the deployment-scanner is a single client for standalone,
>>>>>>> which
>>>>>>> is under our control. So i wanted to see if we there is a way we can
>>>>>>> make this work which also satisfies your use cases.
>>>>>>>
>>>>>>> I have two things in mind which i'll try to explain quickly:
>>>>>>>
>>>>>>> 1) you still link the overlays as part of the <deployment />,
>>>>>>> however
>>>>>>> the deploment-scanner is aware of that and won't remove
>>>>>>> deployments with
>>>>>>> overlays until you use a specific invalidation marker.
>>>>>>>
>>>>>> This requires that the overlays be controlled by the scanner as well.
>>>>>> Why? Because we don't persist the model resource for
>>>>>> scanner-controlled
>>>>>> deployments. So if a user linked a non-scanner-controlled overlay
>>>>>> to a
>>>>>> scanner controlled deployment (which seems quite rational if the
>>>>>> overlay
>>>>>> involves some environmental settings that never change), the link
>>>>>> information will be lost on server reboot.
>>>>>>
>>>>> I know, but that's an ancient limitation. I guess we could just
>>>>> start to
>>>>> persist stuff.
>>>>>
>>>>
>>>> We'd have to persist some association between a deployment and a
>>>> scanner
>>>> (e.g. scanner="default" on the deployment element) and then use that to
>>>> indicate that the server should not auto-deploy on boot but should
>>>> defer
>>>> to the scanner to tell it what to do. And then alter the scanner to add
>>>> logic for it to reconcile the state it sees in the scan directory vs
>>>> what the model shows.
>>>>
>>>> Doable, but a fair bit of work.
>>>>
>>>> I considered this before going with the "don't persist" approach. I
>>>> don't recall though if there was some fundamental problem with it that
>>>> I'm not remembering, or if it was solely that "don't persist" was
>>>> doable
>>>> within the time constraints.
>>>>
>>>>>>> 2) use overlay directories - this is more the file based approach,
>>>>>>> Where
>>>>>>> we just pick the "deployment.name + suffix" for the overlay. So
>>>>>>> there
>>>>>>> would be an automatic overlay/linking done by the
>>>>>>> deployment-scanner.
>>>>>>>
>>>>>> Same issue as 1).
>>>>>>
>>>>> Not really. It can automatically recreate the overlays therefore it
>>>>> does
>>>>> not require persistent information.
>>>>>
>>>>
>>>> It can only recreate the association for overlays that it scans, not
>>>> ones that were added via CLI/console.
>>>>
>>>>>>> 2.1) the overlay directory could be just a link to a virtual file (a
>>>>>>> shared <deployment-overlay />) [1] - so in this case the overlays
>>>>>>> are
>>>>>>> managed by the model and are not linked to the deployment lifecycle.
>>>>>>>
>>>>>> I confess I don't understand this. In particular, how does it
>>>>>> fundamentally differ from what Stuart proposes in terms of how to
>>>>>> organize the data?
>>>>>>
>>>>> Well that is the point. It's basically the same, just that it removes
>>>>> the requirement for the additional linking in the domain.
>>>>>
>>>>
>>>> Ok, then I still don't understand it. If it solves a problem I'd
>>>> like to
>>>> understand. Can you explain further?
>>>>
>>>>>>> On 08/22/2012 01:03 AM, Stuart Douglas wrote:
>>>>>>>> So what are we going to do here? Personally I much prefer the
>>>>>>>> idea of having this stored in the model and linking it to a
>>>>>>>> deployment, so the overlay lifecycle is not tied to that of the
>>>>>>>> deployment.
>>>>>>>>
>>>>>>>> Another idea I had is that we could do something like define the
>>>>>>>> overlays at domain level, but then define the links at server
>>>>>>>> group or server level using the same name. basically something
>>>>>>>> like:
>>>>>>>>
>>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/web.xml
>>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/ejb-jar.xml
>>>>>>>>
>>>>>>>> /server-group=server-group1/deployment-overlay=myOverlay/deployment=test.war
>>>>>>>>
>>>>>>>>
>>>>>>>> /server-group=server-group1/deployment-overlay=myOverlay/deployment=*.war
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> For standalone they would just be combined, similar to what I had
>>>>>>>> below:
>>>>>>>>
>>>>>>>>
>>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/web.xml
>>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/ejb-jar.xml
>>>>>>>> /deployment-overlay=myOverlay/deployment=test.war
>>>>>>>> /deployment-overlay=myOverlay/deployment=*.war
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Stuart
>>>>>>>>
>>>>>>>> On 17/08/2012, at 11:03 PM, Emanuel
>>>>>>>> Muckenhuber<emuckenh at redhat.com>  wrote:
>>>>>>>>
>>>>>>>>> On 08/17/2012 02:52 AM, Stuart Douglas wrote:
>>>>>>>>>> The big problem with this is that it does not work with the
>>>>>>>>>> deployment scanner.
>>>>>>>>>>
>>>>>>>>> Yeah, the deployment-scanner is always a story for itself.
>>>>>>>>> However i think we could actually make it persist information
>>>>>>>>> and be aware of overlays.
>>>>>>>>>
>>>>>>>>> Having overlays as part of the deployment means the lifecycle is
>>>>>>>>> managed together, so the deployment-scanner could define it's
>>>>>>>>> own invalidation policy. May it be based on whether the content
>>>>>>>>> got removed or we have a specific marker for that.
>>>>>>>>>
>>>>>>>>> We could also think of providing a file-based overlay solution,
>>>>>>>>> managed by the deployment scanner only. In the end people use
>>>>>>>>> the deployment-scanner for a reason, so perhaps not having to
>>>>>>>>> worry about the model or "content" folder could be interesting.
>>>>>>>>> This would not even need to access persistent information
>>>>>>>>> though. Anyway just some ideas on the side.
>>>>>>>>>
>>>>>>>>>> It also has to be specified on every re-deploy.
>>>>>>>>> When you do a "remove" and "add" then yes. We do have specific
>>>>>>>>> operations to replace the content and redeploy the deployment
>>>>>>>>> without removing the information from the model.
>>>>>>>>>
>>>>>>>>>> I think the simplest solution here
>>>>>>>>>> is to just move the links under the deployment-overlays element
>>>>>>>>>> and loose the
>>>>>>>>>> flexibility to link at different levels. e.g:
>>>>>>>>>>
>>>>>>>>>> /deployment-overlay=myOverlay/deployment=test.war
>>>>>>>>>> /deployment-overlay=myOverlay/deployment=*.jar
>>>>>>>>>>
>>>>>>>>> Yes, i was not really going for the simplest solution for now -
>>>>>>>>> this would most likely would be it :)
>>>>>>> _______________________________________________
>>>>>>> jboss-as7-dev mailing list
>>>>>>> jboss-as7-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>>>
>>>>> _______________________________________________
>>>>> jboss-as7-dev mailing list
>>>>> jboss-as7-dev at lists.jboss.org
>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>
>>>>
>>>>
>
>