[jboss-as7-dev] problems with deployment overlays and linking

Tue Sep 4 10:01:32 EDT 2012

As part of our solution for audit logging, we're planning on creating a 
hash of the model, which would go into the audit log. In the domain 
case, hashes, one of the domain wide model and then each host has a hash 
of its host-specific stuff. This *may* prove useful in providing a kind 
of optimistic locking feature similar to the way Hibernate uses a 
version field. But I haven't thought about it carefully.

On 9/4/12 7:27 AM, Alexey Loubyansky wrote:
> That'll help, thanks.
>
> There is one other issue I can mention (which is not as critical as the
> original) which makes linking a bit more complicated in the domain mode.
> Standalone works fine.
>
> The linking in the domain mode is done as
>
> server-group=sg_name/deployment-overlay=overlay_name/deployment=app.war
>
> deployment-overlay=overlay_name has to be created explicitly with its
> own 'add' (since we don't implicitly create non-existing parent nodes).
> So, every time a link is added, we have to perform a check whether the
> server-group=sg_name/deployment-overlay=overlay_name part exists. If
> not, create it. Then proceed with the actual linking.
>
> It's doable, of course. But from the tools' side, to compose the final
> composite op, it will need to perform some intermediary ops. Of course,
> adding the non existing parent can and should be put into the same final
> composite op for efficiency.
>
> This is not the only command which behaves this way. For these commands,
> it means:
> - the final composite request may differ from one execution to another
> depending on the state of the model (not only argument values but actual
> requested actions);
> - the model checks (or other necessary intermediate ops) are performed
> not at the same time with the actual final request (i.e. kind of in a
> different transaction) which doesn't guarantee the same state when the
> actual request is executed;
> - the previous one may affect run-time batches, since every command
> added to a batch is translated into a DMR request immediately, not right
> before the batch is actually run (which would be safer and which I
> should do actually).
>
> Alexey
>
>
> On 09/04/2012 12:40 AM, Stuart Douglas wrote:
>> I will fix this to make it work like the deploy operation. Not sure if I
>> will have time to get to it today.
>>
>> Stuart
>>> Alexey Loubyansky <mailto:alexey.loubyansky at redhat.com>
>>> 3 September 2012 10:39 PM
>>> Sorry for the late reply, I'll cherry-pick it, ok.
>>>
>>> One issue that I mentioned and still there is that you can't add a new
>>> overlay with content in a single composite operation (first the content
>>> should be uploaded then its references can be used in the overlay). So,
>>> this command still can't be executed as part of a batch.
>>>
>>> Alexey
>>> _______________________________________________
>>> jboss-as7-dev mailing list
>>> jboss-as7-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>> Brian Stansberry <mailto:brian.stansberry at redhat.com>
>>> 1 September 2012 2:28 AM
>>> Do you see value in cherry-picking his patch and adjusting the command
>>> first?  The issues you find implementing the command are what led to
>>> these changes, so it'd be nice to know your commands work before
>>> merging his stuff.
>>>
>>>
>>>
>>>
>>> Alexey Loubyansky <mailto:olubyans at redhat.com>
>>> 31 August 2012 8:03 PM
>>> Great. Once it's merged I'll adjust the deployment-overlay command and
>>> submit a pull request.
>>>
>>> Thanks,
>>> Alexey
>>>
>>>
>>> Stuart Douglas <mailto:stuart.w.douglas at gmail.com>
>>> 31 August 2012 1:29 PM
>>>
>>> I have done up an initial implementation of this here
>>> https://github.com/jbossas/jboss-as/pull/2989 .
>>>
>>>
>>> Stuart
>>>
>>>
>>>
>>> Brian Stansberry <mailto:brian.stansberry at redhat.com>
>>> 24 August 2012 2:08 AM
>>> On 8/23/12 3:39 AM, Emanuel Muckenhuber wrote:
>>>> On 08/23/2012 01:11 AM, Brian Stansberry wrote:
>>>>> Sorry for the slow responses on this thread.
>>>>>
>>>>> I think we should go with Stuart's approach, particularly if Alexey is
>>>>> comfortable that the CLI can support high level commands associated
>>>>> with it.
>>>>>
>>>> Yeah, i am fine then as well - nevertheless some comments inline :)
>>>>
>>>
>>> Me too. ;)
>>>
>>>>> On 8/22/12 8:22 AM, Emanuel Muckenhuber wrote:
>>>>>> My main concern is regarding the domain mode, since the additional
>>>>>> linking does not seem to be necessary to me and just additional steps
>>>>>> mgmt clients have to take care of.
>>>>>>
>>>>>> Whereas the deployment-scanner is a single client for standalone,
>>>>>> which
>>>>>> is under our control. So i wanted to see if we there is a way we can
>>>>>> make this work which also satisfies your use cases.
>>>>>>
>>>>>> I have two things in mind which i'll try to explain quickly:
>>>>>>
>>>>>> 1) you still link the overlays as part of the <deployment />, however
>>>>>> the deploment-scanner is aware of that and won't remove
>>>>>> deployments with
>>>>>> overlays until you use a specific invalidation marker.
>>>>>>
>>>>> This requires that the overlays be controlled by the scanner as well.
>>>>> Why? Because we don't persist the model resource for
>>>>> scanner-controlled
>>>>> deployments. So if a user linked a non-scanner-controlled overlay to a
>>>>> scanner controlled deployment (which seems quite rational if the
>>>>> overlay
>>>>> involves some environmental settings that never change), the link
>>>>> information will be lost on server reboot.
>>>>>
>>>> I know, but that's an ancient limitation. I guess we could just
>>>> start to
>>>> persist stuff.
>>>>
>>>
>>> We'd have to persist some association between a deployment and a scanner
>>> (e.g. scanner="default" on the deployment element) and then use that to
>>> indicate that the server should not auto-deploy on boot but should defer
>>> to the scanner to tell it what to do. And then alter the scanner to add
>>> logic for it to reconcile the state it sees in the scan directory vs
>>> what the model shows.
>>>
>>> Doable, but a fair bit of work.
>>>
>>> I considered this before going with the "don't persist" approach. I
>>> don't recall though if there was some fundamental problem with it that
>>> I'm not remembering, or if it was solely that "don't persist" was doable
>>> within the time constraints.
>>>
>>>>>> 2) use overlay directories - this is more the file based approach,
>>>>>> Where
>>>>>> we just pick the "deployment.name + suffix" for the overlay. So there
>>>>>> would be an automatic overlay/linking done by the deployment-scanner.
>>>>>>
>>>>> Same issue as 1).
>>>>>
>>>> Not really. It can automatically recreate the overlays therefore it
>>>> does
>>>> not require persistent information.
>>>>
>>>
>>> It can only recreate the association for overlays that it scans, not
>>> ones that were added via CLI/console.
>>>
>>>>>> 2.1) the overlay directory could be just a link to a virtual file (a
>>>>>> shared <deployment-overlay />) [1] - so in this case the overlays are
>>>>>> managed by the model and are not linked to the deployment lifecycle.
>>>>>>
>>>>> I confess I don't understand this. In particular, how does it
>>>>> fundamentally differ from what Stuart proposes in terms of how to
>>>>> organize the data?
>>>>>
>>>> Well that is the point. It's basically the same, just that it removes
>>>> the requirement for the additional linking in the domain.
>>>>
>>>
>>> Ok, then I still don't understand it. If it solves a problem I'd like to
>>> understand. Can you explain further?
>>>
>>>>>> On 08/22/2012 01:03 AM, Stuart Douglas wrote:
>>>>>>> So what are we going to do here? Personally I much prefer the
>>>>>>> idea of having this stored in the model and linking it to a
>>>>>>> deployment, so the overlay lifecycle is not tied to that of the
>>>>>>> deployment.
>>>>>>>
>>>>>>> Another idea I had is that we could do something like define the
>>>>>>> overlays at domain level, but then define the links at server
>>>>>>> group or server level using the same name. basically something like:
>>>>>>>
>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/web.xml
>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/ejb-jar.xml
>>>>>>>
>>>>>>> /server-group=server-group1/deployment-overlay=myOverlay/deployment=test.war
>>>>>>>
>>>>>>> /server-group=server-group1/deployment-overlay=myOverlay/deployment=*.war
>>>>>>>
>>>>>>>
>>>>>>> For standalone they would just be combined, similar to what I had
>>>>>>> below:
>>>>>>>
>>>>>>>
>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/web.xml
>>>>>>> /deployment-overlay=myOverlay/content=WEB-INF/ejb-jar.xml
>>>>>>> /deployment-overlay=myOverlay/deployment=test.war
>>>>>>> /deployment-overlay=myOverlay/deployment=*.war
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Stuart
>>>>>>>
>>>>>>> On 17/08/2012, at 11:03 PM, Emanuel
>>>>>>> Muckenhuber<emuckenh at redhat.com>  wrote:
>>>>>>>
>>>>>>>> On 08/17/2012 02:52 AM, Stuart Douglas wrote:
>>>>>>>>> The big problem with this is that it does not work with the
>>>>>>>>> deployment scanner.
>>>>>>>>>
>>>>>>>> Yeah, the deployment-scanner is always a story for itself.
>>>>>>>> However i think we could actually make it persist information
>>>>>>>> and be aware of overlays.
>>>>>>>>
>>>>>>>> Having overlays as part of the deployment means the lifecycle is
>>>>>>>> managed together, so the deployment-scanner could define it's
>>>>>>>> own invalidation policy. May it be based on whether the content
>>>>>>>> got removed or we have a specific marker for that.
>>>>>>>>
>>>>>>>> We could also think of providing a file-based overlay solution,
>>>>>>>> managed by the deployment scanner only. In the end people use
>>>>>>>> the deployment-scanner for a reason, so perhaps not having to
>>>>>>>> worry about the model or "content" folder could be interesting.
>>>>>>>> This would not even need to access persistent information
>>>>>>>> though. Anyway just some ideas on the side.
>>>>>>>>
>>>>>>>>> It also has to be specified on every re-deploy.
>>>>>>>> When you do a "remove" and "add" then yes. We do have specific
>>>>>>>> operations to replace the content and redeploy the deployment
>>>>>>>> without removing the information from the model.
>>>>>>>>
>>>>>>>>> I think the simplest solution here
>>>>>>>>> is to just move the links under the deployment-overlays element
>>>>>>>>> and loose the
>>>>>>>>> flexibility to link at different levels. e.g:
>>>>>>>>>
>>>>>>>>> /deployment-overlay=myOverlay/deployment=test.war
>>>>>>>>> /deployment-overlay=myOverlay/deployment=*.jar
>>>>>>>>>
>>>>>>>> Yes, i was not really going for the simplest solution for now -
>>>>>>>> this would most likely would be it :)
>>>>>> _______________________________________________
>>>>>> jboss-as7-dev mailing list
>>>>>> jboss-as7-dev at lists.jboss.org
>>>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>>>
>>>> _______________________________________________
>>>> jboss-as7-dev mailing list
>>>> jboss-as7-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/jboss-as7-dev
>>>>
>>>
>>>

-- 
Brian Stansberry
Principal Software Engineer
JBoss by Red Hat