trying to run subsystem extension in host controller
by John Mazzitelli
<TL;DR>
Trying to run a subsystem extension in host controller and it fails because some of its service dependencies are missing. Am I doing something wrong (it would not be a shocker if I was) or are these services really not deployed inside the Host Controller?
</TL;DR>
I'm trying to get a subsystem extension to run inside of the Host Controller. This is a subsystem extension that runs fine in a standalone WildFly container - just trying to get it to run in a host controller (and maybe later in a domain controller as well).
In domain/configuration/host.xml, I add my own <subsystem> in the <profile> - and that seems to run OK (well, not OK, but I know it is getting its initialization method invoked by the container).
Because I need an outbound socket binding defined, I put add a socket-binding-groups in the host.xml (there are no socket-binding-groups in here by default):
<socket-binding-groups>
<socket-binding-group default-interface="public" name="standard-sockets">
<outbound-socket-binding name="hawkular">
<remote-destination host="127.0.0.1" port="8080"/>
</outbound-socket-binding>
</socket-binding-group>
</socket-binding-groups>
That causes the container to abort with "WFLYCTL0198: Unexpected element '{urn:jboss:domain:4.0}socket-binding-groups' encountered" So I remove <socket-binding-groups> so that <socket-binding-group> is a direct child of the top-level <host>. The container starts OK when I do this.
But now the problems - my subsystem registers itself with a few service dependencies that do not exist at the time my service is initialized - specifically:
jboss.naming.context.java (o.j.a.naming.deployment.ContextNames.JAVA_CONTEXT_SERVICE_NAME)
jboss.outbound-socket-binding (o.j.a.network.OutboundSocketBinding.OUTBOUND_SOCKET_BINDING_BASE_SERVICE_NAME)
jboss.server.environment (o.j.a.server.ServerEnvironmentService.SERVICE_NAME)
jboss.as.server-controller (o.j.a.server.Services.JBOSS_SERVER_CONTROLLER)
When the host controller runs, I get this startup error:
ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([
("host" => "master"),
("subsystem" => "hawkular-wildfly-agent")
]) - failure description: {"WFLYCTL0180: Services with missing/unavailable dependencies" => [
"\"org.hawkular.agent.\".hawkular-wildfly-agent is missing [jboss.outbound-socket-binding.hawkular, jboss.server.environment, jboss.as.server-controller]",
"jboss.naming.context.java.abc is missing [jboss.naming.context.java]"
]}
Sooo..... my question is - what happens if a subsystem needs these things? Do I need to do something special to make them available or are they really not deployed in the host controller?
What I need these for is:
jboss.naming.context.java - so I can store something in JNDI (actually, for my use case, I can probably skip this dependency - I don't think I will need it when running in the host controller)
jboss.outbound-socket-binding - my subsystem needs to talk to an external server and this defines what host/port that server is on
jboss.server.environment - I need this to find out where the host controller's data directory is (e.g. domain/data). Is there another way to get this?
jboss.as.server-controller - I need this to obtain information from the management model such as "/:uuid" (which it looks like HCs do not have), the "/:launch-type" attribute and others.
8 years, 7 months
Re: [wildfly-dev] Graceful startup
by gaoyonglu＠126.com
One thing I am not really sure about is if we need a configuration
> switch for hold/reject behavior
I think use reject is better. The weblogic server start graceful use reject and if use hold option we must set queue to hold request.
I very much looking forward to this feature
gaoyonglu(a)126.com
From: wildfly-dev-request
Date: 2016-04-22 07:00
To: wildfly-dev
Subject: wildfly-dev Digest, Vol 37, Issue 9
Send wildfly-dev mailing list submissions to
wildfly-dev(a)lists.jboss.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.jboss.org/mailman/listinfo/wildfly-dev
or, via email, send a message with subject or body 'help' to
wildfly-dev-request(a)lists.jboss.org
You can reach the person managing the list at
wildfly-dev-owner(a)lists.jboss.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of wildfly-dev digest..."
Today's Topics:
1. Re: Graceful startup (David M. Lloyd)
2. question on jboss-app_7_0.xsd intermingling jee versions
(Rob Stryker)
3. Re: question on jboss-app_7_0.xsd intermingling jee versions
(Rob Stryker)
4. Re: Graceful startup (Brian Stansberry)
5. Re: Graceful startup (John Doyle)
6. Re: Graceful startup (Brian Stansberry)
7. Re: Graceful startup (Stuart Douglas)
----------------------------------------------------------------------
Message: 1
Date: Thu, 21 Apr 2016 06:50:47 -0500
From: "David M. Lloyd" <david.lloyd(a)redhat.com>
Subject: Re: [wildfly-dev] Graceful startup
To: wildfly-dev(a)lists.jboss.org
Message-ID: <5718BE97.9010405(a)redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
On 04/20/2016 09:05 PM, Stuart Douglas wrote:
> Hi,
>
> I have come across a few bug reports [1][2] (and a feature request from
> the Swarm team) recently that are essentially caused by an application
> being accessed before it is fully deployed. Basically even though we
> have service dependencies that make sure individual components
> dependencies are up, once a request has been accepted it can potentially
> programmatically access other parts of the deployment that are not up
> yet (basically the same problem we have with graceful shutdown, but in
> reverse).
>
> I propose we solve this using a 'graceful startup' mechanism, that holds
> or rejects new requests until a server or deployment is fully started.
> The specifics of how this would work are:
>
> - If the server is booting all external requests will be held or
> rejected until the the boot process is complete
> - When deploying a new deployment all requests for that deployment will
> be held or rejected until MSC has attained stability
>
> This would be implemented for the following endpoints/subsystems:
>
> - Undertow will hold requests until the deployment is done (so if you
> try and load a page while deployment is happening it could be a bit of a
> wait)
> - Remote EJB will hold requests until deployment is done
> - mod_cluster will not send availability messages until deployment is done
> - JMS will delay message delivery until deployment is done
> - EJB persistent timers will not fire until deployment is done
> - Possibly some other cases I can't think of right now
>
> One thing I am not really sure about is if we need a configuration
> switch for hold/reject behavior. e.g. for Undertow the request holding
> behavior is very developer friendly, as it means they can just hit
> refresh in their browser and as soon as the redeployment is done the
> page will display, however I am worried that it might not be ideal for
> load balancers that may prefer a quick error response that could then be
> attempted on another node (although if mod_cluster is not sending out
> availability till the deployment is 100% complete this may not be a big
> deal).
I think load balancers should not have a server that is being started up
in the rotation anyway; this already probably doesn't work great today.
I like the idea overall.
--
- DML
------------------------------
Message: 2
Date: Thu, 21 Apr 2016 12:08:05 -0400
From: Rob Stryker <rstryker(a)redhat.com>
Subject: [wildfly-dev] question on jboss-app_7_0.xsd intermingling jee
versions
To: "wildfly-dev(a)lists.jboss.org" <wildfly-dev(a)lists.jboss.org>
Message-ID: <5718FAE5.2020307(a)redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Hi guys:
I recently opened https://issues.jboss.org/browse/JBMETA-394 because it
seems our application schema are intermingling jee versions between ee6
and ee7, and it's causing problems for our eclipse validators.
Who's in charge of these schema, specifically? Who would be the right
person to ping to get it fixed?
- Rob Stryker
------------------------------
Message: 3
Date: Thu, 21 Apr 2016 12:13:51 -0400
From: Rob Stryker <rstryker(a)redhat.com>
Subject: Re: [wildfly-dev] question on jboss-app_7_0.xsd intermingling
jee versions
To: wildfly-dev(a)lists.jboss.org
Message-ID: <5718FC3F.4020300(a)redhat.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Ok, seems I missed some responses before.
> Max Anderson wrote: Rob - is the error you see on the
jboss-application.xml you wrote or in the included xsd ?
I opened https://issues.jboss.org/browse/JBMETA-394 for a more clear
explanation as to the specific error. The error is in a user's
jboss-application.xml.
> Marek wrote: I guess you mixed Java EE 6 with JBoss AS 7 versions here.
I'll try to find out where the example is coming from. If it's from one
of our published examples, then I would guess it needs to be fixed
there. Looking at the released versions of jboss-app_x_y.xsd, it would
seem Marek is probably right and the versions there match the jboss/wf
releases, and not the ee version.
I'll dig deeper. Thanks.
On 04/21/2016 12:08 PM, Rob Stryker wrote:
> Hi guys:
>
> I recently opened https://issues.jboss.org/browse/JBMETA-394 because it
> seems our application schema are intermingling jee versions between ee6
> and ee7, and it's causing problems for our eclipse validators.
>
> Who's in charge of these schema, specifically? Who would be the right
> person to ping to get it fixed?
>
> - Rob Stryker
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
------------------------------
Message: 4
Date: Thu, 21 Apr 2016 13:44:48 -0500
From: Brian Stansberry <brian.stansberry(a)redhat.com>
Subject: Re: [wildfly-dev] Graceful startup
To: wildfly-dev(a)lists.jboss.org
Message-ID: <57191FA0.9070800(a)redhat.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
On 4/20/16 9:05 PM, Stuart Douglas wrote:
> Hi,
>
> I have come across a few bug reports [1][2] (and a feature request from
> the Swarm team) recently that are essentially caused by an application
> being accessed before it is fully deployed. Basically even though we
> have service dependencies that make sure individual components
> dependencies are up, once a request has been accepted it can potentially
> programmatically access other parts of the deployment that are not up
> yet (basically the same problem we have with graceful shutdown, but in
> reverse).
>
> I propose we solve this using a 'graceful startup' mechanism, that holds
> or rejects new requests until a server or deployment is fully started.
> The specifics of how this would work are:
>
> - If the server is booting all external requests will be held or
> rejected until the the boot process is complete
> - When deploying a new deployment all requests for that deployment will
> be held or rejected until MSC has attained stability
>
> This would be implemented for the following endpoints/subsystems:
>
> - Undertow will hold requests until the deployment is done (so if you
> try and load a page while deployment is happening it could be a bit of a
> wait)
> - Remote EJB will hold requests until deployment is done
> - mod_cluster will not send availability messages until deployment is done
Isn't this what mod_cluster does on a per-deployment basis anyway? The
basic idea of mod_cluster is the LB isn't notified the context is
available on a backend server until it's....available.
> - JMS will delay message delivery until deployment is done
> - EJB persistent timers will not fire until deployment is done
> - Possibly some other cases I can't think of right now
>
> One thing I am not really sure about is if we need a configuration
> switch for hold/reject behavior. e.g. for Undertow the request holding
> behavior is very developer friendly, as it means they can just hit
> refresh in their browser and as soon as the redeployment is done the
> page will display, however I am worried that it might not be ideal for
> load balancers that may prefer a quick error response that could then be
> attempted on another node (although if mod_cluster is not sending out
> availability till the deployment is 100% complete this may not be a big
> deal).
>
> If you want to see this in action I have a very simple PR at [3] that
> enables this for Undertow at server boot.
>
> Thoughts?
>
> Stuart
>
> [1] https://issues.jboss.org/browse/WFLY-6402
> [2] https://issues.jboss.org/browse/JBEAP-867
> [3] https://github.com/wildfly/wildfly/pull/8858
>
>
>
>
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
------------------------------
Message: 5
Date: Thu, 21 Apr 2016 15:07:18 -0400
From: John Doyle <jdoyle(a)redhat.com>
Subject: Re: [wildfly-dev] Graceful startup
To: Brian Stansberry <brian.stansberry(a)redhat.com>
Cc: wildfly-dev(a)lists.jboss.org
Message-ID:
<CAKSM3ocU+B0gXue4fu6JUZ=wA_Ov9F3kPRhEeAFU-OAO5GgNNQ(a)mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Thu, Apr 21, 2016 at 2:44 PM, Brian Stansberry
<brian.stansberry(a)redhat.com> wrote:
> On 4/20/16 9:05 PM, Stuart Douglas wrote:
>> Hi,
>>
>> I have come across a few bug reports [1][2] (and a feature request from
>> the Swarm team) recently that are essentially caused by an application
>> being accessed before it is fully deployed. Basically even though we
>> have service dependencies that make sure individual components
>> dependencies are up, once a request has been accepted it can potentially
>> programmatically access other parts of the deployment that are not up
>> yet (basically the same problem we have with graceful shutdown, but in
>> reverse).
>>
>> I propose we solve this using a 'graceful startup' mechanism, that holds
>> or rejects new requests until a server or deployment is fully started.
>> The specifics of how this would work are:
>>
>> - If the server is booting all external requests will be held or
>> rejected until the the boot process is complete
>> - When deploying a new deployment all requests for that deployment will
>> be held or rejected until MSC has attained stability
>>
>> This would be implemented for the following endpoints/subsystems:
>>
>> - Undertow will hold requests until the deployment is done (so if you
>> try and load a page while deployment is happening it could be a bit of a
>> wait)
>> - Remote EJB will hold requests until deployment is done
>> - mod_cluster will not send availability messages until deployment is done
>
> Isn't this what mod_cluster does on a per-deployment basis anyway? The
> basic idea of mod_cluster is the LB isn't notified the context is
> available on a backend server until it's....available.
True, but not everyone is using mod_cluster.
>
>> - JMS will delay message delivery until deployment is done
>> - EJB persistent timers will not fire until deployment is done
>> - Possibly some other cases I can't think of right now
>>
>> One thing I am not really sure about is if we need a configuration
>> switch for hold/reject behavior. e.g. for Undertow the request holding
>> behavior is very developer friendly, as it means they can just hit
>> refresh in their browser and as soon as the redeployment is done the
>> page will display, however I am worried that it might not be ideal for
>> load balancers that may prefer a quick error response that could then be
>> attempted on another node (although if mod_cluster is not sending out
>> availability till the deployment is 100% complete this may not be a big
>> deal).
>>
>> If you want to see this in action I have a very simple PR at [3] that
>> enables this for Undertow at server boot.
>>
>> Thoughts?
>>
>> Stuart
>>
>> [1] https://issues.jboss.org/browse/WFLY-6402
>> [2] https://issues.jboss.org/browse/JBEAP-867
>> [3] https://github.com/wildfly/wildfly/pull/8858
>>
>>
>>
>>
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>
>
>
> --
> Brian Stansberry
> Senior Principal Software Engineer
> JBoss by Red Hat
> _______________________________________________
> wildfly-dev mailing list
> wildfly-dev(a)lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/wildfly-dev
------------------------------
Message: 6
Date: Thu, 21 Apr 2016 15:39:56 -0500
From: Brian Stansberry <brian.stansberry(a)redhat.com>
Subject: Re: [wildfly-dev] Graceful startup
To: John Doyle <jdoyle(a)redhat.com>
Cc: wildfly-dev(a)lists.jboss.org
Message-ID: <57193A9C.1040103(a)redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
On 4/21/16 2:07 PM, John Doyle wrote:
> On Thu, Apr 21, 2016 at 2:44 PM, Brian Stansberry
> <brian.stansberry(a)redhat.com> wrote:
>> On 4/20/16 9:05 PM, Stuart Douglas wrote:
>>> Hi,
>>>
>>> I have come across a few bug reports [1][2] (and a feature request from
>>> the Swarm team) recently that are essentially caused by an application
>>> being accessed before it is fully deployed. Basically even though we
>>> have service dependencies that make sure individual components
>>> dependencies are up, once a request has been accepted it can potentially
>>> programmatically access other parts of the deployment that are not up
>>> yet (basically the same problem we have with graceful shutdown, but in
>>> reverse).
>>>
>>> I propose we solve this using a 'graceful startup' mechanism, that holds
>>> or rejects new requests until a server or deployment is fully started.
>>> The specifics of how this would work are:
>>>
>>> - If the server is booting all external requests will be held or
>>> rejected until the the boot process is complete
>>> - When deploying a new deployment all requests for that deployment will
>>> be held or rejected until MSC has attained stability
>>>
>>> This would be implemented for the following endpoints/subsystems:
>>>
>>> - Undertow will hold requests until the deployment is done (so if you
>>> try and load a page while deployment is happening it could be a bit of a
>>> wait)
>>> - Remote EJB will hold requests until deployment is done
>>> - mod_cluster will not send availability messages until deployment is done
>>
>> Isn't this what mod_cluster does on a per-deployment basis anyway? The
>> basic idea of mod_cluster is the LB isn't notified the context is
>> available on a backend server until it's....available.
>
> True, but not everyone is using mod_cluster.
>
Sure, but the statement I was commenting on was about proposed new
behavior of our mod_cluster subsystem. But the proposed new behavior
sounds like what the existing behavior should already be. So I wonder if
I'm missing something.
>>
>>> - JMS will delay message delivery until deployment is done
>>> - EJB persistent timers will not fire until deployment is done
>>> - Possibly some other cases I can't think of right now
>>>
>>> One thing I am not really sure about is if we need a configuration
>>> switch for hold/reject behavior. e.g. for Undertow the request holding
>>> behavior is very developer friendly, as it means they can just hit
>>> refresh in their browser and as soon as the redeployment is done the
>>> page will display, however I am worried that it might not be ideal for
>>> load balancers that may prefer a quick error response that could then be
>>> attempted on another node (although if mod_cluster is not sending out
>>> availability till the deployment is 100% complete this may not be a big
>>> deal).
>>>
>>> If you want to see this in action I have a very simple PR at [3] that
>>> enables this for Undertow at server boot.
>>>
>>> Thoughts?
>>>
>>> Stuart
>>>
>>> [1] https://issues.jboss.org/browse/WFLY-6402
>>> [2] https://issues.jboss.org/browse/JBEAP-867
>>> [3] https://github.com/wildfly/wildfly/pull/8858
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> wildfly-dev mailing list
>>> wildfly-dev(a)lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
>>>
>>
>>
>> --
>> Brian Stansberry
>> Senior Principal Software Engineer
>> JBoss by Red Hat
>> _______________________________________________
>> wildfly-dev mailing list
>> wildfly-dev(a)lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/wildfly-dev
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
------------------------------
Message: 7
Date: Thu, 21 Apr 2016 22:59:54 +0000
From: Stuart Douglas <stuart.w.douglas(a)gmail.com>
Subject: Re: [wildfly-dev] Graceful startup
To: Brian Stansberry <brian.stansberry(a)redhat.com>,
wildfly-dev(a)lists.jboss.org
Message-ID:
<CAAoo=c5DAg+yCzMxpzK4BDxfViQDTJnb_vMZz22k4_WwZrHikg(a)mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
> > - Undertow will hold requests until the deployment is done (so if you
> > try and load a page while deployment is happening it could be a bit of a
> > wait)
> > - Remote EJB will hold requests until deployment is done
> > - mod_cluster will not send availability messages until deployment is
> done
>
> Isn't this what mod_cluster does on a per-deployment basis anyway? The
> basic idea of mod_cluster is the LB isn't notified the context is
> available on a backend server until it's....available.
>
>
At the moment it is sent when the Undertow service comes up. It is still
possible that there could be other services (especially in a multi module
EAR deployment) that have not come up yet.
In practice for a lot of deployments this window will be very small or even
non existent, and the behavior will be basically indistinguishable.
Stuart
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.jboss.org/pipermail/wildfly-dev/attachments/20160421/52d3edb...
------------------------------
_______________________________________________
wildfly-dev mailing list
wildfly-dev(a)lists.jboss.org
https://lists.jboss.org/mailman/listinfo/wildfly-dev
End of wildfly-dev Digest, Vol 37, Issue 9
******************************************
8 years, 7 months
Graceful startup
by Stuart Douglas
Hi,
I have come across a few bug reports [1][2] (and a feature request from the
Swarm team) recently that are essentially caused by an application being
accessed before it is fully deployed. Basically even though we have service
dependencies that make sure individual components dependencies are up, once
a request has been accepted it can potentially programmatically access
other parts of the deployment that are not up yet (basically the same
problem we have with graceful shutdown, but in reverse).
I propose we solve this using a 'graceful startup' mechanism, that holds or
rejects new requests until a server or deployment is fully started. The
specifics of how this would work are:
- If the server is booting all external requests will be held or rejected
until the the boot process is complete
- When deploying a new deployment all requests for that deployment will be
held or rejected until MSC has attained stability
This would be implemented for the following endpoints/subsystems:
- Undertow will hold requests until the deployment is done (so if you try
and load a page while deployment is happening it could be a bit of a wait)
- Remote EJB will hold requests until deployment is done
- mod_cluster will not send availability messages until deployment is done
- JMS will delay message delivery until deployment is done
- EJB persistent timers will not fire until deployment is done
- Possibly some other cases I can't think of right now
One thing I am not really sure about is if we need a configuration switch
for hold/reject behavior. e.g. for Undertow the request holding behavior is
very developer friendly, as it means they can just hit refresh in their
browser and as soon as the redeployment is done the page will display,
however I am worried that it might not be ideal for load balancers that may
prefer a quick error response that could then be attempted on another node
(although if mod_cluster is not sending out availability till the
deployment is 100% complete this may not be a big deal).
If you want to see this in action I have a very simple PR at [3] that
enables this for Undertow at server boot.
Thoughts?
Stuart
[1] https://issues.jboss.org/browse/WFLY-6402
[2] https://issues.jboss.org/browse/JBEAP-867
[3] https://github.com/wildfly/wildfly/pull/8858
8 years, 7 months
Graceful shutdown on SIGTERM?
by Marko Lukša
Any plans to hook graceful shutdown to SIGTERM or is the plan to always
require the CLI for this?
Regards
Marko
8 years, 7 months
Re: [wildfly-dev] Inter Host Controller group communication mesh
by Brian Stansberry
On 4/18/16 10:18 AM, Bela Ban wrote:
> Hey Brian,
>
> On 18/04/16 17:04, Brian Stansberry wrote:
>> As an FYI, copying Bela Ban, who I stupidly forgot to copy on the first
>> post. Sebastian kindly copied him on the other main branch of the thread.
>
> Yes, I read that thread.
>
>> Bela, tl;dr on this branch is it mostly discusses concerns about N^2 TCP
>> connections in a possibly very large cluster. Whether the JGroups
>> cluster would need to get very large depends on what use cases we used
>> it to solve.
>
> When I tested on a 2000+ node cluster running over TCP in Google Compute
> Engine, TCP wasn't that much of an issue.
Good!
> The main drawbacks were that
> every node needed to have ~2000 connections open, which means 1 reader
> thread running per connection. However, connections are closed after a
> configurable idle time.
>
> TCP_NIO2 is much better in that respect as it gets rid of the reader
> threads even if the connection is open.
>
In the POC I did, the transport uses the NIO-based JBoss Remoting
infrastructure we already use for intra-domain communications. All
connections are created using a single remoting Endpoint instance, which
in turn uses a Xnio worker. That worker is configured with two io
threads, and then a pool of 5 min, max 10 threads for handling tasks.
That pool size is just the settings that were already in the code for
the existing uses of the endpoint (CLI requests, intra-domain comms etc)
and I spent zero time when I did the POC thinking about whether those
settings are appropriate if we also add JGroups traffic to the mix.
For the existing management uses of the endpoint, most of the work
actually gets shunted off to threads from a separate pool. A management
request comes in and the xnio worker threads deal with the initial work
of reading it off the wire or writing the response, but the bulk of the
work in between of actually doing the management stuff is done on
another thread. I'd need to refamiliarize myself with TP and the thread
pools JGroups uses to see if we get a similar effect with the JGroups
communications. I have some vague memories of up pools and down pools
and OOB pools and .... ;) All surely out of date.
> The other option is to use UDP without multicasting, ie. ip_mcast=false.
> This would not create N-1 connections and possibly N-1 reader threads
> and sockets, but only 2 sockets (constant) and no reader threads. A
> message would still need to be sent N-1 times though, creating increased
> traffic.
>
I don't think we could do that, at least not with this approach using
the existing JBoss Remoting server sockets. That's all TCP based.
> A potential solution for going from N-1 to a constant number of
> connections/threads would be daisy chaining where you only connect to
> your neighbor and a multicast basically is 1 round across the logical
> overlay, see [1] for details. I'd have to revisit this protocol though
> if you wanted to use it, so let me know asap for me to include this in
> the roadmap.
Ok, good to know. Will do.
> Cheers,
>
> [1] http://belaban.blogspot.ch/2010/08/daisychaining-in-clouds.html
>
>> On 4/11/16 4:20 PM, Brian Stansberry wrote:
>>> On 4/11/16 3:43 PM, Ken Wills wrote:
>>>>
>>>>
>>>> On Mon, Apr 11, 2016 at 11:57 AM, Brian Stansberry
>>>> <brian.stansberry(a)redhat.com <mailto:brian.stansberry@redhat.com>>
>>>> wrote:
>>>>
>>>> Just an FYI: I spent a couple days and worked up a POC[1] of
>>>> creating a
>>>> JGroups-based reliable group communication mesh over the sockets
>>>> our
>>>> Host Controllers use for intra-domain management communications.
>>>>
>>>>
>>>> Nice! I've been thinking about the mechanics of this a bit recently,
>>>> but
>>>> I hadn't gotten to any sort of transport details, this looks
>>>> interesting.
>>>>
>>>> Currently those sockets are used to form a tree of connections;
>>>> master
>>>> HC to slave HCs and then HCs to their servers. Slave HCs don't
>>>> talk to
>>>> each other. That kind of topology works fine for our current use
>>>> cases,
>>>> but not for other use cases, where a full communication mesh is
>>>> more
>>>> appropriate.
>>>>
>>>> 2 use cases led me to explore this:
>>>>
>>>> 1) A longstanding request to have automatic failover of the
>>>> master HC to
>>>> a backup. There are different ways to do this, but group
>>>> communication
>>>> based leader election is a possible solution. My preference,
>>>> really.
>>>>
>>>>
>>>> I'd come to the same conclusion of it being an election. A
>>>> deterministic
>>>> election algorithm, perhaps allowing the configuration to supply some
>>>> sort of weighted value to influence the election on each node, perhaps
>>>> analogous to how the master browser smb election works (version +
>>>> weight
>>>> + etc).
>>>
>>> Yep.
>>>
>>> For sure the master must be running the latest version.
>>>
>>>>
>>>>
>>>> 2) https://issues.jboss.org/browse/WFLY-1066, which has led to
>>>> various
>>>> design alternatives, one of which is a distributed cache of
>>>> topology
>>>> information, available via each HC. See [2] for some of that
>>>> discussion.
>>>>
>>>> I don't know if this kind of communication is a good idea, or if
>>>> it's
>>>> the right solution to either of these use cases. Lots of things
>>>> need
>>>> careful thought!! But I figured it was worth some time to
>>>> experiment.
>>>> And it worked in at least a basic POC way, hence this FYI.
>>>>
>>>>
>>>> Not knowing a lot about jgroups .. for very large domains is the mesh
>>>> NxN in size?
>>>
>>> Yes.
>>>
>>> For thousands of nodes would this become a problem,
>>>
>>> It's one concern I have, yes. There are large JGroups clusters, but they
>>> may be based on the UDP multicast transport JGroups offers.
>>>
>>>> or would
>>>> a mechanism to segment into local groups perhaps, with only certain
>>>> nodes participating in the mesh and being eligible for election?
>>>
>>>
>>> For sure we'd have something in the host.xml that controls whether a
>>> particular HC joins the group.
>>>
>>> I don't think this is a big problem for the DC election use case, as you
>>> don't need a large number of HCs in the group. You'd have a few
>>> "potential" DCs that could join the group, and the remaining slaves
>>> don't need to.
>>>
>>> For use cases where you want slave HCs to be in the cluster though, it's
>>> a concern. The distributed topology cache thing may or may not need
>>> that. It needs a few HCs to provide HA, but those could be the same ones
>>> that are "potential" HCs. But if only a few are in the group, the
>>> servers need to be told how to reach those HCs. Chicken and egg, as the
>>> point of the topology cache is to provide that kind of data to servers!
>>> If a server's own HC is required to be a part of the group though, that
>>> helps cut through the chicken/egg problem.
>>>
>>>
>>>> Ken
>>>>
>>>
>>>
>>
>>
>
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
8 years, 7 months
Inter Host Controller group communication mesh
by Brian Stansberry
Just an FYI: I spent a couple days and worked up a POC[1] of creating a
JGroups-based reliable group communication mesh over the sockets our
Host Controllers use for intra-domain management communications.
Currently those sockets are used to form a tree of connections; master
HC to slave HCs and then HCs to their servers. Slave HCs don't talk to
each other. That kind of topology works fine for our current use cases,
but not for other use cases, where a full communication mesh is more
appropriate.
2 use cases led me to explore this:
1) A longstanding request to have automatic failover of the master HC to
a backup. There are different ways to do this, but group communication
based leader election is a possible solution. My preference, really.
2) https://issues.jboss.org/browse/WFLY-1066, which has led to various
design alternatives, one of which is a distributed cache of topology
information, available via each HC. See [2] for some of that discussion.
I don't know if this kind of communication is a good idea, or if it's
the right solution to either of these use cases. Lots of things need
careful thought!! But I figured it was worth some time to experiment.
And it worked in at least a basic POC way, hence this FYI.
If you're interested in details, here are some Q&A:
Q: Why JGroups?
A: Because 1) I know it well 2) I trust it and 3) it's already used for
this kind of group communications in full WildFly.
Q: Why the management sockets? Why not other sockets?
A: Slave HCs already need configuration for how to discover the master.
Using the same sockets lets us reuse that discovery configuration for
the JGroups communications as well. If we're going to use this kind of
communication in an serious way, the configuration needs to be as easy
as possible.
Q: How does it work?
A: JGroups is based on a stack of "protocols" each of which handles one
aspect of reliable group communications. The POC creates and uses a
standard protocol stack, except it replaces two standard protocols with
custom ones:
a) JGroups has various "Discovery" protocols which are used to find
possible peers. I implemented one that integrates with the HC's domain
controller discovery logic. It's basically a copy of the oft used
TCPPING protocol with about 10-15 lines of code changed.
b) JGroups has various "Transport" protocols which are responsible for
actually sending/receiving over the network. I created a new one of
those that knows how to use the WF management comms stuff built on JBoss
Remoting. JGroups provides a number of base classes to use in this
transport area, so I was able to rely on a lot of existing functionality
and could just focus on the details specific to this case.
Q: What have you done using the POC?
A: I created a master HC and a slave on my laptop and saw them form a
cluster and exchange messages. Typical stuff like starting and stopping
the HCs worked. I see no reason why having multiple slaves wouldn't have
worked too; I just didn't do it.
Q: What's next?
A: Nothing really. We have a couple concrete use cases we're looking to
solve. We need to figure out the best solution for those use cases. If
this kind of thing is useful in that, great. If not, it was a fun POC.
[1]
https://github.com/wildfly/wildfly-core/compare/master...bstansberry:jgro...
. See the commit message on the single commit to learn a bit more.
[2] https://developer.jboss.org/wiki/ADomainManagedServiceRegistry
--
Brian Stansberry
Senior Principal Software Engineer
JBoss by Red Hat
8 years, 7 months
Early Access builds of JDK 9 b113 & JDK 9 with Project Jigsaw b113 (#4848) are available on java.net
by Rory O'Donnell
Hi Jason/Tomaz,
Early Access b113 <https://jdk9.java.net/download/> for JDK 9 is
available on java.net, summary of changes are listed here
<http://download.java.net/java/jdk9/changes/jdk-9+113.html>.
Early Access b113 <https://jdk9.java.net/jigsaw/> (#4664) for JDK 9 with
Project Jigsaw is available on java.net.
* The important change in this build is that root modules when
compiling code in the unnamed module, or when running and the main
class is loaded from the class path, do not include the EE modules.
More on this in JEP 261.
* The other change in this build is that the -Xpatch option is now
aligned with what we have documented in JEP 261, support for the old
form has been removed.
We are very interested in hearing your experiences in testing any Early
Access builds. Have you have begun testing against
JDK 9 and or JDK 9 with Project Jigsaw EA builds, have you uncovered
showstopper issues that you would like to discuss?
We would really like to hear your findings so far, either reply to me or
via the mailing lists [1], [2].
Rgds,Rory
[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/
[2] http://mail.openjdk.java.net/pipermail/jdk9-dev/
--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA, Dublin,Ireland
8 years, 7 months
undertow filters
by Claudio Miranda
Hi, working with wildfly 10.1 SNAPSHOT, with latest update as of Mar/23
The issue is, add an error-page filter, named error3, remove it, add
with the same name, it throws an exception of duplicated name, see
below.
The remove operation, requires a reload, but does it should prevent
the adding of a item with the same name ? I thought the reload
operation is necessary only to apply the settings at boot time.
[standalone@localhost:9990 /]
/subsystem=undertow/configuration=filter/error-page=error3:add(code=402,path="caminho2.html")
{"outcome" => "success"}
[standalone@localhost:9990 /]
/subsystem=undertow/configuration=filter/error-page=error3:remove
{
"outcome" => "success",
"response-headers" => {
"operation-requires-reload" => true,
"process-state" => "reload-required"
}
}
[standalone@localhost:9990 /]
/subsystem=undertow/configuration=filter/error-page=error3:add(code=402,path="caminho2.html")
{
"outcome" => "failed",
"failure-description" => "WFLYCTL0158: Operation handler failed:
org.jboss.msc.service.DuplicateServiceException: Service
jboss.undertow.filter.error3 is already registered",
"rolled-back" => true,
"response-headers" => {"process-state" => "reload-required"}
}
--
Claudio Miranda
claudio(a)claudius.com.br
http://www.claudius.com.br
8 years, 7 months
