[undertow-dev] Lightweight Service Bus for Orchestration of Microservices with mod_cluster in Undertow

[Stuart Douglas]

There are two possible fixes for this:

1) Add mod_cluster to the end of the handler chain, at the moment it is not really possible to do this, I would need to add some extra config to the Undertow subsystem to specify that a filter should be applied only if no deployments are matched, but this would be relatively easy.

2) Do the deployment match at the begining of the chain, then run the filters, then dispatch to the deployment. This would allow Wildfly to introduce some kind of 'deployment-target' predicate that returns true if the request is targeting a local deployment. 

I am thinking I will try and implement 2, as it is more flexible (as you could still combine this new predicate with other predicates, e.g. if you want to route to most local deployments with a few select exclusions). Not sure when I will get time to look at this though, hopefully sometime this month.

Stuart

----- Original Message -----
> From: "Erhard Siegl" <erhard.siegl at gepardec.com>
> To: undertow-dev at lists.jboss.org
> Sent: Wednesday, 3 February, 2016 6:59:33 AM
> Subject: [undertow-dev] Lightweight Service Bus for Orchestration of	Microservices with mod_cluster in Undertow
> 
> Hi,
> 
> I packed all buzzwords into the subject and I promise to talk about real
> issues from now on.
> 
> This is about issue https://issues.jboss.org/browse/UNDERTOW-626 , the
> background, possible solutions and workarounds. But first I’d like to thank
> Steward for his quick response to the issue and also to my problem with
> HotSpot 1.8.0_71. Build problem solved!
> 
> The Background
> 
> I live in a country that has 9 states. In the area I work, they share a
> datacenter. Some applications are organised centrally (one database for all
> states), some are organised locally (each state has its own database and its
> own application server). We have a rather big application that is organised
> locally. Since there are quite a few users and we want to have good
> availability, each state has at least two JBoss servers (EAP6.4 right now).
> So we have about 20 servers in production. The problem now is, that they
> don’t live an isolated life, but sometimes they have to talk to each other.
> And not only with each other but also with a few other applications, some of
> them organised locally, some centrally. Usually they communicate
> synchronously with web services. Usually web services are configured
> point-to-point. This creates quite some configuration challenge. And I’m not
> talking about micro services here.
> 
> To ease the problem, a kind of hub and spoke architecture was introduced.
> Apache httpd with mod_jk is put in front of some applications. This eases
> configuration to some degree. But it has its downsides. When you kill the
> hub, all lights go out. Been there, done that, not fun! Also when you add
> some servers or applications, you have to configure the central mod_jk. The
> later problem can be solved with mod_cluster. I love mod_cluster! You don’t
> have to configure it. Just add servers, deploy or undeploy applications on
> JBoss, no configuration on the central hub necessary.
> 
> Architects recommend a bus-architecture to overcome the downsides of a
> central hub. A service bus should have a decentral organisation similar to a
> hardware bus. Applications connect to the bus and should be able to exchange
> messages without further configuration. However, when I looked at products
> called “Enterprise Service Bus”, I had difficulties to see the bus, most
> look like hub and spoke architectures.
> 
> The Idea
> 
> But then the Undertow team announced the support for mod_cluster in Undertow,
> which got me really excited. Now it could be possible to create a service
> bus to connect WildFly servers. You just configure mod_cluster within
> WildFly and you send web service requests to your own server (localhost)
> instead of the real destination ( https://developer.jboss.org/thread/249311
> ). Because mod_cluster knows about the location of all other servers with
> the same multicast-address (the bus-adress), it sends the request to the
> final destination. A real service bus, almost configuration free and comes
> out of the box with WildFly. Is it only me or do others think as well that
> this could be a real burner?
> 
> The Proof of Concept
> 
> Yes, it does work! We created a setup with two nodes, which shows that the
> idea can work: https://github.com/AdrianFarmadin/modcluster-invmhttp-example
> Unfortunately we ran into a problem, which I was not able to solve properly
> until now.
> 
> The Problem
> 
> The problem is described in https://issues.jboss.org/browse/UNDERTOW-626 .
> When you have local context and modcluster filter, modcluster sends requests
> first to other servers and creates a loop.
> 
> The Solutions or Workarounds
> 
> Excluding certain contexts
> In UNDERTOW-626 Steward suggests:
> <filter-ref name="modcluster" predicate="not path-prefix(/myapp)”/>
> 
> Unfortunately i couldn’t test it until now, but I will soon. It could work
> for our special case, since we have only one application on one server and
> only a few contexts. Something like that would have to work:
> <filter-ref name="modcluster" predicate="not path-prefix(/myapp) and not
> path-prefix(/myappws) and not path-prefix(/myappadmin) ”/>
> Additionally we don’t deploy or undeploy applications at runtime, so we don’t
> have to configure this dynamically.
> 
> The solution will be problematic, if you e.g. undeploy one out of several
> applications on WildFly. Then you would have to reconfigure the predicate
> and remove some parts of the " and not path-prefix(/…)” This makes it rather
> difficult.
> 
> In modcluster filter: Don’t handle context, if its from the local server
> In https://github.com/undertow-io/undertow/pull/356/files we tried to change
> the mod cluster filter such that it doesn’t forward a request if it the
> context exists on the local server.
> We used this for our proof of concept for which it worked.
> The problem is, that is uses conventions from WildFly to answer the question
> “What is my local jvmroute”. The system-property “jboss.node.name” is the
> default in WildFly, but thats not necessarily true otherwise. But we
> couldn’t find how to access the configuration property instance-id within
> undertow.
> 
> Fix it in WildFly
> In https://github.com/undertow-io/undertow/pull/356 Steward pointed out that
> the problem should be solved in WildFly. Could you point us in a direction
> where we could go? We are willing to invest some time into this issue, but
> we don’t have enough inside knowledge from Undertow or WildFly jet.
> 
> Put modcluster at the end of the handler chain
> Right now modcluster is invoked before the deployed applications are invoked.
> Maybe one could do it the other way round and invoke modcluster only if the
> deployed applications return “not found”? I just don’t know enough about
> Undertow to decide whether this is feasible or just a stupid idea.
> 
> Other?
> 
> The End
> Any remarks, ideas, and help are welcome.
> 
> Erhard
> 
> _______________________________________________
> undertow-dev mailing list
> undertow-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/undertow-dev