[undertow-dev] Websocket connections and IO thread affinity

[peter royal]

Understood. 

I'm going to test with increased IO threads, and if that fixes things
I'm good. Using thread user CPU time might be a good metric, as looking
at that the imbalance is clear:

CPU: 2673514 ms
CPU: 31270 ms
CPU: 61962 ms
CPU: 7952561 ms

As I think through this more, optimal balancing requires pushing a lot
of application-specific info down low, because a given WS connection
might be high volume or not. it would be easier to migrate a connection
that is detected to be high volume to another IO thread, but that'd be a
hugely invasive change. The optimal strategy for me might just be to
have 1 thread per connection as the counts aren't very high.

Thanks for the help!

-- 
(peter.royal|osi)@pobox.com - http://fotap.org/~osi

On Thu, Jun 16, 2016, at 02:17 PM, Jason Greene wrote:
> The way our current approach works, which is the same approach as
> SO_REUSEPORT’s impl is that address:port are hashed to select the
> destination, this is mainly so we can transition with no real behavioral
> surprises. If you have some connections lasting significantly longer than
> others, then you will eventually go out of balance because the current
> allocation state isn’t a factor into the decision. It’s possible to do
> more advanced algorithms factoring in state, but once you do that you tie
> yourself to a single threaded acceptor (although thats currently the case
> with our emulated SO_REUSEPORT implementation). For many workloads this
> won’t matter though, as you need massive connection rates to hit the
> accept stability limits.
> 
> Maybe you want to play with modifying QueuedTcpNioServer to compare a few
> different algorithms? You could try balancing active connection count as
> one strategy, and perhaps thread user cpu time as another. For both
> approaches you probably want to have i/o threads individually updating a
> volatile statistic field as part of their standard work, and then the
> accept queuing thread scanning those values to select the best
> destination.
> 
> > On Jun 16, 2016, at 2:01 PM, peter royal <peter.royal at pobox.com> wrote:
> > 
> > Gotcha. I was digging through things and found the change where the new
> > strategy was introduced. With my current # of IO threads it is giving
> > un-even weighings:
> > 
> > thread, connections
> > 0, 6
> > 1, 5
> > 2, 3
> > 3, 2
> > 
> > I'm going to double my IO threads, but it will still be less than
> > optimal, but improved:
> > 
> > thread, connections
> > 0, 2
> > 1, 1
> > 2, 1
> > 3, 1
> > 4, 4
> > 5, 4
> > 6, 2
> > 7, 1
> > 
> > Random is only slightly better, eyeballing things.
> > 
> > I'm using Undertow 1.3.22 which uses XNIO 3.3.6.  Linux kernel 2.6.32
> > though.
> > 
> > Digging into my problem more, I would probably need to balance on more
> > than just connection count per IO thread, as some connections are busier
> > than others. 
> > 
> > Can you point me towards any references about the forthcoming access to
> > native facility? I'm curious as to how that will work
> > 
> > -pete
> > 
> > -- 
> > (peter.royal|osi)@pobox.com - http://fotap.org/~osi
> > 
> > On Thu, Jun 16, 2016, at 01:41 PM, Jason T. Greene wrote:
> >> We recently changed xnio to balance connections by default using a
> >> strategy similar to the new SO_REUSEPORT facility in the Linux kernel
> >> (3.3.3 or later). In the short future,  we will be switching to the
> >> native facility when accessible in the JDK NIO implementation. Older
> >> versions had a feature called balancing tokens that you could use to
> >> balance connections fairly, but it had to be especially configured.  
> >> 
> >> 
> >>> On Jun 16, 2016, at 1:00 PM, peter royal <peter.royal at pobox.com> wrote:
> >>> 
> >>> (I believe the following is true... please correct me if not!)
> >>> 
> >>> I have an application which heavily utilizes web sockets. It is an
> >>> internal application which uses a small number of connections with
> >>> reasonable load on each.
> >>> 
> >>> When a new connection is received by Undertow, there is an
> >>> at-connection-time assignment of an XNIO IO Thread to the connection.
> >>> This is causing uneven load on my IO threads, due to chance. 
> >>> 
> >>> I'm increasing the number of IO threads as a temporary fix, but it might
> >>> be useful to be able to either migrate a long-lived connection to
> >>> another IO thread (harder) or do better load balancing amongst IO
> >>> threads. For the latter, if Undertow was able to provide a strategy for
> >>> picking a thread in NioXnioWorker.getIoThread(hashCode), it could try
> >>> and pick a thread that had fewer connections assigned to it.
> >>> 
> >>> Has anyone else run into this problem? Would a fix be accepted?
> >>> 
> >>> -pete
> >>> 
> >>> -- 
> >>> (peter.royal|osi)@pobox.com - http://fotap.org/~osi
> >>> _______________________________________________
> >>> undertow-dev mailing list
> >>> undertow-dev at lists.jboss.org
> >>> https://lists.jboss.org/mailman/listinfo/undertow-dev
> 
> --
> Jason T. Greene
> WildFly Lead / JBoss EAP Platform Architect
> JBoss, a division of Red Hat
>