Re: [undertow-dev] Websocket connections and IO thread affinity

The way our current approach works, which is the same approach as SO_REUSEPORT’s impl is that address:port are hashed to select the destination, this is mainly so we can transition with no real behavioral surprises. If you have some connections lasting significantly longer than others, then you will eventually go out of balance because the current allocation state isn’t a factor into the decision. It’s possible to do more advanced algorithms factoring in state, but once you do that you tie yourself to a single threaded acceptor (although thats currently the case with our emulated SO_REUSEPORT implementation). For many workloads this won’t matter though, as you need massive connection rates to hit the accept stability limits. Maybe you want to play with modifying QueuedTcpNioServer to compare a few different algorithms? You could try balancing active connection count as one strategy, and perhaps thread user cpu time as another. For both approaches you probably want to have i/o threads individually updating a volatile statistic field as part of their standard work, and then the accept queuing thread scanning those values to select the best destination. > On Jun 16, 2016, at 2:01 PM, peter royal <peter.royal(a)pobox.com&gt; wrote: > > Gotcha. I was digging through things and found the change where the new > strategy was introduced. With my current # of IO threads it is giving > un-even weighings: > > thread, connections > 0, 6 > 1, 5 > 2, 3 > 3, 2 > > I'm going to double my IO threads, but it will still be less than > optimal, but improved: > > thread, connections > 0, 2 > 1, 1 > 2, 1 > 3, 1 > 4, 4 > 5, 4 > 6, 2 > 7, 1 > > Random is only slightly better, eyeballing things. > > I'm using Undertow 1.3.22 which uses XNIO 3.3.6. Linux kernel 2.6.32 > though. > > Digging into my problem more, I would probably need to balance on more > than just connection count per IO thread, as some connections are busier > than others. > > Can you point me towards any references about the forthcoming access to > native facility? I'm curious as to how that will work > > -pete > > -- > (peter.royal|osi)(a)pobox.com - http://fotap.org/~osi > > On Thu, Jun 16, 2016, at 01:41 PM, Jason T. Greene wrote: >> We recently changed xnio to balance connections by default using a >> strategy similar to the new SO_REUSEPORT facility in the Linux kernel >> (3.3.3 or later). In the short future, we will be switching to the >> native facility when accessible in the JDK NIO implementation. Older >> versions had a feature called balancing tokens that you could use to >> balance connections fairly, but it had to be especially configured. >> >> >>> On Jun 16, 2016, at 1:00 PM, peter royal <peter.royal(a)pobox.com&gt; wrote: >>> >>> (I believe the following is true... please correct me if not!) >>> >>> I have an application which heavily utilizes web sockets. It is an >>> internal application which uses a small number of connections with >>> reasonable load on each. >>> >>> When a new connection is received by Undertow, there is an >>> at-connection-time assignment of an XNIO IO Thread to the connection. >>> This is causing uneven load on my IO threads, due to chance. >>> >>> I'm increasing the number of IO threads as a temporary fix, but it might >>> be useful to be able to either migrate a long-lived connection to >>> another IO thread (harder) or do better load balancing amongst IO >>> threads. For the latter, if Undertow was able to provide a strategy for >>> picking a thread in NioXnioWorker.getIoThread(hashCode), it could try >>> and pick a thread that had fewer connections assigned to it. >>> >>> Has anyone else run into this problem? Would a fix be accepted? >>> >>> -pete >>> >>> -- >>> (peter.royal|osi)(a)pobox.com - http://fotap.org/~osi >>> _______________________________________________ >>> undertow-dev mailing list >>> undertow-dev(a)lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/undertow-dev -- Jason T. Greene WildFly Lead / JBoss EAP Platform Architect JBoss, a division of Red Hat