TCP_NIO2: single selector slows down writes and reads ----------------------------------------------------- Key: JGRP-1999 URL: https://issues.jboss.org/browse/JGRP-1999 Project: JGroups Issue Type: Enhancement Reporter: Bela Ban Assignee: Bela Ban Fix For: 3.6.7 TCP_NIO2 has a single selector which looks like this: {noformat} x=select() if(read) -> read data from connection x, de-serialize data, pass to thread pool if(write) -> write data pending on connection x {noformat} This means that any operation (read,write) delays other operations. It seems that especially the de-serialization done in reads delays other reads and writes. A quick test showed that having a reader thread per connection (so that reads don't delay other reads or writes) improved perf from 15'000 reqs/sec to 22'000. The idea is to have a reader thread in {{NioConnection}} which reads and de-serializes as many messages as possible. When no more messages are ready to be read, it blocks for a max wait time and then terminates unless more messages are ready. This means that idle connections will have no threads allocated. Investigate: we might possibly also null the pre-allocated buffer when a thread terminates, reducing memory usage even more.