[infinispan-dev] Remoting package refactor

[Radim Vansa]

F/J tasks should not acquire any locks (or, generally, block) during 
their execution. At least according to JavaDocs. Are we ready for that?

Btw., I really don't like the fact that the commonPool() cannot be 
properly shutdown. This leads to threadlocal variables leaking when the 
component using F/J pool is undeployed (the classloader cannot be GCed 
and you end up with OOME in PermGen space).

Radim

On 11/13/2014 08:28 AM, Galder Zamarreño wrote:
> @Pedro, did you consider using a ForkJoinPool instead?
>
> Traditional JDK pools are known to be very hard to configure and get it “right”. Fork join pools are being used as default thread pools in other libraries, vastly reducing configuration.
>
> Jessitron has published some interesting blog posts on the advantages of traditional ExecutorService vs Fork/Join pools and viceversa. See [1] and [3]. She also did a talk on it, see [4].
>
> Cheers,
>
> p.s. I’ve not studied your use case in depth to decide whether F/J would suite better, but it’s certainly worth a look now that we’re on Java 7.
>
> [1] https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ForkJoinPool.html
> [2] http://blog.jessitron.com/2014/01/choosing-executorservice.html
> [3] http://blog.jessitron.com/2014/02/scala-global-executioncontext-makes.html
> [4] https://www.youtube.com/watch?v=yhguOt863nw
>
> On 07 Nov 2014, at 09:31, Radim Vansa <rvansa at redhat.com> wrote:
>
>> Btw., have you ever considered checks if a thread returns to pool
>> reasonably often? Some of the other datagrids use this, though there's
>> not much how to react upon that beyond printing out stack traces (but
>> you can at least report to management that some node seems to be broken).
>>
>> Radim
>>
>> On 11/07/2014 08:35 AM, Bela Ban wrote:
>>> That's exactly what I suggested. No config gives you a shared global
>>> thread pool for all caches.
>>>
>>> Those caches which need a separate pool can do that via configuration
>>> (and of course also programmatically)
>>>
>>> On 06/11/14 20:31, Tristan Tarrant wrote:
>>>> My opinion is that we should aim for less configuration, i.e.
>>>> threadpools should mostly have sensible defaults and be shared by
>>>> default unless there are extremely good reasons for not doing so.
>>>>
>>>> Tristan
>>>>
>>>> On 06/11/14 19:40, Radim Vansa wrote:
>>>>> I second the opinion that any threadpools should be shared by default.
>>>>> There are users who have hundreds or thousands of caches and having
>>>>> separate threadpool for each of them could easily drain resources. And
>>>>> sharing resources is the purpose of threadpools, right?
>>>>>
>>>>> Radim
>>>>>
>>>>> On 11/06/2014 04:37 PM, Bela Ban wrote:
>>>>>> #1 I would by default have 1 thread pool shared by all caches
>>>>>> #2 This global thread pool should be configurable, perhaps in the
>>>>>> <global> section ?
>>>>>> #3 Each cache by default uses the gobal thread pool
>>>>>> #4 A cache can define its own thread pool, then it would use this one
>>>>>> and not the global thread pool
>>>>>>
>>>>>> I think this gives you a mixture between ease of use and flexibility in
>>>>>> configuring pool per cache if needed
>>>>>>
>>>>>> On 06/11/14 16:23, Pedro Ruivo wrote:
>>>>>>> On 11/06/2014 03:01 PM, Bela Ban wrote:
>>>>>>>> On 06/11/14 15:36, Pedro Ruivo wrote:
>>>>>>>>> * added a single thread remote executor service. This will handle the
>>>>>>>>> FIFO deliver commands. Previously, they were handled by JGroups incoming
>>>>>>>>> threads and with a new executor service, each cache can process their
>>>>>>>>> own FIFO commands concurrently.
>>>>>>>> +1000. This allows multiple updates from the same sender but to
>>>>>>>> different caches to be executed in parallel, and will speed thing up.
>>>>>>>>
>>>>>>>> Do you intend to share a thread pool between the invocations handlers of
>>>>>>>> the various caches, or do they each have their own thread pool ? Or is
>>>>>>>> this configurable ?
>>>>>>>>
>>>>>>> That is question that cross my mind and I don't have any idea what would
>>>>>>> be the best. So, for now, I will leave the thread pool shared between
>>>>>>> the handlers.
>>>>>>>
>>>>>>> Never thought to make it configurable, but maybe that is the best
>>>>>>> option. And maybe, it should be possible to have different max-thread
>>>>>>> size per cache. For example:
>>>>>>>
>>>>>>> * all caches using this remote executor will share the same instance
>>>>>>> <remote-executor name="shared" shared="true" max-threads=4.../>
>>>>>>>
>>>>>>> * all caches using this remote executor will create their own thread
>>>>>>> pool with max-threads equals to 1
>>>>>>> <remote-executor name="low-throughput-cache" shared="false"
>>>>>>> max-threads=1 .../>
>>>>>>>
>>>>>>> * all caches using this remote executor will create their own thread
>>>>>>> pool with max-threads equals to 1000
>>>>>>> <remote executor name="high-throughput-cache" shared="false"
>>>>>>> max-thread=1000 .../>
>>>>>>>
>>>>>>> is this what you have in mind? comments?
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Pedro
>>>>>>> _______________________________________________
>>>>>>> infinispan-dev mailing list
>>>>>>> infinispan-dev at lists.jboss.org
>>>>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>>>>
>>>> _______________________________________________
>>>> infinispan-dev mailing list
>>>> infinispan-dev at lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>>>
>>
>> -- 
>> Radim Vansa <rvansa at redhat.com>
>> JBoss DataGrid QA
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarreño
> galder at redhat.com
> twitter.com/galderz
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev


-- 
Radim Vansa <rvansa at redhat.com>
JBoss DataGrid QA