[infinispan-dev] Adding a Combiner

[Radoslav Husar]

Damn good idea, cats as cheap labor force :-)

On 02/01/2012 10:30 AM, Galder Zamarreño wrote:
> LOL! This is what I got when accessing those pastebins…
>
>
> On Jan 31, 2012, at 6:46 PM, Vladimir Blagojevic wrote:
>
>> Response from Brent Douglas:
>>
>> Hi Vladimir,
>>
>> I'm not sure this is the same thing being discussed however if it is
>> not I had intended to request this anyhow. When I looked into using
>> infinispans's map reduce facility this is the the task I came up with:
>>
>> http://pastebin.com/7GGjVnVt
>>
>> I would prefer to specify it as:
>>
>> http://pastebin.com/HTSq3g66
>>
>> I'm pretty sure this is the not intended use case but it distributes
>> the creation of my reports which is what I want. It's not really a big
>> deal for me as I can get around this limitation by creating a wrapper
>> class such as in the first example but it would be nice if I did not
>> have to. Is this a reasonable request?
>>
>> Also, and this is probably just wrong, when I use this I bundle up the
>> task and execute it via JMS. Would it be reasonable to make Collator
>> extend Serializable?
>>
>> Brent
>>
>>
>> On 12-01-30 11:47 AM, Vladimir Blagojevic wrote:
>>> Guys,
>>>
>>> I was looking at this again recently and I still do not understand
>>> how combiner could have different interface than Reducer! Hadoop
>>> forces a user to implement combiner as a Reducer
>>> http://developer.yahoo.com/hadoop/tutorial/module4.html#functionality
>>> and
>>> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setCombinerClass%28java.lang.Class%29
>>> In addition, the original paper does not mention any change of types.
>>>
>>> What we have admittedly done wrong is to apply Reducer on individual
>>> Mapper without checking if a reduce function is both commutative and
>>> associative! This can lead to problems:
>>> http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4-to-use-or-not-to-use-a-combiner/
>>>
>>> So yes, I am all for adding Combiner (it should do the optional
>>> reducer per mapper we do automatically now) but I do not see why we
>>> have to change the interface!
>>>
>>>
>>> Regards,
>>> Vladimir
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> infinispan-dev mailing list
>>>
>>> infinispan-dev at lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>>
>> _______________________________________________
>> infinispan-dev mailing list
>> infinispan-dev at lists.jboss.org
>> https://lists.jboss.org/mailman/listinfo/infinispan-dev
>
> --
> Galder Zamarreño
> Sr. Software Engineer
> Infinispan, JBoss Cache
>
>
>
> _______________________________________________
> infinispan-dev mailing list
> infinispan-dev at lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/infinispan-dev