Response from Brent Douglas: Hi Vladimir, I'm not sure this is the same thing being discussed however if it is not I had intended to request this anyhow. When I looked into using infinispans's map reduce facility this is the the task I came up with: http://pastebin.com/7GGjVnVt I would prefer to specify it as: http://pastebin.com/HTSq3g66 I'm pretty sure this is the not intended use case but it distributes the creation of my reports which is what I want. It's not really a big deal for me as I can get around this limitation by creating a wrapper class such as in the first example but it would be nice if I did not have to. Is this a reasonable request? Also, and this is probably just wrong, when I use this I bundle up the task and execute it via JMS. Would it be reasonable to make Collator extend Serializable? Brent On 12-01-30 11:47 AM, Vladimir Blagojevic wrote: > Guys, > > I was looking at this again recently and I still do not understand > how combiner could have different interface than Reducer! Hadoop > forces a user to implement combiner as a Reducer > http://developer.yahoo.com/hadoop/tutorial/module4.html#functionality > and > http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred... > In addition, the original paper does not mention any change of types. > > What we have admittedly done wrong is to apply Reducer on individual > Mapper without checking if a reduce function is both /commutative/ > and /associative/! This can lead to problems: > http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4... > > So yes, I am all for adding Combiner (it should do the optional > reducer per mapper we do automatically now) but I do not see why we > have to change the interface! > > > Regards, > Vladimir > > > > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev

LOL! This is what I got when accessing those pastebins… On Jan 31, 2012, at 6:46 PM, Vladimir Blagojevic wrote: > Response from Brent Douglas: > > Hi Vladimir, > > I'm not sure this is the same thing being discussed however if it is > not I had intended to request this anyhow. When I looked into using > infinispans's map reduce facility this is the the task I came up with: > > http://pastebin.com/7GGjVnVt > > I would prefer to specify it as: > > http://pastebin.com/HTSq3g66 > > I'm pretty sure this is the not intended use case but it distributes > the creation of my reports which is what I want. It's not really a big > deal for me as I can get around this limitation by creating a wrapper > class such as in the first example but it would be nice if I did not > have to. Is this a reasonable request? > > Also, and this is probably just wrong, when I use this I bundle up the > task and execute it via JMS. Would it be reasonable to make Collator > extend Serializable? > > Brent > > > On 12-01-30 11:47 AM, Vladimir Blagojevic wrote: >> Guys, >> >> I was looking at this again recently and I still do not understand >> how combiner could have different interface than Reducer! Hadoop >> forces a user to implement combiner as a Reducer >> http://developer.yahoo.com/hadoop/tutorial/module4.html#functionality >> and >> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred... >> In addition, the original paper does not mention any change of types. >> >> What we have admittedly done wrong is to apply Reducer on individual >> Mapper without checking if a reduce function is both commutative and >> associative! This can lead to problems: >> http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4... >> >> So yes, I am all for adding Combiner (it should do the optional >> reducer per mapper we do automatically now) but I do not see why we >> have to change the interface! >> >> >> Regards, >> Vladimir >> >> >> >> >> _______________________________________________ >> infinispan-dev mailing list >> >> infinispan-dev(a)lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/infinispan-dev > > _______________________________________________ > infinispan-dev mailing list > infinispan-dev(a)lists.jboss.org > https://lists.jboss.org/mailman/listinfo/infinispan-dev -- Galder Zamarreño Sr. Software Engineer Infinispan, JBoss Cache _______________________________________________ infinispan-dev mailing list infinispan-dev(a)lists.jboss.org https://lists.jboss.org/mailman/listinfo/infinispan-dev