<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Guys,<br>
<br>
I was looking at this again recently and I still do not understand
how combiner could have different interface than Reducer! Hadoop
forces a user to implement combiner as a Reducer <a
class="moz-txt-link-freetext"
href="http://developer.yahoo.com/hadoop/tutorial/module4.html#functionality">http://developer.yahoo.com/hadoop/tutorial/module4.html#functionality</a>
and
<a class="moz-txt-link-freetext" href="http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setCombinerClass%28java.lang.Class%29">http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#setCombinerClass%28java.lang.Class%29</a>
In addition, the original paper does not mention any change of
types.<br>
<br>
What we have admittedly done wrong is to apply Reducer on individual
Mapper
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
without checking if a reduce function is both <i>commutative</i>
and <i>associative</i>! This can lead to problems:
<a class="moz-txt-link-freetext" href="http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4-to-use-or-not-to-use-a-combiner/">http://philippeadjiman.com/blog/2010/01/14/hadoop-tutorial-series-issue-4-to-use-or-not-to-use-a-combiner/</a><br>
<br>
So yes, I am all for adding Combiner (it should do the optional
reducer per mapper we do automatically now) but I do not see why we
have to change the interface!<br>
<br>
<br>
Regards,<br>
Vladimir<br>
<br>
<br>
</body>
</html>