MapReduce编程模型
MapReduce编程模型
map(String key, String value): //key: document name //value: document contents for each word w in value EmitIntermediate(w,"1"); reduce(String key, Iterator values): //key: a word //values: a list of counts int result = 0; for each v in values: result += ParseInt(v); Emit(AsString(result));
Counter* uppercase;
uppercase = getCounter("uppercase");
map(String name, String contents):
for each word w in contents:
if (IsCapitalized(w))
uppercase->Increment();
EmitIntermediate(w,"1");
计数器的值周期性的从各个单独的worker机器上传递给master。master把执行成功的Map和Reduce任务的计数器值进行累加,但MapReduce操作完成后,返回给用户代码。计数器对于MapReduce操作的完整性检查非常有用。
评论暂时关闭