MapReduce编程模型


map(String key, String value):
	//key: document name
	//value: document contents
	for each word w in value
		EmitIntermediate(w,"1");

reduce(String key, Iterator values):
	//key: a word
	//values: a list of counts
	int result = 0;
	for each v in values:
		result += ParseInt(v);
	Emit(AsString(result));


Counter* uppercase;
uppercase = getCounter("uppercase");

map(String name, String contents):
	for each word w in contents:
		if (IsCapitalized(w))
			uppercase->Increment();
		EmitIntermediate(w,"1");

计数器的值周期性的从各个单独的worker机器上传递给master。master把执行成功的Map和Reduce任务的计数器值进行累加,但MapReduce操作完成后,返回给用户代码。计数器对于MapReduce操作的完整性检查非常有用。

相关内容