1. Combiner概述
2. 自定义Combiner实现步骤
1). 定义一个Combiner继承Reducer,重写reduce方法
public class WordcountCombiner extends Reducer<Text, IntWritable, Text,IntWritable>{ @Override
protected void reduce(Text key, Iterable<IntWritable> values,Context context) throws IOException, InterruptedException { // 1 汇总操作
int count = 0;
for(IntWritable v :values){
count += v.get();
} // 2 写出
context.write(key, new IntWritable(count));
}
}
2). 在Driver类中添加设置
job.setCombinerClass(WordcountCombiner.class);
效果