我有两个映射器类。因此,使用ChainMapper.addMapper
方法添加Mapper,并使用ChainReducer.setReducer
方法设置Reducer。 ChainMapper.addMapper
方法可以,但是Chain.setReducer
方法抛出语法错误
The method setReducer(Job, Class<? extends Reducer>, Class<?>, Class<?>, Class<?>, Class<?>, Configuration) in the type ChainReducer is not applicable for the arguments (JobConf, Class<FileComparisionReduce>, Class<LongWritable>, Class<Text>, Class<LongWritable>, Class<Text>, boolean, JobConf)
这是我的驱动程序类:
package fileComparision;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.lib.ChainMapper;
import org.apache.hadoop.mapreduce.lib.chain.ChainReducer;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
public class DriverComparision extends Configured implements Tool{
@Override
public int run(String[] arg0) throws Exception {
JobConf conf = new JobConf(true);
conf.setJobName("Comaprision of 2 file ");
JobConf Mapper1 = new JobConf(false);
ChainMapper.addMapper(conf, FileComparisionMapper1.class, LongWritable.class, Text.class, LongWritable.class, Text.class, true, Mapper1);
JobConf Mapper2 = new JobConf(false);
ChainMapper.addMapper(conf, FileComparisionMapper2.class, LongWritable.class, Text.class, LongWritable.class, Text.class, true, Mapper2);
JobConf Reduc = new JobConf(false);
ChainReducer.setReducer(conf, FileComparisionReduce.class, LongWritable.class, Text.class, LongWritable.class, Text.class, true , Reduc);
FileInputFormat.setInputPaths(conf, new Path(arg0[0]));
FileOutputFormat.setOutputPath(conf, new Path(arg0[1]));
conf.setMapOutputKeyClass(LongWritable.class);
conf.setMapOutputValueClass(Text.class);
conf.setOutputKeyClass(LongWritable.class);
conf.setOutputValueClass(Text.class);
JobClient.runJob(conf);
return 0;
}
也尝试通过删除 bool(boolean) 参数“true”
JobConf Reduc = new JobConf(false);
ChainReducer.setReducer(conf, FileComparisionReduce.class, LongWritable.class, Text.class, LongWritable.class, Text.class, true , Reduc);
最佳答案
终于,我找到了解决方案。
导入错误的软件包,即。 import org.apache.hadoop.mapreduce.lib.chain.ChainReducer
;代替import org.apache.hadoop.mapred.lib.ChainReducer;
关于java - ChainReducer.setReducer方法抛出错误 “ChainReducer is not applicable for the arguments”,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/36350193/