我正在尝试学习MapReduce并执行此任务。
我的输入如下(州,体育,金额(美元)):California Football 69.09California Swimming 31.5Illinois Golf 8.31Illinois Tennis 15.75Oklahoma Golf 15.44Oklahoma Tennis 8.33Texas Golf 16.71Texas Swimming 71.59Washington Football 50.32000000000001
而且,我希望输出的内容能够显示出哪个运动在特定州受欢迎(取决于运动物品的最高销量)。例如:California Football 69.09Illinois Tennis 15.75Oklahoma Golf 15.44
等等
以下是我的Mapper,Reducer和Driver代码:
映射器代码:
package org.assignment.sports;
import java.io.IOException;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
public class Sports_Mapper2 extends Mapper<LongWritable, Text, Text, Text>{
public void map(LongWritable key, Text value, Context context)throws IOException, InterruptedException{
String[] s= value.toString().split(" ");
String Sport_State = s[0];
String other = s[1]+" "+s[2];
context.write(new Text(Sport_State), new Text(other));
}
}
reducer 代码:
package org.assignment.sports;
import java.io.IOException;
import org.apache.hadoop.io.DoubleWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;
public class Sports_Reducer2 extends Reducer<Text, Text, Text, DoubleWritable>{
private static double MAX=0.00;
public void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException
{
//String[] k= values.toString().split(" ");
for (Text value:values){
String[] k= value.toString().split(" ");
DoubleWritable price = new DoubleWritable(Double.parseDouble(k[1]));
if(price.get()>MAX){
MAX = price.get();
}
else{
continue;
}
String ss = key.toString()+" "+ k[0];
context.write(new Text(ss), new DoubleWritable(MAX));
}
}
}
驱动程式码:
package org.assignment.sports;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.DoubleWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;
public class Sports_Driver2 {
public static void main(String[] args) throws Exception
{
Configuration conf = new Configuration();
Job job = new Job(conf, "Sports_Driver2");
String[] otherArgs =new GenericOptionsParser(conf, args).getRemainingArgs();
job.setJarByClass(Sports_Driver2.class);
job.setMapperClass(Sports_Mapper2.class);
job.setReducerClass(Sports_Reducer2.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(Text.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(DoubleWritable.class);
FileInputFormat.addInputPath(job, new Path(otherArgs[0]));
FileOutputFormat.setOutputPath(job,new Path(otherArgs[1]));
System.exit(job.waitForCompletion(true)? 0: 1);
}
}
我得到的输出如下:
California Football 69.09Texas Swimming 71.59
我要去哪里错了?任何帮助表示赞赏
最佳答案
问题是写入每个特定状态后,Reducer中的MAX值未重置。
String ss = key.toString()+" "+ k[0];
context.write(new Text(ss), new DoubleWritable(MAX));
MAX = 0.00;