本文介绍了在 reudcer 类中使用全局变量的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我需要在我的mapreduce程序中使用全局变量,如何在下面的代码中设置它并在reducer中使用全局变量.
I need to use global variable in my mapreduce program how to set it in following code and use global variable in reducer.
public class tfidf
{
public static tfidfMap..............
{
}
public static tfidfReduce.............
{
}
public static void main(String args[])
{
Configuration conf=new Configuration();
conf.set("","");
}
}
推荐答案
模板代码可能看起来像这样(Reducer 未显示,但原理相同)
Template code could look something like this (Reducer not shown but is the same principal)
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Mapper.Context;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;
public class ToolExample extends Configured implements Tool {
@Override
public int run(String[] args) throws Exception {
Job job = new Job(getConf());
Configuration conf = job.getConfiguration();
conf.set("strProp", "value");
conf.setInt("intProp", 123);
conf.setBoolean("boolProp", true);
// rest of your config here
// ..
return job.waitForCompletion(true) ? 0 : 1;
}
public static class MyMapper extends
Mapper<LongWritable, Text, LongWritable, Text> {
private String strProp;
private int intProp;
private boolean boolProp;
@Override
protected void setup(Context context) throws IOException,
InterruptedException {
Configuration conf = context.getConfiguration();
strProp = conf.get("strProp");
intProp = conf.getInt("intProp", -1);
boolProp = conf.getBoolean("boolProp", false);
}
}
public static void main(String args[]) throws Exception {
System.exit(ToolRunner.run(new ToolExample(), args));
}
}
这篇关于在 reudcer 类中使用全局变量的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!