问题描述
我想访问reducer中的myCounter.my值：
public static class Map扩展Mapper< LongWritable，Text， ImmutableBytesWritable，ImmutableBytesWritable>
{
public static enum myCounter {my};
@Override
public void map（LongWritable key，Text value，Context context）
{
context.getCounter（myCounter.my）.increment（1）;
context.write（new ImmutableBytesWritable（），new ImmutableBytesWritable（））;
public static class Reduce extends Reducer< ImmutableBytesWritable，ImmutableBytesWritable，Text，Text>
{
@Override
public void reduce（ImmutableBytesWritable key，Iterable< ImmutableBytesWritable> result，Context context）
{
}
} b
$ b mappers-counter-from-a-reducer？rq = 1>从reducer访问mapper的计数器（给出旧API）
如何使它适用于新的API？
或
我想知道mapper输出的总数量吗？有没有更好的方法？ （我无法在Reducer中访问计数器：
组名称 - > org.apache.hadoop.mapred.Task $ Counter Counter Name - > MAP_OUTPUT_RECORDS ）
感谢
解决方案 配置conf = context.getConfiguration（ ）;
群集群集=新群集（conf）;
Job currentJob = cluster.getJob（context.getJobID（））;
long val = currentJob.getCounters（）。findCounter（myCounter.my）.getValue（）;
I want to acces the myCounter.my value in reducer :
public static class Map extends Mapper<LongWritable, Text, ImmutableBytesWritable, ImmutableBytesWritable>
{
public static enum myCounter{my};
@Override
public void map(LongWritable key, Text value, Context context)
{
context.getCounter(myCounter.my).increment(1);
context.write( new ImmutableBytesWritable ( ),new ImmutableBytesWritable() );
}
}
public static class Reduce extends Reducer<ImmutableBytesWritable, ImmutableBytesWritable, Text, Text>
{
@Override
public void reduce(ImmutableBytesWritable key,Iterable<ImmutableBytesWritable> result,Context context)
{
}
}
Accessing a mapper's counter from a reducer(for old API is given )how to make it work for new API ?
Or
I want to know the total number of mapper output ? Is there any better way ? (i am not able to access counter in Reducer:
Group Name->org.apache.hadoop.mapred.Task$Counter Counter Name->MAP_OUTPUT_RECORDS)
Thanks
解决方案 You to make it work for new API by accessing the counters via job object.
Configuration conf = context.getConfiguration();
Cluster cluster = new Cluster(conf);
Job currentJob = cluster.getJob(context.getJobID());
long val=currentJob.getCounters().findCounter(myCounter.my).getValue();
这篇关于如何在Reducer中访问映射计数器值？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！