本文介绍了有没有所有标准(非自定义)Hadoop mapreduce计数器的列表?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

是否有可用的计数器列表可以从 [RunningJob] [1] 中使用方法 getCouters()

Is there list of available counters that can be retrieved from [RunningJob][1] using method getCouters()?

第二个问题:有没有计数器告诉我这个工作要执行多长时间?

Second question: Is there any counter that tells me how long did it take for the job to execute?

推荐答案

计数器列表








List of Counters


Source

Group Name                                  Counter Name







org.apache.hadoop.mapred.Task$Counter       MAP_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       MAP_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       MAP_SKIPPED_RECORDS
org.apache.hadoop.mapred.Task$Counter       MAP_INPUT_BYTES
org.apache.hadoop.mapred.Task$Counter       MAP_OUTPUT_BYTES
org.apache.hadoop.mapred.Task$Counter       COMBINE_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       COMBINE_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       REDUCE_INPUT_GROUPS
org.apache.hadoop.mapred.Task$Counter       REDUCE_SHUFFLE_BYTES
org.apache.hadoop.mapred.Task$Counter       REDUCE_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       REDUCE_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter       REDUCE_SKIPPED_GROUPS
org.apache.hadoop.mapred.Task$Counter       REDUCE_SKIPPED_RECORDS
org.apache.hadoop.mapred.JobInProgress$Counter  TOTAL_LAUNCHED_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter  RACK_LOCAL_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter  DATA_LOCAL_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter  TOTAL_LAUNCHED_REDUCES
FileSystemCounters                                  FILE_BYTES_READ
FileSystemCounters                                  HDFS_BYTES_READ
FileSystemCounters                                  FILE_BYTES_WRITTEN
FileSystemCounters                                  HDFS_BYTES_WRITTEN









如何通过API获取计数器?:



How to get counter By API ?:

Ans

新API: context.getCounter (groupName,counterName);

旧API: Reporter.getCounter(groupName,counterName);

Qus:是否任何一个告诉我工作执行多长时间的计数器?

答案:是的,你可以看到你的JobTracker /终端

Ans : Yes you can see that info in your Jobtracker/Terminal

Submit Host Address: xx.xx.xxx.xxx
Job-ACLs: All users are allowed
Job Setup: Successful
Status: Succeeded
Started at: Wed Nov 28 16:29:06 IST 2012
Finished at: Wed Nov 28 16:29:53 IST 2012
Finished in: 47sec
Job Cleanup: Successful

这篇关于有没有所有标准(非自定义)Hadoop mapreduce计数器的列表?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-24 03:21