本文介绍了有没有所有标准(非自定义)Hadoop mapreduce计数器的列表?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
是否有可用的计数器列表可以从 [RunningJob] [1]
中使用方法 getCouters()
?
Is there list of available counters that can be retrieved from [RunningJob][1]
using method getCouters()
?
第二个问题:有没有计数器告诉我这个工作要执行多长时间?
Second question: Is there any counter that tells me how long did it take for the job to execute?
推荐答案
计数器列表
List of Counters
Source
Group Name Counter Name
org.apache.hadoop.mapred.Task$Counter MAP_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter MAP_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter MAP_SKIPPED_RECORDS
org.apache.hadoop.mapred.Task$Counter MAP_INPUT_BYTES
org.apache.hadoop.mapred.Task$Counter MAP_OUTPUT_BYTES
org.apache.hadoop.mapred.Task$Counter COMBINE_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter COMBINE_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter REDUCE_INPUT_GROUPS
org.apache.hadoop.mapred.Task$Counter REDUCE_SHUFFLE_BYTES
org.apache.hadoop.mapred.Task$Counter REDUCE_INPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter REDUCE_OUTPUT_RECORDS
org.apache.hadoop.mapred.Task$Counter REDUCE_SKIPPED_GROUPS
org.apache.hadoop.mapred.Task$Counter REDUCE_SKIPPED_RECORDS
org.apache.hadoop.mapred.JobInProgress$Counter TOTAL_LAUNCHED_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter RACK_LOCAL_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter DATA_LOCAL_MAPS
org.apache.hadoop.mapred.JobInProgress$Counter TOTAL_LAUNCHED_REDUCES
FileSystemCounters FILE_BYTES_READ
FileSystemCounters HDFS_BYTES_READ
FileSystemCounters FILE_BYTES_WRITTEN
FileSystemCounters HDFS_BYTES_WRITTEN
如何通过API获取计数器?:
How to get counter By API ?:
Ans
新API: context.getCounter (groupName,counterName);
旧API: Reporter.getCounter(groupName,counterName);
Qus:是否任何一个告诉我工作执行多长时间的计数器?
答案:是的,你可以看到你的JobTracker /终端
Ans : Yes you can see that info in your Jobtracker/Terminal
Submit Host Address: xx.xx.xxx.xxx
Job-ACLs: All users are allowed
Job Setup: Successful
Status: Succeeded
Started at: Wed Nov 28 16:29:06 IST 2012
Finished at: Wed Nov 28 16:29:53 IST 2012
Finished in: 47sec
Job Cleanup: Successful
这篇关于有没有所有标准(非自定义)Hadoop mapreduce计数器的列表?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!