我正在尝试在Hadoop单节点集群上运行排序示例。首先,我开始进行重载:hadoop@ubuntu:/home/user/hadoop$ bin/start-all.sh
然后,我运行随机编写器示例以将顺序文件生成为输入文件。hadoop@ubuntu:/home/user/hadoop$ bin/hadoop jar hadoop-*-examples.jar randomwriter rand
hadoop @ ubuntu:/ home / user / hadoop $ bin / hadoop jar hadoop-*-examples.jar randomwriter rand
正在运行0张 map 。
Job started: Thu Mar 31 18:21:51 EEST 2011
11/03/31 18:21:52 INFO mapred.JobClient: Running job: job_201103311816_0001
11/03/31 18:21:53 INFO mapred.JobClient: map 0% reduce 0%
11/03/31 18:22:01 INFO mapred.JobClient: Job complete: job_201103311816_0001
11/03/31 18:22:01 INFO mapred.JobClient: Counters: 0
Job ended: Thu Mar 31 18:22:01 EEST 2011
这项工作花了9秒钟。
hadoop@ubuntu:/home/user/hadoop$ bin/hadoop jar hadoop-*-examples.jar sort rand rand-sort
在1个节点上运行以将
hdfs://localhost:54310/user/hadoop/rand
排序为带1的
hdfs://localhost:54310/user/hadoop/rand-sort
减少。Job started: Thu Mar 31 18:25:19 EEST 2011
11/03/31 18:25:20 INFO mapred.FileInputFormat: Total input paths to process : 0
11/03/31 18:25:20 INFO mapred.JobClient: Running job: job_201103311816_0002
11/03/31 18:25:21 INFO mapred.JobClient: map 0% reduce 0%
11/03/31 18:25:32 INFO mapred.JobClient: map 0% reduce 100%
11/03/31 18:25:34 INFO mapred.JobClient: Job complete: job_201103311816_0002
11/03/31 18:25:34 INFO mapred.JobClient: Counters: 9
11/03/31 18:25:34 INFO mapred.JobClient: Job Counters
11/03/31 18:25:34 INFO mapred.JobClient: Launched reduce tasks=1
11/03/31 18:25:34 INFO mapred.JobClient: FileSystemCounters
11/03/31 18:25:34 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=96
11/03/31 18:25:34 INFO mapred.JobClient: Map-Reduce Framework
11/03/31 18:25:34 INFO mapred.JobClient: Reduce input groups=0
11/03/31 18:25:34 INFO mapred.JobClient: Combine output records=0
11/03/31 18:25:34 INFO mapred.JobClient: Reduce shuffle bytes=0
11/03/31 18:25:34 INFO mapred.JobClient: Reduce output records=0
11/03/31 18:25:34 INFO mapred.JobClient: Spilled Records=0
11/03/31 18:25:34 INFO mapred.JobClient: Combine input records=0
11/03/31 18:25:34 INFO mapred.JobClient: Reduce input records=0
Job ended: Thu Mar 31 18:25:34 EEST 2011
这项工作花了14秒。
hadoop@ubuntu:/home/user/hadoop$ bin/hadoop dfs -cat rand-sort/part-00000
SEQ#“org.apache.hadoop.io.BytesWritable” org.apache.hadoop.io.BytesWritablej“mY。#
我是Hadoop的新手。我所做的一切是否正确,还是做错了什么?我的问题是,我怎么能看到randomwritewr生成的数据和sort示例的结果是正确的?从哪里可以看到它们?
最佳答案
问题在于,您尝试运行作业时尚未启动tasktracker,它也不会立即启动。您可以运行bin / hadoop job -list-active-trackers来查看tasktracker是否启动,完成启动可能需要一些时间。 No tasktracker =没有节点可将编写器映射到。
关于sorting - Hadoop在单节点集群上运行排序示例,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/5554152/