我正在将我的一些MapReduce代码从Hadoop 1.0迁移到Hadoop 2.0。我从简单的WordCount工作开始。我在构建路径中添加了以下jar，并删除了相应的Hadoop 1.0 jar。
我保留了带有以下客户端定义的简单WordCount类。
JobConf conf = new JobConf(new Configuration(), WordCount.class);
conf.setJobName("WordCount");
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(WordCountMap.class);
//conf.setCombinerClass(Reduce.class);
conf.setReducerClass(WordCountReduce.class);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf, new Path(params.getInput()));
FileOutputFormat.setOutputPath(conf, new Path(params.getOutput()));
JobClient.runJob(conf);
实例化JobConf时出现以下错误。
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2957)
at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1210)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1690)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
at bigdata.hadoop.WordCount.execute(WordCount.java:55)
at bigdata.hadoop.WordCount.execute(WordCount.java:1)
at bigdata.hadoop.BigDataJobDriver.executeJobDriver(BigDataJobDriver.java:15)
at bigdata.jobs.WordCountJob.executeJob(WordCountJob.java:50)
at bigdata.quartz.BigDataJob.execute(BigDataJob.java:30)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
... 14 more
2016-01-23 03:03:35,408 ERROR [QuartzScheduler_Worker-1] core.ErrorLogger (QuartzScheduler.java:schedulerError(2361)) - Job (Hadoop.Hadoop 7 threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration]
at org.quartz.core.JobRunShell.run(JobRunShell.java:224)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:2957)
at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:1210)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1690)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
at bigdata.hadoop.WordCount.execute(WordCount.java:55)
at bigdata.hadoop.WordCount.execute(WordCount.java:1)
at bigdata.hadoop.BigDataJobDriver.executeJobDriver(BigDataJobDriver.java:15)
at bigdata.jobs.WordCountJob.executeJob(WordCountJob.java:50)
at bigdata.quartz.BigDataJob.execute(BigDataJob.java:30)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
... 1 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
... 14 more
我执行了以下命令，但该命令找不到Configuration.class。 hadoop 1.0上的相同命令显示Configuration.class。
tar tvf hadoop-mapreduce-client-core-2.7.1.jar | grep Configuration.class
Configuration.class是否已移至Hadoop 2.7.1的其他jar文件？
最佳答案
通过配置对象进行检查，并将其传递给Job对象。
喜欢:
配置conf = new Configuration();
Job job = new Job(conf，“WordCount”);
关于hadoop - 在hadoop 2.0中，configuration.class定义是否已移动？，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/34961565/