我正在尝试使用Maven构建Hadoop作业。当我不使用Maven并将Hadoop Jar依赖项直接导入eclipse时,此工作效果很好。
我还可以使用Maven构建一个简单的jar(hello world类型),而无需Hadoop依赖。
但是,当我添加Hadoop依赖项并运行jar时,出现以下错误:
dataException in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
at org.graphhadoop.CreateAdjacency.Adjacency(CreateAdjacency.java:74)
at org.graphhadoop.Driver.main(Driver.java:12)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 2 more
pom文件更改如下:
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.203.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>org.graphhadoop.Driver</mainClass>
</manifest>
</archive>
<manifestEntries>
<Class-Path>../target/classes</Class-Path>
</manifestEntries>
</configuration>
</plugin>
</plugins>
</build>
我要去哪里错了?
编辑:
现在,我在本地运行Jar,而不是在Hadoop上,而是在本地数据上。
java -jar jarname.jar
最佳答案
就像有人遇到相同问题的解决方案一样,以下是解决方案:
这些依赖项需要添加到pom.xml中:
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
<version>1.5.2</version>
</dependency>
<dependency>
<groupId>net.sf.json-lib</groupId>
<artifactId>json-lib</artifactId>
<version>2.3</version>
<classifier>jdk15</classifier>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-core</artifactId>
<version>0.20.203.0</version>
</dependency>
</dependencies>
这些构建插件:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>attached</goal>
</goals>
<phase>package</phase>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>org.lll..</mainClass>
</manifest>
<manifestEntries>
<Class-Path>../target/classes</Class-Path>
</manifestEntries>
</archive>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<mainClass>org...</mainClass>
</manifest>
</archive>
<manifestEntries>
<Class-Path>../target/classes</Class-Path>
</manifestEntries>
</configuration>
</plugin>
</plugins>
</build>