我试图从simple-yarn-app运行简单的yarn应用程序。但是我的应用程序错误日志中出现以下异常。

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/conf/YarnConfiguration
    at java.lang.Class.getDeclaredMethods0(Native Method)
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2531)
    at java.lang.Class.getMethod0(Class.java:2774)
    at java.lang.Class.getMethod(Class.java:1663)
    at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494)
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.conf.YarnConfiguration

但是,如果我在所有数据节点上运行“yarn classpath”命令,则会看到以下输出:
/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-yarn/lib/*

它具有应用程序所需的 yarn 客户端,yarn-api,yarn-common和hadoop-common的路径。谁能指出我可能忘记设置正确的类路径的方向。

最佳答案

我发现Hadoop在迭代YarnConfiguration属性时不会解析$ HADOOP_HOME和$ YARN_HOME环境变量。在您的Yarn Client中运行以下命令将打印未解决的配置,例如,


YarnConfiguration conf = new YarnConfiguration()
  for (String c : conf.getStrings(
                YarnConfiguration.YARN_APPLICATION_CLASSPATH,
                YarnConfiguration.DEFAULT_YARN_APPLICATION_CLASSPATH)) {
      System.out.println(c);
  }

因此,如果您提供yarn.application.classpath属性的完整路径,则可以解决NoClassDefFoundError问题。
<property>
    <description>CLASSPATH for YARN applications. A comma-separated list of CLASSPATH entries</description>
    <name>yarn.application.classpath</name>
    <value>
        /etc/hadoop/conf,
        /usr/lib/hadoop/*,
        /usr/lib/hadoop/lib/*,
        /usr/lib/hadoop-hdfs/*,
        /usr/lib/hadoop-hdfs/lib/*,
        /usr/lib/hadoop-mapreduce/*,
        /usr/lib/hadoop-mapreduce/lib/*,
        /usr/lib/hadoop-yarn/*,
        /usr/lib/hadoop-yarn/lib/*
    </value>
  </property>

关于hadoop - NoClassDefFoundError用于简单的Yarn应用程序,我们在Stack Overflow上找到一个类似的问题:https://stackoverflow.com/questions/23458385/

10-16 03:26