问题描述
我正在从我的eclipse运行一个WordCount程序。我尝试使用Hadoop1.x它运行正常。
在hadoop2.x上运行时遇到问题
我尝试
1）将所有xml添加到我的类路径中。
2）也尝试conf.set（），设置conf对象中的xml属性。
同样在日志中，它说： - 不可用于容器container_1394042163908_0573_01_000001
应用程序应用程序_1394042163908_0573由于AM容器的appattempt_1394042163908_0573_000002退出2次，退出与exitCode：1由于：异常从容器启动：
org.apache.hadoop.util.Shell $ ExitCodeException：/ bin / bash：line 0：fg：no job control
在org.apache.hadoop.util.Shell.runCommand （Shell.java:464）
在org.apache.hadoop.util.Shell.run（Shell.java:379）
在org.apache.hadoop.util.Shell $ ShellCommandExecutor.execute（Shell .java：589）
在org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer（DefaultContainerEx ecutor.java:195）
在org.apache.hadoop.yarn.server.nodemanager。 compermanager.launcher.ContainerLaunch.call（ContainerLaunch.java:283）
在org.apac he.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call（ContainerLaunch.java:79）
在java.util.concurrent.FutureTask $ Sync.innerRun（FutureTask.java:334）
在java.util.concurrent.FutureTask.run（FutureTask.java:166）
在java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1110）`在这里输入代码
在java .util.concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java:603）
在java.lang.Thread.run（Thread.java:722）
。这个尝试..失败的应用程序。
我已经弄清楚了这个问题，为此：
我刚刚添加了YARNRunner.java& MRApps.java到我的项目。
它还需要将以下属性添加到Windows框中的mapred-site.xml，以便作业启动器知道作业运行器将是一个linux：
< property>
< name> mapred.remote.os< / name>
< value> Linux< / value>
< description>远程MapReduce框架的操作系统可以是Linux或Windows< / description>
< / property>
现在MapReduce运行正常。
另请将org.apache.hadoop.util.Shell.java复制到您的项目中。
您可以注释掉下面的一行，以删除winutils.exe错误。 br>
抛出新的IOException（无法在Hadoop二进制文件中找到可执行文件+ fullExeName +。）;
I am running a WordCount program from my eclipse.I tried with Hadoop1.x it is running fine.Facing issue while running on hadoop2.x
i tried1)added all xml into my classpath.2)also tried conf.set(), setting xml properties in conf object.
Also in logs it says :-No logs available for container container_1394042163908_0573_01_000001
Application application_1394042163908_0573 failed 2 times due to AM Container for appattempt_1394042163908_0573_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException: /bin/bash: line 0: fg: no job control
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerEx ecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)`enter code here`
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
.Failing this attempt.. Failing the application.
I have figured out the issue, there is a jira for it:https://issues.apache.org/jira/browse/MAPREDUCE-5655
I just added YARNRunner.java & MRApps.java to my project .It also reqires to add the following property to mapred-site.xml on the windows box, so that the job launcher knows, that the job runner will be a linux:
<property>
<name>mapred.remote.os</name>
<value>Linux</value>
<description>Remote MapReduce framework's OS, can be either Linux or Windows</description>
</property>
And MapReduce is running fine now.
Also Copy org.apache.hadoop.util.Shell.java into your project.
You can comment out the below line,to remove the winutils.exe Error.
throw new IOException("Could not locate executable " + fullExeName + " in the Hadoop binaries.");
这篇关于从Windows eclipse运行Mapreduce（纱线）时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！