本文介绍了如何将Jar文件传递到OOZIE shell节点中的shell脚本的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
你好我在脚本中运行java程序时遇到了错误，该脚本正在oozie shell操作工作流中执行。
Stdoutput 2015-08-25 03：36：02,636信息[pool-1-thread-1]（ProcessExecute.java:68） - 线程main中的异常java.io.IOException：打开作业jar时出错：/ tmp / jars / first.jar
Stdoutput 2015-08-25 03：36：02,636 INFO [pool-1-thread-1]（ProcessExecute.java:68） - at org.apache.hadoop.util.RunJar.main （RunJar.java:124）
Stdoutput 2015-08-25 03：36：02,636 INFO [pool-1-thread-1]（ProcessExecute.java:68） - 由：java.io.FileNotFoundException引起：/ tmp / jars / first.jar（没有这样的文件或目录）
Stdoutput 2015-08-25 03：36：02,636 INFO [pool-1-thread-1]（ProcessExecute.java:68） - 在java。 util.zip.ZipFile.open（本地方法）
Stdoutput 2015-08-25 03：36：02,637 INFO [pool-1-thread-1]（ProcessExecute.java:68） - 位于java.util.zip .ZipFile。< init>（ZipFile.java:215）
Stdoutput 2015-08-25 03：36：02,637 INFO [pool-1 < init>（ZipFile.java:145）
Stdoutput 2015-08-25 03：36：02,637 INFO [ （JarFile.java:154）
Stdoutput 2015-08-25 03:36： 02,637 INFO [pool-1-thread-1]（ProcessExecute.java:68） - 位于java.util.jar.JarFile。< init>（JarFile.java:91）
Stdoutput 2015-08-25 03 ：36：02,640 INFO [pool-1-thread-1]（ProcessExecute.java:68） - at org.apache.hadoop.util.RunJar.main（RunJar.java:122）
Shell的退出代码命令1
以下是文件详细信息：
job.properties：
nameNode = maprfs：///
jobTracker = maprfs：///
queueName = nitin
EXEC = execution.jar
ozie.libpath = $ {nameNode} / user / oozie / share / lib
oozie.use.system.libpath = true
oozie.wf.application.path = maprfs：/ dev / user / oozieTest
workflow.xml
<？xml version =1.0encoding =UTF-8standalone =yes？>
< workflow-app name =testxmlns =uri：oozie：workflow：0.4>
< start to =first/>
< shell xmlns =uri：oozie：shell-action：0.1>
< job-tracker> $ {jobTracker}< / job-tracker>
< name-node> $ {nameNode}< / name-node>
<配置>
<属性>
< name> mapred.job.queue.name< / name>
<值> $ {queueName}< /值>
< / property>
< / configuration>
< exec>脚本< / exec>
< argument> -type mine< / argument>
<参数> -cfg config.cfg< /参数>
< file>脚本< / file>
< file> $ {EXEC}＃$ {EXEC}< / file>
< file> config.cfg< / file>
< file> first.jar＃first.jar< / file>
< file> second.jar＃second.jar< / file>
< / shell>
< error to =fail/>
< / action>
< kill name =fail>
Workflow failed，error message [$ {wf：errorMessage（wf：lastErrorNode（））}]< / message>
< / kill>
< end name =end/>
< / workflow-app>
script：
＃！/ bin / bash
＃执行脚本的用户
EXECUTING_USER =user1
＃开始时间
NOW = $（date +％T）
＃获取主机名称
HOST =$ HOSTNAME
ARGUMENTSTRING = $ @ -user user1 -startTime $ NOW
echo传递下列参数：$ ARGUMENTSTRING
java -cp execution.jar com.hadoop.test.Main$ ARGUMENTSTRING
出口$？
我在/ tmp / jars目录下执行我的execution.jar文件中的first.jar，原因是此目录不会为oozie工作流用户创建任何权限问题。
任何方向/建议都会非常有用。
- 我想在oozie shell动作节点中执行脚本。
- 从oozie shell动作节点执行的脚本将运行一个java程序
- 基于参数的java程序将运行first.jar或second.jar $
- 任何Oozie动作都无法引用内容在
节点的本地文件系统上，它只能引用HDFS上的内容 - Java二进制命令只能引用本地文件系统上的文件。 b
因此，按照以下步骤，可以帮助您协调您的期望。
Hi I am getting below error while running a java program in a script which is getting executed in oozie shell action workflow.
Stdoutput 2015-08-25 03:36:02,636 INFO [pool-1-thread-1] (ProcessExecute.java:68) - Exception in thread "main" java.io.IOException: Error opening job jar: /tmp/jars/first.jar Stdoutput 2015-08-25 03:36:02,636 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at org.apache.hadoop.util.RunJar.main(RunJar.java:124) Stdoutput 2015-08-25 03:36:02,636 INFO [pool-1-thread-1] (ProcessExecute.java:68) - Caused by: java.io.FileNotFoundException: /tmp/jars/first.jar (No such file or directory) Stdoutput 2015-08-25 03:36:02,636 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at java.util.zip.ZipFile.open(Native Method) Stdoutput 2015-08-25 03:36:02,637 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at java.util.zip.ZipFile.<init>(ZipFile.java:215) Stdoutput 2015-08-25 03:36:02,637 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at java.util.zip.ZipFile.<init>(ZipFile.java:145) Stdoutput 2015-08-25 03:36:02,637 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at java.util.jar.JarFile.<init>(JarFile.java:154) Stdoutput 2015-08-25 03:36:02,637 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at java.util.jar.JarFile.<init>(JarFile.java:91) Stdoutput 2015-08-25 03:36:02,640 INFO [pool-1-thread-1] (ProcessExecute.java:68) - at org.apache.hadoop.util.RunJar.main(RunJar.java:122) Exit code of the Shell command 1Following are the files details:
job.properties:
nameNode=maprfs:/// jobTracker=maprfs:/// queueName=nitin EXEC=execution.jar ozie.libpath=${nameNode}/user/oozie/share/lib oozie.use.system.libpath=true oozie.wf.application.path=maprfs:/dev/user/oozieTestworkflow.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?> <workflow-app name="test" xmlns="uri:oozie:workflow:0.4"> <start to="first" /> <action name="first"> <shell xmlns="uri:oozie:shell-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <configuration> <property> <name>mapred.job.queue.name</name> <value>${queueName}</value> </property> </configuration> <exec>script</exec> <argument>-type mine</argument> <argument>-cfg config.cfg</argument> <file>script</file> <file>${EXEC}#${EXEC}</file> <file>config.cfg</file> <file>first.jar#first.jar</file> <file>second.jar#second.jar</file> </shell> <ok to="end" /> <error to="fail" /> </action> <kill name="fail"> <message>Workflow failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> </kill> <end name="end" /> </workflow-app>script:
#!/bin/bash #get the user who executed the script EXECUTING_USER="user1" # get start time NOW=$(date +"%T") #get the host name HOST="$HOSTNAME" ARGUMENTSTRING="$@ -user user1 -startTime $NOW" echo "Passing the following arguments : $ARGUMENTSTRING" java -cp execution.jar com.hadoop.test.Main "$ARGUMENTSTRING" exit $?I am taking first.jar in my execution.jar file from /tmp/jars directory, reason is this directory will not create any permission issue to oozie workflow user.
Any direction/suggestions will be really helpful.
My question in nut shell:
- I want to execute a script in oozie shell action node.
- Script which gets executed from oozie shell action node will run a java program
- That java program based upon the arguments will run the first.jar or second.jar
解决方案I would suggest you to somehow shift the dependency out of shell script into java code and run it using the oozie java action node, which will simplify the process to good extend.
Incase if running the Java jar from oozie shell action node is your last option, then we shall very well do that, but it is little bit complicated as for as I know.
Main concerns are,
- Any Oozie action can not refer contents on local file system of thenode, where it can refer only the contents on HDFS
- The Java binary command can only refer files on local file system.
So follow the below steps which might help you to co-ordinate what you expect.
这篇关于如何将Jar文件传递到OOZIE shell节点中的shell脚本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！
解决方案
我建议你以某种方式将shell脚本中的依赖关系转换为java代码并使用oozie运行它如果从oozie shell动作节点运行Java jar是最后一个选项，那么我们将很好地处理这个问题。那个，但是就我而言，它有点复杂
$ b
主要关注的是，