本文介绍了在oozie中覆盖hadoop的mapreduce.fileoutputcommitter.marksuccessfuljobs的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
<property>
<name>mapreduce.fileoutputcommitter.marksuccessfuljobs</name>
<value>false</value>
</property>
我想覆盖上述属性为true。该属性对于群集中的其余作业需要为false，但在我的oozie工作流程中，需要hadoop在作业完成后在输出目录中创建_SUCCESS文件。它是写入输出的工作流程中的配置单元操作。
I want to override the above property to true. The property needs to be false for the rest of the jobs on the cluster, but I need, in my oozie workflow, hadoop to create _SUCCESS file in the output directory after the completion of job. Its a hive action in the workflow which writes output. Please help.
推荐答案
Hive不幸地通过设置它自己的NullOutputComter来覆盖此功能：
Hive unfortunately overrides this capability by setting it's own NullOutputComitter:
conf.setOutputCommitter(NullOutputCommitter.class);
见
src/shims/src/0.20/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java
src/shims/src/common-secure/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java
看起来您必须手动创建标志。我们已经为此提交了。
Looks like you'll have to create the flag manually. We have filed HIVE-3700 for this.
这篇关于在oozie中覆盖hadoop的mapreduce.fileoutputcommitter.marksuccessfuljobs的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！