本文介绍了使用Oozie使用Hive / Hadoop安排即席查询的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

Oozie是否支持用户通过REST API调度特定的Hive查询?



我们正在构建一个系统,用户可以在其中搜索文档Hadoop,支持用户(可选)指定要搜索的数据的某些属性,使用Hive执行针对Hadoop的查询。由于对可选字段的这种支持,我们并不知道Hive查询会是什么样子(就Hive查询中将使用哪些表而言)。我们有一个服务,在运行时,我们处理用户的查询以生成相应的Hive查询。



我们希望能够通过Oozie,但我一直无法找到有关如何通过Oozie执行此操作的文档。我认为这是可能的。是否有示例Java代码可用于描述如何执行此操作?使用Oozie协调器安排作业,Apache文档

a href =https://oozie.apache.org/docs/4.0.1/CoordinatorFunctionalSpec.html#a1._Coordinator_Overview =nofollow>这里和一个例子为Oozie协调员。另外,请看看Azkaban(,)进行排程。


Does Oozie support a user scheduling, via a REST API, an ad-hoc Hive query?

We're building a system where a user can search documents in Hadoop, with support for the user (optionally) specifying some attributes of the data to be searched, using Hive to perform the query against Hadoop. Because of this support for optional fields, we don't know ahead of time what the Hive query will look like (in terms of which tables will be used in the Hive query). We have a service where, at run-time, we process the user's query to generate the corresponding Hive query.

We'd like to be able to schedule these queries via Oozie, but I haven't been able to find documentation on how to perform this via Oozie. I assume this is possible. Is there sample Java code available to describe how to perform this operation?

解决方案

Use the Oozie Coordinator to schedule jobs, Apache documentation here and an example here for Oozie Coordinator. Also, take a look at Azkaban (1, 2) for scheduling.

这篇关于使用Oozie使用Hive / Hadoop安排即席查询的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

09-15 22:09