本文介绍了Boto3 EMR-蜂巢步骤的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
是否可以使用Boto 3执行蜂巢步骤?我一直在使用AWS CLI,但是从文档( http://boto3.readthedocs.org/en/latest/reference/services/emr.html#EMR.Client.add_job_flow_steps ),似乎只接受了jar.如果可能采取Hive步骤,那么资源在哪里?
Is it possible to carry out hive steps using boto 3? I have been doing so using AWS CLI, but from the docs (http://boto3.readthedocs.org/en/latest/reference/services/emr.html#EMR.Client.add_job_flow_steps), it seems like only jars are accepted. If Hive steps are possible, where are the resources?
谢谢
推荐答案
我能够使用Boto3使它起作用:
I was able to get this to work using Boto3:
# First create your hive command line arguments
hive_args = "hive -v -f s3://user/hadoop/hive.hql"
# Split the hive args to a list
hive_args_list = hive_args.split()
# Initialize your Hive Step
hiveEmrStep=[
{
'Name': 'Hive_EMR_Step',
'ActionOnFailure': 'CONTINUE',
'HadoopJarStep': {
'Jar': 'command-runner.jar',
'Args': hive_args_list
}
},
]
# Create Boto3 session and client
session = boto3.Session(region_name=AWS_REGION,profile_name=AWS_PROFILE)
client = session.client('emr')
# Submit and execute EMR Step
client.add_job_flow_steps(JobFlowId=cluster_id,Steps=hiveEmrStep)
#Where cluster_id is the ID of your cluster from AWS EMR (ex: j-2GS7xxxxxx)
这篇关于Boto3 EMR-蜂巢步骤的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!