本文介绍了重用 Amazon Elastic MapReduce 实例的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我使用 Amazon Elastic MapReduce 尝试了一个简单的 Map/Reduce 任务,完成任务只用了 3 分钟.是否可以重用同一个实例来运行另一个任务.

I have tried a simple Map/Reduce task using Amazon Elastic MapReduce and it took just 3 mins to complete the task. Is it possible to re-use the same instance to run another task.

即使我只使用了 3 分钟的实例,亚马逊也会收取 1 小时的费用,所以我想用剩余的 57 分钟来运行其他几个任务.

Even though I have just used the instance for 3 mins Amazon will charge for 1 hr, so I want to use the balance 57 mins to run several other tasks.

推荐答案

答案是肯定的.

以下是使用命令行客户端的方法:

here's how you do it using the command line client:

当您创建实例时,传递 --alive 标志,这会告诉 emr 在您的作业运行后保持集群运行.

When you create an instance pass the --alive flag, this tells emr to keep the cluster around after your job has run.

然后就可以向集群提交更多的任务了:

Then you can submit more tasks to the cluster:

elastic-mapreduce --jobflow <job-id> --stream --input <s3dir> --output <s3dir> --mapper <script1> --reducer  <script2>

要稍后终止集群,只需运行:

To terminate the cluster later, simply run:

elastic-mapreduce <jobid> --terminate

尝试运行 elastic-mapreduce --help 以查看您可以运行的所有命令.

try running elastic-mapreduce --help to see all the commands you can run.

如果您没有命令行客户端,请在此处获取.

If you don't have the command line client, get it here.

这篇关于重用 Amazon Elastic MapReduce 实例的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

05-29 04:39
查看更多