问题描述
我正在Amazon AWS上使用Elastic Mapreduce基础架构. jowflow自动终止.根据Amazon Console的最后一个状态更改原因是:作业流中的所有从属均已终止".
I am using Elastic Mapreduce infrastructure on Amazon AWS. A jowflow got terminated automatically. Last state change reason according Amazon Console is : "All slaves in the job flow were terminated".
创建作业流程命令:
elastic-mapreduce --create --name MyCluster --alive --instance-group master --instance-type m1.xlarge --instance-count 1 --bid-price 2.0 --instance-group core --instance-type m1.xlarge --instance-count 10 --bid-price 2.0 --hive-interactive --enable-debugging
有关工作流程的详细信息:
Details about jobflow :
最近几行日志...
Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201310231204_0099, Tracking URL = http://ip-10-197-16-105.us-west-1.compute.internal:9100/jobdetails.jsp?jobid=job_201310231204_0099
Kill Command = /home/hadoop/bin/hadoop job -Dmapred.job.tracker=10.197.16.105:9001 -kill job_201310231204_0099
2013-10-23 14:11:38,618 Stage-1 map = 0%, reduce = 0%
2013-10-23 14:11:48,741 Stage-1 map = 100%, reduce = 0%
正如您在上面的日志中看到的那样，不会引发任何错误.
As you can see above in logs, no error is thrown as such.
我认为的原因是
我认为这是由于现货价格突然上涨的缘故.有关更多详细信息，请参见下面的答案.
I think that this happened because of sudden increase in price od spot instances. More details in my answer below.
推荐答案
我在这里回答我自己的问题.
Here I am answering my own question.
我认为发生这种情况是因为现货实例价格突然上涨.对于m1.xlarge实例，我的出价为每实例每小时2美元.
I think that this happened because of sudden increase in price of spot instances. My bid price was $2 per instance per hour for a m1.xlarge instance.
aws控制台竞价型实例定价的快照:
您会注意到定价出现蓝色跳跃.我的出价是2美元，实际价格跳到11美元.因此，我的集群被自动杀死.
You can notice the blue jumps in pricing. My bid was $2 and the actual price jumped to $11. Hence my cluster was killed automatically.
这篇关于AWS EMR错误:作业流中的所有从属终端均已终止的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！