exception - Hadoop作业未被杀死

我的工作在减少 map 步骤之前有一些异常(exception)，但是工作没有被杀死。如何配置hadoop，使作业在异常后被杀死？

立即调用主类

心跳
心跳

主类(class)的招生完成

Oozie Launcher结束

标准日志

org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Io exception: Unknown host specified )
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:82)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:577)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:792)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:815)
    at com.seven.crcs.export.dao.ReportDAOImpl.recreateReportEntity(ReportDAOImpl.java:151)
    at com.seven.crcs.export.dao.ReportDAOImpl.saveActiveUserCount(ReportDAOImpl.java:93)
    at com.seven.crcs.export.ReportJdbcExporter.saveActiveUserCount(ReportJdbcExporter.java:55)
    at com.seven.dataprocessor.oc.jobs.reports.export.day.ExportDailyUserReducer.exportUserCounts(ExportDailyUserReducer.java:32)
    at com.seven.dataprocessor.oc.jobs.reports.export.ExportActiveUser
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Io exception: Unknown host specified )

和

2013-02-28 06:06:46,487 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201302270945_0181_r_000000_0, Status : FAILED
2013-02-28 06:07:00,600 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201302270945_0181_r_000000_1, Status : FAILED
2013-02-28 06:07:16,650 INFO org.apache.hadoop.mapred.JobClient: Task Id : attempt_201302270945_0181_r_000000_2, Status : FAILED
2013-02-28 06:07:31,731 INFO org.apache.hadoop.mapred.JobClient: Job complete: job_201302270945_0181

但工作已成功完成

最佳答案

您的工作实际上已终止，但仅在map任务3次失败尝试后(任务ID显示):

try_201302270945_0181_r_000000_ 0

try_201302270945_0181_r_000000_ 1

try_201302270945_0181_r_000000_ 2

您可以通过将parameter mapred.map.max.attempts设置为1或使用JobConf#setMaxMapAttempts(int)JobConf#setMaxMapAttempts来限制每个任务的最大尝试次数。

这将导致您的 map task 在第一个异常上失败，从而更快地终止您的工作。