我正在使用Hadoop MapReduce计算每年的最小值和最大值,但是当我运行该程序时,出现错误:FAILED Error: java.lang.ArrayIndexOutOfBoundsException: 5
我认为这是因为我的数据中包含空值,因为当没有空值时程序可以正常运行。
因此,在我的map函数中,我编写了if语句来检查是否有标题和空值:
public static class ExposureMapper
extends Mapper<Object, Text, Text, MinMaxExposure> {
private Text year = new Text();
private double minexposure;
private Double maxexposure;
private MinMaxExposure outPut = new MinMaxExposure();
public void map(Object key, Text value, Context context
) throws IOException, InterruptedException {
try {
//Some condition satisfying it is header
if (value.toString().contains("Product")) {
return;
} else if(value.toString()==null) {
return;
}
else{
}
} catch (Exception e) {
e.printStackTrace();
}
String[] solarFields = value.toString().split(",");
year.set(solarFields[2]);
minexposure = Double.parseDouble(solarFields[5]);
maxexposure = Double.parseDouble(solarFields[5]);
try {
outPut.setMinExposure(minexposure);
outPut.setMaxExposure(maxexposure);
context.write(year, outPut);
} catch (IOException e) {
e.printStackTrace();
}
}
但是发生同样的错误...
是否因为
value.toString()==null
不是检查空值的正确方法?编辑:
19/06/07 00:14:30 INFO mapreduce.Job: Task Id : attempt_1527224104960_0812_m_000000_1, Status : FAILED
Error: java.lang.ArrayIndexOutOfBoundsException: 5
at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:50)
at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:23)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
最佳答案
如果value.toString().split(",");
的元素少于六个,则solarFields[5]
将不是元素,因此您将看到ArrayIndexOutOfBoundsException
。
创建solarFields
后,您应立即检查其长度:
if (solarFields == null || solarFields.length < 6) {
return;
}
您还想确保
Double.parseDouble(solarFields[5]);
不会抛出NumberFormatException
:Double exposure;
try {
exposure = Double.parseDouble(solarFields[5]);
} catch (NumberFormatException e) {
return;
}