我正在使用Hadoop MapReduce计算每年的最小值和最大值,但是当我运行该程序时,出现错误:FAILED Error: java.lang.ArrayIndexOutOfBoundsException: 5
我认为这是因为我的数据中包含空值,因为当没有空值时程序可以正常运行。

因此,在我的map函数中,我编写了if语句来检查是否有标题和空值:

 public static class ExposureMapper
        extends Mapper<Object, Text, Text, MinMaxExposure> {

    private Text year = new Text();
    private double minexposure;
    private Double maxexposure;

    private MinMaxExposure outPut = new MinMaxExposure();

    public void map(Object key, Text value, Context context
    ) throws IOException, InterruptedException {
        try {
            //Some condition satisfying it is header
            if (value.toString().contains("Product")) {
                return;
            } else if(value.toString()==null) {
               return;
            }
            else{
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        String[] solarFields = value.toString().split(",");

        year.set(solarFields[2]);
        minexposure = Double.parseDouble(solarFields[5]);
        maxexposure = Double.parseDouble(solarFields[5]);

        try {
            outPut.setMinExposure(minexposure);
            outPut.setMaxExposure(maxexposure);
            context.write(year, outPut);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

但是发生同样的错误...
是否因为value.toString()==null不是检查空值的正确方法?

编辑:
19/06/07 00:14:30 INFO mapreduce.Job: Task Id : attempt_1527224104960_0812_m_000000_1, Status : FAILED
Error: java.lang.ArrayIndexOutOfBoundsException: 5
    at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:50)
    at com.mycompany.hw1.SolarMinMax$ExposureMapper.map(SolarMinMax.java:23)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:793)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)

最佳答案

如果value.toString().split(",");的元素少于六个,则solarFields[5]将不是元素,因此您将看到ArrayIndexOutOfBoundsException

创建solarFields后,您应立即检查其长度:

if (solarFields == null || solarFields.length < 6) {
    return;
}

您还想确保Double.parseDouble(solarFields[5]);不会抛出NumberFormatException:
Double exposure;
try {
    exposure = Double.parseDouble(solarFields[5]);
} catch (NumberFormatException e) {
    return;
}

10-08 19:14