我从mapper发出TwoDArrayWritable。我已经实现了扩展TwoDArrayWritable的类，以使其成为默认构造函数。但是当我尝试发出它时，它给了我以下异常:
Error: java.lang.RuntimeException: java.lang.InstantiationException: org.apache.hadoop.io.TwoDArrayWritable
at org.apache.hadoop.io.TwoDArrayWritable.readFields(TwoDArrayWritable.java:75)
at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)
如何发出TwoDArrayWritable？我需要一些帮助。
这是扩展TwoDArrayWritable的类:
public class TwoDArrayWritables extends TwoDArrayWritable
{
public TwoDArrayWritables() {
super(TwoDArrayWritable.class);
}
public TwoDArrayWritables(Class valueClass) {
super(valueClass);
// TODO Auto-generated constructor stub
}
这是映射器:
public class JaccardMapper extends Mapper<LongWritable, Text, IntTextPair, TwoDArrayWritables> {
Hashtable movieInfo = new Hashtable<String, String>();
String[] genres, actors, entities;
String[] attributes = new String[] {"genre", "actors", "directors", "country", "year", "ratings"};
double p,q,r,s;
double result = 0.0;
String input[] = null;
Set<String> keys;
TwoDArrayWritables array2d = new TwoDArrayWritables();
//TwoDArrayWritable array2d = new TwoDArrayWritable(IntWritable.class);
IntWritable[][] jaccard = new IntWritable[2][];
//int[][] jaccard = new int[2][];
public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException
{
p = 0;
q = 0;
r = 0;
s = 0;
input = value.toString().toLowerCase().split(",");
keys = movieInfo.keySet();
//the jaccards 2d array column length depends on the user input best case is 6 but the worst case depends on the sub attributes count like more than one actor/director/genre/country.
int columnlength = input[1].split("\\|").length + input[2].split("\\|").length + input[3].split("\\|").length + input[4].split("\\|").length + 2;
jaccard = new IntWritable[2][columnlength];
for (int i = 0; i < jaccard.length; i++)
{
for (int j = 0; j < jaccard[i].length; j++)
{
jaccard[i][j] = new IntWritable(0);
}
}
if (input.length > 0)
{
//iterate through the dataset in cache
for(String keyy : keys)
{
//iterate to user's input attributes
for (int attribute = 1; attribute < attributes.length; attribute++)
{
if (!input[attribute].equals("-"))
{
entities = input[attribute].toLowerCase().split("\\|");
int subattributecount = 0;
for(String entity : entities)
{
subattributecount += 1;
}
}
}
IntTextPair pair = new IntTextPair(Integer.parseInt(input[0].toString()), movieInfo.get(keyy).toString());
array2d.set(jaccard);
context.write(pair, array2d);
}
}
}
}
这是 reducer :
公共(public)类JaccardReducer扩展Reducer {
double p,q,r,s;
double result = 0.0;
//IntWritable[][] jaccard = null;
IntWritable[][] jaccard;
int temp1 = 0, temp2 = 0;
//Jaccard distance 1.0 emplies that the user's criteria doesn't meet at all
public static final double nonacceptvalue = 1.0;
public void reduce(IntTextPair key, Iterable<TwoDArrayWritables> values, Context context) throws IOException, InterruptedException
{
IntDoublePair pair = new IntDoublePair(key.getFirst().get(), Double.parseDouble("110.00"));
for (TwoDArrayWritable value : values)
{
if (value != null)
{
context.write(pair, new Text("Is not null"));
}
else
{
context.write(pair, new Text("Is null"));
}
}
}
}
最佳答案
您在默认构造函数( super )中指定的类应该是值的类，而不是整体类本身。因此，您可能想要:
public TwoDArrayWritables() {
super(IntWritable.class);
}