背景
我试图以一种这样的方式编写Kryo反序列化：如果对象数组包含某些对象（由于代码更改）而不能被反序列化，则该数组中的那些引用将变为null而不是引发异常。允许回收剩余的物体。我以前一直在使用Java的内置序列化，并且在其中，我能够通过在数组的每个项目之间写入一个“已知良好”整数，然后在流中查找该错误（如果发现错误）来实现这一点，从而实现了这一目标。下一个对象。这是详细的问题Deserializing an array that contains some non-deserializable objects (salvaging the deserializable parts)。
由于效率原因，我现在移至Kryo序列化，并尝试重新创建此方法，但是在Kryo中，此错误恢复似乎只能工作一次，但之后无法正确恢复。
我尝试过的
我试图在序列化期间在END_OF_APPLE_MAGIC数组中的每个对象之间写入一个已知的好整数（Apple）。在反序列化过程中，当找到无法反序列化的BadApple时，将其替换为ErrorApple（分析越来越弱），并搜索END_OF_APPLE_MAGIC以查找在哪里寻找下一个苹果。如果数组中只有一个BadApple并且BadApple不是第一个条目，则此方法有效。但是如果数组中有多个BadApple或第一个Apple是BadApple，则会以各种方式失败（请参阅详细分析）
public class AppleHolder implements Serializable,KryoSerializable{
static int END_OF_APPLE_MAGIC=1234467895; //if this just "turns up" in the stream we will have a bad day; however this is only the case in recovery mode, so is an acceptable risk
int numberOfApples=6;
Apple[] apples=new Apple[numberOfApples];
double otherData=15;
//these are just for debug
int dividers=0; //counts number of times END_OF_APPLE_MAGIC is found
int problems=0; //counts number of times an apple fails to load
int badIntegers=0; //counts number of times END_OF_APPLE_MAGIC is looked for and a different Integer is found (I have never seen this happen)
public AppleHolder(){
Apple goodApple=new Apple("GoodApple","tastyGood");
BadApple badApple=new BadApple("BadApple","untastyBad");
apples[0]=goodApple;
apples[1]=badApple;
apples[2]=goodApple;
apples[3]=goodApple; // multiple references to same object intentional
apples[4]=goodApple;
apples[5]=goodApple;
}
public void write (Kryo kryo, Output output) {
for(int i=0;i<apples.length;i++){
//kryo.writeObject(output, apples[i]);
kryo.writeClassAndObject(output, apples[i]);
kryo.writeClassAndObject(output, END_OF_APPLE_MAGIC);
}
kryo.writeObject(output,otherData);
}
public void read (Kryo kryo, Input input) {
try{
apples =new Apple[numberOfApples];
for(int i=0;i<apples.length;i++){
try{
Object ob=kryo.readClassAndObject(input);
apples[i]=(Apple)ob;
}catch(Exception e){
apples[i]=new ErrorApple();
problems++;
}
//Search for next Apple Boundary (except in recovery mode
//it will be the next entry)
boolean atBoundary=false;
while (atBoundary==false){ //should probably put a limit on this just in case
try{
int appleMagic =(Integer)kryo.readClassAndObject(input);
if (appleMagic == END_OF_APPLE_MAGIC){
atBoundary=true;
dividers++;
}else{
badIntegers++;
}
}catch(Exception e){
//painful byte reading mode only entered in recovery mode; under good
//situations it does not represent an efficiency problem
input.skip(1); //consume byte of bad input
//Where buffer underflow exceptions occur they occur here
}
}
}
otherData = kryo.readObject(input, Double.class);
}catch(Exception e){
//something when wrong (always a Buffer underflow so far), print what we have
for(int i=0;i<apples.length;i++){
System.out.println(apples[i]);
}
throw e;
}
}
public static void main(String[] args)
throws Exception {
/*
* (1) First run serialize()
* (2) Rename/delete badApple such that it cannot be found for deserialization
* (3) Run deSerialize(()
*/
serialize();
//deSerialize();
}
public static void serialize() throws Exception{
AppleHolder testWrite = new AppleHolder();
/*FileOutputStream fos = new FileOutputStream("testfile");
ObjectOutputStream oos = new ObjectOutputStream(fos);
oos.writeObject(testWrite);
oos.flush();
oos.close();
*/
Kryo kryo = new Kryo();
Output output = new Output(new FileOutputStream("testfile"));
kryo.writeObject(output, testWrite);
output.close();
}
public static void deSerialize() throws Exception{
/*AppleHolder testRead;
FileInputStream fis = new FileInputStream("testfile");
ObjectInputStream ois = new ObjectInputStream(fis);
testRead = (AppleHolder) ois.readObject();
ois.close();
*/
Kryo kryo = new Kryo();
Input input = new Input(new FileInputStream("testfile"));
AppleHolder testRead = kryo.readObject(input, AppleHolder.class);
input.close();
for(int i=0;i<testRead.apples.length;i++){
System.out.println(testRead.apples[i]);
}
System.out.println("otherData: " + testRead.otherData);
}
}
public class Apple implements Serializable {
private String propertyOne;
private String propertyTwo;
public Apple(){}
public Apple(String propertyOne, String propertyTwo) {
this.propertyOne = propertyOne;
this.propertyTwo = propertyTwo;
validate();
}
private void writeObject(ObjectOutputStream o)
throws IOException {
o.writeObject(propertyOne);
o.writeObject(propertyTwo);
}
private void readObject(ObjectInputStream o)
throws IOException, ClassNotFoundException {
propertyOne = (String) o.readObject();
propertyTwo = (String) o.readObject();
validate();
}
private void validate(){
if(propertyOne == null ||
propertyOne.length() == 0 ||
propertyTwo == null ||
propertyTwo.length() == 0){
throw new IllegalArgumentException();
}
}
public String getPropertyOne() {
return propertyOne;
}
public String getPropertyTwo() {
return propertyTwo;
}
@Override
public String toString() {
return "goodApple";
}
}
public class BadApple extends Apple {
public BadApple(){}
public BadApple(String propertyOne, String propertyTwo) {
super(propertyOne, propertyTwo);
}
@Override
public String toString() {
return "badApple";
}
}
public class ErrorApple extends Apple {
public ErrorApple(){}
public ErrorApple(String propertyOne, String propertyTwo) {
super(propertyOne, propertyTwo);
}
@Override
public String toString() {
return "errorApple";
}
}
题
如何挽救仅某些对象可反序列化的Kyro序列化数组？从而为不可反序列化的部分获得一个带有
ErrorApple条目的数组。在我的数组中，我在单个数组中有多个对同一个对象的引用，这必须在反序列化过程中保留。所以要进行序列化
[GoodApple]
[GoodApple]
[GoodApple]
[BadApple]
[BadApple]
[GoodApple]
并从反序列化中脱颖而出（因为糟糕的Apple已更改且无法反序列化
[GoodApple]
[GoodApple]
[GoodApple]
[ErrorApple]
[ErrorApple]
[GoodApple]
我希望这提供了一个无法实现向后兼容性的后备功能，或者删除了先前安装的对我的程序的第三者修改
详细分析
本节概述了现有程序失败的方式。
一般来说
在数组第一个位置以外的某个地方的单个
BadApple将正常运行数组中第一个位置处的
BadApple会导致下一个Apple正确读取，然后正确读取ErrorApples（即使对于良好的Apple）如果第二个
BadApple之后的第一个好Apple多于1个BadApple，则将正确读取，但可以在数组中向前移动，然后从此向前移动ErrorApples（即使对于良好的Apple）。将有一个KryoException: Buffer underflow，并且在数组末尾可能有空条目。我使用的输入和输出如下所示：
进出
[goodApple] [goodApple]
[goodApple] [goodApple]
[badApple] [badApple]
[goodApple] [goodApple]
[goodApple] [goodApple]
[goodApple] [goodApple]
进出
[badApple] [errorApple]
[goodApple] [goodApple]
[goodApple] [errorApple]
[goodApple] [errorApple]
[goodApple] [errorApple]
[goodApple] [errorApple]
进出
[goodApple] [goodApple]
[badApple] [errorApple]
[goodApple] [goodApple]
[badApple] [errorApple]
[goodApple] [goodApple]
[goodApple] [errorApple]
KryoException：缓冲区下溢。 （发生在input.skip（1）;上）
进出
[goodApple] [goodApple]
[goodApple] [goodApple]
[badApple] [errorApple]
[badApple] [errorApple]
[goodApple] [goodApple]
[goodApple] [errorApple]
KryoException：缓冲区下溢（发生在input.skip（1）;处）
进出
[goodApple] [goodApple]
[badApple] [errorApple]
[badApple] [errorApple]
[badApple] [goodApple]
[goodApple] [errorApple]
[goodApple] [null]
KryoException：缓冲区下溢。 （发生在input.skip（1）;上）
最佳答案
我发现的解决方案是编写第二个恢复序列化文件，该文件仅在添加了每个项目后才保留文件长度。然后，如果在反序列化数据时遇到问题，程序就会知道在哪里可以找到“下一个好”值。
但是有一个陷阱，一旦您读取了一个字节，就无法重新读取它，而无需重新开始（.reset()抛出一个UnsupportedOperationException），有时Kryo在意识到下一个阻塞之前就开始读入下一个好的对象。坏对象。我对此的解决方案是使用单独文件中的数据来检测每个对象要读取的字节数，将它们读取为字节，然后将其传递给Kryo进行反序列化。
所有这些都有一些开销，但是对于我的测试而言，整个过程仍然比标准Java反序列化要快得多，并且只能在“恢复模式”下使用。
下面的程序演示了这一点，该程序将错误的值保留为空（但可以执行任何所需的操作）。
public class AppleHolder implements Serializable,KryoSerializable{
int numberOfApples=10;
Apple[] apples=new Apple[numberOfApples];
double otherData=15;
public AppleHolder(){
BadApple sharedBad=new BadApple(0);
for(int i=0;i<numberOfApples;i++){
if (i>3 && i<=6){
apples[i]=new Apple(i);
}else{
apples[i]=new BadApple();
}
}
}
public void write (Kryo kryo, Output output) {
int[] recoveryTrack=new int[apples.length+1]; //last one for after the last entry
for(int i=0;i<apples.length;i++){
recoveryTrack[i]=output.total();
kryo.writeClassAndObject(output, apples[i]);
}
recoveryTrack[recoveryTrack.length-1]=output.total();
kryo.writeObject(output,otherData);
Output outputRecovery;
try {
outputRecovery = new Output(new FileOutputStream("testfile.recovery"));
kryo.writeObject(outputRecovery, recoveryTrack);
outputRecovery.close();
} catch (FileNotFoundException ex) {
//I guess hopefully we won't need the recovery track
Logger.getLogger(AppleHolder.class.getName()).log(Level.SEVERE, null, ex);
}
}
public void read (Kryo kryo, Input input) {
int[] readRecoveryTrack=null;
try {
Kryo kryoRecovery = new Kryo();
Input inputRecovery = new Input(new FileInputStream("testfile.recovery"));
readRecoveryTrack =kryoRecovery.readObject(inputRecovery, int[].class);
inputRecovery.close();
} catch (FileNotFoundException ex) {
Logger.getLogger(AppleHolder.class.getName()).log(Level.SEVERE, null, ex);
}
apples=new Apple[numberOfApples];
for(int j=0;j<apples.length;j++){
int actualPos=input.total();
int desiredPos=readRecoveryTrack[j];
int desiredBytes=readRecoveryTrack[j+1]-readRecoveryTrack[j];
byte[] bytes=input.readBytes(desiredBytes);
ByteArrayInputStream byteStream =new ByteArrayInputStream(bytes);
try{
apples[j]=(Apple)kryo.readClassAndObject(new Input(byteStream));
}catch(Exception e){
//don't care, leave null
}
}
}
public static void main(String[] args)
throws Exception {
/*
* (1) First run serialize()
* (2) Rename/delete badApple such that it cannot be found for deserialization
* (3) Run deSerialize(()
*/
serialize();
//deSerialize();
}
public static void serialize() throws Exception{
AppleHolder testWrite = new AppleHolder();
Kryo kryo = new Kryo();
Output output = new Output(new FileOutputStream("testfile"));
kryo.writeObject(output, testWrite);
output.close();
}
public static void deSerialize() throws Exception{
Kryo kryo = new Kryo();
Input input = new Input(new FileInputStream("testfile"));
AppleHolder testRead = kryo.readObject(input, AppleHolder.class);
input.close();
for(int i=0;i<testRead.apples.length;i++){
System.out.println(testRead.apples[i]);
}
System.out.println(testRead.otherData);
}
}
public class Apple implements Serializable {
protected int index;
public Apple(){}
public Apple(int index) {
this.index = index;
}
@Override
public String toString() {
return "goodApple " + index;
}
}
public class BadApple extends Apple {
private static final long serialVersionUID = 7;
public BadApple(){}
public BadApple(int index){
super(index);
}
@Override
public String toString() {
return "badApple " + index;
}
}