单声道下的protobuf-net反序列化System.IO.EndOfStreamException

问题描述

我一直在使用protobuf-net通过网络发送一些对象，并且到目前为止一切都很好.但是，我遇到了我的类的一个特殊实例，该实例在mono下运行时无法反序列化.完全相同的对象反序列化可以在.net下正确运行.我已经通过检查md5的总和验证了在单声道和.net下运行时通过电线收到的完全相同的byte[].这表明该问题必须与protobuf-net反序列化有关.这是我用来反序列化byte[]的代码:

I have been using protobuf-net to send some objects over the wire and everything has been working nicely until now. However, I have come across a particular instantiation of my class that fails to deserialize when running under mono. The exact same object deserializes correctly running under .net. I have verified that it is the exact same byte[] that I receive over the wire when I run under mono and under .net by checking the md5 sum. This would indicate that the problem must be with protobuf-net deserialization. Here is the code I am using to deserialize the byte[]:

using (MemoryStream ms = new MemoryStream(serializedByteArray))
{
    return (MyProtoBufDto)Serializer.Deserialize<MyProtoBufDto>(ms);
}

这是我得到的例外:

System.IO.EndOfStreamException: Failed to read past end of stream.
at ProtoBuf.ProtoReader.Ensure (int,bool) <0x00167>
at ProtoBuf.ProtoReader.ReadString () <0x0005b>
at (wrapper dynamic-method) System.Collections.Generic.KeyValuePair`2<string, string>.proto_18 (object,ProtoBuf.ProtoReader) <0x000c8>
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) <0x0002d>
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) <0x00112>
at ProtoBuf.ProtoReader.ReadTypedObject (object,int,ProtoBuf.ProtoReader,System.Type) <0x00056>
at ProtoBuf.ProtoReader.ReadObject (object,int,ProtoBuf.ProtoReader) <0x0001b>
at (wrapper dynamic-method) System.Collections.Generic.KeyValuePair`2<string, System.Collections.Generic.List`1<System.Collections.Generic.KeyValuePair`2<string, string>>>.proto_16 (object,ProtoBuf.ProtoReader) <0x00220>
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) <0x0002d>
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) <0x00112>
at ProtoBuf.ProtoReader.ReadTypedObject (object,int,ProtoBuf.ProtoReader,System.Type) <0x00056>
at ProtoBuf.ProtoReader.ReadObject (object,int,ProtoBuf.ProtoReader) <0x0001b>
at (wrapper dynamic-method) MyGroupDto.proto_14 (object,ProtoBuf.ProtoReader) <0x001bc>
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) <0x0002d>
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) <0x00112>
at ProtoBuf.ProtoReader.ReadTypedObject (object,int,ProtoBuf.ProtoReader,System.Type) <0x00056>
at ProtoBuf.ProtoReader.ReadObject (object,int,ProtoBuf.ProtoReader) <0x0001b>
at (wrapper dynamic-method) System.Collections.Generic.KeyValuePair`2<MyGroupNameDto, MyGroupDto>.proto_12 (object,ProtoBuf.ProtoReader) <0x00197>
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) <0x0002d>
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) <0x00112>
at ProtoBuf.ProtoReader.ReadTypedObject (object,int,ProtoBuf.ProtoReader,System.Type) <0x00056>
at ProtoBuf.ProtoReader.ReadObject (object,int,ProtoBuf.ProtoReader) <0x0001b>
at (wrapper dynamic-method) MyResultProtoBufDto.proto_8 (object,ProtoBuf.ProtoReader) <0x002b9>
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) <0x0002d>
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) <0x00112>
at ProtoBuf.ProtoReader.ReadTypedObject (object,int,ProtoBuf.ProtoReader,System.Type) <0x00056>
at ProtoBuf.ProtoReader.ReadObject (object,int,ProtoBuf.ProtoReader) <0x0001b>
at (wrapper dynamic-method) MyProtoBufDto.proto_6 (object,ProtoBuf.ProtoReader) <0x00116>
at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read (object,ProtoBuf.ProtoReader) <0x0002d>
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize (int,object,ProtoBuf.ProtoReader) <0x00112>
at ProtoBuf.Meta.TypeModel.DeserializeCore (ProtoBuf.ProtoReader,System.Type,object,bool) <0x0005b>
at ProtoBuf.Meta.TypeModel.Deserialize (System.IO.Stream,object,System.Type,ProtoBuf.SerializationContext) <0x00097>
at ProtoBuf.Meta.TypeModel.Deserialize (System.IO.Stream,object,System.Type) <0x0001f>
at ProtoBuf.Serializer.Deserialize<MyProtoBufDto> (System.IO.Stream) <0x00033>
at MyClient.Serialization.ProtoBufSerialization.DecompressAndDeserialize<MyProtoBufDto> (byte[]) <0x0008b>

我尝试了Mono 2.10.9和最新的Mono 3.2.3，并收到了相同的异常.我正在使用最新的protobuf-net dll版本(2.0.0.666).我尝试从protobuf-net-mono目录(使用mono编译器编译吗?)和常规.net编译版本中使用dll，但在mono下运行时始终遇到此异常.

I have tried mono 2.10.9 and the latest mono 3.2.3 and have received the same exception. I am using the latest protobuf-net dll version (2.0.0.666). I tried the dll from protobuf-net-mono directory (compiled with mono compiler?) and the regular .net compiled version but keep getting this exception when running under mono.

MyProtoBufDto类的详细信息:

[DataContract]
public class MyProtoBufDto
{
    [DataMember(Order = 1)]
    public List<MyResultProtoBufDto> Result { get; set; }
}

[DataContract]
public class MyResultProtoBufDto
{
    [DataMember(Order = 1)]
    public Dictionary<MyGroupNameDto, MyGroupDto> Groups { get; set; }
}

[DataContract]
public class MyGroupDto
{
    [DataMember(Order = 1)]
    public Dictionary<string, List<KeyValuePair<string, string>>> Group { get; set; }
}

MyGroupNameDto只是一个enum

更多信息:这是在Linux上使用mono.我尚未在Windows上测试过单声道.我在单声道和Windows上都验证了脱机后的byte []和解压缩后的byte []相同，因此我们应该向protobuf-net提供相同的准确数据.

More Info:This is using mono on linux. I have not tested mono on windows yet. I verified both the byte[] off the wire and the byte[] after decompression is identical under mono and on windows so we should be feeding protobuf-net the same exact data.

更新我们在服务器端进行了更改，以始终从已序列化的对象中删除空集合，此后一直没有遇到此错误.我知道protobuf不会区分空集合和空集合，但是仍然奇怪的是，单声道与.net上的行为有所不同，并且触发了此错误.

UpdateWe made a change on the server side to always remove empty collections from the objects that are serialized and since then have not encountered this error. I understand that protobuf does not make a distinction between empty collections and null collections, but it is still peculiar that somehow the behavior was different on mono vs .net and it triggered this error.

推荐答案

我能够弄清楚这个奇怪问题的原因.显然，最初调试此问题时我犯了一个错误.我曾以为我要发送反序列化的byte []在.net和mono下运行时是相同的.但是，事实并非如此，并且错误是与DeflateStream结合反序列化一起使用的.在单声道下运行时，DeflateStream不会返回protobuf-net期望的所有字节.

I was able to figure out the cause of this weird problem. Apparently I made an error when debugging this problem originally. I had thought the byte[] I was sending for deserialization was identical when running in .net and under mono. However, this was not the case and the error was with the use of DeflateStream in conjunction with deserialization. When running under mono the DeflateStream was not return all the bytes that protobuf-net was expecting.

损坏的代码如下:

using (MemoryStream ms = new MemoryStream(compressedByteArray))
using (DeflateStream ds = new DeflateStream(ms, CompressionMode.Decompress, true))
     return (MyProtoBufDto)Serializer.Deserialize<MyProtoBufDto>(ds);

工作代码如下:

using (MemoryStream msDecompressed = new MemoryStream())
{
     using (MemoryStream ms = new MemoryStream(compressedByteArray))
     using (DeflateStream ds = new DeflateStream(ms , CompressionMode.Decompress, true))
           ds.CopyTo(msDecompressed);

     msDecompressed.Seek(0, SeekOrigin.Begin);
     return (MyProtoBufDto)Serializer.Deserialize<MyProtoBufDto>(msDecompressed);
}

单声道和.net之间存在这种差异是非常糟糕的.我了解DeflateStream使用基础缓冲区. Mono可能正在使用惰性实现，即使请求更多的字节，也只返回缓冲区中的解压缩字节.只要DeflateStream.ReadBytes返回至少1个字节，从技术上讲，这可能不会破坏任何.net规范.更改为protobuf-net时，如果没有获得预期的数目，则继续从流中读取字节，则可能会解决此问题.仅当ReadBytes返回零字节时，protobuf-net才会引发我们所看到的异常.

It's too bad that this difference between mono and .net exists. I understand that DeflateStream uses underlying buffers. Mono is probably using the lazy implementation of only returning the decompressed bytes in the buffer even though more bytes are requested. As long as DeflateStream.ReadBytes returns at least 1 byte, this technically might not break any .net specifications. A change to protobuf-net that continues reading bytes from the stream when it does not get the expected number back would probably fix this issue. Only if ReadBytes returns zero bytes should protobuf-net throw the exception we are seeing.

这篇关于单声道下的protobuf-net反序列化System.IO.EndOfStreamException的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！