问题描述
为了支持带有进度报告的大文件(实际上是非常大的文件，高达几GB)的上传，我们开始将HttpClient与PushStreamContent结合使用，如此处.它很简单，我们在两个流之间复制字节，这是一个代码示例:
In order to support upload of large (actually very large, up to several gigabytes) files with progress report we started using HttpClient with PushStreamContent, as described here. It works straightforward, we copy bytes between two streams, here is a code example:
private void PushContent(Stream src, Stream dest, int length)
{
const int bufferLength = 1024*1024*10;
var buffer = new byte[bufferLength];
var pos = 0;
while (pos < length)
{
var bytes = Math.Min(bufferLength, length - pos);
src.Read(buffer, 0, bytes);
dest.Write(buffer, 0, bytes);
pos += bufferLength;
dest.Flush();
Console.WriteLine($"Transferred {pos} bytes");
}
dest.Close();
}
但是在开始时，此代码在传输320 MB之后引发了OutOfMemory异常，即使该进程的内存消耗不是很高(约500 MB).解决此问题的方法是设置TransferEncodingChunked标志:
But in the beginning this code raised OutOfMemory exception after transferring 320 MB even though the memory consumption of the process was not very high (ca. 500 MB). What fixed this problem is setting TransferEncodingChunked flag:
request.Headers.TransferEncodingChunked = true;
不仅我们能够使用此标志设置传输大文件，而且内存消耗减少了90％.
Not only we were able to transfer huge files with this flag set, the memory consumption decreased by 90%.
我还没有找到任何需要使用TransferEncodingChunked的文档，这更多是一个尝试和失败的过程，但是在这种情况下似乎至关重要.仍然令我感到困惑的是，为什么根本引发异常-内存消耗不是很高，是什么原因造成的?
I haven't found any documentation requiring use of TransferEncodingChunked, it was more of a trial and failure process but it seems crucial in this scenario. Still I am puzzled why the exception is thrown at all - the memory consumption is not very high, what causes it?
推荐答案
每个块的大小在块本身之前发送，这样 接收方可以告知接收数据的完成时间 块.数据传输以最后一个长度的块终止 零.
The size of each chunk is sent right before the chunk itself so that the receiver can tell when it has finished receiving data for that chunk. The data transfer is terminated by a final chunk of length zero.
如果我们从逻辑上考虑，该文件将以小块发送，这意味着当您完成一个块时，便会将其从内存中释放出来.最后，您将减少内存消耗，因为您要处理多个小块.
If we think logically, the file is send on little chunks, that means when you finish with a chunk you are freeing it from the memory. In the end you have less memory consumption, because you are dealing with multiple small chunks.
这篇关于未设置TransferEncodingChunked时HttpClient抛出OutOfMemory异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！