流式传输时节点 Zlib 创建无效的 .gz 文件

问题描述

我正在尝试通过逐渐将字符串(由换行符分隔的 JSON)流式传输到 .gz 文件来创建它.

I am trying to create a .gz file by streaming strings (JSONs separated by newlines) to it gradually.

使用 Node 0.11 和 0.12(结果相同，.gz 文件无法打开).

Using Node 0.11 and 0.12 (both with same reasult, .gz file won't open).

我将代码简化为:

var fs = require('fs');
var output = fs.createWriteStream('test.gz');
var zlib = require('zlib');
var compress = zlib.createGzip();
var myStream = compress.pipe(output);

myStream.write('Hello, World!');
myStream.end();

文件已创建，但我无法打开它.我做错了什么?

The file is created, but I cannot open it. What am I doing wrong?

推荐答案

好的，解决方法如下:

var fs = require('fs');
var output = fs.createWriteStream('test.gz');
var zlib = require('zlib');
var compress = zlib.createGzip();
/* The following line will pipe everything written into compress to the file stream */
compress.pipe(output);
/* Since we're piped through the file stream, the following line will do:
   'Hello World!'->gzip compression->file which is the desired effect */
compress.write('Hello, World!');
compress.end();

以及解释:管道用于将流从一个上下文转发到另一个上下文，每个上下文根据自己的规范操作流(即 STDOUT->gzip 压缩->加密->文件将导致打印到 STDOUT 的所有内容通过 gzip 压缩、加密并最终写入归档).

And the explanation:Piping is used to forward streams from one context to another, each context manipulating the stream according to it's own specification (i.e STDOUT->gzip compression->encryption->file will cause everything printed to STDOUT to pass gzip compression, encryption and eventually write to file).

在您的原始示例中，您正在写入管道的 end，这意味着无需操作即可写入文件，因此您获得了要求写入的纯 ASCII.这里的困惑在于 myStream 是什么.您假设它是整个链(gzip->file)，但实际上，它只是结尾(文件).

In your original example, you are writing to the end of the pipe, this means writing to the file with no manipulations and hence you got the plain ASCII you asked to write. The confusion here is about what myStream is. You assumed it is the entire chain (gzip->file) but in fact, it is just the end (file).

一旦将管道设置为流对象，当您写入原始流时，对该流的所有进一步写入都将自动通过管道.

Once a pipe is set to a stream object, all further writes to that stream will pipe through automatically when you write to the original stream.

我认为有用的一些参考资料:
http://codewinds.com/blog/2013-08-20-nodejs-transform-streams.html#what_are_transform_streams_

Some references I found useful:
http://codewinds.com/blog/2013-08-20-nodejs-transform-streams.html#what_are_transform_streams_

http://www.sitepoint.com/basics-node-js-streams/

https://nodejs.org/api/process.html

这篇关于流式传输时节点 Zlib 创建无效的 .gz 文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！