问题描述
标准库中是否存在 io.Pipe 的缓冲版本.第三方图书馆)，然后再推出自己的图书?
Is there a buffered version of io.Pipe (either in the standard library or a third-party library) before I roll my own?
上下文:我正在尝试使用此解决方案来解析使用bzip2压缩的JSON数据，以便进行解压缩和解析是并行进行的，但是发现加速非常小.解析未压缩的数据每百万条记录大约需要22秒.解压缩那么多数据大约需要相同的时间.如预期的那样，在单线程上完成它们大约需要44秒.使用上述解决方案大约需要41秒钟.
Context: I'm trying to use this solution for parsing of JSON data compressed with bzip2, so that the decompression and parsing happen in parallel, but finding that the speed-up is very small. Parsing uncompressed data takes ~22 sec per million records. Decompressing that much data takes about the same time. Doing them on a single thread takes ~44 seconds, as expected. Using the solution above takes ~41 seconds.
io.Pipe的文档说:
The documentation for io.Pipe says:
我怀疑这可能是个问题，具体取决于bzip2解压缩器写入数据的方式以及JSON解析器读取数据的方式，因此我想尝试一个缓冲版本.
I suspect this could be a problem, depending on the way the bzip2 decompressor writes data and the way the JSON parser reads it, so I'd like to try a buffered version.
推荐答案
这就是 bufio 包适用于.它允许您使用 NewReader将任何 或任何 io.Reader 转换为缓冲的阅读器. io.Writer 放入具有 NewWriter .
That's what the bufio package is for. It lets you turn any io.Reader into a buffered reader with NewReader, or any io.Writer into a buffered writer with NewWriter.
(缓冲的IO是否真的可以解决您的特定问题，我不知道...)
(Whether buffered IO will actually help with your specific problem, I have no idea...)
这篇关于Go io.Pipe的缓冲版本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！