Kotlin可以链接来自不同InputStream的多个序列?

问题描述

说我是否要解析这样的大文件:

Say if I want to parse a large file like this :

val iStream = MyFile::class.java
    .getResourceAsStream("largeFile.txt")

iStream.bufferedReader(Charsets.UTF_8).useLines { lines ->
    lines.filterNot { it.startsWith("#") }
    // parseing
    .toSet()
}

但是如果我想将largeFile拆分为多个较小的文件，该如何链接序列?

But if I want to split the largeFile to multiple smaller files , how to chain the sequences ?

例如:

val seq1 = MyFile::class.java.getResourceAsStream("file1.txt")
    .use { it.bufferedReader(Charsets.UTF_8).lineSequence() }
val seq2 = MyFile::class.java.getResourceAsStream("file2.txt")
    .use { it.bufferedReader(Charsets.UTF_8).lineSequence() }

sequenceOf(seq1, seq2).flatten()
  .filterNot { it.startsWith("#") }
  // parsing
  .toSet()

它将抛出java.io.IOException: Stream closed，这是合理的，因为解析在use块的scope之外.

It will throw java.io.IOException: Stream closed , which is reasonable , because the parsing is outside the scope of the use block.

如何解决问题?

我知道可能会有一些嵌套解决方案(嵌套useLines ...)，但是我认为这很丑陋.还有其他flat解决方案吗?

I know there may be some nesting solution (nesting useLines ... ) , but I think that is ugly . Is there any other flat solutions ?

推荐答案

您可以颠倒您的逻辑.重要的是，所有内容都在use内获取或处理，否则，将无法正常工作.

You could invert your logic. Important is, that everything is got or handled within the use otherwise that will not work, as you already know.

一个这样的〜反转看起来像:

One such ~invertion could look like:

setOf("file1.txt", "file2.txt")
  .map { MyFile::class.java.getResourceAsStream(it) }
  .flatMap {
    it.use {
      it.bufferedReader(Charsets.UTF_8)
        .lineSequence()
        .filterNot { it.startsWith("#") }
        .toSet()
    }
  }

或者，如果您想从外部传递链式转换或过滤器，也许是这样的:

Or if you want to pass the chain transformation or filter from outside, maybe something like:

val handleLine : (Sequence<String>) -> Sequence<String> = {
  it.filterNot { it.startsWith("#") }
  // .map { ... whatever }
}
setOf("file1.txt", "file2.txt")
  .map { MyFile::class.java.getResourceAsStream(it) }
  .flatMap {
    it.use {
      handleLine(it.bufferedReader(Charsets.UTF_8).lineSequence())
        .toSet()
    }
  }

另一种选择是打开流，忽略use并最终自己关闭它们，就像@MarkoTopolnik在评论中指出的那样:

The other alternative is to open up the streams, omit use and finally close them yourself as also @MarkoTopolnik pointed out in the comments:

val inputStreams = sequenceOf("file1.txt", "file2.txt")
  .map { MyFile::class.java.getResourceAsStream(it) }

inputStreams.flatMap { it.bufferedReader(Charsets.UTF_8).lineSequence() }
  .filterNot { it.startsWith("#") }
  .toSet()

然后使用:

inputStreams.forEach(InputStream::close) // but this will fail on the first error...

或安全"方式:

inputStreams.forEach { try { it.close() } catch (e: Exception) { e.printStackTrace() } }

这篇关于Kotlin可以链接来自不同InputStream的多个序列?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！