我正试图使用此代码(在stackoverflow上找到)删除无效的utf-8字符:
def text = file.text
CharsetDecoder utf8Decoder = Charset.forName("UTF-8").newDecoder();
utf8Decoder.onMalformedInput(CodingErrorAction.IGNORE);
utf8Decoder.onUnmappableCharacter(CodingErrorAction.IGNORE);
ByteBuffer bytes = ByteBuffer.allocate(text.getBytes().length * 2)
CharBuffer cbuf = bytes.asCharBuffer()
cbuf.put(text)
cbuf.flip()
CharBuffer parsed = utf8Decoder.decode(bytes);
println parsed.toString()
我得到的输出如下:
< d o c u m e n t >
< t i t l e > S o me T i t l e < / t i t l e >
< s i t e > A S i t e < / s i t e >
你知道它为什么会这样吗?
最佳答案
不知道为什么这不起作用,但这就是解决问题的方法(代码是用groovy编写的,而不是用java编写的):
file.withInputStream { stream ->
CharsetDecoder utf8Decoder = Charset.forName("UTF-8").newDecoder();
utf8Decoder.onMalformedInput(CodingErrorAction.IGNORE);
utf8Decoder.onUnmappableCharacter(CodingErrorAction.IGNORE);
def reader = new BufferedReader(new InputStreamReader(stream, utf8Decoder))
def line = null
def sb = new StringBuilder()
while ( (line = reader.readLine()) != null) {
sb.append("$line\n")
}
reader.close()
}