我们需要在通过Webhdfs将文件移动到Hadoop（HDFS）之后验证校验和吗？我想确保文件在复制后HDFS没有损坏。但是检查校验和是否必要？
在数据写入HDFS之前，我读取客户端校验和
有人可以帮忙我了解如何确保Linux系统上的源文件与使用webhdfs的Hdfs中的文件相同。
解决方案Checksum可以使用hadoop fs命令来计算文件。
用法：hadoop fs -checksum URI
返回
示例：
hadoop fs -checksum hdfs：//nn1.example。 com / file1
hadoop fs -checksum file：/// path / in / linux / file1
请参阅：了解更多详情
所以如果你想在linux和hdfs中comapre file1，你可以使用上面的工具。
Do we need to verify checksum after we move files to Hadoop (HDFS) from a Linux server through a Webhdfs ?
I would like to make sure the files on the HDFS have no corruption after they are copied. But is checking checksum necessary?
I read client does checksum before data is written to HDFS
Can somebody help me to understand how can I make sure that source file on Linux system is same as ingested file on Hdfs using webhdfs.
Checksum for a file can be calculated using hadoop fs command.
Usage: hadoop fs -checksum URI
Returns the checksum information of a file.
Example:
hadoop fs -checksum hdfs://nn1.example.com/file1hadoop fs -checksum file:///path/in/linux/file1
Refer : Hadoop documentation for more details
So if you want to comapre file1 in both linux and hdfs you can use above utility.
这篇关于Hadoop中的校验和验证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！
