问题描述
我已经配置了启用加密的EMR集群(emrfs-site.xml中的属性)我正在使用dataframe savemode.append写入S3n://my-bucket/path/保存在s3中.
I have configured encryption enabled EMR cluster (properties in emrfs-site.xml)I am using dataframe savemode.append to write into S3n://my-bucket/path/ to save in s3.
但是我看不到该对象对AWS KMS进行了加密.但是，当我从EMR的配置单元中进行简单插入时，我能够看到对象已被aws kms加密.
But I am not able to see the object getting AWS KMS encrypted.However, when I do a simple insert from hive from EMR, I am able to see the objects getting aws kms encrypted.
如何使用sse kms从S3中的数据帧加密文件?
How can I encrypt files from dataframe in S3 using sse kms?
推荐答案
问题是我们正在使用s3a将文件从spark程序保存到EMR. AWS官方不支持在EMR上使用s3a.尽管我们能够在S3中保存数据，但它并未对数据进行加密.我尝试使用s3://和s3n://加密对两者都起作用.
The problem was we were using s3a to save the files from spark program to EMR. AWS officially doesn't support use of s3a on EMR. Though we were able to save data in S3, it was not encrypting the data. I tried using s3:// and s3n:// The encryption works with both.
这篇关于Spark/数据帧中的AWS SSE-KMS加密的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！