这是从Google Cloud Storage下载文件的代码:
@Override
public void write(OutputStream outputStream) throws IOException {
try {
LOG.info(path);
InputStream stream = new ByteArrayInputStream(GoogleJsonKey.JSON_KEY.getBytes(StandardCharsets.UTF_8));
StorageOptions options = StorageOptions.newBuilder()
.setProjectId(PROJECT_ID)
.setCredentials(GoogleCredentials.fromStream(stream)).build();
Storage storage = options.getService();
final CountingOutputStream countingOutputStream = new CountingOutputStream(outputStream);
byte[] read = storage.readAllBytes(BlobId.of(BUCKET, path));
countingOutputStream.write(read);
} catch (Exception e) {
e.printStackTrace();
} finally {
outputStream.close();
}
}
这行得通,但是这里的问题是,它必须先缓冲所有字节,然后再流回此方法的客户端。这会导致很多延迟,尤其是当存储在GCS中的文件很大时。
有没有一种方法可以从GCS获取文件,然后将直接将其流传输到OutputStream ,这里的OutputStream是用于Servlet的。
最佳答案
只是为了澄清一下,您需要OutputStream
还是InputStream
?一种查看方式是,将Google Cloud Storage对象中存储的数据作为文件存储,并且您有一个InputStream可以读取该文件。如果可行,请继续阅读。
Storage API中没有提供InputStream
或OutputStream
的现有方法。但是有2 APIs in the Cloud Storage client library公开了ReadChannel
对象,该对象是从ReadableByteChannel
(来自Java NIO API)扩展的。
ReadChannel reader(String bucket, String blob, BlobSourceOption... options);
ReadChannel reader(BlobId blob, BlobSourceOption... options);
一个简单的例子(取自StorageSnippets.java):
/**
* Example of reading a blob's content through a reader.
*/
// [TARGET reader(String, String, BlobSourceOption...)]
// [VARIABLE "my_unique_bucket"]
// [VARIABLE "my_blob_name"]
public void readerFromStrings(String bucketName, String blobName) throws IOException {
// [START readerFromStrings]
try (ReadChannel reader = storage.reader(bucketName, blobName)) {
ByteBuffer bytes = ByteBuffer.allocate(64 * 1024);
while (reader.read(bytes) > 0) {
bytes.flip();
// do something with bytes
bytes.clear();
}
}
// [END readerFromStrings]
}
您还可以使用
newInputStream()
方法将InputStream
包装在ReadableByteChannel
上。public static InputStream newInputStream(ReadableByteChannel ch)
即使您需要
OutputStream
,您也应该能够将数据从InputStream
复制或更好地从ReadChannel
对象复制到OutputStream
中。完整的例子
将此示例运行为:
PROGRAM_NAME <BUCKET_NAME> <BLOB_PATH>
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.Channels;
import java.nio.channels.WritableByteChannel;
import com.google.cloud.ReadChannel;
import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
/**
* An example which reads the contents of the specified object/blob from GCS
* and prints the contents to STDOUT.
*
* Run it as PROGRAM_NAME <BUCKET_NAME> <BLOB_PATH>
*/
public class ReadObjectSample {
private static final int BUFFER_SIZE = 64 * 1024;
public static void main(String[] args) throws IOException {
// Instantiates a Storage client
Storage storage = StorageOptions.getDefaultInstance().getService();
// The name for the GCS bucket
String bucketName = args[0];
// The path of the blob (i.e. GCS object) within the GCS bucket.
String blobPath = args[1];
printBlob(storage, bucketName, blobPath);
}
// Reads from the specified blob present in the GCS bucket and prints the contents to STDOUT.
private static void printBlob(Storage storage, String bucketName, String blobPath) throws IOException {
try (ReadChannel reader = storage.reader(bucketName, blobPath)) {
WritableByteChannel outChannel = Channels.newChannel(System.out);
ByteBuffer bytes = ByteBuffer.allocate(BUFFER_SIZE);
while (reader.read(bytes) > 0) {
bytes.flip();
outChannel.write(bytes);
bytes.clear();
}
}
}
}