我有以下方法可以分段上传到amazonS3：
public static void uploadToS3UserPath(String filePath, String bucket, String userPath) {
ClientConfiguration config = new ClientConfiguration();
config.setProxyHost("host");
config.setProxyPort(3128);
config.setProtocol(Protocol.HTTP);
TransferManager tm = new TransferManager(new AmazonS3Client(credentials, config));
TransferManagerConfiguration conf = new TransferManagerConfiguration();
conf.setMinimumUploadPartSize(50 * 1024 * 1024); //use 50 megabytes parts;
tm.setConfiguration(conf);
PutObjectRequest req = new PutObjectRequest(bucket, userPath, new File(filePath));
req.setCannedAcl(CannedAccessControlList.PublicRead);
req.setStorageClass(StorageClass.ReducedRedundancy); //we do it to decrease storage costs
Upload up = tm.upload(req);
try {
up.waitForCompletion();
tm.shutdownNow();
System.out.println("Upload completed successfully");
} catch (AmazonClientException ex) {
Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, ex.getLocalizedMessage(), ex);
tm.shutdownNow();
} catch (InterruptedException ex) {
Logger.getLogger(Utilities.class.getName()).log(Level.SEVERE, ex.getLocalizedMessage(), ex);
}
}
有时候我会得到：
com.amazonaws.AmazonClientException: Unable to complete transfer: null
at
com.amazonaws.services.s3.transfer.Transfer.unwrapExecutionException(Transfer.java:226)
at
com.amazonaws.services.s3.transfer.Transfer.rethrowExecutionException(Transfer.java:210)
at
com.amazonaws.services.s3.transfer.Transfer.waitForCompletion(Transfer.java:116)
at
xx.xxx.xxx.xxx.agent.Utilities.uploadToS3UserPath(Utilities.java:373)
at
xx.xxx.xxx.xxx.agent.AbstractAgent.uploadOutputToS3(AbstractAgent.java:312)
at
uk.org.infectogenomics.agent.AbstractAgent.uploadOutputToS3(AbstractAgent.java:305)
at xx.xxx.xxx.xxx.hostEl.HostEl.run(HostEl.java:598)
at xx.xxx.xxx.xxx.agent.Agent.main(Agent.java:98)
Caused by: java.util.concurrent.RejectedExecutionException
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1759)
at
java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at
java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:216)
at
java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:378)
at
com.amazonaws.services.s3.transfer.internal.UploadMonitor.reschedule(UploadMonitor.java:210)
at
com.amazonaws.services.s3.transfer.internal.UploadMonitor.upload(UploadMonitor.java:197)
at
com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:148)
at
com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:49)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
问题是我不知道为什么这只是为了随机上传而表现出来。显然原因是RejectedExecutionException吗？可能是因为我过早关闭了TransferManager吗？
最佳答案
可能是因为我过早关闭了TransferManager吗？
这确实是此行为的触发因素，但是根本原因是AWS SDK for Java中的错误，请参见Eric Milas类似的问题Amazon AWS Java SDK TransferManager issue：
TransferManager包含一个UploadMonitor，而UploadMonitor具有一个
名为timedThreadPoool的静态ScheduledExecutorService。
TransferManager.shutdownNow（）调用静态方法
UploadMonitor.shutdownNow（）调用
timedThreadPoool.shutdownNow（）。下次致电
使得UploadMonitor.reschedule（）的RejectedExecutionException为
抛出。
His own answer将AWS Team响应引用到TransferManager Thread Issue ，这确认了此错误，并且从Release 1.3.2开始已实施相应的修复程序）：
Amazon S3 TransferManager class已更新，以解决两个错误：
从InputStream和
在上传的某些部分遇到IO错误。干净地
在调用shutdownNow（）时关闭计时器线程
TransferManager的实例