我正在使用Executorsevice从数据库生成文件。我正在使用jdbc
和核心java
将表数据转换为文件。
用10个线程创建Executorservice后,我在for循环中提交60个线程以并行获取60个文件。这适用于小数据和具有很少列的表。但是,如果文件很大且表具有更多列,则正在处理巨大表数据/更多列表的线程将在其他线程完成时停止,而不会在日志中提供任何信息。
ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
for (String filename : filenames) {
EachFileThread worker = new EachFileThread(destdir, converter,
filename, this);
executor.execute(worker);
}
executor.shutdown();
在每个文件线程内部,我正在读取xml并获取列,表和形成查询,并执行查询并格式化数据并将数据放入文件中
forTable = (FileData) converter.convertFromXMLToObject( filename + ".xml");
String query = getQuery(forTable);
statement = connection.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
resultSet = statement.executeQuery(query);
resultSet.setFetchSize(3000);
WriteData(resultSet, filepath, forTable);(formatting the data from db and then writing to a file)
最佳答案
问题是您没有等待所有作业完成他们的工作。正如@msandiford在评论中建议的那样,应在调用awaitTermination(..)
之后添加对shutdown()
的调用,就像在https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html的示例shutdownAndAwaitTermination()
方法中一样
例如,您可以尝试这样做:
ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
for (String filename : filenames) {
EachFileThread worker = new EachFileThread(destdir, converter, filename, this);
executor.execute(worker);
}
executor.shutdown();
try {
// Wait a while for existing tasks to terminate
if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
executor.shutdownNow(); // Cancel currently executing tasks
// Wait a while for tasks to respond to being cancelled
if (!executor.awaitTermination(60, TimeUnit.SECONDS))
System.err.println("Executor did not terminate");
}
} catch (InterruptedException ie) {
// (Re-)Cancel if current thread also interrupted
executor.shutdownNow();
// Preserve interrupt status
Thread.currentThread().interrupt();
}