我正在使用Executorsevice从数据库生成文件。我正在使用jdbc和核心java将表数据转换为文件。

用10个线程创建Executorservice后,我在for循环中提交60个线程以并行获取60个文件。这适用于小数据和具有很少列的表。但是,如果文件很大且表具有更多列,则正在处理巨大表数据/更多列表的线程将在其他线程完成时停止,而不会在日志中提供任何信息。

ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
        for (String filename : filenames) {
            EachFileThread worker = new EachFileThread(destdir, converter,
                    filename, this);
            executor.execute(worker);
        }
        executor.shutdown();


在每个文件线程内部,我正在读取xml并获取列,表和形成查询,并执行查询并格式化数据并将数据放入文件中

    forTable = (FileData) converter.convertFromXMLToObject( filename + ".xml");
    String query = getQuery(forTable);
    statement = connection.createStatement(ResultSet.TYPE_SCROLL_SENSITIVE, ResultSet.CONCUR_READ_ONLY);
    resultSet = statement.executeQuery(query);
resultSet.setFetchSize(3000);
    WriteData(resultSet, filepath, forTable);(formatting the data from db and then writing to a file)

最佳答案

问题是您没有等待所有作业完成他们的工作。正如@msandiford在评论中建议的那样,应在调用awaitTermination(..)之后添加对shutdown()的调用,就像在https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html的示例shutdownAndAwaitTermination()方法中一样

例如,您可以尝试这样做:

ExecutorService executor = Executors.newFixedThreadPool(THREAD_COUNT);
for (String filename : filenames) {
    EachFileThread worker = new EachFileThread(destdir, converter, filename, this);
    executor.execute(worker);
}
executor.shutdown();
try {
    // Wait a while for existing tasks to terminate
    if (!executor.awaitTermination(60, TimeUnit.SECONDS)) {
        executor.shutdownNow(); // Cancel currently executing tasks
        // Wait a while for tasks to respond to being cancelled
        if (!executor.awaitTermination(60, TimeUnit.SECONDS))
            System.err.println("Executor did not terminate");
    }
} catch (InterruptedException ie) {
    // (Re-)Cancel if current thread also interrupted
    executor.shutdownNow();
    // Preserve interrupt status
    Thread.currentThread().interrupt();
}

10-07 19:04
查看更多