本文介绍了我如何在Java中对Hive进行异步调用?的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我想以异步方式在服务器上执行Hive查询。 Hive查询可能需要很长时间才能完成,因此我不希望阻止该呼叫。我目前使用Thirft进行阻塞调用(在client.execute()上的阻塞),但我还没有看到如何进行非阻塞调用的示例。这里是阻止代码:
TSocket transport = new TSocket(hive.example.com,10000);
transport.setTimeout(999999999);
TBinaryProtocol protocol = new TBinaryProtocol(transport);
客户端客户端=新ThriftHive.Client(协议);
transport.open();
client.execute(hql); //省略HQL
List< String>行; $(行= client.fetchN(1000))!= null){
(字符串行:rows){
//做行的元素
}
}
transport.close();
上面的代码缺少try / catch块来保持简短。
有人有任何想法如何做一个异步调用? Hive / Thrift可以支持它吗?有没有更好的方法?
谢谢! 解决方案
到Hive邮件列表,Hive不支持使用Thirft的异步调用。
I would like to execute a Hive query on the server in an asynchronous manner. The Hive query will likely take a long time to complete, so I would prefer not to block on the call. I am currently using Thirft to make a blocking call (blocks on client.execute()), but I have not seen an example of how to make a non-blocking call. Here is the blocking code:
TSocket transport = new TSocket("hive.example.com", 10000);
transport.setTimeout(999999999);
TBinaryProtocol protocol = new TBinaryProtocol(transport);
Client client = new ThriftHive.Client(protocol);
transport.open();
client.execute(hql); // Omitted HQL
List<String> rows;
while ((rows = client.fetchN(1000)) != null) {
for (String row : rows) {
// Do stuff with row
}
}
transport.close();
The code above is missing try/catch blocks to keep it short.
Does anyone have any ideas how to do an async call? Can Hive/Thrift support it? Is there a better way?
Thanks!
解决方案
After talking to the Hive mailing list, Hive does not support async calls using Thirft.
这篇关于我如何在Java中对Hive进行异步调用?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!