我正在尝试将我的Glue目录连接到EMR中的Presto和Hive。在presto-cli中运行查询时，我收到NullPointerException，而则同一查询在hive-cli中成功。
开始如下的cli
presto-cli --catalog hive
执行查询的异常:
Query 20180814_174636_00003_iika5 failed: java.lang.NullPointerException: parameters is null
EMR配置如下所示:
[
{
"classification": "presto-connector-hive",
"properties": {
"hive.metastore": "glue"
},
"configurations": []
},
{
"classification": "hive-site",
"properties": {
"hive.metastore.client.factory.class": "com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory"
},
"configurations": []
}
]
EMR版本:
5.16.0 Presto版本: 0.203参考文件:https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto-glue.html
调试日志
Query 20180816_060942_00001_m9i52 failed: java.lang.NullPointerException: parameters is null
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.NullPointerException: parameters is null
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2052)
at com.google.common.cache.LocalCache.get(LocalCache.java:3943)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3967)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4952)
at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:4958)
at com.facebook.presto.hive.metastore.CachingHiveMetastore.get(CachingHiveMetastore.java:207)
at com.facebook.presto.hive.metastore.CachingHiveMetastore.getPartitionNamesByParts(CachingHiveMetastore.java:499)
at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.doGetPartitionNames(SemiTransactionalHiveMetastore.java:467)
at com.facebook.presto.hive.metastore.SemiTransactionalHiveMetastore.getPartitionNamesByParts(SemiTransactionalHiveMetastore.java:445)
at com.facebook.presto.hive.HivePartitionManager.getFilteredPartitionNames(HivePartitionManager.java:284)
at com.facebook.presto.hive.HivePartitionManager.getPartitions(HivePartitionManager.java:146)
at com.facebook.presto.hive.HiveMetadata.getTableLayouts(HiveMetadata.java:1305)
at com.facebook.presto.spi.connector.classloader.ClassLoaderSafeConnectorMetadata.getTableLayouts(ClassLoaderSafeConnectorMetadata.java:73)
at com.facebook.presto.metadata.MetadataManager.getLayouts(MetadataManager.java:346)
at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout.planTableScan(PickTableLayout.java:203)
at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout.access$200(PickTableLayout.java:61)
at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout$PickTableLayoutWithoutPredicate.apply(PickTableLayout.java:186)
at com.facebook.presto.sql.planner.iterative.rule.PickTableLayout$PickTableLayoutWithoutPredicate.apply(PickTableLayout.java:153)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.transform(IterativeOptimizer.java:168)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreNode(IterativeOptimizer.java:141)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:104)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:193)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:106)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:193)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:106)
at com.facebook.presto.sql.planner.iterative.IterativeOptimizer.optimize(IterativeOptimizer.java:95)
at com.facebook.presto.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:140)
at com.facebook.presto.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:129)
at com.facebook.presto.execution.SqlQueryExecution.doAnalyzeQuery(SqlQueryExecution.java:327)
at com.facebook.presto.execution.SqlQueryExecution.analyzeQuery(SqlQueryExecution.java:312)
at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:268)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: parameters is null
at java.util.Objects.requireNonNull(Objects.java:228)
at com.facebook.presto.hive.metastore.Partition.<init>(Partition.java:54)
at com.facebook.presto.hive.metastore.Partition$Builder.build(Partition.java:180)
at com.facebook.presto.hive.metastore.glue.converter.GlueToPrestoConverter.convertPartition(GlueToPrestoConverter.java:141)
at com.facebook.presto.hive.metastore.glue.GlueHiveMetastore.lambda$getPartitions$8(GlueHiveMetastore.java:558)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at com.facebook.presto.hive.metastore.glue.GlueHiveMetastore.getPartitions(GlueHiveMetastore.java:558)
at com.facebook.presto.hive.metastore.glue.GlueHiveMetastore.getPartitionNamesByParts(GlueHiveMetastore.java:541)
at com.facebook.presto.hive.metastore.CachingHiveMetastore.loadPartitionNamesByParts(CachingHiveMetastore.java:504)
at com.google.common.cache.CacheLoader$FunctionToCacheLoader.load(CacheLoader.java:165)
at com.google.common.cache.CacheLoader$1.load(CacheLoader.java:188)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3524)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2273)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2156)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2046)
... 33 more
最佳答案
好像presto 0.203就有这个bug，我也遇到了，我切换到了新版本，它起作用了。
在我编写此答案时，EMR 5.17已发布，并且已存储为0.206，从而解决了此问题。
关于amazon-emr - Presto-Glue-EMR集成:presto-cli提供NullPointerException，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/51847204/