本文介绍了Spark Streaming和Phoenix Kerberos问题的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

对于这个问题抱歉,因为这已经引起了无数次的讨论,但是我仍然无法在阅读每篇相关文章后找到解决方案。
我在HDP 2.4.2上的Kerberos env上使用Spark Streaming(1.6.1)和Phoenix(4.4),当尝试从HBase读取或写入时获取低于异常。即使跳过spark-submit中的key-ph.conf文件,我也会遇到同样的问题。



我在下面的帖子中看到了与我的问题相同的问题,但仍然存在我无法为我的问题找到解决方案:





Spark无法连接到安全凤凰

以下是我的Spark-submit命令。

  spark-submit \ 
--verbose \
- 主管簇群\
- 执行程序2 \
- 执行程序 - 内存8g \
--executor-cores 4 \
--conf spark.driver.memory = 1024m \
--files key-ph.conf#key-ph.conf,user.headless .keytab#user.headless.keytab,在/ etc / HBase的/ 2.4.2.0 -258 / 0 / hbase-site.xml \
--jars /usr/hdp/2.4.2.0-258/hbase/lib/hbase-common-1.1.2.2.4.2.0-258.jar, /usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar,/usr/hdp/2.4.2.0-258/hbase/lib/hbase-server- 1.1.2.2.4.2.0-258.jar,在/ usr / HDP / 2.4.2.0-258 / HBase的/ LIB / HBase的 - 协议 - 1.1.2.2.4.2.0-258.jar,在/ usr / HDP / 2.4。 2.0-258 / HBase的/ LIB / HTRACE核-3.1.0-incubating.jar,在/ usr / HDP / 2.4.2.0-258 / HBase的/ LIB /番石榴12.0.1.jar,在/ usr / HDP / 2.4。 2.0-258 /凤凰/ LIB /凤芯4.4.0.2.4.2.0-258.jar,在/ usr / HDP / 2.4.2.0-258 /凤凰/凤4.4.0.2.4.2.0-258客户端-spark.jar \
--driver-java-options-Djava.security.auth.login.config =。/ key-ph.conf -Dhttp.proxyHost = proxy-host -Dhttp.proxyPort = 8080 -Dhttps.proxyHost = proxy-host -Dhttps.proxyPort = 8080 -Dlog4j.configuration = file:/home/user/spark-log4j/log4j-phoenix-driver.properties\
--confspark。 executor.extraJavaOptions = -Djava.security.auth.login.config = ./key-ph.conf -Dlog4j.configuration = file:/ home / user / spark-log4j / log4j-phoenix-executor。属性\
--class com.spark.demo.SampleInsert /home/user/test-ph.jar tableName ZK_IP:2181:/ hbase-secure:user@CLIENT.LAN:/ home / user / user .headless.keytab

Spark代码:

  demoArrDataFrame.write 
.format(org.apache.phoenix.spark)
.options(Map(table - > tableName.toUpperCase,
zkUrl - > ZK_IP:2181:/ hbase-secure:user@FORSYS.LAN:/home/user/user.headless.keytab))
.mode(SaveMode.Overwrite)
.save

16/12/05 16:11:36 WARR AbstractRpcClient:连接到服务器时遇到异常:javax.security.sasl.SaslException:GSS启动失败[由GSSException引起:没有提供有效的凭据(机制级别:失败找到任何Kerberos tgt)]
16/12/05 16:11:36错误AbstractRpcClient:SASL身份验证失败。最可能的原因是丢失或无效的凭证。考虑'kinit'。
javax.security.sasl.SaslException:GSS启动失败[由GSSException引起:没有提供有效的凭据(机制级别:无法找到任何Kerberos tgt)]
at com.sun.security.sasl.gsskerb .GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
at org.apache.hadoop.hbase.ipc .RpcClientImpl $ Connection.setupSaslConnection(RpcClientImpl.java:611)
at org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.access $ 600(RpcClientImpl.java:156)
at org.apache。 hadoop.hbase.ipc.RpcClientImpl $ $连接2.run(RpcClientImpl.java:737)在org.apache.hadoop.hbase.ipc.RpcClientImpl $ $连接2.run
(RpcClientImpl.java:734)
在java.security.AccessController.doPrivileged(Native方法)$ b $在javax.security.auth.Subject.doAs(Subject.java:422)
在org.apache.hadoop.security.UserGroupInformation.doAs (UserGroupInformation.java:1709)在org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.setupIOstreams
(RpcClientImpl.java:734)
在org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.writeRequest(RpcClientImpl。 Java的:887)
在org.apache.hadoop.hbase.ipc.RpcClientImpl $ Connection.tracedWriteRequest(RpcClientImpl.java:856)
在org.apache.hadoop.hbase.ipc.RpcClientImpl.call( RpcClientImpl.java:1200)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient $ BlockingRpcChannelImplementation。 callBlockingMethod(AbstractRpcClient.java:287)
at org.apache.hadoop.hbase.protobuf.generated.MasterProtos $ MasterService $ BlockingStub.isMasterRunning(MasterProtos.java:58152)
at org.apache.hadoop。 hbase.client.ConnectionManager $ HConnectionImplementation $ MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1571)
at org.apache.hadoop.hbase.client.ConnectionManager $ HConnec tionImplementation $ StubMaker.makeStubNoRetries(ConnectionManager.java:1509)
at org.apache.hadoop.hbase.client.ConnectionManager $ HConnectionImplementation $ StubMaker.makeStub(ConnectionManager.java:1531)
at org.apache。 hadoop.hbase.client.ConnectionManager $ HConnectionImplementation $ MasterServiceStubMaker.makeStub(ConnectionManager.java:1560)
在org.apache.hadoop.hbase.client.ConnectionManager $ HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1711)
。在org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
在org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:528)$ b $ org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4083)
b在org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:550)
在org.apache.p hoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:810)
处org.apache.phoenix.query org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1174)
。 DelegateConnectionQueryServices.createTable(DelegateConnectionQueryServices.java:112)
at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1974)
at org.apache.phoenix.schema.MetaDataClient.createTable( MetaDataClient.java:770)
在org.apache.phoenix.compile.CreateTableCompiler $ 2.execute(CreateTableCompiler.java:186)
在org.apache.phoenix.jdbc.PhoenixStatement $ 2.call(PhoenixStatement。 java:305)
at org.apache.phoenix.jdbc.PhoenixStatement $ 2.call(PhoenixStatement.java:297)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53 )
在org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:295)
在org.apache.phoenix.jdb c.PhoenixStatement.executeUpdate(PhoenixStatement.java:1244)
在org.apache.phoenix.query.ConnectionQueryServicesImpl $ 12.call(ConnectionQueryServicesImpl.java:1850)
在org.apache.phoenix.query.ConnectionQueryServicesImpl $ 12.call(ConnectionQueryServicesImpl.java:1819)
在org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)
在org.apache.phoenix.query.ConnectionQueryServicesImpl.init(
at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:180)
at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:
at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:151)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java .sql.DriverManager.getConnection(DriverManager.java:208)
at org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(Connecti onUtil.java:99)
at org.apache.phoenix.mapreduce.util.ConnectionUtil.getOutputConnection(ConnectionUtil.java:82)
at org.apache.phoenix.mapreduce.util.ConnectionUtil.getOutputConnection( ConnectionUtil.java:70)
在org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.getUpsertColumnMetadataList(PhoenixConfigurationUtil.java:232)
在org.apache.phoenix.spark.DataFrameFunctions $$ anonfun $ 2。应用(DataFrameFunctions.scala:45)
在org.apache.phoenix.spark.DataFrameFunctions $$ anonfun $ 2.apply(DataFrameFunctions.scala:41)
at org.apache.spark.rdd.RDD $ $ anonfun $ mapPartitions $ 1 $$ anonfun $ apply $ 22.apply(RDD.scala:717)
at org.apache.spark.rdd.RDD $$ anonfun $ mapPartitions $ 1 $$ anonfun $ apply $ 22.apply(RDD .scala:717)
在org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
在org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:313 )org.apache.spark.rdd.RDD.iterator
(RDD.scala:277)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
在org.apache.spark.scheduler.Task.run(Task.scala:89)
在org.apache。 spark.executor.Executor $ TaskRunner.run(Executor.scala:214)
。在java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
。在java.util.concurrent.ThreadPoolExecutor中$ Worker.run(ThreadPoolExecutor.java:617)在java.lang.Thread.run处的
(Thread.java:745)
引起:GSSException:没有提供有效的凭据(机制级别:无法找到任何Kerberos tgt)
at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
。在sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
处太阳sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
。 security.jgss.GSSContextImpl.initSecContext(GSS ContextImpl.java:212)
at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java: 192)
... 62 more


我可以通过执行以下步骤来解决此问题:
$ b

1)将所需的hbase,phoenix jars传递到spark额外classpath选项:

   -  confspark.executor.extraClassPath = / usr / hdp / 2.4.2.0-258 / hbase / lib / hbase-common-1.1.2.2。 4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/ HBase的/ lib目录/ HBase的服务器 -  1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-protocol-1.1.2.2.4.2.0-258.jar :/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.2.0-258/hbase/lib/guava-12.0.1.jar :/usr/hdp/current/spark-client/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar:在/ usr / HDP / CURR ent / phoenix-client / phoenix-client.jar\ 

--confspark.driver.extraClassPath = / usr / hdp / 2.4.2.0-258 / hbase / lib / hbase-common -1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4 .2.0-258 / HBase的/ lib目录/ HBase的服务器 - 1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-protocol-1.1.2.2.4.2。 0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.2.0-258/hbase/lib/guava- 12.0.1.jar:/usr/hdp/current/spark-client/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar:在/ usr / HDP /当前/ phoenix-client / phoenix-client.jar\


$ b 2)有效的keytab和jaas conf在Spark额外的java选项中:

   -  confspark.driver.extraJavaOptions = -XX:+ UseG1GC -Djava.security。 auth.login.config = ./ kafka_jaas.conf -Dhttp.proxyHost = PROXY.IP -Dhttp.proxyPort = 8080 -Dhttps.proxyHost = PROXY.IP2 -Dhttps.proxyPort = 8080\ 

--confspark.executor.extraJavaOp tions = -XX:+ UseG1GC -Djava.security.auth.login.config =。/ kafka_jaas.conf\

kafka-jaas.conf文件为例:

  KafkaServer {
com.sun.security .auth.module.Krb5LoginModule需要
useKeyTab = true
keyTab =/ etc / security / keytabs / kafka.service.keytab
storeKey = true
useTicketCache = false
serviceName =kafka
principal =kafka/IP@REALM.LAN;
};
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache = true
renewTicket = true
serviceName =kafka;
};
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab = true
keyTab =/ etc / security / keytabs / kafka.service.keytab
storeKey = true
useTicketCache = false
serviceName =zookeeper
principal =kafka/IP@REALM.LAN;
};



配置 - 文件火花选项

   - 文件kafka_jaas.conf#kafka_jaas.conf,user.headless.keytab#user.headless.keytab,/ etc / hbase / conf / hbase-site.xml#hbase-site .xml \ 

如果您仍然无法连接到安全凤凰,请执行以下步骤。

  export SPARK_CLASSPATH = / usr / hdp / 2.4.2.0- 258 / HBase的/ LIB / HBase的-共1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258的.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-server-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase - 方案 -  1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.2.0 -258 / HBase的/ LIB /番石榴12.0.1.jar:/usr/hdp/current/spark-client/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0- 258.jar:/usr/hdp/current/phoenix-client/phoenix-client.jar 


Sorry for this question as this has been raised numerous time on SO, But I am still not able to find a solution for my issue after going through each relevant post.I am using Spark Streaming (1.6.1) with Phoenix (4.4) on Kerberos env on HDP 2.4.2 getting below exception when try to read or write from HBase. I get the same issue even after skipping key-ph.conf file from spark-submit.

I had a look on below post which has identical problem as mine but still I am not able to find solution for my issue:

https://community.hortonworks.com/questions/56848/spark-cant-connect-to-secure-phoenix.html

Spark can't connect to secure phoenix

Below is my Spark-submit command.

spark-submit \
--verbose \
--master yarn-cluster \
--num-executors 2  \
--executor-memory 8g \
--executor-cores 4 \
--conf spark.driver.memory=1024m  \
--files key-ph.conf#key-ph.conf,user.headless.keytab#user.headless.keytab,/etc/hbase/2.4.2.0-258/0/hbase-site.xml \
--jars /usr/hdp/2.4.2.0-258/hbase/lib/hbase-common-1.1.2.2.4.2.0-258.jar,/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar,/usr/hdp/2.4.2.0-258/hbase/lib/hbase-server-1.1.2.2.4.2.0-258.jar,/usr/hdp/2.4.2.0-258/hbase/lib/hbase-protocol-1.1.2.2.4.2.0-258.jar,/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar,/usr/hdp/2.4.2.0-258/hbase/lib/guava-12.0.1.jar,/usr/hdp/2.4.2.0-258/phoenix/lib/phoenix-core-4.4.0.2.4.2.0-258.jar,/usr/hdp/2.4.2.0-258/phoenix/phoenix-4.4.0.2.4.2.0-258-client-spark.jar \
--driver-java-options "-Djava.security.auth.login.config=./key-ph.conf -Dhttp.proxyHost=proxy-host -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy-host -Dhttps.proxyPort=8080 -Dlog4j.configuration=file:/home/user/spark-log4j/log4j-phoenix-driver.properties" \
--conf "spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./key-ph.conf -Dlog4j.configuration=file:/home/user/spark-log4j/log4j-phoenix-executor.properties" \
--class com.spark.demo.SampleInsert /home/user/test-ph.jar tableName ZK_IP:2181:/hbase-secure:user@CLIENT.LAN:/home/user/user.headless.keytab

Spark Code:

 demoArrDataFrame.write
        .format("org.apache.phoenix.spark")
        .options(Map("table" -> tableName.toUpperCase,
          "zkUrl" -> "ZK_IP:2181:/hbase-secure:user@FORSYS.LAN:/home/user/user.headless.keytab"))
        .mode(SaveMode.Overwrite)
        .save

16/12/05 16:11:36 WARN AbstractRpcClient: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
16/12/05 16:11:36 ERROR AbstractRpcClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
    at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:211)
    at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:611)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:156)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:737)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:734)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:734)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:887)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:856)
    at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1200)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
    at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
    at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:58152)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.isMasterRunning(ConnectionManager.java:1571)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStubNoRetries(ConnectionManager.java:1509)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1531)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1560)
    at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1711)
    at org.apache.hadoop.hbase.client.MasterCallable.prepare(MasterCallable.java:38)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:124)
    at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:4083)
    at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:528)
    at org.apache.hadoop.hbase.client.HBaseAdmin.getTableDescriptor(HBaseAdmin.java:550)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:810)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1174)
    at org.apache.phoenix.query.DelegateConnectionQueryServices.createTable(DelegateConnectionQueryServices.java:112)
    at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:1974)
    at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:770)
    at org.apache.phoenix.compile.CreateTableCompiler$2.execute(CreateTableCompiler.java:186)
    at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:305)
    at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:297)
    at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:295)
    at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1244)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1850)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:1819)
    at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:77)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:1819)
    at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:180)
    at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.connect(PhoenixEmbeddedDriver.java:132)
    at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:151)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:208)
    at org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(ConnectionUtil.java:99)
    at org.apache.phoenix.mapreduce.util.ConnectionUtil.getOutputConnection(ConnectionUtil.java:82)
    at org.apache.phoenix.mapreduce.util.ConnectionUtil.getOutputConnection(ConnectionUtil.java:70)
    at org.apache.phoenix.mapreduce.util.PhoenixConfigurationUtil.getUpsertColumnMetadataList(PhoenixConfigurationUtil.java:232)
    at org.apache.phoenix.spark.DataFrameFunctions$$anonfun$2.apply(DataFrameFunctions.scala:45)
    at org.apache.phoenix.spark.DataFrameFunctions$$anonfun$2.apply(DataFrameFunctions.scala:41)
    at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$22.apply(RDD.scala:717)
    at org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$22.apply(RDD.scala:717)
    at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:313)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:277)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
    at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
    at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
    at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
    at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
    at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
    at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192)
    ... 62 more
解决方案

I am able to fix this issue by performing below steps as follows:

1) Pass required hbase,phoenix jars into spark extra classpath option:

--conf "spark.executor.extraClassPath=/usr/hdp/2.4.2.0-258/hbase/lib/hbase-common-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-server-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-protocol-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.2.0-258/hbase/lib/guava-12.0.1.jar:/usr/hdp/current/spark-client/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar:/usr/hdp/current/phoenix-client/phoenix-client.jar" \

--conf "spark.driver.extraClassPath=/usr/hdp/2.4.2.0-258/hbase/lib/hbase-common-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-server-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-protocol-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.2.0-258/hbase/lib/guava-12.0.1.jar:/usr/hdp/current/spark-client/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar:/usr/hdp/current/phoenix-client/phoenix-client.jar" \

2) Valid keytab and jaas conf in spark extra java options:

--conf "spark.driver.extraJavaOptions=-XX:+UseG1GC -Djava.security.auth.login.config=./kafka_jaas.conf -Dhttp.proxyHost=PROXY.IP -Dhttp.proxyPort=8080 -Dhttps.proxyHost=PROXY.IP2 -Dhttps.proxyPort=8080" \

--conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -Djava.security.auth.login.config=./kafka_jaas.conf" \

kafka-jaas.conf file as an example:

KafkaServer {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/etc/security/keytabs/kafka.service.keytab"
   storeKey=true
   useTicketCache=false
   serviceName="kafka"
   principal="kafka/IP@REALM.LAN";
};
KafkaClient {
   com.sun.security.auth.module.Krb5LoginModule required
   useTicketCache=true
   renewTicket=true
   serviceName="kafka";
};
Client {
   com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="/etc/security/keytabs/kafka.service.keytab"
   storeKey=true
   useTicketCache=false
   serviceName="zookeeper"
   principal="kafka/IP@REALM.LAN";
};

3) Configuring --files spark options

--files kafka_jaas.conf#kafka_jaas.conf,user.headless.keytab#user.headless.keytab,/etc/hbase/conf/hbase-site.xml#hbase-site.xml \

After following the steps if you are still not able to connect to secured phoenix. Please set below SPARK_CLASSPATH and then run/execute spark-submit command.

export SPARK_CLASSPATH=/usr/hdp/2.4.2.0-258/hbase/lib/hbase-common-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-client-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-server-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/hbase-protocol-1.1.2.2.4.2.0-258.jar:/usr/hdp/2.4.2.0-258/hbase/lib/htrace-core-3.1.0-incubating.jar:/usr/hdp/2.4.2.0-258/hbase/lib/guava-12.0.1.jar:/usr/hdp/current/spark-client/lib/spark-assembly-1.6.1.2.4.2.0-258-hadoop2.7.1.2.4.2.0-258.jar:/usr/hdp/current/phoenix-client/phoenix-client.jar

这篇关于Spark Streaming和Phoenix Kerberos问题的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

09-09 10:26
查看更多