问题描述
我是新来的星火和蜂巢。我正在星火V1.0.1与内置的蜂巢(星火安装带有SPARK_HIVE =真SBT / SBT组件/组装)
I am new to Spark and Hive. I am running Spark v1.0.1 with build-in Hive (Spark install with SPARK_HIVE=true sbt/sbt assembly/assembly)
我也配置蜂巢存储Metastore在PostgreSQL数据库的指令:
I also config Hive to store Metastore in PostgreSQL database as instruction:
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_18_4.html
我可以配置蜂房(不建,与星火)来使用PostgreSQL,但我不知道如何得到它与蜂巢Spark中工作
I could config Hive (not build-in with Spark) to use PostgreSQL but I don't know how to get it work with Hive in Spark
在该指令,我知道我需要把链接或PostgreSQL的-了jdbc.jar蜂巢/ lib下,这样蜂巢可能包括PostgreSQL的-JDBC,当它运行
In the instruction, I see that I need to put or link postgresql-jdbc.jar to hive/lib so that Hive could include the postgresql-jdbc when it run
$ sudo yum install postgresql-jdbc
$ ln -s /usr/share/java/postgresql-jdbc.jar /usr/lib/hive/lib/postgresql-jdbc.jar
通过内置的配置单元的星火,我应该在哪里把PostgreSQL的-了jdbc.jar得到它的工作?
With Build-in Hive in Spark, where should I put the postgresql-jdbc.jar to get it work?
推荐答案
我找到我的问题的解决方案。我需要添加 CLASSPATH
为 SPARK
,使内置的蜂巢可以使用 PostgreSQL相关jdbc4的.jar
I find the solution for my problem. I need to add CLASSPATH
for SPARK
so that build-in Hive could use postgresql-jdbc4.jar
我加3的环境变量:
export CLASSPATH="$CLASSPATH:/usr/share/java/postgresql-jdbc4.jar"
export SPARK_CLASSPATH=$CLASSPATH
export SPARK_SUBMIT_CLASSPATH=$CLASSPATH
SPARK_CLASSPATH 用于火花壳
SPARK_SUBMIT_CLASSPATH 用于火花提交(我不知道)
现在我可以用火花壳
与内置的蜂巢要使用的配置 Metastore Postgres里
Now I could use spark-shell
with build-in Hive which config to use Metastore in Postgres
这篇关于星火运行与内置的配置单元和配置为配置单元Metastore远程PostgreSQL数据库的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!