Hive 中 metastore(元数据存储)的三种方式:
a)内嵌 Derby 方式
b)Local 方式
c)Remote 方式
第一种方式:本地 mysql
注: 这种存储方式需要在本地运行一个 mysql 服务器,并作如下配置,使用 mysql 的方 式,需要将 mysql 的 jar 包拷贝到$HIVE_HOME/lib 目录下
1. 安装mysql, 修改权限
yum install mysqld-server
chkconfig mysqld on 设置开机启动
service mysqld start 启动mysql
进入mysql
mysql
use mysql;
show tables;
desc;
select host, user, password from user;
修改mysql权限
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
delete from user;
flush privileges; 刷新权限
quit
mysql
mysql -u root -p
此时需要输入密码,进入mysql, 如果能够进入,则mysql修改成功
2. 同一台主机上安装 hive
上传hive压缩包,并解压到home目录下
tar zxvf apache-hive-1.2.1-bin.tar.gz -C /home
配置hive环境变量
vi /etc/profile
source /etc/profile
修改hive的配置文件
vi hive-site.xml
hive
发现报如下错误,只需要拷贝mysql的jar包到hive的lib下即可
hive中的高版本替换hadoop中的低版本
hive中的高版本
hadoop中的jline为低版本-->替换为高版本
进入hive中的lib目录,进行copy到hadoop中的lib下
cp jline-2.12.jar /root/hadoop-2.5.1/share/hadoop/yarn/lib
然后 hive 就成功进入了!!!
测试hive
注:使用 derby 存储方式时,运行 hive会在当前目录生成一个 derby 文件和一个metastore_db 目录。这种存储方式的弊端是在同一个目录下同时只能有一个 hive 客户端能使用数据库, 否则会提示如下错误 [html] view plaincopyprint? hive> show tables; FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Failed to start database 'metast ore_db', see the next exception for details. NestedThrowables: java.sql.SQLException: Failed to start database 'metastore_db', see the next exception for details. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask hive> show tables; FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Failed to start database 'metastore_db', see the next exception for details. NestedThrowables: java.sql.SQLException: Failed to start database 'metastore_db', see the next exception for details. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
第二种方式:远端mysql--remote分开 亲测安装
假定node1为客户端,node2为服务端
node1和node2上传hive压缩包,并解压到home路径下,配置hive环境变量
服务端node2:服务端需要mysql jar包
拷贝mysql-connecor-java.jar到hive的lib下
cd conf
mv hive-default.template.xml hive-site.xml
vi hive-site.xml
<configuration> <property>
<name>hive.metastore.warehouse.dir</name>
<value>/opt/hive/warehouse</value>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.1.33:3306/hive?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property> <property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property> <property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property> <property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value>
<description>password to use against metastore database</description>
</property>
</configuration>
客户端node1:
首先替换hadoop中jline的版本为hive中的高版本,步骤参考第一种方式
cd conf
mv hive-default.template.xml hive-site.xml
vi hive-site.xml
<configuration>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/opt/hive/warehouse</value>
</property>
<property>
<name>hive.metastore.local</name>
<value>false</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value>thrift://node1:9083</value>
</property>
</configuration>
客户端服务端修改hive-env.sh配置其中的$hadoop_home
测试hive:
node2:
hive --service metastore
node1:
hive
create table tbl (id int, name string);
insert into tbl values(1, 'zhangsan');
select * from tbl;
第三种方式:本地derby
这种方式是最简单的存储方式,只需要在 hive-site.xml 做如下配置便可
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=metastore_db;create=true</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
</property>
<property>
<name>hive.metastore.local</name>
<value>true</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
</configuration>
注:使用 derby 存储方式时,运行 hive会在当前目录生成一个 derby 文件和一个metastore_db 目录。这种存储方式的弊端是在同一个目录下同时只能有一个 hive 客户端能使用数据库, 否则会提示如下错误 [html] view plaincopyprint? hive> show tables; FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Failed to start database 'metast ore_db', see the next exception for details. NestedThrowables: java.sql.SQLException: Failed to start database 'metastore_db', see the next exception for details. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask hive> show tables; FAILED: Error in metadata: javax.jdo.JDOFatalDataStoreException: Failed to start database 'metastore_db', see the next exception for details. NestedThrowables: java.sql.SQLException: Failed to start database 'metastore_db', see the next exception for details. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask