R使用RJDBC写表到Hive | R使用RJDBC写表到Hive

本文介绍了R使用RJDBC写表到Hive的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我已经使用rjdbc成功连接了本地R3.1.2(win7 64bit rstudio)和远程hive服务器，

I have successfully connected local R3.1.2( win7 64bit rstudio) and remote hive server using rjdbc,

library(RJDBC)
.jinit()
dir = "E:/xxx/jars/hive/"
for(l in list.files(dir)) {
  .jaddClassPath(paste( dir ,l,sep="")) }
options( java.parameters = "-Xmx8g" )
drv <- JDBC("org.apache.hadoop.hive.jdbc.HiveDriver",
        "E:/xxx/jars/hive/hive-jdbc-0.11.0.jar")

conn <- dbConnect(drv,  "jdbc:hive://10.127.130.162:10002/default", "", "" )
dbGetQuery(conn, "select * from test.test limit 10 ")

已成功从 hive 读取数据，但我无法使用
写入 R 数据帧dbWriteTable :

successfully read data from hive ,but I cannot write R data frame using
dbWriteTable :

data(iris)
dbWriteTable(conn, iris , "test.dc_test")

错误返回:

.jcall(md, "Ljava/sql/ResultSet;", "getTables", .jnull("java/lang/String"), 中的错误:带有签名的方法 getTables (Ljava/lang/String;Ljava/lang/String;[Ljava/lang/String;)Ljava/sql/ResultSet;未找到

是我的误用还是需要其他方法?

Either my misuse or other methods needed?

推荐答案

我有部分答案.您对 dbWriteTable 的参数被颠倒了.模式是 dbWriteTable(connection, tableName, data)，文档读取 dbWriteTable(conn, name, value, ...).话虽如此，我也没有发现正确"的形式有效，而是产生以下错误消息:

I have a partial answer. Your arguments to dbWriteTable are reversed. The pattern is dbWriteTable(connection, tableName, data), the docs read dbWriteTable(conn, name, value, ...). That being said, I don't find that the 'correct' form works either, instead yielding the following error message:

Error in .local(conn, statement, ...) :
  execute JDBC update query failed in dbSendUpdate ([Simba][HiveJDBCDriver](500051) ERROR processing query/statement. Error Code: 40000, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:42000, errorCode:40000, errorMessage:Error while compiling statement: FAILED: ParseException line 1:41 mismatched input 'PRECISION' expecting ) near 'DOUBLE' in create table statement), Query: CREATE TABLE iris (`Sepal.Length` DOUBLE PRECISION,`Sepal.Width` DOUBLE PRECISION,`Petal.Length` DOUBLE PRECISION,`Petal.Width` DOUBLE PRECISION,Species VARCHAR(255)).)

(至少在将 Amazon 的 JDBC 驱动程序用于 Hive 时).该错误至少看起来不言自明，为使数据插入表生成的查询未在 HiveQL 中正确解析.除了手动修复，我不确定.

(at least when using Amazon's JDBC driver for Hive). That error at least seems self apparent, the query generated to make the table for data insertion didn't parse correctly in HiveQL. The fix, other than doing it manually, I'm not sure about.

这篇关于R使用RJDBC写表到Hive的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！