我尝试使用自动创建表从MySQL将数据库从mysql导入到Hive,并使用来自sqoop的“--hive import”命令将数据加载到hive。
我使用贝娄命令使用sqoop执行导入
./sqoop-import --connect jdbc:mysql://localhost/paman -table bibis -m 1 -hive-import
执行此命令时:
hadoop@dewi:/opt/sqoop/bin$ ./sqoop-import --connect jdbc:mysql://localhost/paman -table bibis -m 1 -hive-import
12/06/11 16:08:47 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
12/06/11 16:08:47 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
12/06/11 16:08:47 INFO tool.CodeGenTool: Beginning code generation
12/06/11 16:08:47 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
at com.cloudera.sqoop.manager.SqlManager.makeConnection(SqlManager.java:597)
at com.cloudera.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:51)
at com.cloudera.sqoop.manager.MySQLManager.execute(MySQLManager.java:201)
at com.cloudera.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:177)
at com.cloudera.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:161)
at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:908)
at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:337)
at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:423)
at com.cloudera.sqoop.Sqoop.run(Sqoop.java:144)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:180)
at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:218)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:228)
hadoop@dewi:/opt/sqoop/bin$
我的sqoop命令出了什么问题?还是在sqoop或hive中有其他配置?
请帮帮我
最佳答案
首先,sqoop检查 jar 的位置
$SQOOP_HOME/lib
您必须将mysql-connector jar添加到此位置。并且必须在sqoop import命令中包括--hive-table参数,如下所示。
./sqoop-import --connect jdbc:mysql://localhost:<port_number>/paman -table bibis -m 1 -hive-import --hive-table hive_table_name
在这里hive_table_name是动态创建的。