本文介绍了Hadoop和hbase java客户端问题与HbaseRPC的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我试图将数据插入hbase.I从远程计算机运行java程序。

  try {
Configuration conf = HBaseConfiguration.create();
conf.clear();
conf.set(hbase.zookeeper.quorum,< HOST_IP>:2181);
conf.set(hbase.zookeeper.property.clientPort,2181);
conf.set(hbase.zookeeper.dns.nameserver,< HOST_IP>);
conf.set(hbase.regionserver.port,60020);
conf.set(hbase.master,< HOST_IP>:9000);
HTable table = new HTable(conf,test);
Put put = new Put(Bytes.toBytes(row5));
put.add(Bytes.toBytes(colfam1),Bytes.toBytes(qual1),
Bytes.toBytes(val1));
put.add(Bytes.toBytes(colfam1),Bytes.toBytes(qual2),
Bytes.toBytes(val2));
table.put(put);
} catch(MasterNotRunningException e){
e.printStackTrace();
} catch(ZooKeeperConnectionException e){
e.printStackTrace();
} catch(TableNotFoundException e){
e.printStackTrace();
} catch(IOException e){
e.printStackTrace();
}

我收到以下错误:

 信息ipc.HbaseRPC:1次尝试后无法达到localhost / 127.0.0.1:60020处的服务器,放弃。 
org.apache.hadoop.hbase.client.RetriesExhaustedException:在尝试= 1
org后,将代理接口org.apache.hadoop.hbase.ipc.HRegionInterface设置为localhost / 127.0.0.1:60020失败.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:355)
at org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1176)
org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1195)
at org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:898)
at org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.locateRegion(HConnectionManager.java:797)
at org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.relocateRegion(HConnectionManager。 java:772)
在org.apache.hadoop.hbase.client.HConnectionM anager $ HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1002)
位于org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.locateRegion(HConnectionManager.java:801)
位于org.apache.hadoop。 hbase.client.HConnectionManager $ HConnectionImplementation.locateRegion(HConnectionManager.java:766)
at org.apache.hadoop.hbase.client.HTable。< init>(HTable.java:189)
at org .apache.hadoop.hbase.client.HTable。< init>(HTable.java:163)
at com.tcs.hbase.HbaseSample.insertData(HbaseSample.java:30)
at com。 tcs.hbase.HbaseSample.main(HbaseSample.java:82)
导致:java.net.ConnectException:连接被拒绝:没有更多信息
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method )
在sun.nio.ch.SocketChannelImpl.finishConnect(未知源)
在org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
在org.apache .hadoop.net.NetUtils.con nect(NetUtils.java:406)
位于org.apache.hadoop.hbase.ipc.HBaseClient $ Connection.setupIOstreams(HBaseClient.java:328)
位于org.apache.hadoop.hbase.ipc。 HBaseClient.getConnection(HBaseClient.java:883)
位于org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
位于org.apache.hadoop.hbase.ipc。 HBaseRPC $ Invoker.invoke(HBaseRPC.java:257)
at $ Proxy4.getProtocolVersion(Unknown Source)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
... 12 more

当我在安装hbase的机器上运行相同的代码时,它的工作正常..从日志中可以清楚地看到rpc解析为localhost ip。我想知道如何配置rpc ip到安装hbase的机器的ip。 解决方案

这个问题驻留在你的/ etc / hosts文件中。如果你检查你的/ etc / hosts文件,你会发现一个如下所示的条目(在我的情况下,mu机器被命名为domainnameyouwanttogive)

127.0.0.1 localhost
127.0.1.1 domainnameyouwanttogive


对于支持IPv6的主机,以下几行是可取的: b
$ b

:: 1 ip6- localhost ip6-loopback
fe00 :: 0 ip6-localnet
ff00 :: 0 ip6-mcastprefix
ff02 :: 1 ip6-allnodes
ff02 :: 2 ip6-allrouters



根本原因是domainnameyouwanttogive解析为127.0.1.1,这是不正确的,因为它应该解析为127.0.0.1(或外部IP)。由于我的外部IP是192.168.58.10,我创建了以下/ etc / hosts配置;

127.0.0.1 localhost
192.168.43.3 domainnameyouwanttogive



对于支持IPv6的主机来说,以下几行是可取的:



:: 1 ip6-localhost ip6-loopback
fe00 :: 0 ip6-localnet
ff00 :: 0 ip6-mcastprefix
ff02 :: 1 ip6-allnodes
ff02 :: 2 ip6-allrouters




I am trying to insert the data into hbase.I am running java program from remote machine. I have mentioned the code below.

try {
    Configuration conf = HBaseConfiguration.create();
    conf.clear();
    conf.set("hbase.zookeeper.quorum", "<HOST_IP>:2181");
    conf.set("hbase.zookeeper.property.clientPort", "2181");
    conf.set("hbase.zookeeper.dns.nameserver", "<HOST_IP>");
    conf.set("hbase.regionserver.port","60020");
    conf.set("hbase.master", "<HOST_IP>:9000");
    HTable table = new HTable(conf, "test");
    Put put = new Put(Bytes.toBytes("row5"));
    put.add(Bytes.toBytes("colfam1"), Bytes.toBytes("qual1"),
    Bytes.toBytes("val1"));
    put.add(Bytes.toBytes("colfam1"), Bytes.toBytes("qual2"),
    Bytes.toBytes("val2"));
    table.put(put);
} catch (MasterNotRunningException e) {
    e.printStackTrace();
} catch (ZooKeeperConnectionException e) {
    e.printStackTrace();
} catch (TableNotFoundException e) {
    e.printStackTrace();
} catch (IOException e) {
    e.printStackTrace();
}

I am getting the following error

INFO ipc.HbaseRPC: Server at localhost/127.0.0.1:60020 could not be reached after 1 tries, giving up.
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy interface org.apache.hadoop.hbase.ipc.HRegionInterface to localhost/127.0.0.1:60020 after attempts=1
    at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:355)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1176)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1195)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:898)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:797)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:772)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1002)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:801)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:766)
    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:189)
    at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:163)
    at com.tcs.hbase.HbaseSample.insertData(HbaseSample.java:30)
    at com.tcs.hbase.HbaseSample.main(HbaseSample.java:82)
Caused by: java.net.ConnectException: Connection refused: no further information
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
    at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
    at $Proxy4.getProtocolVersion(Unknown Source)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
    at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
    ... 12 more

When i run this same code in the machine where hbase is installed its workin fine.. from the logs it is clear the rpc is resolving to localhost ip. i wanted to know how to configure the rpc ip to the ip of the machine where hbase is installed.

解决方案

The root cause of this issue resides in your /etc/hosts file. If you check your /etc/hosts file you will find a entry something like the one below (in my case mu machine is named domainnameyouwanttogive)

127.0.0.1 localhost127.0.1.1 domainnameyouwanttogive

The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopbackfe00::0 ip6-localnetff00::0 ip6-mcastprefixff02::1 ip6-allnodesff02::2 ip6-allrouters

The root cause is that domainnameyouwanttogive resolves to 127.0.1.1 which is incorrect as it should resolve to 127.0.0.1 (or a external IP). As my external IP is 192.168.58.10 I created the following /etc/hosts configuration;

127.0.0.1 localhost192.168.43.3 domainnameyouwanttogive

The following lines are desirable for IPv6 capable hosts

::1 ip6-localhost ip6-loopbackfe00::0 ip6-localnetff00::0 ip6-mcastprefixff02::1 ip6-allnodesff02::2 ip6-allrouters

This will ensure that the resolving of your host processes on your localhost will be done correctly and you can start your HBase installation correctly on your development system.

这篇关于Hadoop和hbase java客户端问题与HbaseRPC的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!

08-13 12:24