本文介绍了在HDFS文件路径的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我想读取Hadoop的文件系统中。
I want to read the file from the Hadoop File System.
为了实现文件的正确路径,我需要的主机名和端口地址 HDFS
。
In order to achieve the correct path of the file, I need host name and port address of the hdfs
.
所以最后文件我前进的道路看起来像
so finally my path of the file will look something like
Path path = new Path("hdfs://123.23.12.4344:9000/user/filename.txt")
现在我想知道提取的主机名=123.23.12.4344&放大器;端口:9000
Now I want to know to extract the HostName = "123.23.12.4344" & port: 9000?
基本上,我想访问文件系统在Amazon EMR,但是,当我使用
Basically, I want to access the FileSystem on Amazon EMR but, when I use
FileSystem fs = FileSystem.get(getConf());
我得到
You possibly called FileSystem.get(conf) when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your path
所以,我决定使用URI。 (我不得不使用URI),但我不知道如何访问的URI。
So I decided to use URI. (I have to use URI) but I am not sure how to access the URI.
推荐答案
您可以使用两种方式来解决您的错误。
You can use either of the two ways to solve your error.
1
String infile = "file.txt";
Path ofile = new Path(infile);
FileSystem fs = ofile.getFileSystem(getConf());
2
Configuration conf = getConf();
System.out.println("fs.default.name : - " + conf.get("fs.default.name"));
// It prints uri as : hdfs://10.214.15.165:9000 or something...
String uri = conf.get("fs.default.name");
FileSystem fs = FileSystem.get(uri,getConf());
这篇关于在HDFS文件路径的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持!