如何使用Java中的Eclipse在Windows上本地运行Spark

问题描述

限时删除！！

我正在尝试测试Mllib的SVM实现.我想使用Eclipse在Windows本地运行其Java示例.我已经下载了为Hadoop 2.6预先构建的Spark 1.3.1.当我尝试运行示例代码时，我得到:

I'm trying to test Mllib's implementation of SVM. I want to run their java example locally on windows, using eclipse.I've downloaded Spark 1.3.1 pre-built for Hadoop 2.6 .When i try to run the example code, i get:

15/06/11 16:17:09 ERROR Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

我应该更改什么才能在此设置中运行示例代码?

What should i change in order to be able to run the example code in this setup?

推荐答案

1. 创建以下目录结构:"C:\ hadoop_home \ bin"(或将"C:\ hadoop_home"替换为您喜欢的任何内容)

1. Create the following directory structure: "C:\hadoop_home\bin" (or replace "C:\hadoop_home" with whatever you like)

下载以下文件: http://public-repo-1.hortonworks.com/hdp-win-alpha/winutils.exe

将第2步中的文件放入第1步中的"bin"目录.

Put the file from step 2 into the "bin" directory from step 1.

将"hadoop.home.dir"系统属性设置为"C:\ hadoop_home"(或您在第1步中创建的任何目录，结尾都没有"\ bin").

Set the "hadoop.home.dir" system property to "C:\hadoop_home" (or whatever directory you created in step 1, without the "\bin" at the end).

这篇关于如何使用Java中的Eclipse在Windows上本地运行Spark的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

1403页，肝出来的..