如何修复此错误运行 Nutch 1.15 ERROR fetcher.Fetcher - Fetcher 作业未成功，作业状态:FAILED，原因:NA

问题描述

当我使用 Nutch 1.15 开始抓取时:

When I'm starting a crawl using Nutch 1.15 with this:

/usr/local/nutch/bin/crawl --i -s urls/seed.txt crawldb 5

然后它开始运行，当它尝试获取时出现此错误:

Then it starts to run and I get this error when it tries to fetch:

2019-02-10 15:29:32,021 INFO  mapreduce.Job - Running job: job_local1267180618_0001
2019-02-10 15:29:32,145 INFO  fetcher.FetchItemQueues - Using queue mode : byHost
2019-02-10 15:29:32,145 INFO  fetcher.Fetcher - Fetcher: threads: 50
2019-02-10 15:29:32,145 INFO  fetcher.Fetcher - Fetcher: time-out divisor: 2
2019-02-10 15:29:32,149 INFO  fetcher.QueueFeeder - QueueFeeder finished: total 1 records hit by time limit : 0
2019-02-10 15:29:32,234 WARN  mapred.LocalJobRunner - job_local1267180618_0001
java.lang.Exception: java.lang.NullPointerException
        at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NullPointerException
        at org.apache.nutch.net.URLExemptionFilters.<init>(URLExemptionFilters.java:39)
        at org.apache.nutch.fetcher.FetcherThread.<init>(FetcherThread.java:154)
        at org.apache.nutch.fetcher.Fetcher$FetcherRun.run(Fetcher.java:222)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2019-02-10 15:29:33,023 INFO  mapreduce.Job - Job job_local1267180618_0001 running in uber mode : false
2019-02-10 15:29:33,025 INFO  mapreduce.Job -  map 0% reduce 0%
2019-02-10 15:29:33,028 INFO  mapreduce.Job - Job job_local1267180618_0001 failed with state FAILED due to: NA
2019-02-10 15:29:33,038 INFO  mapreduce.Job - Counters: 0
2019-02-10 15:29:33,039 ERROR fetcher.Fetcher - Fetcher job did not succeed, job status:FAILED, reason: NA
2019-02-10 15:29:33,039 ERROR fetcher.Fetcher - Fetcher: java.lang.RuntimeException: Fetcher job did not succeed, job status:FAILED, reason: NA
        at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:503)
        at org.apache.nutch.fetcher.Fetcher.run(Fetcher.java:543)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:517)

我在控制台中收到此错误，这是它运行的命令:

And I get this error in the console which is the command it runs:

Error running:
  /usr/local/nutch/bin/nutch fetch -D mapreduce.job.reduces=2 -D mapred.child.java.opts=-Xmx1000m -D mapreduce.reduce.speculative=false -D mapreduce.map.speculative=false -D mapreduce.map.output.compress=true -D fetcher.timelimit.mins=180 crawlsites/segments/20190210152929 -noParsing -threads 50

推荐答案

我不得不删除 nutch 文件夹并重新安装，然后它就可以工作了.

I had to delete the nutch folder and do a new install and it worked after this.

这篇关于如何修复此错误运行 Nutch 1.15 ERROR fetcher.Fetcher - Fetcher 作业未成功，作业状态:FAILED，原因:NA的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！