MiNiFi

MiNiFi

关注
发信
关注(28)粉丝(399)

MiNiFi - NiFi 无法与主机通信:端口连接超时

本文介绍了MiNiFi - NiFi 无法与主机通信:端口连接超时的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

我在我公司防火墙后面的 Windows 服务器中运行 MiNiFi.我的 NiFi 在 Hortonworks 集群上运行(3 个节点，例如:sj46、sj47、sj48).

I am running MiNiFi in a windows server which is behind my company's firewall. My NiFi is running on Hortonworks cluster (3 nodes ex:sj46,sj47,sj48).

我正在尝试将数据从 MiNiFi 发送到 NiFi.

I am trying to send data from the MiNiFi to NiFi.

  • MiNiFi 版本:0.4
  • NiFi 版本:1.5

在 MiNiFi 中，我有 ListFile 处理器、FetchFile 处理器和 RemoteProcessGroup.

In MiNiFi, i have ListFile processor, FetchFile processor and RemoteProcessGroup.

远程进程组:

NiFi.properties:

nifi.remote.input.host=
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.input.secure=True
nifi.remote.input.socket.port=6698

nifi.cluster.node.protocol.port=6699
nifi.web.http.port=
nifi.web.https.host=sj46.xxxx
nifi.web.https.port=6689

MiNiFi config.yml

MiNiFi Config Version: 3
Flow Controller:
  name: min2510
  comment: ''
Core Properties:
  flow controller graceful shutdown period: 10 sec
  flow service write delay interval: 500 ms
  administrative yield duration: 30 sec
  bored yield duration: 10 millis
  max concurrent threads: 1
  variable registry properties: ''
FlowFile Repository:
  partitions: 256
  checkpoint interval: 2 mins
  always sync: false
  Swap:
    threshold: 20000
    in period: 5 sec
    in threads: 1
    out period: 5 sec
    out threads: 4
Content Repository:
  content claim max appendable size: 10 MB
  content claim max flow files: 100
  always sync: false
Provenance Repository:
  provenance rollover time: 1 min
  implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository
Component Status Repository:
  buffer size: 1440
  snapshot frequency: 1 min
Security Properties:
  keystore: ''
  keystore type: ''
  keystore password: ''
  key password: ''
  truststore: ''
  truststore type: ''
  truststore password: ''
  ssl protocol: ''
  Sensitive Props:
    key:
    algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
    provider: BC
Processors:
- id: 50420aac-ef66-30ac-0000-000000000000
  name: FetchFile
  class: org.apache.nifi.processors.standard.FetchFile
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list:
  - failure
  - not.found
  - permission.denied
  Properties:
    Completion Strategy: None
    File to Fetch: ${absolute.path}/${filename}
    Log level when file not found: ERROR
    Log level when permission denied: ERROR
    Move Conflict Strategy: Rename
    Move Destination Directory:
- id: ae4799d5-073c-3dd6-0000-000000000000
  name: ListFile
  class: org.apache.nifi.processors.standard.ListFile
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list: []
  Properties:
    File Filter: '[^\.].*'
    Ignore Hidden Files: 'true'
    Include File Attributes: 'true'
    Input Directory: C:\minifi to EFT
    Input Directory Location: Local
    Maximum File Age:
    Maximum File Size:
    Minimum File Age: 0 sec
    Minimum File Size: 0 B
    Path Filter:
    Recurse Subdirectories: 'true'
    target-system-timestamp-precision: auto-detect
Controller Services: []
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: 95dde9e7-96a4-3bea-0000-000000000000
  name: FetchFile/success/eef78a5f-cc55-1d33-0000-0000481921e3
  source id: 50420aac-ef66-30ac-0000-000000000000
  source relationship names:
  - success
  destination id: eef78a5f-cc55-1d33-0000-0000481921e3
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
- id: 2ba13196-5ac6-3966-0000-000000000000
  name: ListFile/success/FetchFile
  source id: ae4799d5-073c-3dd6-0000-000000000000
  source relationship names:
  - success
  destination id: 50420aac-ef66-30ac-0000-000000000000
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
Remote Process Groups:
- id: 6bd4b280-646b-3b93-0000-000000000000
  name: ''
  url: https://sj46.xxxx:6689/nifi
  comment: ''
  timeout: 30 sec
  yield period: 10 sec
  transport protocol: HTTP
  proxy host: ''
  proxy port: ''
  proxy user: ''
  proxy password: ''
  local network interface: ''
  Input Ports:
  - id: eef78a5f-cc55-1d33-0000-0000481921e3
    name: From MiNiFi
    comment: ''
    max concurrent tasks: 1
    use compression: false
  Output Ports: []
NiFi Properties Overrides: {}

在MiNiFi windows server中，打开了6688、6689、6699等端口.

in the MiNiFi windows server, the ports such as 6688,6689 and 6699 are opened.

但是如果我尝试在 MiNiFi 和 NiFi 上 telnet，它没有建立.

but if i try to telnet on MiNiFi and NiFi, it was not established.

minifi-app.log

2019-03-27 15:50:09,152 WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://sj46.xxxx:6689/nifi-api due to org.apache.http.conn.HttpHostConnectException: Connect to sj46.xxxx:6689 [sj46.xxxx/x.x.x.x] failed: Connection timed out: connect
2019-03-27 15:50:09,160 WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.remote.StandardRemoteProcessGroup Unable to communicate with remote instance RemoteProcessGroup[https://sj46.xxxx:6689/nifi]
org.apache.nifi.controller.exception.CommunicationsException: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://sj46.xxxx:6689/nifi due to: Connect to sj46.xxxx:6689 [sj46.xxxx/x.x.x.x] failed: Connection timed out: connect
    at org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:882)
    at org.apache.nifi.remote.StandardRemoteProcessGroup.lambda$initialize$0(StandardRemoteProcessGroup.java:193)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://sj46.xxxx:6689/nifi due to: Connect to sj46.xxxx:6689 [sj46.xxxx/x.x.x.x] failed: Connection timed out: connect
    at org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:844)
    ... 8 common frames omitted

有关如何解决此问题的任何指示?

Any pointers on how to troubleshoot this issue?

通过从 MiNiFi 服务器禁用防火墙解决了上述问题.现在我遇到了另一个问题，如下所示:

the above issue got resolved by disabling the firewall from MiNiFi server.now i am getting another issue like below:

WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://sj46.xxxx:6689/nifi-api due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.remote.StandardRemoteProcessGroup Unable to connect to RemoteProcessGroup[https://sj46.xxxx:6689/nifi] due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

上述 SSLHandshakeException 问题已通过将 SSL 证书(您可以在 NiFi.properties 尤其是 keystore.jks 和 truststore.jks 中引用)导入 MiNiFi 解决.我们需要手动修改MiNiFi中的config.yml.

the above issue of SSLHandshakeException has been resolved by importing the SSl certificate (which you can refer in NiFi.properties especially keystore.jks and truststore.jks) to the MiNiFi. we need to manually modify config.yml in MiNiFi.

在这个问题之后，我面临着以下问题:

after this issue, i was facing the below one:

ERROR [Timer-Driven Process Thread-6] o.a.n.c.t.ContinuallyRunConnectableTask RemoteGroupPort[name=From MiNiFi,targets=https://sj46.xxxx:6689/nifi] failed to process session due to java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;

ERROR [Timer-Driven Process Thread-6] o.a.n.c.t.ContinuallyRunConnectableTask
java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;

通过将 MiNiFi 版本从 0.4 更改为 0.3，已解决 SSLContext 问题.

the SSLContext issue has been resolved by changing the MiNiFi version from 0.4 to 0.3.

现在我可以成功地从 MiNiFi 服务器获取数据到 NiFi.

Now i am successfully able to fetch data from MiNiFi server to the NiFi.

谢谢.

推荐答案

我看到您的远程端口配置为 6698.

I see that your remote port is configured at 6698.

  1. 尝试从运行 MiNiFi 的机器 telnet 6698 上的 NiFi 主机.尝试查看 NiFi 主机上配置的入站规则.(通过特定端口 Telnet 到远程主机是为了检查您是否能够通过该特定端口与该主机通信.这与您机器上的特定端口是否打开无关.)

  1. Try to telnet the NiFi Host on 6698 from the machine where MiNiFi is running. Try to see the inbound rules configured on the NiFi Host. (Telnetting to a remote host on a specific port is to check whether you are able to communicate with that host over that specific port. It has got nothing to do with whether the specific port on your machine is open or not. )

在 #Site to Site Properties 下的 nifi.properties 中添加 NiFi 主机名

Add the NiFi Host name in nifi.properties under #Site to Site Properties

nifi.remote.input.host=运行 NiFi 的主机

我建议，为了消除对 NiFi Web URL 端口和远程输入端口的常见模式的混淆，请尝试在 nifi.properties 中更改此设置

I would suggest, just to remove the mist of confusion over the common patterns of NiFi Web URL Port and Remote Input Port, try changing this in nifi.properties

nifi.remote.input.socket.port=1026

还在 config.yml 的 Input Ports 下添加以下部分:

Also add the below section under Input Ports in config.yml:

属性:
端口:6698主机名:

对于 config.yml 和 nifi.properties 修改参考，检查 这个.

For config.yml and nifi.properties modification references, check this.

这篇关于MiNiFi - NiFi 无法与主机通信:端口连接超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！

09-12 09:24
Powered by LMLPHP ©2024 MiNiFi 0.052077