问题描述
我在新的Linux centos 7上安装了kafka confluent oss 4.0，但kafka connect无法启动.
复制步骤:
- Install Oracle JDK 8
- Copy confluent-4.0.0 folder on opt/confluent-4.0.0
- Run /opt/confluent-4.0.0/confluent start
结果:
Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
schema-registry is [UP]
Starting kafka-rest
kafka-rest is [UP]
Starting connect
\Kafka Connect failed to start
connect is [DOWN]
错误日志(connect.stderr):
Exception in thread "main" java.lang.NoClassDefFoundError: io/confluent/connect/storage/StorageSinkConnectorConfig
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:54)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructor0(Class.java:3075)
at java.lang.Class.newInstance(Class.java:412)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:279)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:260)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:201)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:193)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:153)
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:70)
Caused by: java.lang.ClassNotFoundException: io.confluent.connect.storage.StorageSinkConnectorConfig
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:62)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 22 more
其他信息:
Java版本:
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode
Centos版本:
centos-release-7-4.1708.el7.centos.x86_64
在每个属性文件中编辑plugin.path变量并不能解决问题.
包含"plugin.path"变量的文件列表:
./etc/schema-registry/connect-avro-distributed.properties:84:plugin.path=/opt/confluent-4.0.0/share/java
./etc/schema-registry/connect-avro-standalone.properties:51:plugin.path=/opt/confluent-4.0.0/share/java
./etc/kafka/connect-distributed.properties:95:plugin.path=/opt/confluent-4.0.0/share/java
./etc/kafka/connect-standalone.properties:50:plugin.path=/opt/confluent-4.0.0/share/java
对于Confluent 4.0.0，使用 plugin.path 默认情况下已为Kafka Connect启用.
从deb或rpm软件包安装Confluent Platform时，预先知道plugin.path的默认位置.
但是，当您在文件系统中的某个位置下载并提取Confluent Platform的zip或tar.gz存档时，它设置为:
plugin.path=share/java
这是一条相对路径，因为当您将Confluent Platform下载为存档文件(zip或tar.gz)时，提取存档文件的位置未知(在上面的示例中为/opt/confluent-4.0.0/)./p>
如果从提取Confluent平台的目录中运行它，CLI或Connect的bin脚本将能够猜测该位置:
例如，在上面的示例中:
cd /opt/confluent-4.0.0
./bin/confluent start
鉴于Confluent Platform的bin目录位于您的PATH中，为了使您能够从文件系统中的任何目录启动Connect，您需要将属性plugin.path设置为绝对值插件的路径位置:
要使用Confluent CLI编辑，请执行以下操作:
etc/schema-registry/connect-avro-distributed.properties
并适当地设置您的plugin.path(在这里:plugin.path=/opt/confluent-4.0.0/share/java)
对于常规bin脚本，请
./etc/kafka/connect-distributed.properties
和
./etc/kafka/connect-standalone.properties
并按上述设置您的plugin.path(同样，在您的示例中为plugin.path=/opt/confluent-4.0.0/share/java).
I installed kafka confluent oss 4.0 on a fresh linux centos 7 but kafka connect failed to start.
Steps to reproduce :
- Install Oracle JDK 8
- Copy confluent-4.0.0 folder on opt/confluent-4.0.0
- Run /opt/confluent-4.0.0/confluent start
Result :
Starting zookeeper
zookeeper is [UP]
Starting kafka
kafka is [UP]
Starting schema-registry
schema-registry is [UP]
Starting kafka-rest
kafka-rest is [UP]
Starting connect
\Kafka Connect failed to start
connect is [DOWN]
Error Log (connect.stderr) :
Exception in thread "main" java.lang.NoClassDefFoundError: io/confluent/connect/storage/StorageSinkConnectorConfig
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:54)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
at java.lang.Class.getConstructor0(Class.java:3075)
at java.lang.Class.newInstance(Class.java:412)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:279)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:260)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:201)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:193)
at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:153)
at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:47)
at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:70)
Caused by: java.lang.ClassNotFoundException: io.confluent.connect.storage.StorageSinkConnectorConfig
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:62)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 22 more
Additional informations :
Java version :
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.151-b12, mixed mode
Centos version :
centos-release-7-4.1708.el7.centos.x86_64
[Edit : 30/11/2017]Editing plugin.path variables in every properties files didn't fix the problem.
List of files containing 'plugin.path' variable :
./etc/schema-registry/connect-avro-distributed.properties:84:plugin.path=/opt/confluent-4.0.0/share/java
./etc/schema-registry/connect-avro-standalone.properties:51:plugin.path=/opt/confluent-4.0.0/share/java
./etc/kafka/connect-distributed.properties:95:plugin.path=/opt/confluent-4.0.0/share/java
./etc/kafka/connect-standalone.properties:50:plugin.path=/opt/confluent-4.0.0/share/java
With Confluent 4.0.0, classloading isolation with plugin.path is enabled by default for Kafka Connect.
When you install Confluent Platform from deb or rpm packages the default location of your plugin.path is known beforehand.
However, when you download and extract the zip or tar.gz archive of Confluent Platform somewhere in your filesystem, it's set to:
plugin.path=share/java
This is a relative path, because when you download Confluent Platform as an archive (zip or tar.gz), the location where you extract the archive is not known (in your example above it's /opt/confluent-4.0.0/).
The CLI or Connect's bin scripts will be able to guess this location if you run it from the directory where you extracted Confluent platform:
For instance, in the example above:
cd /opt/confluent-4.0.0
./bin/confluent start
In order for you to be able to start Connect from any directory within your filesystem, given that the bin directory for Confluent Platform is in your PATH, you will need to set the property plugin.path to the absolute path location of your plugins:
To use Confluent CLI edit:
etc/schema-registry/connect-avro-distributed.properties
and set your plugin.path appropriately (here: plugin.path=/opt/confluent-4.0.0/share/java)
For the regular bin scripts edit:
./etc/kafka/connect-distributed.properties
and
./etc/kafka/connect-standalone.properties
and set your plugin.path as above (again, in your example: plugin.path=/opt/confluent-4.0.0/share/java).
这篇关于Kafka Connect无法启动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！