Kafka依赖Zookeeper,虽然Kafka自带zookeeper,但是建议单独部署,所以先部署Zookeeper。
测试环境
citus1,citus2,citus3三台机器。对主机名和ip在/etc/hosts文件中进行映射自行完成。
部署zookeeper
Zookeeper版本:zookeeper-3.4.6
1.解压缩,修改配置
cp zookeeper-3.4.6/conf/zoo_sample.cfg zookeeper-3.4.6/conf/zoo.cfg
vi zoo.cfg中修改配置,没有的添加:
tickTime=
dataDir=/var/lib/zookeeper
clientPort=
initLimit=
syncLimit=
server.=citus1::
server.=citus2::
server.=citus3::
2.分发其他机器
scp -r zookeeper-3.4. citus2:/opt/
scp -r zookeeper-3.4. citus3:/opt/
3.设置myid
[root@citus1 opt]# mkdir /var/lib/zookeeper
[root@citus1 opt]# echo "" > /var/lib/zookeeper/myid
[root@citus2 opt]# mkdir /var/lib/zookeeper
[root@citus2 opt]# echo "" > /var/lib/zookeeper/myid
[root@citus3 opt]# mkdir /var/lib/zookeeper
[root@citus3 opt]# echo "" > /var/lib/zookeeper/myid
4.启动zookeeper
[root@citus1 zookeeper-3.4.]# bin/zkServer.sh start
JMX enabled by default
Using config: /opt/zookeeper-3.4./bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@citus2 zookeeper-3.4.]# bin/zkServer.sh start
JMX enabled by default
Using config: /opt/zookeeper-3.4./bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@citus3 zookeeper-3.4.]# bin/zkServer.sh start
JMX enabled by default
Using config: /opt/zookeeper-3.4./bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
5.查看状态
[root@citus1 zookeeper-3.4.]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4./bin/../conf/zoo.cfg
Mode: follower
[root@citus1 zookeeper-3.4.]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4./bin/../conf/zoo.cfg
Mode: leader
[root@citus1 zookeeper-3.4.]# bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper-3.4./bin/../conf/zoo.cfg
Mode: follower
部署kafka
1.解压缩,修改配置
tar -xzvf kafka_2.-0.10.2.1.tgz -C /opt
vi /opt/kafka_2.-0.10.2.1/config/server.properties
修改zookeeper配置如下:
Broker.id为当前机器在集群中的唯一标识,每台服务器的broker.id都不能相同。
2.从citus1上分发其他机器
scp -r kafka_2.-0.10.2.1/ citus2:/opt
scp -r kafka_2.-0.10.2.1/ citus3:/opt
3.修改配置
citus2上修改broker.id为1
citus3上修改broker.id为2
4.启动kafka
每台机器都执行:
bin/kafka-server-start.sh config/server.properties &
5.测试
新建一个topic:
./bin/kafka-topics.sh --zookeeper citus1:,citus2:,citus3: --topic test --replication-factor --partitions --create
查看当前topic:
./bin/kafka-topics.sh --zookeeper citus1:,citus2:,citus3: --list
在一个节点创建一个producer:
./bin/kafka-console-producer.sh --broker-list citus1:,citus2:,citus3: --topic test
在另外一个节点创建consumer:
./bin/kafka-console-consumer.sh --zookeeper citus1:,citus2:,citus3: --from-beginning -topic test
在producer端输入信息,在consumer端能接收到信息。