1.部署JDK
参考:https://www.jianshu.com/p/c3a93c01b3b1
2.部署zookeeper
下载zookeeper
https://zookeeper.apache.org/releases.html
解压
tar -xzvf apache-zookeeper-3.7.0-bin.tar.gz /usr/java
配置环境变量
JAVA_HOME=/usr/java/jdk1.8.0_291
JRE_HOME=/usr/java/jdk1.8.0_291/jre
CLASS_PATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
KAFKA_HOME=/usr/java/kafka_2.12-2.7.0
PATH=$PATH:$KAFKA_HOME/bin
修改配置
conf目录下提供了配置的样例zoo_sample.cfg,要将zk运行起来,需要将其名称修改为zoo.cfg。
配置存储快照文件的目录。如果没有配置dataLogDir,那么事务日志也会存储在此目录
cp zoo_sample.cfg zoo.cfg
dataDir=/opt/modules/zookeeper
启动zookeeper
./zkServer.sh start
3.单节点-单broker 部署
下载kafka
http://kafka.apache.org/downloads
解压
tar -xzvf kafka_2.12-2.7.0.tgz /usr/java
修改配置
修改conf路径下的server.properties
broker.id=0
port:9092
# 不加上advertised,启动报错
advertised.listeners=PLAINTEXT://1.116.235.191:9092
# 调整数据过期时间
log.retention.hours=1680000
# 配置kafka链接
zookeeper.connect=localhost:2181
# 配置数据存储路径
log.dirs=/opt/modules/data/0
启动kafka
参考:https://www.jianshu.com/p/7bdd44bb866f
4.单节点-多broker 部署
修改配置
复制server.properties到server1.properties
修改server1.properties配置
broker.id=1
port:9093
启动kafka
& 不要丢掉且要换行
kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties &
kafka-server-start.sh -daemon $KAFKA_HOME/config/server1.properties
5.错误
启动生产者时报错,找不到broker,且已经设置advertised.listeners=PLAINTEXT://ip:9092
[root@VM-12-4-centos ~]# kafka-console-producer.sh --broker-list localhost:9092 --topic topicname
[2021-02-22 16:20:39,752] WARN [Consumer clientId=consumer-console-consumer-18786-1, groupId=console-consumer-18786] Bootstrap broker hadoop102:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2021-02-22 16:20:39,869] WARN [Consumer clientId=consumer-console-consumer-18786-1, groupId=console-consumer-18786] Bootstrap broker hadoop102:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
经排查发现,是9092端口未开启,使用的是腾讯云,需要在控制台开启9092端口