Kafka环境搭建及使用教程

admin

kafka里自带了一个zookeeper，可以不需要再去下载zookeeper。在启动kafka前，需要先启动zookeeper。

（1）首先去修改config目录下的zookeeper.properties
（2）修改config目录下的server.properties，修改log.dirs和zookeeper.connect。log.dirs是日志存放文件夹，zookeeper.connect是zookeeper连接地址（端口和clientPort保持一致）。
+ B* F3 r+ \- k6 u9 Q3 n
, l' E4 Z1 v& |& a6 W7 j6 F9 \6 {至此，kafka单机版环境已搭建完毕。（如果对里面的参数没修改的话，可不修改，直接用默认的）



# G6 d0 a% Q  Z( fkafka使用教程
（1）启动kafka自带的zookeeper(加了-daemon，可以让这个进程挂在后台进行，否则会发现是个阻塞进程，只能再开一个终端进行其他操作)
bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
* ^- T. w7 ?; S9 |3 u% w! p# c" l
$ g$ c5 h& f6 Y: r) v& h/ x, y（2）启动kafka
bin/kafka-server-start.sh -daemon config/server.properties

! R% P2 j: D+ b（3）创建一个topic
* \& ~6 s3 L& r) _bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
2 D- x9 A% J9 ?# r' e3 n! F
2 S, _/ @) p! j7 B0 |9 D5 y5 m+ Z' F
（4）查看topic列表
* J- M/ K+ J, R/ Sbin/kafka-topics.sh -list -zookeeper localhost:2181
; a* h* u! r2 d& ?0 F7 q
（5）创建生产者进程  （9092是默认的broker-list端口）
  c- k& v3 K4 V; g6 gbin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
0 p  Y$ D5 k$ ^5 [+ a1 J
（6）创建消费者进程
9 M6 q* y: m8 t0 o2 f" p0 F) rbin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
# r  I$ d- I  d+ W6 ^5 a9 K1 y% |9 p
（7）外网访问 vi config/server.properties
advertised.listeners=PLAINTEXT://192.168.8.65:9092
) s( F, \" y# [% p

admin

cat config/zookeeper.properties
6 ^. s: M7 |7 c/ G# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
# f1 U3 V3 U8 w9 V  G6 v0 M% V* E% V: ?#
( d2 ^. x4 Z7 R& P, @; D; Y" \#    http://www.apache.org/licenses/LICENSE-2.0
2 C4 g4 ~+ z6 f, j7 U/ K#
" k! R$ I; @4 v# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
. z0 l* |" N; }' O9 B( M4 X# limitations under the License.
# the directory where the snapshot is stored.
dataDir=/mnt/kafka_2.13-3.3.1/data/zookeeper
# the port at which the clients will connect
/ _2 W% f9 s" j" p1 ^) I6 b5 qclientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
4 y/ L. ^; A1 l+ B1 i& g$ D8 L2 QmaxClientCnxns=0
# Disable the adminserver by default to avoid port conflicts.
/ V/ d7 U( X) D3 C% y# Set the port to something non-conflicting if choosing to enable this
admin.enableServer=false
# admin.serverPort=8080
* R, w* w) n3 }4 N

nohup bin/zookeeper-server-start.sh config/zookeeper.properties &

admin

[root@ceph2 kafka_2.13-3.3.1]# egrep -v "^#|^$" config/server.properties
broker.id=0
/ ?. u' j# B5 O  }" `% v8 glisteners=PLAINTEXT://192.168.0.233:9092
% ?+ r- }4 C8 ?6 v1 w$ }num.network.threads=3
8 U2 }9 L/ ?3 `0 E' s5 |2 inum.io.threads=8
- ^1 r! X6 ?9 ^7 g! r/ ^# b1 Ksocket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
( D' D; a1 `2 u; ]' X, wsocket.request.max.bytes=104857600
6 o  v% X+ C$ S8 clog.dirs=/var/log/kafka-logs
num.partitions=1
* }+ g; {# ]1 i- x/ ?! Nnum.recovery.threads.per.data.dir=1
9 k' H. G  D) p/ i8 {offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
5 O7 R5 E1 s' ~% T" O4 ~* Y4 Blog.retention.check.interval.ms=300000
; V: l" ?' f, m8 q, D9 g! Vzookeeper.connect=localhost:2181
! B4 \& U$ W0 o8 fzookeeper.connection.timeout.ms=18000
8 H; ~$ b0 }/ c5 B$ ]0 ~group.initial.rebalance.delay.ms=0
& m* h5 V. \4 R+ N! e+ u

4 ?& |. A% r. B# L6 E# X8 G[root@ceph2 kafka_2.13-3.3.1]# nohup bin/kafka-server-start.sh config/server.properties &



netstat  -ntlp |grep java
0 `- b' E, V3 r3 E; a. vtcp6       0      0 192.168.0.233:9092      :::*                    LISTEN      21222/java         
tcp6       0      0 :::2181                 :::*                    LISTEN      20780/java         
tcp6       0      0 :::41467                :::*                    LISTEN      21222/java         
tcp6       0      0 :::40571                :::*                    LISTEN      20780/java         

$ t/ e9 w6 Z  x3 S