Kafka环境搭建及使用教程

admin

kafka里自带了一个zookeeper，可以不需要再去下载zookeeper。在启动kafka前，需要先启动zookeeper。
9 Z9 |0 C' q) c  K( S3 _3 k, B
! N9 d# C8 L: t* E! q# u+ n& Q9 }（1）首先去修改config目录下的zookeeper.properties
（2）修改config目录下的server.properties，修改log.dirs和zookeeper.connect。log.dirs是日志存放文件夹，zookeeper.connect是zookeeper连接地址（端口和clientPort保持一致）。
9 L; Q; ~0 V1 Q7 N( D  @
至此，kafka单机版环境已搭建完毕。（如果对里面的参数没修改的话，可不修改，直接用默认的）



kafka使用教程
（1）启动kafka自带的zookeeper(加了-daemon，可以让这个进程挂在后台进行，否则会发现是个阻塞进程，只能再开一个终端进行其他操作)
bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

# Z3 ?1 V' x6 A  U& f: g" T（2）启动kafka
bin/kafka-server-start.sh -daemon config/server.properties

（3）创建一个topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
: Q1 d8 P# i3 `0 \
: ?8 }" ?. f8 S: S# b
（4）查看topic列表
  Q, a) s' e" x( I% r9 cbin/kafka-topics.sh -list -zookeeper localhost:2181
7 R# y* w* l9 f) g# }( d
（5）创建生产者进程  （9092是默认的broker-list端口）
  @# X/ c! N$ Q) K" Qbin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
7 R/ N/ W( I: c4 ]) s  K
（6）创建消费者进程
. l( X; c) w  }; [bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
- C+ X6 ]7 R1 n8 o5 [. k9 j8 i: p$ W/ F
（7）外网访问 vi config/server.properties
/ l* _3 n! X6 q, Jadvertised.listeners=PLAINTEXT://192.168.8.65:9092
& @' Y8 S: l, h( U2 v, ]; W( j

admin

cat config/zookeeper.properties
# Licensed to the Apache Software Foundation (ASF) under one or more
; _. o$ x5 p( {$ J* R" }: a# contributor license agreements.  See the NOTICE file distributed with
6 x8 r+ }/ ~' X' e# this work for additional information regarding copyright ownership.
% o& h9 t* V) M& u  H# The ASF licenses this file to You under the Apache License, Version 2.0
: o/ K' b6 L/ Q# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
7 _6 P! T3 ?8 H2 f4 P9 v- M3 Z7 Z5 b#    http://www.apache.org/licenses/LICENSE-2.0
$ b* q" R: w# A$ v$ d#
, d6 c9 _! k' g8 _- T# Unless required by applicable law or agreed to in writing, software
0 [/ m- E. l& G7 s# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
% i, g) `2 }! b0 W# See the License for the specific language governing permissions and
# limitations under the License.
. i2 c' g' [0 U5 z( m) i# the directory where the snapshot is stored.
dataDir=/mnt/kafka_2.13-3.3.1/data/zookeeper
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
; H: M! Q8 a9 `1 imaxClientCnxns=0
. T4 f* C, s$ Z: Y) W, ^# Disable the adminserver by default to avoid port conflicts.
# Set the port to something non-conflicting if choosing to enable this
2 `) s8 P  F! M$ o+ n+ Jadmin.enableServer=false
8 K* p8 O6 x% m  Y: Z) I# admin.serverPort=8080

) P; T2 c; p2 [, l$ D, c
nohup bin/zookeeper-server-start.sh config/zookeeper.properties &
0 S2 T- S& g. H0 z+ R4 G8 m$ ?

admin

[root@ceph2 kafka_2.13-3.3.1]# egrep -v "^#|^$" config/server.properties
broker.id=0
listeners=PLAINTEXT://192.168.0.233:9092
num.network.threads=3
9 V" Z% K2 p4 w8 \; Lnum.io.threads=8
' G/ b) X8 o4 w) F: [socket.send.buffer.bytes=102400
1 |4 ~  p  V7 dsocket.receive.buffer.bytes=102400
0 J6 X1 z# D6 X. k/ |  J) Esocket.request.max.bytes=104857600
log.dirs=/var/log/kafka-logs
num.partitions=1
8 ]! j1 x, r! p' }+ g2 ?- Hnum.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
! L* }# P. P. w, g8 d1 a+ `) s& u0 y/ Y/ ztransaction.state.log.min.isr=1
) p) Q# x% c- }7 G- @2 [. v% mlog.retention.hours=168
log.retention.check.interval.ms=300000
zookeeper.connect=localhost:2181
# _* v/ a" I3 M# w( k/ jzookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0


[root@ceph2 kafka_2.13-3.3.1]# nohup bin/kafka-server-start.sh config/server.properties &
5 ?  W% l4 k; J& M
0 E, N; i- d" B
1 N" V9 ^5 x0 O  U+ t* w/ ]
' n; K. L- ]2 {- Hnetstat  -ntlp |grep java
tcp6       0      0 192.168.0.233:9092      :::*                    LISTEN      21222/java         
tcp6       0      0 :::2181                 :::*                    LISTEN      20780/java         
6 P6 y* ]) C. V3 E4 Ttcp6       0      0 :::41467                :::*                    LISTEN      21222/java         
2 g' S+ }2 J/ I. D; G0 h- ztcp6       0      0 :::40571                :::*                    LISTEN      20780/java         

1 F$ {" m% P) E/ o# G