Kafka环境搭建及使用教程

admin

kafka里自带了一个zookeeper，可以不需要再去下载zookeeper。在启动kafka前，需要先启动zookeeper。

6 {) g, W, R% M% `（1）首先去修改config目录下的zookeeper.properties
/ ?6 B$ r: y0 w. [! o（2）修改config目录下的server.properties，修改log.dirs和zookeeper.connect。log.dirs是日志存放文件夹，zookeeper.connect是zookeeper连接地址（端口和clientPort保持一致）。
' u; \1 a9 ^3 J" H- |. j
至此，kafka单机版环境已搭建完毕。（如果对里面的参数没修改的话，可不修改，直接用默认的）

6 n$ c7 z' g6 ?; h8 S

kafka使用教程
（1）启动kafka自带的zookeeper(加了-daemon，可以让这个进程挂在后台进行，否则会发现是个阻塞进程，只能再开一个终端进行其他操作)
4 _# u! R$ I7 z  S7 y: B) Z  k% }bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

5 S( @" k8 y$ ?: K（2）启动kafka
bin/kafka-server-start.sh -daemon config/server.properties

4 ]1 d. ~) f0 u4 a5 d2 B（3）创建一个topic
! g: b9 u6 C  Y, U, nbin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
5 ^9 i, U& u" D! \- Z' E
6 w+ W: }+ m/ R
, j. I# x8 w$ A! C. c（4）查看topic列表
! D8 Z2 H- u. x) g4 \6 c( X, ]' Rbin/kafka-topics.sh -list -zookeeper localhost:2181
( n6 [& w+ l3 Y) B% ~5 S
（5）创建生产者进程  （9092是默认的broker-list端口）
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
  N# V  K) y* n# o
（6）创建消费者进程
9 k( P4 B1 B8 h( W) kbin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

（7）外网访问 vi config/server.properties
advertised.listeners=PLAINTEXT://192.168.8.65:9092

admin

cat config/zookeeper.properties
  y) I% G9 k: o0 A7 X# Licensed to the Apache Software Foundation (ASF) under one or more
7 j1 N/ g8 S( N5 n9 x# contributor license agreements.  See the NOTICE file distributed with
  r0 q# g2 k' I( j6 P( T( w* s/ N# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
0 f# F* k* X! _4 x4 c# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
1 l9 M5 d! _+ b! R  {, P% r! k#    http://www.apache.org/licenses/LICENSE-2.0
8 \" q% [, N* D" N  x% j6 P#
" ?* S" j# O- K- N7 N% A3 c! t# Unless required by applicable law or agreed to in writing, software
. J/ ^3 X' a, [6 O# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* C1 H) m2 w- i1 A+ y+ [1 A# B7 j# See the License for the specific language governing permissions and
/ p: v" g6 Q$ x- Q# limitations under the License.
# the directory where the snapshot is stored.
dataDir=/mnt/kafka_2.13-3.3.1/data/zookeeper
3 `, N# x3 u6 X1 `5 ]9 J" P) t# the port at which the clients will connect
5 b$ u: A& c+ X2 f/ LclientPort=2181
$ ^) q: g5 N  T- w# n' ^( X# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0
# Disable the adminserver by default to avoid port conflicts.
# Set the port to something non-conflicting if choosing to enable this
. S  y3 N% \0 `% V% X, s  e5 ]$ w& Hadmin.enableServer=false
3 d8 v$ o: a/ Y8 K4 T3 h" B" O# ]# admin.serverPort=8080
% d; M) m/ X6 W2 `& W

% z" ^. t! p+ T2 q* S& @nohup bin/zookeeper-server-start.sh config/zookeeper.properties &

admin

[root@ceph2 kafka_2.13-3.3.1]# egrep -v "^#|^$" config/server.properties
5 M( ]) e, g3 h5 E% t& U! Zbroker.id=0
listeners=PLAINTEXT://192.168.0.233:9092
num.network.threads=3
! ~* s- A9 J6 q0 |$ W& jnum.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
7 S& f8 a. o7 l# o5 D6 E: plog.dirs=/var/log/kafka-logs
9 a9 p* B/ s' `  q, N% ]0 j# Tnum.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
% D9 W+ f2 {8 H% t2 ^( P4 k( gtransaction.state.log.replication.factor=1
# A/ w7 q0 Q- I% Etransaction.state.log.min.isr=1
2 {9 b4 @1 v9 T7 R5 ^log.retention.hours=168
9 E$ L/ o& `' D$ c: a; Glog.retention.check.interval.ms=300000
, c4 N4 K- j: w- l# ]' T, czookeeper.connect=localhost:2181
zookeeper.connection.timeout.ms=18000
( ?' [1 f3 u6 [9 n8 ?' n, igroup.initial.rebalance.delay.ms=0


[root@ceph2 kafka_2.13-3.3.1]# nohup bin/kafka-server-start.sh config/server.properties &



netstat  -ntlp |grep java
tcp6       0      0 192.168.0.233:9092      :::*                    LISTEN      21222/java         
3 P0 L& a4 L. L6 v4 U' `( k5 A$ Ltcp6       0      0 :::2181                 :::*                    LISTEN      20780/java         
% O( D# {5 z- z0 P9 y' F4 @/ R5 stcp6       0      0 :::41467                :::*                    LISTEN      21222/java         
8 F  R# t9 T3 T6 b$ m! wtcp6       0      0 :::40571                :::*                    LISTEN      20780/java