|
|
楼主 |
发表于 2025-12-18 08:51:30
|
显示全部楼层
2、网络服务Neutron! J' q& u$ f# y# T' V
Neutron基于软件定义网络的思想，实现了网络虚拟化下的资源管理。Neutron的设计目标是实现网络即服务（NaaS），在设计上遵循SDN(Software Defined Network，软件定义网络)架构来管理的。% [: J- P' K$ P$ r: w t
Neutron主要包含Neutron server、Plugin和Agent等组件。Neutron server对外提供 OpenStack网络 API，接收请求，并调用Plugin处理请求；Plugin处理 Neutron Server发来的请求，维护OpenStack逻辑网络的状态， 并调用 Agent 处理请求；Agent处理Plugin的请求，负责在network provider上真正实现各种网络功能；此外还有database，用来存放OpenStack的网络状态信息，包括Network、Subnet、Port、Router等。
$ [) ]- ?5 r5 q, i; M, X- |
* z; r# x; v" h) _* ` T2 U3、OVS* l+ f9 `3 _! {; t$ ^
OVS(Open vSwitch)是虚拟交换机，遵循SDN(Software Defined Network，软件定义网络)架构来管理的。5 p, e9 }( w+ n5 M, c3 V1 g; B
OVS介绍参考：https://mp.weixin.qq.com/s?__biz ... 189#wechat_redirect
& b) V1 t# L) p. B在这里插入图片描述+ [1 I( a: a5 F; d. I. [# t
ovs由三个组件组成：dataPath、vswitchd和ovsdb。
% a9 I y- @) i, `$ o0 EdataPath(opevswitch.ko)：openvswitch.ko是ovs的内核模块，当openvswitch.ko模块被加载到内核时，会在网卡上注册一个钩子函数，每当网络包到达网卡时这个钩子函数就会被调用。openvswitch.ko模块在处理网络包时，会先匹配内核中能不能匹配到策略（内核流表）来处理，如果匹配到了策略，则直接在内核态根据该策略做网络包转发，这个过程全程在内核中完成，处理速度非常快，也称之为fast path（快速通道）；如果内核中没有匹配到相应策略，则把数据包交给用户态的vswitchd进程处理，此时叫作slow path（慢通道）。dataPath模块可以通过ovs-dpctl命令来配置。7 b5 P) E- b2 P' E4 c, A
vswitchd：vswitchd是ovs的核心模块，它工作在用户空间(user space)，负责与OpenFlow控制器、第三方软件通信。vswitchd接收到数据包时，会去匹配用户态流表，如果匹配成功则根据相关规则转发；如果匹配不成功，则会根据OpenFlow协议规范处理，把数据包上报给控制器（如果有）或者丢弃。6 F3 Y2 s$ \) K) ~2 S Q
ovsdb：ovs数据库，存储整个ovs的配置信息，包括接口、交换内容、vlan、虚拟交换机信息等。
, z ~5 V" z+ R% c6 Vovs相关术语解释：
# \( J B4 U c" e+ r1、Bridge：网桥，也就是交换机（不过是虚拟的，即vSwitch），一台主机中可以创建多个网桥。当数据包从网桥的某个端口进来后，网桥会根据一定的规则把该数据包转发到另外的端口，也可以修改或者丢弃报文。Bridge桥指的是虚拟交换机。3 F( {; Q0 \- I/ r$ A0 L# \: j/ S
2、Port：交换机的端口，有以下几种类型：
+ W. Z3 e: T9 d: KNormal: 将物理网卡添加到bridge时它们会成为Port，类型为Normal。此时物理网卡配置ip已没有意义，它已经“退化成一根网线”只负责数据报文的进出。Normal类型的Port常用于vlan模式下多台物理主机相连的那个口，交换机的一端属于Trunk模式。3 s+ j% }% H* J
Internal: 此类型的Port，ovs会自动创建一个虚拟网卡接口(Interface)，此端口收到数据都会转发给这块网卡，从网卡发出的数据也会通过Port交给ovs处理。当ovs创建一个新的Bridge时，会自动创建一个与网桥同名的Internal Port，同时也会创建一个与网桥同名的Interface。另外，Internal Port可配置IP地址，然后将其up，即可实现ovs三层网络。
5 M. Z3 d/ I* }3 G0 E4 pPatch: 与veth pair功能类似，常用于连接两个Bridge。veth pair：两个网络虚拟端口（设备）7 P5 U$ U; M. F% _9 j T
Tunnel: 实现overlay网络，支持GRE、vxlan、STT、Geneve和IPSec等隧道协议。Tunnel：隧道，三层
5 L! k' H# E: C4 D3、Interface：网卡，虚拟的（TUN/TAP）或物理的都可以。TAP：单个网络虚拟端口（设备），基于二层；TUN：单个网络虚拟端口（设备），基于三层。veth pair：两个网络虚拟端口（设备），常用于连接两个Bridge。8 A3 N3 o% h! z: L' |. b V
4、Controller：控制器，ovs可以接收一个或多个OpenFlow控制器的管理，主要功能为下发流表来控制转发规则。) x/ _- \; d; h# U
5、FlowTable：流表，ovs进行数据转发的核心功能，定义了端口之间的转发数据规则。每条流表规则可以分为匹配和动作两部分，“匹配”决定哪些数据将被处理，“动作”则决定了这些数据将被如何处理。5 R9 O9 y: t, R: {! u: u2 H- S* p) I
在这里插入图片描述3 S( d/ b7 y2 k, I- y* X2 J
ens160的ip地址没有了，用的是br-ex的ip地址出去的。9 ~% }! J4 E) c. O
在这里插入图片描述
! Y& C5 W1 _, F+ T4 n8 J! m% t. N2 |/ Bovs安装# p1 x* V# O+ i# g. Q% A! f& a4 `
1.开启一台新的linux/ x+ H7 n$ u( y% F. _1 h9 e
2.配置在线yum源（openstack那个在线yum源）5 N& i( [4 J Z
$ F/ }* Z$ ?& R4 c7 Y
配置yum源（先把原有的备份后清空）8 V/ ~0 |- \% W G* @+ f
# cd /etc/yum.repos.d/ # rm -rf *
( q& F( t# [ S# cat cloud.repo
3 q- K6 d1 ]9 \/ B6 [3 V( d% I, x1 C% [( I* _& K$ ~, S
[highavailability]
& M: {# m, @9 Q x6 F( ^name=CentOS Stream 8 - HighAvailability
. B+ l: x3 G2 _% ^- nbaseurl=https://mirrors.aliyun.com/centos/8-stream/HighAvailability/x86_64/os/
8 H+ G% d/ r9 c8 P% m; ygpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
8 i6 I" E5 N7 k% A0 egpgcheck=1/ R" k5 M# m+ r+ I# p$ ^# l0 T
repo_gpgcheck=0
3 C8 M, H) {/ ]+ I+ z3 a( Cmetadata_expire=6h
+ w9 i. E& x0 `4 m6 jcountme=1$ `5 {" }7 W9 l& z5 T
enabled=1
$ B* `, X0 p" K
1 ]) b' N/ r' w: K9 `2 |. N[nfv]% r- `( t6 i) p) I0 ?
name=CentOS Stream 8 - NFV
8 G) K$ G6 X w+ kbaseurl=https://mirrors.aliyun.com/centos/8-stream/NFV/x86_64/os/4 w8 w% E' T) Y% Z5 b
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
) l: m& [, ^8 y2 q {! P# C; ~ f7 Lgpgcheck=1
: j0 A7 `) E: L& G! N5 Z) e( e [repo_gpgcheck=0
$ D* p+ f0 l! [1 |/ X. [metadata_expire=6h
7 W; e$ j' s9 n4 S: u' o; icountme=1
; V, s' T* ~# B+ }# ^enabled=1
$ [4 B* e" s. |4 T6 g! f7 ` S9 W: }# Z! S# @8 q
[rt]
% q7 z6 e( X9 G9 `6 w% \name=CentOS Stream 8 - RT
9 w! a2 f' \1 a* M% Z* s8 t% |baseurl=https://mirrors.aliyun.com/centos/8-stream/RT/x86_64/os/
( b. m2 `. r: p6 T; M8 \gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
0 D' E- d3 B. I2 X3 J. H1 sgpgcheck=1
' K; C" I' B( j) Prepo_gpgcheck=0$ a& m7 L/ h( \+ n2 k9 i
metadata_expire=6h1 J: ^7 ~ o/ y9 S
countme=1) g' f1 c- v! f
enabled=10 E. | I5 }5 h* u. V2 s
) P Y* h5 y: h$ Y4 g( d[resilientstorage]
F6 W( a$ t0 M; \) cname=CentOS Stream 8 - ResilientStorage# c; q3 x9 ]- C4 T P. }% n) c
baseurl=https://mirrors.aliyun.com/centos/8-stream/ResilientStorage/x86_64/os/0 }+ G2 v& _) K- o4 ]; T3 p
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial5 d" q+ A; F, c
gpgcheck=1, R; Q! k/ W Y8 I2 c6 k u
repo_gpgcheck=01 N- Y3 C9 j. E2 [* C4 j! _
metadata_expire=6h
2 ?+ K% F! M [/ ucountme=1
" x4 `4 @6 h% O' G+ G: ienabled=1
% M; k- |+ `/ ?( F, R1 ^; p6 {; z2 z# c- X# G
[extras-common]- F( ]- |: c" v; @. h- x
name=CentOS Stream 8 - Extras packages3 Q7 Z$ K! x1 ]& ], M( X
baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/extras-common/
9 j' G8 Y0 N8 y6 v4 M: K2 Zgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA5125 `/ r) Z( S& k0 j% O
gpgcheck=1
' N+ d2 H# o. J* F+ ^repo_gpgcheck=0
# c) X* I. E# W6 G; u# hmetadata_expire=6h
: S( L* n: t% @/ c# b* |, Qcountme=1
& D0 F9 R8 ~) U0 q' u" n* {enabled=1$ x5 x U9 j% h% y+ T+ e
( S; U/ l; R* r6 T- A4 r( G
[extras]- n5 `1 o8 O# T, V6 k+ k" B
name=CentOS Stream - Extras
+ V+ {) r/ f+ K* |mirrorlist=http://mirrorlist.centos.org/?release=&arch=&repo=extras&infra=- X8 o) o1 \( \
#baseurl=http://mirror.centos.org///extras//os/, k7 v/ F2 B7 n1 T
baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/os/
! I. m9 M, p, ~( e3 l2 v8 T' Lgpgcheck=1% ?& a0 e$ y5 d5 f
enabled=1+ f W8 D9 s" ~7 h3 J
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
- F( S2 X) g2 E. ]5 B# U$ Z1 f" E: w. V1 V2 @- z9 }/ r: c
[centos-ceph-pacific]
1 A: G8 p9 j# r' c3 J3 kname=CentOS - Ceph Pacific8 H1 u7 A7 ^+ y+ _/ Z
baseurl=https://mirrors.aliyun.com/centos/8-stream/storage/x86_64/ceph-pacific/
. U! n( O6 F# Mgpgcheck=0* I: M7 { d" k- _
enabled=1' ^4 A5 Z7 g& u$ q
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
) S' c5 L& p$ Q7 V7 f ^7 G# [. _; I7 s3 T
[centos-rabbitmq-38]. v0 c; j n( r: A* G
name=CentOS-8 - RabbitMQ 38: N! Z0 C5 V! W" J( M8 W
baseurl=https://mirrors.aliyun.com/centos/8-stream/messaging/x86_64/rabbitmq-38/
' _) v/ [8 ], F/ Zgpgcheck=1
( E( I9 B5 p; genabled=1
4 n) ?- N9 a; w. E3 w# Qgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging+ i6 A) q& N% n$ L3 y. \9 l
0 N$ f1 y5 \% P: j
[centos-nfv-openvswitch]' f4 N' d. p. w( q& \3 g' t* r& b
name=CentOS Stream 8 - NFV OpenvSwitch
/ A4 E! R* O6 l6 s+ \3 _baseurl=https://mirrors.aliyun.com/centos/8-stream/nfv/x86_64/openvswitch-2/$ J' Z; w& p, u' c
gpgcheck=1
$ B6 x; _9 a0 Zenabled=1' a3 I; \$ |8 `
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV6 R, r* J% D# B/ \
module_hotfixes=11 ^) g) \( ?& G" e7 A
1 P6 V7 A% G; Z' f5 J) e[baseos]9 M4 N- ]) D f
name=CentOS Stream 8 - BaseOS
t! ~! V" n+ l& z; Y& w9 qbaseurl=https://mirrors.aliyun.com/centos/8-stream/BaseOS/x86_64/os/
5 z5 ?2 r, L0 H0 y% U( [% dgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial! Z3 V, v" V. K( R i+ n
gpgcheck=1
# Y( S6 B' U9 {3 c% x9 M6 V6 Nrepo_gpgcheck=0
3 S% n/ |: X# X) E. o& Pmetadata_expire=6h% |% [9 w( j. C
countme=1. Q! K/ W8 V, G) a- D1 }5 {% |
enabled=1' v$ ~+ \2 _% z
+ F* A: U( O+ X* Q% K
[appstream]
1 l5 P7 ^/ _( U5 xname=CentOS Stream 8 - AppStream% l( n, t3 y) @# m
baseurl=https://mirrors.aliyun.com/centos/8-stream/AppStream/x86_64/os/9 B( e9 U6 |- b# l c& c/ e
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial8 S- H ~9 z1 Q* p/ v- _
gpgcheck=10 D ^6 E. r# `! q* d
repo_gpgcheck=0! b+ n* ~8 D% j% `2 i7 i1 S( K
metadata_expire=6h
- v) t+ J( N( Q( Pcountme=1
! T/ [% D [2 v' \/ u2 qenabled=11 J# @* _8 L) b' S
) C* C) f9 n3 U' B. |
[centos-openstack-victoria]6 G6 ]9 |$ r* B( ]. `: X! j
name=CentOS 8 - OpenStack victoria: O# H8 b6 c& K$ x+ Y
baseurl=https://mirrors.aliyun.com/centos/8-stream/cloud/x86_64/openstack-victoria/
) @/ i$ U+ z* W! Q7 M, K6 |$ V#baseurl=https://repo.huaweicloud.com/centos/8-stream/cloud/x86_64/openstack-yoga/
p6 \$ D- h6 c2 ^gpgcheck=1) j7 A9 a0 d+ e3 C5 \& n! U- X) {, B
enabled=1# z$ S" A$ @$ Z6 h
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud" L( w. X7 @4 |& K6 M' W
module_hotfixes=1
, I; x/ ]$ W$ s: @* G+ o6 [4 D1 ]+ ]! R+ ^" Q
[powertools]% U! W5 m3 g1 w2 g9 w
name=CentOS Stream 8 - PowerTools# H) e6 ^' S& N
#mirrorlist=http://mirrorlist.centos.org/?release=&arch=&repo=PowerTools&infra=# _4 [, F8 r) A
baseurl=https://mirrors.aliyun.com/centos/8-stream/PowerTools/x86_64/os/% ^/ N8 D5 x6 L w
gpgcheck=1
m- i+ {& y- E* [* denabled=1" w. D0 P/ T+ y3 O9 m4 b b2 N7 ?
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial# g: v$ K8 b! N$ A* R" G. |2 W4 y5 a" w
. Y; K" {- l( z% ^- H8 K5 \
# yum clean all 清理缓存4 ~. V" M" i7 k2 K( V" ^9 j, q% K
# yum makecache 重新建立缓存
' a. C8 u2 S! l# yum repolist all 列出yum仓库（13个）
' z4 D b8 q" J! |6 s: P+ w& b3.安装基础包及ovs（Tab补全命令，安装bash-completion包后执行bash就行）
8 r4 w$ _0 G- q+ n% ?安装openvswitch3.1过程报错说找不到gpgkey文件就禁用gpgcheck=0再次安装就行了
% X0 [- D2 E5 S" Y: B8 L' Y; Qyum install -y vim net-tools bash-completion centos-release-openstack-victoria.noarch tcpdump openvswitch3.1% \/ h" J1 o: h4 v: I" _; i4 C7 ]
或再单独安装yum install -y openvswitch3.1*% B( h# T. N5 R. n1 M
查看安装版本：[root@ovs ~]# ovs-vsctl --version
6 p1 Y# [- i' E4 E- _4.启动ovs服务3 E1 l, z! ~. H
[root@ovs ~]# systemctl start openvswitch
% l( o! m6 Z' K; A1 t- R9 _# G. L[root@ovs ~]# systemctl enable openvswitch
# M, y' [+ J: E H |$ w! f[root@ovs ~]# ps -ef | grep openvswitch- J# Q' z- h" V
[root@ovs ~]# ovs-vsctl show 查看ovs虚拟交换机信息
) C, W* k& {9 D, F' X6 Z[root@ovs ~]# ovs-vsctl --help 求帮助 或[root@ovs ~]# man ovs-vsctl5 @* J }, G8 `
5、创建ovs虚拟交换机( e1 s* f0 q4 @+ C% A( Z- _, o
当创建一个虚拟交换机会生成一个和虚拟交换机同名的Port 和Interface，type为internal（内部的）
& k/ F: E8 `; _5 D
3 p0 ~) W3 ]' l: j* D[root@ovs ~]# ovs-vsctl add-br br-int
2 c. y: h7 H' d- p; n! n& t& {3 k& x" Y[root@ovs ~]# ovs-vsctl add-br br-memeda 添加3 c0 X/ e8 E6 x8 R7 |( B
[root@ovs ~]# ovs-vsctl del-br br-memeda 删除3 R! |9 f+ [6 ?) L7 k/ ]$ a
[root@ovs ~]# ovs-vsctl list-br 查看
; K2 C- g% u3 r Y! Gbr-int
) }0 c2 L R: j X- abr-memeda" O' n. M. l* f v h
[root@ovs ~]# ovs-vsctl show 查询ovs虚拟交换机信息，Bridge桥指的是虚拟交换机& |0 [. @( |) j) O/ O. T
54c67146-9a9f-40be-8cb7-e8792879aafa
# @" d: ~$ ]5 [' c9 G Bridge br-memeda
2 O- Z+ M5 m. u) p. b; `: w1 q Port br-memeda* @( g( `2 F, o1 {/ h" c
Interface br-memeda
% }& [( Q+ b4 P* {5 }0 u& S type: internal
, J& A% W. d- _" j/ W Bridge br-int4 S; J- W7 h# m/ [$ L& u0 Z
Port br-int
& c* x6 s0 ^. [- Q2 w- r }; t3 D, O Interface br-int
, ^% ?7 p! N: r8 Z2 b type: internal8 }8 J7 Y. \0 v$ `: L3 Y( k6 x
ovs_version: "3.1.3"$ p& v) O4 N k% G( k( l' t
用轻量级namespace网络命名空间模拟虚拟机4 m% ^$ E: [0 i O* A) F" @# |
在这里插入图片描述
; y: ~5 X \3 V. S9 k0 d' q+ m6 x
. k$ V6 U+ Y: [7 v& P" V[root@ovs ~]# ip netns 查看网络命名空间
8 `2 A5 Q: I+ A3 E# D8 H4 p[root@ovs ~]# ip netns add ns1 添加网络命名空间
7 E0 u1 U: z, g2 P- F+ r[root@ovs ~]# ip netns add ns2+ t) S; Z1 C0 J$ M' d+ ~$ q
[root@ovs ~]# ip netns. D5 T$ R5 J; C- n' s2 J" y( R
ns2
# E# q' G- Y: w. Zns1
( s! r2 h( J; I: t- y7 i创建两个veth pair（一个veth pair有两个网络虚拟接口，veth可理解为网卡端口） 并将一端虚拟接口（veth1和veth2）连接到两个网络命名空间里面。veth pair：两个网络虚拟端口（设备）。6 V. m' z0 H+ {" E; s3 E
在这里插入图片描述/ f8 X0 \# ?: D: x, l0 x [* k
* E y6 P q4 i* B- v2 v$ v创建两个veth pair，并分别把这两个veth pair的一端放到上述两个网络命名空间
% J3 z: o2 e/ _% F+ A6 M3 Q2 K N# ip link help 或# man ip link 求帮助+ y0 K6 j) }% E7 p" C' e2 S
第一个网络命名空间配置
/ O* ?0 m4 z4 C+ B[root@ovs ~]# ip link add veth11 type veth peer name veth16 i; s' v/ p5 {* \6 A
[root@ovs ~]# ip link set veth1 netns ns1& c$ |( V$ m' v& |; w: {/ Z
[root@ovs ~]# ip netns exec ns1 ip link set veth1 up
3 j ~ J; B+ `7 M第二个网络命名空间配置
9 B- Y% B; T2 h0 f% a$ v( Y8 X[root@ovs ~]# ip link add veth22 type veth peer name veth2% F0 b/ ^* @$ f& H- X( S; I( V
[root@ovs ~]# ip link set veth2 netns ns2
& V5 K# Q# O% E& f0 b+ \ a[root@ovs ~]# ip netns exec ns2 ip link set veth2 up
2 D7 O }3 l& S! p1 [: _8 g将另外一端虚拟接口（veth11和veth22）连接到ovs虚拟交换机上
1 C, M* U/ X8 M! }2 B在这里插入图片描述1 f* W- _4 N. }1 R
6 X, z3 \8 l; L) `' x m2 r[root@ovs ~]# ip link set veth11 up0 G2 {* e3 i7 Z/ z' x& O
[root@ovs ~]# ip link set veth22 up# ~6 N' c" _; I# [2 B: Q% {
[root@ovs ~]# ovs-vsctl add-port br-memeda veth11: ]+ v D4 v- Z0 ^) }+ P6 X: y
[root@ovs ~]# ovs-vsctl add-port br-memeda veth221 E: K6 B$ ^1 O1 q" C+ |9 N* @
[root@ovs ~]# ovs-vsctl show 发现br-memeda虚拟交换机多了2个Port（Port veth22、Port veth11）2 V% L% f; `+ m2 J. S; s! K5 l
3b79f2e1-f433-4015-905e-8945dcada530# x/ w( k$ ]: g
Bridge br-memeda
7 f/ _7 f u' Z( W% T Port br-memeda
5 n1 [% i3 [) u* O2 V Interface br-memeda' \" r$ c% G$ G+ `" P, L) y
type: internal
) C, w+ n- N, @' ~. [* g Port veth22
! R2 z* K, K1 j3 B' l" I/ T Interface veth22! ?' X, M7 l& V) s: t
Port veth11, m* j* ?# i3 d7 C: N" A
Interface veth11* S: a: T( Z3 F0 @' Q" u$ W) x. r1 T! M
Bridge br-int
* w) d8 S, T) F' H Port br-int F$ G& O# B& [5 P+ d& q" C4 R
Interface br-int1 u! L9 r+ t) b; i- j; e' h
type: internal+ [$ c# |9 ]4 Z
ovs_version: "3.1.3"/ }% r8 o; N* s* ^/ j
为两个网络命名空间手动设置ip地址
) j' C) Y# T6 l( S在这里插入图片描述8 G+ f9 T9 O, u: b. I- `. q3 A
; P# n! I2 C. M w( q4 O" G[root@ovs ~]# ip netns exec ns1 ip addr add 1.1.1.1/24 dev veth1
* U8 P1 o9 v3 Y) c[root@ovs ~]# ip netns exec ns1 ip a
" }- B' ]9 W, T$ L! n. e1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
* V j2 Z, m; V# k$ q/ `) a link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00& {2 r# y b5 A1 B+ n
7: veth1@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group d efault qlen 1000$ t7 i. {, w$ z1 ^8 F2 m6 R1 D$ k
link/ether fe:f9:3b:cb:9b:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0( M) P& y& k5 P6 ]6 j
inet 1.1.1.1/24 scope global veth1% {3 i' d, M, z8 B, C- @
valid_lft forever preferred_lft forever4 e4 k5 O3 F* L
inet6 fe80::fcf9:3bff:fecb:9bc5/64 scope link& t, D5 M1 T8 L% ^1 o; g: n
valid_lft forever preferred_lft forever
- X+ s7 S/ G% T1 U% N[root@ovs ~]# ip netns exec ns2 ip addr add 1.1.1.2/24 dev veth2$ @( [/ H2 {) @6 n# t$ J
[root@ovs ~]# ip netns exec ns2 ip a
/ B' Z) \1 y# b1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000$ T3 }' X6 Y. W) D. C4 I L+ t9 z
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
7 Q1 e- r' f) o$ X$ t5 r9: veth2@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
3 r4 Y8 H4 w8 h link/ether 0a:e3:ac:a8:f3:bc brd ff:ff:ff:ff:ff:ff link-netnsid 0
6 h; t4 M& r( n inet 1.1.1.2/24 scope global veth2
5 L: T9 Z* S9 v& G valid_lft forever preferred_lft forever
( R7 l4 ?. O# R9 @# } inet6 fe80::8e3:acff:fea8:f3bc/64 scope link. h L7 ^8 Y. T2 u
valid_lft forever preferred_lft forever( H4 ?: ^+ N' {
两个网络命名空间测试连通性
4 X" M$ |% d+ x9 m Z[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2' \, u e7 @6 T6 Y1 B( v
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
8 S8 u* f& q5 ^6 Q) b" {) m64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=2.98 ms/ L6 w3 E0 e3 a" }& w1 H
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.167 ms& E6 h; a3 g' _
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.081 ms
+ f- p0 N) g& [6 h& d( H) z( f) _* p; T
--- 1.1.1.2 ping statistics ---$ b0 S8 F" C7 \. {4 j1 u5 a
3 packets transmitted, 3 received, 0% packet loss, time 2065ms
! V; t$ ]3 G+ u6 U5 p h& Wrtt min/avg/max/mdev = 0.081/1.075/2.979/1.346 ms
( u$ }0 A# p0 M; V& y0 q) p[root@ovs ~]# ip netns exec ns2 ping -c 3 1.1.1.1
# g* _4 v5 J6 ?: C$ ^! g) OPING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.% j% ^9 V; ^. C
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.923 ms4 M* w0 ` ^' B Q
64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.084 ms1 z" l) W) ^: ]
64 bytes from 1.1.1.1: icmp_seq=3 ttl=64 time=0.091 ms
+ j1 T& D! q1 \8 m# }( G0 h$ R- G, ^% R4 a# Z& h0 j
--- 1.1.1.1 ping statistics ---
1 z2 j# s! \% n4 O/ g3 h4 N3 packets transmitted, 3 received, 0% packet loss, time 2007ms1 a. X5 V1 h+ s+ ^9 S, O' a
rtt min/avg/max/mdev = 0.084/0.366/0.923/0.393 ms
- F; o T# v: I. \5 |/ p) ovlan虚拟的本地局域网，vlan隔离为了减少网络阻塞和数据包安全 A T0 H! `) x! X3 v; ?$ P
ovs虚拟交换机能和物理交换机一样定义vlan，一个vlan10（tag10）,一个vlan20（tag20），把插在ovs交换机上的两个虚拟网络设备对端口分别打上不同的tag（默认是0），也就是配置到不同的vlan里，再验证网络连通性。
# }( ?( p" x' M/ E! C在这里插入图片描述
3 e' |3 D. j, I; s* ?
; W/ }" M l8 o[root@ovs ~]# ovs-vsctl set port veth11 tag=10- ~5 \* }. w) i9 a2 s
[root@ovs ~]# ovs-vsctl set port veth22 tag=205 k" j( g' u8 P
[root@ovs ~]# ovs-vsctl show 发现br-memeda虚拟交换机的Port veth22和Port veth11下面多了tag标签
/ `$ X! Z# n c9 t; n3b79f2e1-f433-4015-905e-8945dcada530
0 I( a, l8 p/ R& T$ I Q- n Bridge br-memeda
4 i! w! Y3 V, G" {$ j7 h" B Port br-memeda! D: m' I1 u1 Y- u( @# n
Interface br-memeda
6 w7 a% u1 i8 w0 S0 S3 U& S type: internal4 x7 O% t% W8 x: `1 [$ i
Port veth22; _. O: F* i# A2 Z4 ]
tag: 20
% g- x, L j7 _& k Interface veth22
+ z" X+ F4 |, j* d. x) D Port veth117 W4 u5 Y4 C1 N+ m( w$ g5 M
tag: 102 W: y f" t8 P# k) v3 J( t
Interface veth11
/ ^, j+ a; u3 k p3 X" [8 _& I+ V Bridge br-int! ]9 F. R, u, R) I3 g
Port br-int
* N7 | ^0 i5 z* \$ w0 r7 l1 l" ] Interface br-int
8 T) \! K6 @. B2 s! v/ f: j type: internal
: W D, D6 a! {% {. M) L& _ ovs_version: "3.1.3"
9 N& S& Z, L3 _+ [& a. r1 F添加不同vlan（tag标签）后ping不通，需借助路由或物理三层交换机
$ o4 n; Y( q: b; j2 X* r0 c7 v8 Z1 g: ?7 G. \5 x
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2+ a, }5 V0 E+ b- q+ H8 Q- u
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.6 V/ _0 A$ w F3 x$ R" j
2 h$ e. I6 X! U1 \1 |
--- 1.1.1.2 ping statistics --- j( X- g' @2 Y
3 packets transmitted, 0 received, 100% packet loss, time 2064ms
& k. k2 z- R7 o: w/ }# [在这里插入图片描述
, P5 W% I; P. F9 n+ v* n- L2 x) T+ X( _
[root@ovs ~]# ovs-vsctl set port veth22 tag=10 把veth22也改成tag=10就相当于同一个vlan二层互通了2 B5 F3 X$ \' N C
[root@ovs ~]# ovs-vsctl show
/ L% N) y* r* o3 y3 t0 V5 t3b79f2e1-f433-4015-905e-8945dcada5303 e4 ?5 e z: |" [4 {; O
Bridge br-memeda3 ?8 `* m, i2 O. [# O% C
Port br-memeda3 g+ p8 i I- }& d. K0 ^. y$ j/ P
Interface br-memeda
! N2 ?8 {8 D R S type: internal
" R, r3 i- e/ o4 K0 M- ` Port veth228 d% z, O+ B& R! V+ }9 r; N
tag: 10+ G/ T" V4 r& p v
Interface veth225 i7 r9 }+ `; j2 [1 J& ~$ s
Port veth11
( h( Y: m3 D% Z2 @ tag: 10
( b6 f1 @4 d: z: F/ a4 q6 | Interface veth11+ Y7 R$ U' y( ^
Bridge br-int; L& I; F7 Y$ V! X
Port br-int
2 v1 I2 v3 ^4 H s6 F- S2 [- m! K Interface br-int
* m, A$ S% I4 b( @& d8 |1 { type: internal- U" c6 Y R" X
ovs_version: "3.1.3"
5 q* T5 a( m5 f% d[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2 同一个vlan（tag标签）能ping通进行二层通信 o2 S. {6 O E! I- J& ^
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.- I1 D$ |- b3 A- d% m
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=1.43 ms$ T; U3 \7 d# b8 \+ A
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.093 ms; \* M% o# F: i- g, e
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.086 ms8 x: f4 E$ U% U5 Y- i
6 A. B0 q' r9 p @--- 1.1.1.2 ping statistics ---, X1 X( |8 l4 n; r. d. m+ q4 Y
3 packets transmitted, 3 received, 0% packet loss, time 2051ms w- E. T1 R" B
rtt min/avg/max/mdev = 0.086/0.535/1.426/0.630 ms
2 a5 N$ U+ i/ v) v* `FlowTable：流表，ovs进行数据转发的核心功能，定义了端口之间的转发数据规则。每条流表规则可以分为匹配和动作两部分，“匹配”决定哪些数据将被处理，“动作”则决定了这些数据将被如何处理。5 N8 ~" @1 h# l: |! o9 l
流量走向，添加流表，针对流量进口添加规则。" u& ^/ z2 W' o9 M, J) F* V7 N
在这里插入图片描述
# n! y8 Z! B$ p! u' r( P在这里插入图片描述# f, W$ Y; a' Y- L1 x8 O
$ T9 s+ Z- T5 x. _% ?% ~* C
查看ovs默认的流表# v6 J1 O* G- A$ G
[root@ovs ~]# ovs-ofctl dump-flows br-memeda 查看虚拟交换机的流规则
1 H! a* \8 I) t6 g cookie=0x0, duration=2161.884s, table=0, n_packets=49, n_bytes=3682, priority=0 action s=NORMAL- L, b- T& F3 m/ k2 b
此时ovs就类似于传统交换机，我们给ovs交换机添加一条优先级为2（数字越大优先级越高，高于默认表项的0优先级）的流表项，把veth11进来的请求都drop掉，发现ns1不能ping通ns2。
7 w" s% @3 h6 Z1 m0 n% w3 w4 h4 @* W1 t[root@ovs ~]# ovs-ofctl add-flow br-memeda "priority=2,in_port=veth11,actions=drop" 添加流规则
0 ^0 @2 W; z) D! E[root@ovs ~]# ovs-ofctl dump-flows br-memeda! N5 R8 W% O/ [) ~# @- y) ]9 e6 ~
cookie=0x0, duration=2.578s, table=0, n_packets=0, n_bytes=0, priority=2,in_port=veth11 actions=drop
: V. h4 W _, R! d1 e z cookie=0x0, duration=2217.329s, table=0, n_packets=49, n_bytes=3682, priority=0 actions=NORMAL6 u. L) y( q. A; z4 ]
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2! l+ v7 X+ i$ y- |, o
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
& ~5 i8 }3 J+ _ }4 p3 Y$ e3 A& B e# D8 y6 K! h7 g) F
--- 1.1.1.2 ping statistics ---
W8 f7 x# L# q, s( j: a4 ]3 packets transmitted, 0 received, 100% packet loss, time 2076ms6 v; {# u) h" s1 N7 R
删除刚添加的表项，ns1与ns2又能正常通信
( b8 I8 E0 G8 e8 ?& @[root@ovs ~]# ovs-ofctl del-flows br-memeda "in_port=veth11" 删除刚添加的流规则就互通了; {% S2 T# p: L) |; H5 s
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2$ q; x5 e0 T9 |8 x" R
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
! W+ y& Z! [% g% |+ h+ r* R64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.766 ms7 j; c3 h! b( e0 c# _+ v
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.096 ms
7 l0 [% X4 i8 l7 B6 D# ?64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.088 ms0 K( i* N* q0 Z/ }3 [8 _
, d" d7 [3 @ m% a* j( Q0 R--- 1.1.1.2 ping statistics ---
$ S4 G# K+ r$ }3 packets transmitted, 3 received, 0% packet loss, time 2043ms
+ l! l% N. ~& ^1 _* ~rtt min/avg/max/mdev = 0.088/0.316/0.766/0.318 ms& l( Y+ }+ K* C+ T
[root@ovs ~]# ovs-ofctl dump-flows br-memeda
{4 F3 R: t: f5 A6 [9 g cookie=0x0, duration=2315.744s, table=0, n_packets=59, n_bytes=4438, priority=0 action s=NORMAL
6 u4 x8 ?$ p! f7 P. h3 U Y& l4、OVN0 }) Q4 G; `+ }) f7 |' J ~
OVN建立在OVS之上的，遵循SDN(Software Defined Network，软件定义网络)架构来管理的，用软件将控制面和转发面分离，OVN做控制面，OVS做转发面。
# G% q0 I a" j9 d8 g6 Fovn是建立在ovs之上的，ovn必须有底层的ovs，ovs可理解为二层交换机，ovn可理解为三层交换机。
" a3 U8 {" G" G1 y( [: z. T( j6 |5 @4 SOVS介绍参考：https://mp.weixin.qq.com/s?__biz ... 189#wechat_redirect. o9 E; c, b% N. i& Y' S5 L
单纯的ovs在云计算领域还存在着一些问题，例如：! n& |7 i& K* p9 ?0 z7 ]
1、ovs只能做二层转发，没有三层的能力，无法在ovs上进行路由配置等操作；
2 x) X( V# G1 w2、ovs没有高可用配置；
0 m' ^7 M9 i: |3、在虚拟化领域vm从一台物理机迁移到另一台物理机，以及容器领域container从一个节点迁移到另一个节点都是非常常见的场景，而单纯的ovs的配置只适用于当前节点。当发生上述迁移过程时，新的节点因对应的ovs没有相关配置，会导致迁移过来的vm或者container无法正常运作。, m6 t, x8 p# n# Z6 s5 m" e
针对这些问题，出现了ovn(Open Virtual Network)，ovn提供的功能包括：6 O; G9 g6 h% p7 ^7 r
1、分布式虚拟路由器（distributed virtual routers）
6 ]- q+ |/ s( l* Q- M6 ]2、分布式虚拟交换机（distributed logical switches）
7 y& V$ c% u/ T1 f! X$ N3、访问控制列表（ACL）
: R7 |$ b ]# G4、DHCP) J1 ^# _1 ^$ R" [* h% U5 ~
5、DNS server/ q; _; M- b8 N
在openstack里面，创建一个网络，就相当于创建了一个逻辑虚拟交换机，这个逻辑交换机（网络）信息会被保存到北向数据库里面。openstack创建一个网络，会以逻辑交换机（switch）的形式保存到北向数据库。
7 n/ }3 s4 H4 y在这里插入图片描述4 Z# x1 l, }" [5 J) l7 V. @) }
在这里插入图片描述
4 H: n' q- K9 kovn官网对ovn的逻辑架构如下所示：
; V n }+ w5 e" c
+ U6 F$ e% M; i+ \2 F! m( e CMS2 R& g! y+ Y! K, g% i, [% v
|
4 p+ S' l4 q. P# Q S |
5 g7 p2 T9 h) H. Q, @ +-----------|-----------+/ F9 S3 R2 H' } i/ X) Z: c* ^4 U
| | |0 J4 l, r$ M6 f
| OVN/CMS Plugin |% a0 l5 Q S/ t1 R# P! ?# o
| | |
) c; d2 b. t( b; _: E | | |0 C# c" P+ k, o, _+ g
| OVN Northbound DB |4 g1 @( Q6 i: L$ w8 s# T
| | |6 [- ?3 d; b1 ^9 U/ h
| | |
1 O. B) l1 q9 \5 w/ j9 j | ovn-northd |# D% N$ Q' g; e' ?6 p+ O
| | |4 Y! L k, X- z( b! {7 F
+-----------|-----------+9 w, m( r/ R: V) c! `* Z7 f m) o
|1 @4 u: K$ g; b( h+ W) V
|3 A( t* u5 U2 n) d% S# @
+-------------------++ f3 @# h, t* u2 q
| OVN Southbound DB |
" G! q8 t4 e0 U+ h7 F +-------------------+
2 l" p2 p' L0 u) @' S |
2 X) V1 H y; J$ r3 g |
, l6 D/ @: [' F +------------------+------------------+
T1 N+ [6 I2 d' e2 V/ x- w, P | | |6 W! J9 t6 [! Q( W. r9 P$ E: m
HV 1 | | HV n |8 i. Y4 I6 @7 }3 p9 |" |; L
+---------------|---------------+ . +---------------|---------------+
9 R0 |4 s! D8 _! o | | | . | | |
& w. K4 p8 b7 I$ F | ovn-controller | . | ovn-controller |
1 D1 H+ a$ J3 |; V& h | | | | . | | | |- [5 x1 H8 H6 {, I
| | | | | | | |. k9 z, w* X- ?, ] t
| ovs-vswitchd ovsdb-server | | ovs-vswitchd ovsdb-server |4 V- B1 f% N" o: v$ D8 w! @2 v, ?
| | | |- ~# o8 Q! U2 ^) w8 ?, Q% ?
+-------------------------------+ +-------------------------------+
# Q. g3 F% _: Uovn根据功能可以把节点分为两类：: [/ X/ F% u9 g) S. `3 y
central: 可以看做中心节点，central节点组件包括OVN/CMS plugin、OVN Northbound DB、ovn-northd、OVN Southbound DB。2 ?* P9 u4 ?% B$ p5 m$ S2 a& F
hypervisor（hv）: 可以看做工作节点，hypervisor节点组件包括ovn-controller、ovs-vswitchd、ovsdb-server。
: m Z4 _; D& G8 l3 C# B) Ecentral节点相关组件和hypervisor组件运行在同一个物理节点上。 K+ \3 [1 Z- J
相关组件的功能如下：
' E, X- Z/ c, V1、CMS: 云管软件（Cloud Management Software），例如openstack（ovn最初就是设计给openstack用的）。, l* [# Z$ m" }: D2 @/ ^. }
2、OVN/CMS plugin: 云管软件插件，例如openstack的neutron plugin。它的作用是将逻辑网络配置转换成OVN理解的数据，并写到北向数据库（OVN Northbound DB）中。5 \* U `+ _ F# |* m% ^. \
3、OVN Northbound DB: ovn北向数据库，保存CMS plugin下发的配置，它有两个客户端CMS plugin和ovn-northd。通过ovn-nbctl命令直接操作它。北向数据库保存逻辑网络信息（交换机和路由器等）
& `& G1 o4 @7 n% G3 m7 s L. G4、ovn-northd: 北向进程将OVN Northbound DB中的数据进行转换并保存到OVN Southbound DB。所有信息经过北向数据库通过ovn-northd北向进程和南向数据库互通。
7 c$ l, Q* R2 ?7 V- J" E5、OVN Southbound DB: ovn南向数据库，它也有两个客户端: 上面的ovn-northd和下面的运行在每个hypervisor上的ovn-controller。通过ovn-sbctl命令直接操作它。南向数据库保存各个节点的物理网络信息。& _7 v* i; Z: V- P) i
6、ovn-controller: 相当于OVN在每个hypervisor上的agent（代理）。北向它连接到OVN Southbound Database学习最新的配置转换成openflow流表，南向它连接到ovs-vswitchd下发转换后的流表，同时也连接到ovsdb-server获取它需要的配置信息。/ @, W; [5 K: b8 z& x: S, t( F# e0 X
7、ovs-vswitchd和ovs-dbserver: ovs用户态的两个进程。
! m7 @8 T# Q0 H每个节点都有个ovn-controller控制器，这个ovn-controller控制器是管理ovs（ovs-vswitchd、ovsdb-server）的，ovn-controller对接到南向数据库，经过ovn-northd北向进程和北向数据库互通，之后和openstack互通。 }$ M7 v# j% F/ l3 X v
南向数据库保存物理网络状态信息，北向数据库保存逻辑网络状态信息。# U: V; T7 W$ I5 C2 w) K
在这里插入图片描述& j0 e2 L7 v8 K$ U/ X; ]+ f
克隆出两台虚拟机，安装ovs、ovn0 y: A. R: z7 g/ N* h
' B+ J! n# o2 D v3 }CentOS Stream 8 版本. [* Q; y. N# R' G2 y
! q9 _- D4 h, f! {
systemctl stop firewalld.service
* ]* H1 r+ G; p$ jsystemctl disable firewalld.service/ e2 F* l1 ]2 I/ L8 s
setenforce 09 x8 ~( q$ j6 r [$ b+ l" m ?
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
$ X J4 U3 S9 s2 v e6 ~0 tmkdir /etc/yum.repos.d/bak
' ]: m. J5 \. F e( E1 Dmv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/
& ]7 u9 O1 R& _2 b. B& R" f! k/ l) p9 S+ p) o: b6 q
cat <<EOF > /etc/yum.repos.d/cloudcs.repo9 e) v# u. R' o
[ceph]* H! H; Q" q5 H6 s
name=ceph
2 ?" F+ s1 T& X" jbaseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/x86_64/
* `4 N( M& U4 ^; G- ygpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
7 Y+ U+ K# ?9 X3 ~0 q' f3 b& m$ cgpgcheck=1
( G! a& n4 K+ f6 d% n( Penabled=1
# A3 _" h' X0 e' y: w) B: K' n: [9 |5 Z) O! A
[ceph-noarch]
4 u z3 g8 p9 W) ?+ Iname=ceph-noarch
! p: q% q+ z; r! E, D) y/ v% g" R3 qbaseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/noarch/9 j5 N1 S9 {( y, R
gpgcheck=1% h0 @, p9 C& A' d4 }
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
& b% s x$ l0 F0 Denabled=1; f% `/ M3 c/ m+ | B7 w0 t5 n$ @
& }1 G! c6 S7 s7 j4 Z- Y+ Z3 l4 w[ceph-SRPMS]
% c0 h( i7 E& Y6 \; @name=SRPMS
' B+ P( ~7 L2 A: w3 ?, Qbaseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/SRPMS/) Y( h/ _. @0 A
gpgcheck=1
9 P. x" g; ^; p$ mgpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc6 T& ?, Z6 B- T, Y5 Z
enabled=1" J' Z9 O3 m, t" E% f
1 ^& ~0 A8 M# V3 K8 |[highavailability]( L6 w) T! v' r$ h/ r# O
name=CentOS Stream 8 - HighAvailability6 D2 j2 s1 ?8 W" J! }+ z
baseurl=https://mirrors.aliyun.com/centos/8-stream/HighAvailability/x86_64/os/
. l; B! |+ c3 R& E5 s* wgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
0 g' i! ^6 d! l6 d/ a+ I* Dgpgcheck=1
- l" }: n3 a/ V4 M3 ]0 V9 @repo_gpgcheck=05 Y$ O+ m" V6 Z; y, T
metadata_expire=6h
, {! {) c6 n% @countme=12 n2 d7 R( L8 ~
enabled=1
7 \9 M/ ~8 a! p! H/ |7 W E7 } M1 p$ ~" N5 j. s! {
[nfv]6 z7 ^; }4 g/ M% U0 v# M
name=CentOS Stream 8 - NFV
: S e; {6 P! z( `: ^. hbaseurl=https://mirrors.aliyun.com/centos/8-stream/NFV/x86_64/os/: I5 P, g& A3 [6 C' R% [7 v f8 H
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial' A. ?; P0 u2 U0 u
gpgcheck=1) W2 c. |- U* r9 U2 _8 D% Z
repo_gpgcheck=0( M" v6 x, o( E
metadata_expire=6h' z0 ~" U0 a) [" O7 V$ U9 y F$ v
countme=1% e( X, S, R4 e) P% i7 z
enabled=1
) H: A* q" i* i2 L
) o/ O# \' o' I ~8 O[rt]
! Y# y; l8 q8 K% ^. e9 zname=CentOS Stream 8 - RT" z. K# j f7 U r; [, |
baseurl=https://mirrors.aliyun.com/centos/8-stream/RT/x86_64/os/% W3 E$ A, w8 V3 {2 ?9 W5 K( @2 O
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
4 V k2 L( A$ ]2 ?% ]3 mgpgcheck=19 ^* u4 O1 S+ u# V( S
repo_gpgcheck=0" N( D8 D. m" |+ o
metadata_expire=6h" d0 Q; o/ G* V
countme=1! Z0 x1 U9 Q! Y+ F$ t
enabled=1' P8 A/ k$ Q7 M7 ?% j( @2 K
& u% }/ w- l! b, v[resilientstorage]0 ]) }$ E. D5 W. u# f& ~# ^
name=CentOS Stream 8 - ResilientStorage
2 H' \# \ ~5 O4 u% @baseurl=https://mirrors.aliyun.com/centos/8-stream/ResilientStorage/x86_64/os/6 l+ B# B/ p0 C: \" B0 ?$ R
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial4 [2 y- F. G" ?! J+ @
gpgcheck=1
4 G) { V4 u1 x- U' orepo_gpgcheck=0
$ w# \# d9 x# A' lmetadata_expire=6h
3 o( Y& ?# ?9 U4 u+ _1 l" Ocountme=1
, W( Q1 w# i* [enabled=1! ~, X$ ]' x1 V7 K5 G
# C+ N' t: b. A$ ^- {0 M[extras-common]/ _' t" H1 q, w3 a9 S
name=CentOS Stream 8 - Extras packages" r @$ ~# n# _
baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/extras-common/1 K: d' w& w9 A r, e+ u
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512& G. H: `& E' M4 o; X3 L
gpgcheck=1/ a7 [0 v' ^5 G- g" ?
repo_gpgcheck=0
0 d( O, U7 f( o p8 \' Kmetadata_expire=6h6 U/ D6 I1 v! D) a; O! @
countme=1
: z/ d3 s# d i( |enabled=1
?; ]* c( K0 }9 b) Z/ P4 C% N" ^- k, }7 ^$ x+ o
[extras]
5 s6 N* ]* i7 l: ]1 Jname=CentOS Stream $releasever - Extras1 W# w8 p4 I7 c9 v, H6 K' O
mirrorlist=http://mirrorlist.centos.org/?release=$stream&arch=$basearch&repo=extras&infra=$infra! Y9 ?/ _. v4 X: s3 O
#baseurl=http://mirror.centos.org/$contentdir/$stream/extras/$basearch/os/( B3 B% _+ U0 \' U' d* Q
baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/os/- ^3 b4 [% B) B6 e& n' g1 E6 [
gpgcheck=1
3 H2 n2 f# G0 Z2 X9 venabled=1
/ Y0 `3 S. B0 e% U7 y8 X/ r& S/ n; Hgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial8 A4 v. I; M3 |' Z
0 K2 J n( c' [* M
[centos-ceph-pacific]
, e; I7 y. B* Bname=CentOS - Ceph Pacific
& L6 F- n' M4 H2 y* ?baseurl=https://mirrors.aliyun.com/centos/8-stream/storage/x86_64/ceph-pacific/+ m9 S3 d6 |/ u0 ~
gpgcheck=0
% N! G9 ^6 f: [9 J# Wenabled=13 ~$ p) X6 Q3 u+ F
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
! y/ _' ]0 K! z& w. ^; d- Y5 b* k& G+ M0 \
[centos-rabbitmq-38] |1 Y, e' }0 A0 g# j4 k! E! g h6 o
name=CentOS-8 - RabbitMQ 383 ~- R" x# } P1 w7 j% \+ y4 O; ?
baseurl=https://mirrors.aliyun.com/centos/8-stream/messaging/x86_64/rabbitmq-38/
3 Y, g ~& ?; w+ K+ I+ E; Ugpgcheck=1: ^7 U1 D' @( L8 j% a* Y
enabled=1) a* ] f' g @# ^- n' e% V- V
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging
$ R# C$ ?: H g; F2 a! C, K( z* ?1 F5 u, M* f
[centos-nfv-openvswitch]
- l4 F4 p8 R, i o3 N: Iname=CentOS Stream 8 - NFV OpenvSwitch
6 k) H \1 W) gbaseurl=https://mirrors.aliyun.com/centos/8-stream/nfv/x86_64/openvswitch-2/
3 L% i; k' I5 n2 |' i7 I7 }" f2 Igpgcheck=1
. e' G, T' U; `+ V" }enabled=11 o3 I/ V: K9 Y3 e) f
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV* A& v* E0 R2 T. u# {
module_hotfixes=14 L3 S- v+ q& Q* q5 Q
9 o% c! j4 P0 Z8 V[baseos]% }2 H) Y2 n2 h( V
name=CentOS Stream 8 - BaseOS
' [7 m/ O( G! s1 j! Z) {baseurl=https://mirrors.aliyun.com/centos/8-stream/BaseOS/x86_64/os/% q( y6 z# K4 ~* z
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial: S# B' Z2 C1 |5 Q" F0 y
gpgcheck=1, t. G- `' Y) M! Q
repo_gpgcheck=0
4 S7 u5 @- \% B2 `metadata_expire=6h
' t) E8 d+ r: P; w2 icountme=1
% T+ e) @: _4 n& k3 {2 g# G6 R' Ienabled=1
/ U- L/ V1 J0 v9 n1 U7 k" |3 t2 p* r, G5 \* E& I% u
[appstream]3 J3 D2 H1 I, V% U C3 K
name=CentOS Stream 8 - AppStream7 _3 O. f- R/ O% t$ b
baseurl=https://mirrors.aliyun.com/centos/8-stream/AppStream/x86_64/os/
( h& u S$ J% l. C. ygpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
; ?8 O R8 C9 Pgpgcheck=1
! ]" B" x1 y# c$ E7 X) Prepo_gpgcheck=0! S0 c& s$ S7 h
metadata_expire=6h6 ?5 n y- y( ~0 h7 A
countme=12 c) X7 \5 `: ~8 j; X: W8 ?
enabled=1# P( z& o$ W! Y: s
3 a Z7 ~' F# A- A8 g7 E: m
[centos-openstack-victoria]2 T0 r1 c/ \9 j2 E
name=CentOS 8 - OpenStack victoria
: L$ _( ^, S$ |+ G4 }/ dbaseurl=https://mirrors.aliyun.com/centos/8-stream/cloud/x86_64/openstack-victoria/- O; R9 C( ~$ k4 ], W3 [) C7 T
#baseurl=https://repo.huaweicloud.com/centos/8-stream/cloud/x86_64/openstack-yoga/
7 R1 _) Z/ p2 K, vgpgcheck=1
; E; p3 A$ O& i) E% N3 ?% kenabled=1
7 i d( x {* G9 T! B/ |6 x' w, `gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud7 j: A! X2 h/ O
module_hotfixes=1
) \8 q% ]7 X" I! _! N9 C: w% Y5 W
. \- R" m, {! c[powertools]4 R8 H9 G2 e( z) J6 N$ ?
name=CentOS Stream 8 - PowerTools
3 i% t9 A$ q" \% R( B. |* M- {0 r( r% E; M& v#mirrorlist=http://mirrorlist.centos.org/?release=$stream&arch=$basearch&repo=PowerTools&infra=$infra$ ]3 m* n# j/ |" f$ ^. @
baseurl=https://mirrors.aliyun.com/centos/8-stream/PowerTools/x86_64/os/& J. F! O! m6 z
gpgcheck=14 K0 \' H# ] R7 W5 T, P/ \: n
enabled=1 j' C2 ?' L G
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
$ j; ]5 h' y0 C' }. p; ZEOF+ c' ?* k4 C z0 k% Z' o* M7 V8 e9 \
2 R6 s2 q4 n% d2 S: v
yum install -y vim net-tools bash-completion git tcpdump autoconf automake libtool make python3 centos-release-openstack-victoria.noarch! |) p0 _5 y& [% ~9 ?$ z
yum install -y openvswitch3.1*6 g+ v% a( Y1 x/ e0 S! Q& S
yum install -y ovn22.12*
% g. O8 j$ \- p查看安装版本来检查ovn是否安装成功，# ovn-appctl --version, I$ m6 L& b% A' U9 |" \. @6 \
echo 'export PATH=$PATH:/usr/share/ovn/scripts:/usr/share/openvswitch/scripts' >> /etc/profile
& m1 L& `; _0 }$ |4 q( b! R3 xsource /etc/profile 重新读取配置文件让配置文件立即生效
) J; `$ m3 p/ P( W7 ?% l在这里插入图片描述
) ?# L# N& s" E% {central相关组件启动：把node1作为central节点，安装central必需的三个组件：OVN Northbound DB、ovn-northd、OVN Southbound DB。
7 s! C9 w4 V' ]在控制节点启动central，只用在一个控制节点上启动即可（node1或node2上开启都行，这里是在node1开启），central只需要一套即可。5 M, ^, x+ i0 k& f
6 A/ l) z) p$ y3 ~3 Lovn-ctl start_northd命令会自动启动北桥数据库、ovn-northd、南桥数据库三个服务
9 I- D6 f. z& |9 T" @7 p( Q[root@node1 ~]# ovn-ctl start_northd$ X( |& \; w$ b
/etc/ovn/ovnnb_db.db does not exist ... (warning)./ d, \ h- }' ]
Creating empty database /etc/ovn/ovnnb_db.db [ OK ]
+ ]( z6 s9 l$ g4 d( C, G# yStarting ovsdb-nb [ OK ]9 `' Z1 a; e+ H2 D
/etc/ovn/ovnsb_db.db does not exist ... (warning).
# Q" k" o f7 K$ _- r4 ]# T) c# NCreating empty database /etc/ovn/ovnsb_db.db [ OK ]
! w1 E9 Q" {1 T: y3 y* L0 a, VStarting ovsdb-sb [ OK ]- v4 T5 M' ]. |+ `1 y
Starting ovn-northd [ OK ]- H" C, K. W/ h+ o- [# Q' h
: d3 M8 t0 R! E, [! J% i' x- B[root@node1 ~]# ps -ef | grep ovn4 D8 v( v+ J) I
root 34102 34101 0 21:02 ? 00:00:00 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --detach --monitor --remote=db:OVN_Northbound,NB_Global,connections --private-key=db:OVN_Northbound,SSL,private_key --certificate=db:OVN_Northbound,SSL,certificate --ca-cert=db:OVN_Northbound,SSL,ca_cert --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /etc/ovn/ovnnb_db.db
( d. E- p( d2 Q; }9 v; W9 iroot 34118 34117 0 21:02 ? 00:00:00 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-sb.log --remote=punix:/var/run ovn/ovnsb_db.sock --pidfile=/var/run/ovn/ovnsb_db.pid --unixctl=/var/run/ovn/ovnsb_db.ctl --detach --monitor --remote=db:OVN_Southbound,SB_Global,connections --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers /etc/ovn/ovnsb_db.db% u( C6 h. X! M
root 34128 1 0 21:02 ? 00:00:00 ovn-northd: monitoring pid 34129 (healthy)
! l* Q/ I0 G' s" q% e+ droot 34129 34128 0 21:02 ? 00:00:00 ovn-northd -vconsole:emer -vsyslog:err -vfile:info --ovnnb-db=unix:/var/run/ovn/ovnnb_db.sock --ovnsb-db=unix:/var/run/ovn/ovnsb_db.sock --no-chdir --log-file=/var/log/ovn/ovn-northd.log --pidfile=/var/run/ovn/ovn-northd.pid --detach --monitor
7 s9 w) Z) e' yroot 34302 34259 0 21:07 pts/0 00:00:00 grep --color=auto ovn
) Z6 o( S3 l2 v: S在这里插入图片描述/ f" a t5 M& i f; Z( r0 l
hypervisor相关组件启动：hypervisor节点包含三个组件：ovn-controller、ovs-vswitchd和ovsdb-server。5 b* k# ?6 p' m: Z' s5 x
启动hypervisor（hv）相关组件：node1和node2两台节点上都要启动，首先启动两个节点上的 ovs-vswitchd 和 ovsdb-server
5 l( ]% y) \, t) \
# L: a1 V( a0 B8 @+ Q4 i- @: L[root@node1 ~]# ovs-ctl start --system-id=random8 C. d* W7 c- E* V( g6 ~" S
/etc/openvswitch/conf.db does not exist ... (warning).3 L- m. P3 ~6 r# A, b
Creating empty database /etc/openvswitch/conf.db [ OK ]
p: r2 ]% G$ NStarting ovsdb-server [ OK ]
@' ~2 V* |2 UConfiguring Open vSwitch system IDs [ OK ]9 [/ g' T# m) N( P1 B: P
Inserting openvswitch module [ OK ]; y$ V: {1 O# G3 k( A
Starting ovs-vswitchd [ OK ]
8 p3 B3 _: _4 C9 E! A. ^: g6 cEnabling remote OVSDB managers [ OK ]9 C; M) t; p5 R; h0 k: ? W- u
& Z& [0 x# r8 g% R+ f: m
[root@node2 ~]# ovs-ctl start --system-id=random; Q& D* ?( a) k- @& U3 f4 [
/etc/openvswitch/conf.db does not exist ... (warning).' j) H+ o8 |3 m: p- u
Creating empty database /etc/openvswitch/conf.db [ OK ]8 h0 s; o# f9 M, S+ x) a
Starting ovsdb-server [ OK ], v) q5 q2 Q4 v( k. z' ?
Configuring Open vSwitch system IDs [ OK ]. |( W9 D& L( e3 {2 \( }0 v6 T) g
Inserting openvswitch module [ OK ]
0 ~9 A6 {( Y z7 b; qStarting ovs-vswitchd [ OK ] o- R7 u: X5 i
Enabling remote OVSDB managers [ OK ]2 a3 Q# b# E' U4 G8 Z; O7 l
在这里插入图片描述3 \# B; a. ~* j x4 b
两个节点分别启动ovn-controller& m& j& {2 W" N, [! D& n9 C
' w r0 D* S( i$ e$ o9 I[root@node1 ~]# ovn-ctl start_controller- l* O! u. }$ @. R
Starting ovn-controller [ OK ]
$ E6 _& ?3 l/ c0 {' _5 w[root@node1 ~]# ovs-vsctl show ovn-controler启动后会自动创建br-int网桥
4 Q( b# R' r0 P2 g+ bed157e0c-cac3-46b9-830c-f2d710b475d52 l7 Z1 S2 h2 L7 k0 X6 m
Bridge br-int+ B) B7 M: ?8 \
fail_mode: secure
5 x8 t! C* {6 A: A1 `8 |+ O datapath_type: system, i+ `- L3 E8 ~) z( o
Port br-int
% ~/ j# T) ?# m: i% w* y Interface br-int- \% S0 |. q5 r* B
type: internal
+ d: [" }" z& j$ [ ovs_version: "3.1.3"
* f# k: x' J, \4 c
+ ~) q1 @: L! B' L) H[root@node2 ~]# ovn-ctl start_controller) D& b1 y" p7 V/ x" P
Starting ovn-controller [ OK ]- ` Q4 u, I& u+ F) k+ u `
[root@node2 ~]# ovs-vsctl show ovn-controler启动后会自动创建br-int网桥/ t1 O' N5 E. K5 f/ }
f6669675-b42d-47de-be95-b26bf6d1e069# i5 D$ E _% }6 \/ g
Bridge br-int1 w- N; s- B/ P: E x) X4 w. |/ {
fail_mode: secure
' j( A4 O" G, t1 `% X datapath_type: system
% m# ]/ @$ e: p: Z- o, U Port br-int
$ _& J" S) O+ ] Interface br-int* @( N' d/ B( P* A" z ^
type: internal4 Q$ \% m1 J; V3 [+ C% y
ovs_version: "3.1.3"
' L6 }3 C; u; y8 l" A5 v9 w$ m8 L1 X3 W在这里插入图片描述
5 }; y3 n9 z& |+ q8 x; B可以看出此时hypervisor并没有和central关联起来（也就是ovn-controller没有和南向数据库连接）。可以在node1上验证：[root@node1 ~]# ovn-nbctl show1 l" N$ {& i% J m5 Q* X8 K& {* b
hypervisor连接central，开放南北数据库端口：5 J* ]5 i5 |* G
% c% l) |- x2 D6 O
ovn-northd之所以能连上南向数据和北向数据库，是因为它们部署在同一台机器上，通过unix sock连接# E, G4 L* R1 e9 E
central节点开放北向数据库端口6441，该端口主要给CMS plugins连接使用
& z9 ]9 ?! _, R9 L! I6 _. L6 jcentral节点开放南向数据库端口6442，该端口给ovn-controller连接- ]2 t, c1 t. j3 |1 O7 P5 F% Y/ b& P
[root@node1 ~]# ovn-nbctl set-connection ptcp:6641:10.1.1.41
, n" _ I* G9 U4 S- S8 d[root@node1 ~]# ovn-sbctl set-connection ptcp:6642:10.1.1.41. ]) X: ^: K" _& y, l" P
[root@node1 ~]# netstat -tulnp |grep 6644 X/ z) ? o0 n: X5 g" w) K! L
tcp 0 0 10.1.1.41:6641 0.0.0.0:* LISTEN 34102/ovsdb-server6 S- u9 J* j4 t1 b; b
tcp 0 0 10.1.1.41:6642 0.0.0.0:* LISTEN 34118/ovsdb-server. E' c8 I& W/ b: g, U. z% B
node1上ovn-controller连接南向数据库; S% {8 m8 E- v7 O
ovn-remote：指定南向数据库连接地址
! S' Q+ ]2 a) B1 X6 Bovn-encap-ip：指定ovs/controller本地ip
% N+ v# c! h9 z0 bovn-encap-type：指定隧道协议，这里用的是geneve
3 f. B7 g3 y5 M! ?system-id：节点标识
, E6 E; {) ~; w. z$ Q[root@node1 ~]# ovs-vsctl set Open_vSwitch . external-ids:ovn-remote="tcp:10.1.1.41:6642" external-ids:ovn-encap-ip="10.1.1.41" external-ids:ovn-encap-type=geneve external-ids:system-id=node1$ F. W# p/ b, I
0 T7 h* W. A4 g1 w( v' ]
node2上ovn-controller连接南向数据库
C) ?3 w) ?3 a' x! U[root@node1 ~]# ovs-vsctl set Open_vSwitch . external-ids:ovn-remote="tcp:10.1.1.41:6642" external-ids:ovn-encap-ip="10.1.1.42" external-ids:ovn-encap-type=geneve external-ids:system-id=node2
, j; K7 R8 B2 ~4 k2 Y
, F4 W3 n; s8 M" K, L4 t在node1查看南向数据库信息
* ~/ l& y0 W$ J: o9 B |[root@node1 ~]# ovn-sbctl show
1 d$ F/ m' D4 j9 d; nChassis node2
* p5 P; p) F+ |" K% R% N* C hostname: node2
( v7 O! P" ^: v5 ~, a! x5 \% V5 r$ ? Encap geneve# O% }6 `6 I2 \! k, h; e# k' s
ip: "10.1.1.42"
/ J+ `& ^; m) C: p options: {csum="true"}
: \* B, W1 a8 B2 VChassis node1
6 \8 l( j2 f- f$ j/ T- ~ K3 y hostname: node1/ k0 {: @% o( f* n* w: B4 a1 F
Encap geneve
i) I- \0 O4 Y) G ip: "10.1.1.41"
: t i5 d6 K- N options: {csum="true"}# W5 Z# [* Y0 G) d+ l
在这里插入图片描述
- V7 X6 W, A1 g# J U以上的逻辑架构是站在底层组件和服务的角度来看的。7 Y. e' S8 [ Q/ }
接下来换一种角度，站在逻辑网络的角度来看。
0 e" q! l. n( A7 @* ^( I$ O在这里插入图片描述- f3 P1 w9 ^' q3 m( m" z6 @9 h
geneve隧道：ovn-controller连接南向数据库时，指定了external-ids:ovn-encap-type=geneve参数，此时看看两个节点上的ovs信息如下，会发现两个节点上都有一个ovn创建的ovs交换机br-int，而且br-int交换机上添加的节点port/interface类型都为geneve
+ A& A& k3 O* Y( u8 b; X4 O6 a! i; M6 t& S
[root@node1 ~]# ovs-vsctl show node1上查看ovs信息
. z* E4 a3 X( K; Ved157e0c-cac3-46b9-830c-f2d710b475d50 R7 @3 B9 ]* E* U* x. H
Bridge br-int4 S9 r0 N! ~1 \* d/ [
fail_mode: secure* a6 N6 u+ Z+ U
datapath_type: system
4 H; s T7 P$ _# c; V- ~ Port br-int% o9 q) s/ D8 V0 f Y. }1 [
Interface br-int% g+ O. \6 {) e4 z: p( I8 Q
type: internal$ m, L3 Q$ F1 A
Port ovn-node2-0
# H4 B. G. U+ Z Interface ovn-node2-0
5 u0 S: V' I O5 } type: geneve
9 J3 D9 R. B3 Y, d8 ~! c, N options: {csum="true", key=flow, remote_ip="10.1.1.42"}
0 I/ n2 t4 R6 {0 h/ r! h ovs_version: "3.1.3"
) S0 I8 d5 o6 w5 v( R% X
( A( |2 ~, |1 }8 {, Z9 h[root@node2 ~]# ovs-vsctl show node2上查看ovs信息! l) O1 H- o. o1 H* S( T
f6669675-b42d-47de-be95-b26bf6d1e069+ A) w6 q( l9 g
Bridge br-int
3 X1 ]' E2 X# x6 h fail_mode: secure/ F$ I# F# _4 X' i% T5 X5 l* @
datapath_type: system
, ~0 \( Z2 T5 j9 c; @% g& C3 p& { Port ovn-node1-00 g+ L- H% p4 H9 n1 [! K8 |0 g. r
Interface ovn-node1-0; c/ T. a# M- B% k
type: geneve
3 I. b6 j0 p; q( N' a& F1 M options: {csum="true", key=flow, remote_ip="10.1.1.41"}
2 }9 Q6 C! q1 Y4 Q( D Port br-int. d! S- r5 X$ t! }/ e
Interface br-int
! E# P2 q1 ]$ _" Q' B) ?" T type: internal
9 [( d% M# Q: U/ M, L ovs_version: "3.1.3"* J' ^9 H* g# e6 H5 v" ^4 E
[root@node1 ~]# ip link | grep gene 查看geneve隧道link L: k7 t* r6 \# ?6 k! W2 b1 q! c
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
- A# n; P- }5 I% a查看geneve隧道link详情，从dstport 6081可以看出geneve隧道udp端口是60813 W- Q, `6 Y9 D6 v$ w/ t' m
[root@node1 ~]# ip -d link show genev_sys_6081 6 _1 R! B# q3 x5 V' n. }
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
8 b/ H- o8 m2 ~" z) J/ L link/ether 6a:e3:ff:a5:cc:d6 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 654650 n0 Z6 }4 D' G3 ]
geneve external id 0 ttl auto dstport 6081 udp6zerocsumrx
. ?3 }+ ]. }8 _+ \" `2 i: D( A" z openvswitch_slave addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 655351 @ j, L, \- y' J8 A8 ^
查看geneve隧道udp端口，最后一列为“-”表示这个端口是内核态程序监听
1 M( T2 R; B; c% N0 Q6 W[root@node1 ~]# netstat -nulp|grep 6081
" n7 U$ U2 B+ D9 |udp 0 0 0.0.0.0:6081 0.0.0.0:* -
/ P7 u% D# d! ^5 i& Q: ?$ l" Audp6 0 0 :::6081 :::* -
, M6 F6 Y5 C8 d9 h: l0 e1 D: Z3 ?; H% W6 B7 e9 w
[root@node2 ~]# ip link | grep gene
& {1 m0 G! X0 O& \5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
/ u9 T+ M, e6 ~& V+ v[root@node2 ~]# ip -d link show genev_sys_6081$ n) \4 L ^% D" `7 C9 _2 U
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000( f9 n5 [. `; x+ V: r8 \/ z
link/ether 4e:db:f1:e4:43:94 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65465
4 j% z8 q& Q* v geneve external id 0 ttl auto dstport 6081 udp6zerocsumrx% P& w& J) b! v% W; l# c7 e
openvswitch_slave addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 655354 V9 _" f; A' f' T) p2 t
[root@node2 ~]# netstat -nulp|grep 60819 o% G8 \- d& j! v9 G
udp 0 0 0.0.0.0:6081 0.0.0.0:* -% w8 F0 O# q6 Z1 v- b4 |8 z: e
udp6 0 0 :::6081 :::* -
" J6 Q# |( d' i; k3 N* v- b在这里插入图片描述* I4 V8 e. c( S- o1 a( D
$ l a- [) F2 Y6 l+ i! r在做以下实验验证时需要注意MAC地址的合法性，不要误配置。MAC地址分为三类：) T8 f. U. I# n; }# ]
广播地址（全F）
* l' T" `& ^) S5 Y% G( yFF:FF:FF:FF:FF:FF
* B; G- L6 S, |9 J. f% K主播地址（第一个字节为奇数）
* U7 N6 H+ r- ?8 T4 zX1:XX:XX:XX:XX:XX
( |" x8 I7 q4 n8 ^; W7 Y9 B# pX3:XX:XX:XX:XX:XX2 l, i/ H1 c9 _) k1 g! ~+ w/ g/ C7 {/ }
X5:XX:XX:XX:XX:XX
/ B. Q7 @: q7 t8 g: ?' ^5 kX7:XX:XX:XX:XX:XX
1 C" C4 q# D* _7 g, g( nX9:XX:XX:XX:XX:XX1 d- H: V! t7 \0 y9 K
XB:XX:XX:XX:XX:XX4 A6 Y- f l& b& i7 x- m
XD:XX:XX:XX:XX:XX1 O4 w/ x6 E7 I" c& a! V
XF:XX:XX:XX:XX:XX
' D3 B. I( K' y0 Z可用MAC地址（第一个字节为偶数）
% m- ^: I1 _4 c/ {/ c" i9 F% O- xX0:XX:XX:XX:XX:XX
! ]" \/ S+ D5 S# f3 X5 q$ w9 e5 kX2:XX:XX:XX:XX:XX
/ n& M0 N% D2 p a" PX4:XX:XX:XX:XX:XX( m) g- {3 T; y
X6:XX:XX:XX:XX:XX$ R2 O( @2 z. `
X8:XX:XX:XX:XX:XX
, z- ?+ U) m7 A* u2 M1 EXA:XX:XX:XX:XX:XX
$ h) H+ J' E- \, \0 o3 A) ]$ h3 `XC:XX:XX:XX:XX:XX1 T( D% s3 [! z% K- E
XE:XX:XX:XX:XX:XX
5 \- v _: V+ ]. S7 u8 z( b* @在每个节点上创建一个网络命名空间ns1（因为在两个节点上所以同名ns1不会冲突），网络命名空间可理解为虚拟机，并且在ovs交换机上创建一组port和interfacce，然后把interface放到网络命名空间下。veth pair：两个网络虚拟端口（设备），veth可理解为网卡端口，一个端口在虚拟机上，一个端口在br-int虚拟交换机上。' L6 p( |/ W5 M, a* a1 z
/ e. |, f6 ~3 ]6 P* Dnode1上执行
. H# b- D0 u* |( x4 ?+ ^[root@node1 ~]# ip netns add ns1% f u5 `1 P+ o
[root@node1 ~]# ip link add veth11 type veth peer name veth12 ^4 Q+ I. \, j' d* b
[root@node1 ~]# ip link set veth12 netns ns1
% d5 }$ n$ c0 e q* W; b[root@node1 ~]# ip link set veth11 up( ?: h' `" G7 f* e- I
[root@node1 ~]# ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:015 x3 N9 p8 _6 p) o( f
[root@node1 ~]# ip netns exec ns1 ip link set veth12 up( l% f# G& t, s( i S; ?! _
[root@node1 ~]# ovs-vsctl add-port br-int veth11: o, U W, r# F4 E7 l
[root@node1 ~]# ip netns exec ns1 ip addr add 192.168.1.10/24 dev veth12
6 H( P, |' b( e2 L1 k* b4 c
0 T2 _3 L1 @; _2 q0 D* snode2上执行，注意veth12的ip和和node1上veth12 ip在同一个子网
0 t! n5 E0 C8 I* A[root@node2 ~]# ip netns add ns1% Q4 k4 [5 O" n1 D d
[root@node2 ~]# ip link add veth11 type veth peer name veth12
$ o7 @% u( W+ j% `$ q[root@node2 ~]# ip link set veth12 netns ns1/ q9 W& x0 B# A
[root@node2 ~]# ip link set veth11 up+ W# @$ B! ~% d
[root@node2 ~]# ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:02
0 }% p" \6 u% _! w7 p. P! v2 A[root@node2 ~]# ip netns exec ns1 ip link set veth12 up0 g9 q+ j t0 ?: ~& G: W" A0 s
[root@node2 ~]# ovs-vsctl add-port br-int veth11, m4 j) X& [3 l+ D% c
[root@node2 ~]# ip netns exec ns1 ip addr add 192.168.1.20/24 dev veth127 n+ z. X; I/ Y$ L" {) v: @
/ B8 L6 c- D8 l5 U4 y; ^
查看node1上br-int交换机信息6 |" m( k6 g: p4 C' G% k& J: [7 G
[root@node1 ~]# ovs-vsctl show- K% v4 S" ]# {2 H
ed157e0c-cac3-46b9-830c-f2d710b475d5' Q8 K7 U9 g% R( p
Bridge br-int
7 ^: ^' p# a3 x# w1 A fail_mode: secure0 |0 G8 _; \5 }6 P2 z
datapath_type: system1 [5 `' T( V* y9 G9 G+ L- r! t
Port br-int
) S/ ?2 H% `5 Z4 j Interface br-int1 A9 e0 }2 g; X' O" j. X8 d0 K3 f
type: internal
" m4 c: S, A3 M7 [5 l Port veth11
6 D+ p. G) ?8 N Interface veth11% x; A+ |: O( |0 {
Port ovn-node2-0
3 g! Q0 V; c2 s Interface ovn-node2-0
+ M( S) ]9 X: \9 m3 r q6 r type: geneve
6 A9 T1 E- \5 n+ T8 S options: {csum="true", key=flow, remote_ip="10.1.1.42"}2 o* e0 f5 q- }8 q" P5 l0 t
ovs_version: "3.1.3"9 L, c, E' Y# W7 ~0 t/ L
查看node2上br-int交换机信息$ t& @& [& o. b; X+ a5 m! K
[root@node2 ~]# ovs-vsctl show
+ `0 C, `% u+ T: k, [f6669675-b42d-47de-be95-b26bf6d1e069* d. _$ h0 G4 x7 b
Bridge br-int8 v. M9 {7 l+ w1 J6 Z# a' n* L' R
fail_mode: secure- F8 o2 J/ w4 V3 \- e
datapath_type: system9 C# ]) D' w, J& l. K& g9 j
Port veth11: b/ H* W( J( F3 p
Interface veth11( g3 T y7 g' F
Port ovn-node1-0 T& ]1 h, X4 i: h0 @
Interface ovn-node1-0
w& g5 K \: J3 u type: geneve
' ^# f- X5 L$ N0 V. V& y; R options: {csum="true", key=flow, remote_ip="10.1.1.41"}
9 i, [4 C8 g0 O6 H( y5 i; C% J Port br-int
% B9 s& c( v6 n9 @* c, y# I Interface br-int/ V( f1 Y* s4 ^1 a, C i
type: internal
& [: G8 \ |. a0 P" l ovs_version: "3.1.3"
- w3 E- e8 P: k. k; e+ q/ i8 |/ a, F: W2 F, Z1 z0 F! Y/ ?
现在从node1上的ns1 ping node2上的ns1是不通的，因为它们是不同主机上的网络，二/三层广播域暂时还不可达。
a% E" r" z5 N0 A# ]3 m[root@node1 ~]# ip netns exec ns1 ping -c 3 192.168.1.20; S2 X5 G; P5 O/ V! K2 r
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
8 D6 X- [8 ]9 j( k1 t( O
/ q2 E* }: h: Q; J0 N# M--- 192.168.1.20 ping statistics ---
# F4 G# @. C }# f5 `3 packets transmitted, 0 received, 100% packet loss, time 2047ms
+ _- J8 @( e8 N" {在这里插入图片描述
- N Z$ G7 l4 g( m/ f% _! Y; F查看openstack的控制节点发现，ovn的北向数据库中有逻辑交换机信息。& O% }( O* D2 ?. u3 }4 U+ l
在openstack里面，创建一个网络，就相当于创建了一个逻辑虚拟交换机，这个逻辑交换机（网络）信息会被保存到北向数据库里面。一个网络就是一个逻辑交换机。
1 T R0 K% y$ E! T9 Y# H& B( x( |在这里插入图片描述; C$ {% }% g+ X" V+ l2 b" [9 q3 U
在node1中查看发现，ovn的北向数据库中没有逻辑交换机信息
' j* v. u! M! z5 |+ o3 h* e在这里插入图片描述0 R" P8 P5 L/ y0 h) }2 z
在openstack不同节点的虚拟机ip互通，这两个虚拟机ip连的是同一个网络，是同一个逻辑交换机上的同一个子网不同ip所以互通。" u. O7 m# V; a" D _8 S
这两个节点的虚拟机ns1的ip是手工配置的独立的、不互通，这两个虚拟机ip没有连到逻辑交换机上，加个逻辑交换机就能互通。
4 M' R. L& n* |2 Y" p9 K在这里插入图片描述
. A: L# g, p5 |3 F) k W/ Y逻辑交换机（Logical Switch）：为了使node1和node2上两个连接到ovs交换机的ns能正常通信，需借助ovn的逻辑交换机，注意逻辑交换机是北向数据库概念。4 V) d; \# W* l1 m5 q
- V3 o0 b) R& Z9 l! ^! J5 l在node1上创建逻辑交换机6 m' a5 I n( V* f* ^1 a9 S
[root@node1 ~]# ovn-nbctl ls-add ls18 T, G" m2 S. ?+ M3 }
[root@node1 ~]# ovn-nbctl show
! d* \! W5 U( y3 Q* z* }( a8 `switch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
5 ^* L3 j1 F7 R$ `在逻辑交换机上添加端口1 E# |" C& }& x( X+ a9 v
添加并设置用于连接node1的端口，注意mac地址要和veth pair网络命名空间内的那端匹配起来
1 \% V" u9 L# K+ L: Q0 K N[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-node1-ns1
0 G" o9 M" |; r, w, ?4 ^; B[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node1-ns1 00:00:00:00:00:01
9 ]$ B" A! F/ }1 C2 n[root@node1 ~]# ovn-nbctl lsp-set-port-security ls1-node1-ns1 00:00:00:00:00:01
* B9 x) X2 c8 H# J8 |0 w添加并设置用于连接node2的端口，注意mac地址要匹配起来 Z+ A" w3 {! q
[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-node2-ns1/ r; k! `2 s9 H3 L k P
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node2-ns1 00:00:00:00:00:02- ]$ O& D: r3 D! w) @, k. P
[root@node1 ~]# ovn-nbctl lsp-set-port-security ls1-node2-ns1 00:00:00:00:00:02+ D& V+ v$ x4 q( U
查看逻辑交换机信息/ m3 h8 h" X% w
[root@node1 ~]# ovn-nbctl show
. L: x- |! b2 r9 `6 ^# r Pswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1): ?" v- ?- f: P1 ?5 M
port ls1-node1-ns1
" ]) @, d* \3 K# @% W: f# ^% r1 P addresses: ["00:00:00:00:00:01"]
7 e- |- D* R9 \3 }, X3 s1 x" ] port ls1-node2-ns1) U ~/ S* g0 y
addresses: ["00:00:00:00:00:02"]
8 W1 B$ N0 l+ |3 `: p
2 B. A: ~& \* h# A: N; Knode1上执行，veth11端口连接逻辑交换机端口4 A: j- j8 ]# _; G
[root@node1 ~]# ovs-vsctl set interface veth11 external-ids:iface-id=ls1-node1-ns1/ o/ e/ p% g# Z& w4 l
node2上执行，veth11端口连接逻辑交换机端口
6 o5 }$ P* f* O4 f8 ~[root@node2 ~]# ovs-vsctl set interface veth11 external-ids:iface-id=ls1-node2-ns1
) ^2 t$ g! @% D, e' H0 b8 _再次查看南向数据库信息，发现端口已连接2 }, w8 x/ K7 w; |8 ]
[root@node1 ~]# ovn-sbctl show( G$ j: ~% c4 k& d. l2 \ B# k
Chassis node2! ]1 U7 X7 P/ g- [
hostname: node2: w2 f3 e6 |0 _, j
Encap geneve6 o; v1 |' R6 V' N
ip: "10.1.1.42"8 Q9 t" f: l3 B$ E1 @" ~5 w3 Y- \
options: {csum="true"}$ |+ y, A* @- M
Port_Binding ls1-node2-ns17 L( m" `, [4 R: a( A* ]
Chassis node1# }6 G( g. Z1 Y7 e
hostname: node13 T s. m, @% @9 f, ]1 J5 y! C
Encap geneve1 H/ c% A% L2 E1 o
ip: "10.1.1.41"# D' z; A, {2 f$ E
options: {csum="true"}
5 s- ^, }: y! ~; }, K% t; `( A* j4 V Port_Binding ls1-node1-ns1
4 X+ T% m5 P9 `& E. V; Nnode1上验证网络连通性
2 {; n3 `: s9 O& u3 l$ M[root@node1 ~]# ip netns exec ns1 ping -c 3 192.168.1.20- Q: X) M' ]( X
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data., \$ u# W" Z2 H8 T1 T
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=4.68 ms
3 m. u6 i& z7 H# |64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.908 ms- a) c# t8 K: a
64 bytes from 192.168.1.20: icmp_seq=3 ttl=64 time=0.756 ms9 j1 i* p. v% N, I8 B
3 P' s* j0 H7 r- ]--- 192.168.1.20 ping statistics ---7 t9 r" C' d. ]% e
3 packets transmitted, 3 received, 0% packet loss, time 2004ms" H: p/ L( V: ~) R7 Z) a# \% S
rtt min/avg/max/mdev = 0.756/2.115/4.682/1.816 ms% r- w/ Z2 Q( C+ |5 A
node2上验证网络连通性+ I4 C p4 c- K/ U) @, ~2 T
[root@node2 ~]# ip netns exec ns1 ping -c 3 192.168.1.10. l$ e" [$ N2 }! i- x1 Y
PING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.6 Q! G$ v9 h: K8 G+ c3 b6 Q
64 bytes from 192.168.1.10: icmp_seq=1 ttl=64 time=3.34 ms
4 ]. `0 |# I2 G" @2 j64 bytes from 192.168.1.10: icmp_seq=2 ttl=64 time=0.863 ms8 K) Z6 h+ B W \% p7 v) R
64 bytes from 192.168.1.10: icmp_seq=3 ttl=64 time=0.372 ms
9 s" d+ @: K5 k" ?
* g- S' }" a+ w. V6 C9 M--- 192.168.1.10 ping statistics ---* {, z9 v; h) O& Z6 N( ?
3 packets transmitted, 3 received, 0% packet loss, time 2003ms, y& T+ A: ]! @% J* A3 n. H' v- J
rtt min/avg/max/mdev = 0.372/1.525/3.342/1.300 ms7 d# A6 l; j2 K5 @5 A
现在node1和node2的ns1互通了，相当于创建了两个实例，这两个实例ip用的子网是连在同一个逻辑交换机上的，是同一个逻辑交换机上的同一个子网不同ip所以互通。
4 e0 l$ d) @# Z& O1 m在这里插入图片描述
/ D/ f& U! V! F3 C8 V, d$ j在这里插入图片描述
. G7 f7 q2 O/ l+ f- l0 D! C7 Ngeneve隧道验证：从node1上的ns1 ping node2上的ns1的例子，抓包看看各个相关组件报文，验证geneve隧道封解包。通过抓包分析，可以看出geneve隧道在ovn/ovs跨主机通信的重要作用，同时也能看到ovn逻辑交换机可以把不同宿主机上的二层网络打通，或者说ovn逻辑交换机可以把ovs二层广播域扩展到跨主机。
+ N6 t. v) @, G3 F- f- v/ q M/ F* z0 ?- A; a
// node1上ns1 ping node2上ns1, G+ X% H3 K3 p( H+ t9 z) V
# ip netns exec ns1 ping -c 1 192.168.1.20
9 Q B9 S8 ]7 J) W. m; Y! F6 D2 WPING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.2 y, V* G J3 e) ~3 u+ X
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=1.00 ms) [/ ~$ d Y. \+ k$ k4 m8 Q
--- 192.168.1.20 ping statistics ---/ o/ F! F2 Y# [- j
1 packets transmitted, 1 received, 0% packet loss, time 0ms3 [5 a# M. k2 ]) F3 i! y" B% D/ P
rtt min/avg/max/mdev = 1.009/1.009/1.009/0.000 ms! Y/ _! p; i7 F
- o+ L9 e: {( H" ^9 l6 h
// node1上ns1中的veth12抓包
! c$ k. g, d% \# ^9 Y1 p, [# ip netns exec ns1 tcpdump -i veth12 -n
4 ]* u$ c% {8 ~, W% F* G% etcpdump: verbose output suppressed, use -v or -vv for full protocol decode
6 k S& p. o1 Y n$ q# _, j6 Ulistening on veth12, link-type EN10MB (Ethernet), capture size 262144 bytes
2 Q/ {* ~& Z: N& }/ S& x22:23:11.364011 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 24275, seq 1, length 644 |" k, D& k" P, g8 |' k1 i
22:23:11.365000 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 24275, seq 1, length 64 S. J! g4 S: `0 C1 G9 g- Q
22:23:16.364932 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28
4 p+ Z5 r- x- i5 Q' s% ~- Z2 H22:23:16.365826 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28- o- n# j1 D [& \7 L* N$ e' R! g. {
& \5 V4 ]+ h+ X8 D3 f3 D3 K
// node1上veth12的另一端veth11抓包, ]! ~! @9 S( g' O6 `- J9 t
# tcpdump -i veth11 -n
6 e- Y0 y* W4 S/ Y4 itcpdump: verbose output suppressed, use -v or -vv for full protocol decode& r* N ^8 R5 [' w% N0 o
listening on veth11, link-type EN10MB (Ethernet), capture size 262144 bytes
9 h9 b6 K! }' s: z6 S22:25:11.225987 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 25166, seq 1, length 64* i1 R0 l7 j, \# W0 @" N
22:25:11.226914 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 25166, seq 1, length 645 V" j- G8 ]/ T# u1 K% a% S
22:25:16.236933 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28' F0 I4 M$ C. v ~2 L
22:25:16.237563 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28! c$ i1 z c) h; s7 `/ F
22:25:16.237627 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
1 m! `6 L" ~; f0 [( h22:25:16.237649 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
9 \5 H$ r( e$ ~% ]& o/ D# s- n. k% X% H4 s, ~
// node1上genev_sys_6081网卡抓包 w4 `, ?4 J+ @
# tcpdump -i genev_sys_6081 -n$ f( x9 P$ s8 G. V) M) n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode1 X2 z$ @ L' ~9 d
listening on genev_sys_6081, link-type EN10MB (Ethernet), capture size 262144 bytes
s2 e- ]$ l: o" } K* g3 N22:28:15.872064 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 26492, seq 1, length 64
' g3 F$ _/ y+ O ?% I5 {7 e* E22:28:15.872717 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 26492, seq 1, length 64
! ^- r! g9 M9 C/ @# z1 y* I22:28:20.877100 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28# e# w8 q T! r% y0 `
22:28:20.877640 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 288 d- }% x! M$ v1 [ J; O
22:28:20.877654 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
p+ N- ^! X* h9 g0 y# x+ Y6 A22:28:20.877737 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28( E* K5 |5 v3 O% {0 T' {
, ^# {4 y" C7 y0 L- n/ ~
// node1上eth0抓包，可以看出数据包经过genev_sys_6081后做了geneve封装
( n3 B& J5 Y3 I; j# tcpdump -i eth0 port 6081 -n
, N# v, W: f T% g+ itcpdump: verbose output suppressed, use -v or -vv for full protocol decode
$ t4 A% m' f( }+ K& L D: V, }listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes4 _; S3 ?1 f+ d% Y! s$ k
22:30:23.446147 IP 10.0.12.7.51123 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 27458, seq 1, length 649 x) h" V' U8 W+ l' o4 H6 F* p
22:30:23.446659 IP 10.0.12.11.50319 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 27458, seq 1, length 64% g4 c+ c) k5 k! f
22:30:28.461137 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28: J) {; @6 T2 x5 ]
22:30:28.461554 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
- f/ i, g, e8 }: J" Z" Q6 P2 G22:30:28.461571 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28# e5 P: p& |0 s" s
22:30:28.461669 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28% v5 o/ r9 f) c6 _7 q
9 l# _1 U: v. F, |===================跨主机===================0 P; S; l) e% p% z; @( u9 i
- A; o; i% `& ?7 @ o, Q: ~: u// node2上eth0抓包
3 D2 c% M( ~6 Y9 e+ A# [# tcpdump -i eth0 port 6081 -n
! E" g( s8 x$ A* `8 g5 f. O6 Btcpdump: verbose output suppressed, use -v or -vv for full protocol decode
1 n+ W/ y2 r, e* C8 E* Clistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
) t% z& ]( O' t1 V6 v22:23:11.364189 IP 10.0.12.7.51123 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 24275, seq 1, length 64
m% M' x& g3 d: c22:23:11.364662 IP 10.0.12.11.50319 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 24275, seq 1, length 64
) M1 F' i5 W6 G8 q7 b0 X* T22:23:16.365086 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28# Q0 n" ~: D0 p9 W8 L0 M4 e4 A+ x
22:23:16.365487 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28% E. W6 c6 u9 J. I) R
, x, y$ Y8 f5 i
// node2上genev_sys_6081网卡抓包，可以看到数据包从genev_sys_6081出来后做了geneve解封1 j9 [, [8 ^1 N% Z) }' v
# tcpdump -i genev_sys_6081 -n5 D# d. }4 K# P# }
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode2 R1 Y& I- R% f+ T+ Z) y
listening on genev_sys_6081, link-type EN10MB (Ethernet), capture size 262144 bytes
! d% I# ~) K) \$ f {* F5 X/ F22:25:11.226186 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 25166, seq 1, length 64+ v2 ^' W( [5 ` P0 m
22:25:11.226553 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 25166, seq 1, length 642 R% g% I/ D. m+ y
22:25:16.237070 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28
7 m! Z! g9 R& C2 n, I22:25:16.237162 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28) @6 j, Y( T3 g4 A9 u( j8 l( p
22:25:16.237203 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
* `# S% W, a& j22:25:16.237523 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
0 D9 H4 \9 {# \1 b" U& G3 N. p1 x: X, m6 @. a0 i$ g
// node2上veth11抓包2 a9 H3 n& k$ |5 D3 o/ a- r2 C' l! `
# tcpdump -i veth11 -n8 l5 Z: o6 ~: x
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode* c1 p! @2 P4 h1 f/ G
listening on veth11, link-type EN10MB (Ethernet), capture size 262144 bytes/ l* S# a+ i, ]' U( {
22:28:15.872198 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 26492, seq 1, length 64
4 G2 C/ B6 \ }) a9 N0 C* |22:28:15.872235 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 26492, seq 1, length 64
6 t" w' z% e. {1 k4 t22:28:20.876913 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
* y% K2 J4 [6 Z' J0 K22:28:20.877274 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28& j( U. H9 l1 S, K: \( u4 E
22:28:20.877287 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28& v1 [. Q' p: S7 ~
22:28:20.877613 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28) t+ b" a6 x' N% {
2 U8 y; T4 c" E2 h {
// node2上ns1中的veth12抓包
3 g# `# r6 b: Q0 k0 h# ip netns exec ns1 tcpdump -i veth12 -n' d: ~6 ~! w9 N( |. V5 j9 B" h) y
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
: b% o5 e# r3 j v! [; c5 ^listening on veth12, link-type EN10MB (Ethernet), capture size 262144 bytes/ f# S0 @/ y) \2 M* `% U4 y/ g
22:30:23.446212 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 27458, seq 1, length 64 W6 Z1 t/ I% @ e$ Z+ H
22:30:23.446242 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 27458, seq 1, length 64$ B9 B! V6 V* F2 j# p& `
22:30:28.460912 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 281 ]5 O$ i& i1 }$ {% y
22:30:28.461260 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28 g" J3 q Z* D5 ^) u) e6 v
22:30:28.461272 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 286 ]& m8 e V4 e4 X t5 ^
22:30:28.461530 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
- `& i3 Q5 O) j逻辑路由器（Logical Router）：
0 L }1 n/ B1 a! w" Q* u前面验证了ovn逻辑交换机跨主机同子网的通信，那不同子网间又该如何通信呢？这就要用到ovn的逻辑路由器了。
- Q# B" }2 ?% I3 {1 J" t先在node2上再创建个网络命名空间ns2，ip设置为另外一个子网192.168.2.30/24，并且再增加一个逻辑交换机。. T) | h) Z5 i0 i Q( r
在这里插入图片描述
$ c6 `. Z( w; u) M* N! {& f4 O! T1 U- N* D
node2上执行
; |3 u9 K% l( p- M: z[root@node2 ~]# ip netns 查看网络命名空间& g; L) F# \6 z% ]2 _ w+ k
ns1 (id: 0)
8 A1 n* c. n- q3 \5 a& C# R$ t[root@node2 ~]# ip netns add ns2
6 \8 A) a) ^. U* w; r2 [1 h/ i[root@node2 ~]# ip link add veth21 type veth peer name veth22
! W3 k' I. ^! w* z" h" t! p3 s[root@node2 ~]# ip link set veth22 netns ns2
/ c5 v6 e9 f$ ]1 r# a[root@node2 ~]# ip link set veth21 up
8 a3 \" ~! Q7 p! C2 m: Z/ J: B- N[root@node2 ~]# ip netns exec ns2 ip link set veth22 address 00:00:00:00:00:03
+ N. M. N5 t* @' f6 G[root@node2 ~]# ip netns exec ns2 ip link set veth22 up9 Z- h c0 S) ]3 D3 O
[root@node2 ~]# ovs-vsctl add-port br-int veth216 ?1 f- h% ?2 Q
[root@node2 ~]# ip netns exec ns2 ip addr add 192.168.2.30/24 dev veth227 I: K2 V. O3 ?: x
[root@node2 ~]# ip netns
. @; t8 N0 b- F6 Lns2 (id: 1)
! ?: R# z( L% N# j" R+ [ns1 (id: 0)) U" S# Z8 @/ J3 v% u& Y
$ t* P% b/ N1 T' a Z, L& b6 s
node1上用ovn命令新增一个逻辑交换机，并配置好端口
: H, d: N/ G) {2 T# M* ?. @[root@node1 ~]# ovn-nbctl ls-add ls23 G7 ^( v& C. Y4 J. \' s
[root@node1 ~]# ovn-nbctl lsp-add ls2 ls2-node2-ns2& L" m2 R8 G; u" U- U# Q0 K
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-node2-ns2 00:00:00:00:00:03" E- A6 [7 B. ?' w4 a2 k
[root@node1 ~]# ovn-nbctl lsp-set-port-security ls2-node2-ns2 00:00:00:00:00:03
- ?$ {) M% C5 f, _4 V6 O$ r/ G/ C- R6 i
node2上ovs交换机端口和ovn逻辑交换机端口匹配起来: |# N+ G* u$ B# [* U0 K
[root@node2 ~]# ovs-vsctl set interface veth21 external-ids:iface-id=ls2-node2-ns2
' y6 R( B* n2 Q) R6 ?# j( Q+ i8 h/ N+ K I! A
查看北向数据库和南向数据库信息
) f" s: [# w, |5 {% s[root@node1 ~]# ovn-nbctl show
; S2 N% n0 E% R5 u0 mswitch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)
8 X& ~" V' p! Z$ \! Y port ls2-node2-ns2" R! Y( T0 E! B
addresses: ["00:00:00:00:00:03"]
2 _6 A6 H2 w2 B- J1 c, Iswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
( Z( [: N- L T5 @& {5 k! h port ls1-node1-ns1, N- }, q/ H$ o
addresses: ["00:00:00:00:00:01"]
; Q9 j5 t7 P& c' V. c port ls1-node2-ns1" l$ o0 a' H6 `' n
addresses: ["00:00:00:00:00:02"]1 {0 Q# Y: ~, m; m8 u! M
[root@node1 ~]# ovn-sbctl show2 m/ `1 Q+ { r R
Chassis node2
: ?4 Y: F) S I# c. L hostname: node20 ^& ]7 c9 k1 s
Encap geneve
: F) F, ]- V e& N) f7 S ip: "10.1.1.42"
3 K/ l! |5 {! D2 f- T y options: {csum="true"}
6 Q" t* s- ^) {5 b# S# Y Port_Binding ls2-node2-ns2 R3 V1 g3 T; F6 m d6 u3 o
Port_Binding ls1-node2-ns19 C, h: M0 b) q" N* ]
Chassis node1
+ T" D% U6 n/ c( P8 [. B* ], l3 \ hostname: node16 D: F+ p- Q" S+ p
Encap geneve$ }/ x3 O- R1 V6 i: G
ip: "10.1.1.41"
- S% @5 z5 l# L options: {csum="true"}
9 J- b b7 w+ J& Z6 t2 q" n' X5 P Port_Binding ls1-node1-ns1
1 O9 p6 d# |( d8 ?. z创建ovn逻辑路由器连接两个逻辑交换机0 q' n+ n4 [$ T$ D3 u" ^" g- Z4 Q
2 \- b; F- a Q. b! Y添加逻辑路由器，路由信息保存在北向数据库
8 W) j% \; t* q0 Y5 G4 Q[root@node1 ~]# ovn-nbctl lr-add lr1
8 \1 U# x* c3 T+ u% r逻辑路由器添加连接交换机ls1的端口
+ T+ v1 P1 q6 h" O V# [/ n' o[root@node1 ~]# ovn-nbctl lrp-add lr1 lr1-ls1 00:00:00:00:11:00 192.168.1.1/24% s) x0 f$ l8 V# m( @& M% h
逻辑路由器添加连接交换机ls2的端口 w. ?7 [4 {* z7 Q& K7 [9 G* F
[root@node1 ~]# ovn-nbctl lrp-add lr1 lr1-ls2 00:00:00:00:12:00 192.168.2.1/245 i3 w, ^9 m" a' s4 N) E
7 I( l3 [! _! a! j5 y* }
逻辑路由器连接逻辑交换机ls1& m" ]7 R4 q, m
[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-lr1
$ F8 R! G6 R7 V1 o: x& N+ E% D[root@node1 ~]# ovn-nbctl lsp-set-type ls1-lr1 router
9 h* l7 e1 L: X; D[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-lr1 00:00:00:00:11:006 T8 O) \$ x/ W4 f0 ?0 ^% u
[root@node1 ~]# ovn-nbctl lsp-set-options ls1-lr1 router-port=lr1-ls16 T$ C) e0 P+ D1 p5 D* V2 m) W$ Z
* V0 Q; I$ V) u& Y! n7 O% ]7 C逻辑路由器连接逻辑交换机ls2
+ J [. K q$ X& |; v" m[root@node1 ~]# ovn-nbctl lsp-add ls2 ls2-lr17 H* |' s/ h) d9 D6 L1 X
[root@node1 ~]# ovn-nbctl lsp-set-type ls2-lr1 router4 R4 t6 v- P( I- _0 {7 }6 w6 x
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-lr1 00:00:00:00:12:00
8 f" W6 _- c" T& c2 j[root@node1 ~]# ovn-nbctl lsp-set-options ls2-lr1 router-port=lr1-ls2) z' E2 K- B' d' E
! [/ a; ]4 Q2 ]+ t: x4 E
查看北向数据库和南向数据库信息
# G6 x; G, T( l0 _( c: a6 y[root@node1 ~]# ovn-nbctl show
0 z; a. }5 Z/ f pswitch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)
8 w5 |& K, U- g/ A- v0 f" B M port ls2-node2-ns2
4 q+ M- O7 ]5 s1 Y5 E addresses: ["00:00:00:00:00:03"]
2 p5 f% D; z( g8 i* e port ls2-lr1" y- }, O5 W& u0 N; G$ E1 ^
type: router6 y# Y8 |4 O( _: P9 M" f
addresses: ["00:00:00:00:12:00"]
9 _* m8 H4 W U, S; t2 {8 a router-port: lr1-ls2( P# ~5 u' U7 J6 B* p
switch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)/ _7 M+ q7 N9 S" n4 p( u
port ls1-node1-ns1" \8 x1 b. a1 M& }+ t
addresses: ["00:00:00:00:00:01"]. j, w; R' z" f% O7 I
port ls1-node2-ns1
% k! Y% s) M9 B; @* l# X addresses: ["00:00:00:00:00:02"]" V0 i( ~6 @5 D
port ls1-lr1/ c2 \6 H3 @0 n- h7 X- C$ F
type: router' \# J" R9 S& d+ A: s- l
addresses: ["00:00:00:00:11:00"]" l& c) y1 o: d3 T' R
router-port: lr1-ls1. ~* ^3 w! t, O, |
router e9c151a0-5db7-4af6-91bd-89049c4bbf9f (lr1)
3 @7 @! |, k( y; ^; F) X7 w3 m port lr1-ls26 m, g7 b& G+ [( @' T' b; q5 U' r- c
mac: "00:00:00:00:12:00"
1 U3 c2 Z& {6 ?% l0 m networks: ["192.168.2.1/24"]9 ]$ O- }: U5 @: E+ @
port lr1-ls1
% p) I+ B' N+ c3 O mac: "00:00:00:00:11:00"
; A; m3 x/ O/ i" p2 a9 G networks: ["192.168.1.1/24"]
2 l. H. b; v. z[root@node1 ~]# ovn-sbctl show, V" B: ]* W) ?6 X0 [
Chassis node2
+ u) j$ B- M6 M- }: z8 O& G hostname: node22 P6 e4 @# H$ O/ T9 t' [3 J7 Q* @* `9 w% M
Encap geneve: A0 _# B6 t6 Q% u% u
ip: "10.1.1.42"' J9 }" Z! M8 U( s
options: {csum="true"}
$ l( R/ O/ Z3 ^1 m8 ?# h1 a5 ` Port_Binding ls2-node2-ns2- w! C2 \. X3 B$ L5 P+ y# M
Port_Binding ls1-node2-ns1
~8 f) t9 Q) {Chassis node1
. F. g9 ~3 A- e: q% z) r4 d hostname: node1
\3 F7 u4 [. }/ Y) \9 i Encap geneve$ y. c e/ S/ `% {( P! `; F
ip: "10.1.1.41"
! f* |6 [3 z2 o. }5 O6 Q options: {csum="true"}
* j+ Q+ z% Q( c1 `7 P Port_Binding ls1-node1-ns1
4 P$ E# M' ^9 t3 x9 [在这里插入图片描述: P/ @/ B% ?2 ~% H- U; y* w+ W
从node1的ns1(192.168.1.10/24) ping node2的ns2(192.168.2.30)，验证跨节点不同子网的连通性。/ b n# a" `6 U5 E) x _
2 u) y7 g: _, S0 Y8 C) j[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.2.30
! }5 g1 h- I7 D; O1 }( r. oconnect: Network is unreachable connect: 网络不可达
% D+ ?" t* H( f# c查看ns1上的路由配置，显然此时没有到192.168.2.0/24网段的路由! p& `& J2 B& v! _) I
[root@node1 ~]# ip netns exec ns1 ip route show
9 U6 X* \. w: @. p Z192.168.1.0/24 dev veth12 proto kernel scope link src 192.168.1.10. [* O H& J, a7 J1 Q
[root@node1 ~]# ip netns exec ns1 route -n, Z: |4 o* i2 q& U
Kernel IP routing table
' L: j5 Y, q" `! }9 UDestination Gateway Genmask Flags Metric Ref Use Iface$ ~5 y: p" o) c" Q0 g
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 veth12; n b; E3 \6 `$ z5 y
因为路由器是三层概念，要先给ovs的相关port配置上ip( h* U7 u9 P8 k( p$ Q
; \6 J1 B6 v# R. y) F, M9 s[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node1-ns1 00:00:00:00:00:016 d# t* ^2 N+ d5 U" q: n3 X
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node2-ns1 00:00:00:00:00:02
" y: ]8 X+ r. g[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-node2-ns2 00:00:00:00:00:03
' z5 ^/ }* A) K+ c$ c再给三个网络命名空间添加默认路由，网关为ovn逻辑路由器对应的port ip5 B! e7 V: }2 e, K Z/ v* t' j4 K' y
$ I, l. f& a) g$ k. ~9 d
node1上ns1/ a! l& E6 P1 g
[root@node1 ~]# ip netns exec ns1 ip route add default via 192.168.1.1 dev veth12: m$ `& \, ]+ V9 ~
node2上ns1
* o) u3 D+ y' b8 o- K [root@node2 ~]# ip netns exec ns1 ip route add default via 192.168.1.1 dev veth12
$ @7 ?9 W; V% E' C/ ~ node2上ns2
; C/ w- m5 k% R( s6 L [root@node2 ~]# ip netns exec ns2 ip route add default via 192.168.2.1 dev veth22
6 m. ?5 R; Y5 [& M: `6 `; A/ j再次查看下南北向数据库信息
) \' E( A3 ?/ x5 m* q, q! w+ s9 N8 c. q$ X- x( k( w1 ~+ ` K
[root@node1 ~]# ovn-nbctl show' d6 b/ n) }- I3 z( D
switch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)
7 p. U; S& L( U* K; J* Y' g port ls2-node2-ns2
3 B; n- o( ?( Z3 H% ^' G addresses: ["00:00:00:00:00:03"]
2 L9 H4 S" C4 Q$ d, I" t6 X port ls2-lr1
# R% [# {9 e) V0 @ type: router
2 s, t( L: n# x addresses: ["00:00:00:00:12:00"]
}( V! T! R( u& w5 t( S* F# r router-port: lr1-ls2: h/ P, t5 d$ C A' `
switch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
5 Z' v7 l3 ~; \! M5 q port ls1-node1-ns11 ]+ D. E! s8 n3 Z+ t
addresses: ["00:00:00:00:00:01"]
6 U$ I3 D8 u- I2 t- u port ls1-node2-ns1& o/ |9 v$ h4 _2 n4 ^! J; r
addresses: ["00:00:00:00:00:02"]/ d3 C. Y1 Y4 t8 q: o( L1 M
port ls1-lr1
- ~: G5 Z2 v& @( _ type: router
+ Y" v% F4 \, W addresses: ["00:00:00:00:11:00"]
' v) g& {: L7 f! \ router-port: lr1-ls1
5 N2 H! f% j$ V W+ P! Srouter e9c151a0-5db7-4af6-91bd-89049c4bbf9f (lr1)! s5 U7 Q5 d* O8 h2 o7 v0 t
port lr1-ls2
( E: i t: q0 ~% w1 ~ mac: "00:00:00:00:12:00"
! L* w5 S( G& s, p. G6 H" _. J e5 _ networks: ["192.168.2.1/24"]# _6 _9 j3 ]1 F5 b& W. B/ T) V; L
port lr1-ls19 M; Z& h( R' U2 @( J" Y+ W
mac: "00:00:00:00:11:00"
z" T; W# a0 P$ A4 i networks: ["192.168.1.1/24"]5 O& ?* c5 Y; G# k; A; I, L
[root@node1 ~]# ovn-sbctl show
0 m0 \6 t, p4 p5 a6 CChassis node2" m) l2 _0 i Y k2 _+ ?7 X
hostname: node29 O& D4 W5 J6 ^1 ]6 A
Encap geneve
) Z" P5 |3 V! |- ^3 M ip: "10.1.1.42"8 m; S1 C4 ^! r2 R8 y7 ]
options: {csum="true"}
4 a C3 ~# W: e: F- p+ D Port_Binding ls2-node2-ns2
; Y: _. g% S: D1 } Port_Binding ls1-node2-ns1; V9 a7 O8 _4 d2 w. E) H, e. C
Chassis node1
\" O$ k1 a% _/ i* S# J4 _ hostname: node1 T+ J V7 V" ?$ N1 W" p# P
Encap geneve
* }; v h/ D; D ip: "10.1.1.41"
5 a& C- f/ j9 W9 v- F options: {csum="true"}
- P5 G" S8 r* b9 r& T3 }/ O- U- X Port_Binding ls1-node1-ns1# P1 w4 q9 Q; D9 h0 P2 G* d6 d1 |
在这里插入图片描述
& s' X4 m/ F. N# Y$ k验证网络连通性2 H9 W/ h( H. A1 ^. V2 X" E
. l4 q3 I, i. ?: \7 Q# S. j4 D
node1上ns1连通网关% p( @" {2 K1 X
[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.1.1' p$ K2 s5 e3 S9 j- Z" N4 Q* W, V
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data., j7 ^8 g* H( I1 D1 k
64 bytes from 192.168.1.1: icmp_seq=1 ttl=254 time=20.10 ms
# I* d0 o0 E" x6 a) K4 J/ M: [) j2 H* x5 _1 i. F. l* _
--- 192.168.1.1 ping statistics ---& P/ b6 u. }, a( I! D5 E6 _) F& `4 l- n
1 packets transmitted, 1 received, 0% packet loss, time 0ms: M: u7 ]. k7 V1 J$ i9 R" u
rtt min/avg/max/mdev = 20.950/20.950/20.950/0.000 ms( p. t% O6 h- z/ ~ q; D8 o: ~
- @1 m2 u& v8 c2 Znode2上ns2连通网关* V4 p0 X* F: J8 \
[root@node2 ~]# ip netns exec ns2 ping -c 1 192.168.2.1: z5 `$ z4 v% S, Z+ b1 d# v
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
5 `0 | g" } h! K64 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=38.5 ms, g" z- C4 q$ a
- M! |% m$ q0 o7 [: X
--- 192.168.2.1 ping statistics ---
# \' u% d' I7 L5 U% n( I9 N1 packets transmitted, 1 received, 0% packet loss, time 0ms
& }: t9 C& E2 }9 ?8 v! t& Brtt min/avg/max/mdev = 38.477/38.477/38.477/0.000 ms8 P2 z5 Y: e( U6 |8 I
% S: K! k4 m! k$ x% B7 Bnode1上ns1 ping node2上ns2+ y3 X6 T# m: y* U. L8 R8 X/ }* ~5 O
[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.2.30* j0 K* t! H: ]) A
PING 192.168.2.30 (192.168.2.30) 56(84) bytes of data.
1 v1 X0 q4 V- H5 }8 H64 bytes from 192.168.2.30: icmp_seq=1 ttl=63 time=1.23 ms$ Z# J* m" ~: Z' x
@9 d- g& N) I
--- 192.168.2.30 ping statistics ---
+ v# W% ^: }0 S6 n0 ?) ~0 C1 packets transmitted, 1 received, 0% packet loss, time 0ms ^1 I3 w3 M8 f& J8 u& Q0 ]
rtt min/avg/max/mdev = 1.225/1.225/1.225/0.000 ms2 @9 O" r; [6 T
复制- y) k% K; x$ f2 b
注意：ovn逻辑交换机/逻辑路由器是北向数据库概念，这两个逻辑概念经过ovn-northd“翻译”到了南向数据库中，再通过hypervisor上的ovn-controller同步到ovs/ovsdb-server，最终形成ovs的port和流表等数据。" Y( ~4 p r. d+ w
ovn逻辑交换机通过geneve隧道，把二层广播域扩展到了不同主机上的ovs；而ovn逻辑路由器则是把三层广播域扩展到了不同主机上的ovs，从而实现跨主机的网络通信。
: Q2 V5 P; C6 }# q; _ovn逻辑交换机和逻辑路由器都会在所有的hypervisor中生成对应的流表配置，这也是ovn网络高可用以及解决实例迁移等问题的原理。 |
|
发表于 2025-12-18 08:50:57
6 g+ _$ ^0 }3 D8 g
. [$ W3 [% N5 s' i2 g3 B
8 D F# o% }( C& i' B$ Q4 w# X
# R. \; `+ [3 n8 \& n+ ]' |' K
0 u* @0 t8 c- M, z
; w- o% ~* O+ P, Z; {& ?5 E# h
7 d9 S1 F& n6 s/ n3 U6 g' `
3 a2 G3 m# \& Y. p' c6 K, I+ `: l
, H0 w' l+ y7 p* S& F! R
0 w8 N9 `0 ^! _# u& l
5 K) [8 I; v2 I) }0 J
* V5 F/ ^. S# o7 D- r. H5 N
; O7 \1 A- y0 ]