|
|
楼主 |
发表于 2025-12-18 08:51:30
|
显示全部楼层
2、网络服务Neutron
y/ i# e3 m7 MNeutron基于软件定义网络的思想，实现了网络虚拟化下的资源管理。Neutron的设计目标是实现网络即服务（NaaS），在设计上遵循SDN(Software Defined Network，软件定义网络)架构来管理的。 d$ a9 S7 g2 j: Y# O; r5 [; h, P( f
Neutron主要包含Neutron server、Plugin和Agent等组件。Neutron server对外提供 OpenStack网络 API，接收请求，并调用Plugin处理请求；Plugin处理 Neutron Server发来的请求，维护OpenStack逻辑网络的状态， 并调用 Agent 处理请求；Agent处理Plugin的请求，负责在network provider上真正实现各种网络功能；此外还有database，用来存放OpenStack的网络状态信息，包括Network、Subnet、Port、Router等。
7 O# }/ j! x$ ^3 u1 q% z* |! k) M$ u* |9 _6 I& r% h
3、OVS
+ s' g* q1 {9 qOVS(Open vSwitch)是虚拟交换机，遵循SDN(Software Defined Network，软件定义网络)架构来管理的。
) ]! q3 {) G- |$ I8 _OVS介绍参考：https://mp.weixin.qq.com/s?__biz ... 189#wechat_redirect6 |/ X3 z1 c% H. Z
在这里插入图片描述
; t7 n, F) i; F/ o7 Tovs由三个组件组成：dataPath、vswitchd和ovsdb。7 M, C$ l! J( J. z
dataPath(opevswitch.ko)：openvswitch.ko是ovs的内核模块，当openvswitch.ko模块被加载到内核时，会在网卡上注册一个钩子函数，每当网络包到达网卡时这个钩子函数就会被调用。openvswitch.ko模块在处理网络包时，会先匹配内核中能不能匹配到策略（内核流表）来处理，如果匹配到了策略，则直接在内核态根据该策略做网络包转发，这个过程全程在内核中完成，处理速度非常快，也称之为fast path（快速通道）；如果内核中没有匹配到相应策略，则把数据包交给用户态的vswitchd进程处理，此时叫作slow path（慢通道）。dataPath模块可以通过ovs-dpctl命令来配置。5 q. s/ U- s9 w- B# g! U
vswitchd：vswitchd是ovs的核心模块，它工作在用户空间(user space)，负责与OpenFlow控制器、第三方软件通信。vswitchd接收到数据包时，会去匹配用户态流表，如果匹配成功则根据相关规则转发；如果匹配不成功，则会根据OpenFlow协议规范处理，把数据包上报给控制器（如果有）或者丢弃。
2 I2 p7 i/ i- ~ovsdb：ovs数据库，存储整个ovs的配置信息，包括接口、交换内容、vlan、虚拟交换机信息等。
" x9 c1 K' X6 Z+ ^2 w8 g; rovs相关术语解释：
0 C" A; ~) V* R( q0 m6 d$ g1、Bridge：网桥，也就是交换机（不过是虚拟的，即vSwitch），一台主机中可以创建多个网桥。当数据包从网桥的某个端口进来后，网桥会根据一定的规则把该数据包转发到另外的端口，也可以修改或者丢弃报文。Bridge桥指的是虚拟交换机。$ m! P9 c+ g+ D* v; ~3 a" @
2、Port：交换机的端口，有以下几种类型：) ~" d4 U% @7 x- t
Normal: 将物理网卡添加到bridge时它们会成为Port，类型为Normal。此时物理网卡配置ip已没有意义，它已经“退化成一根网线”只负责数据报文的进出。Normal类型的Port常用于vlan模式下多台物理主机相连的那个口，交换机的一端属于Trunk模式。
* A5 ]' Z: O4 @+ {9 J6 L: ?, RInternal: 此类型的Port，ovs会自动创建一个虚拟网卡接口(Interface)，此端口收到数据都会转发给这块网卡，从网卡发出的数据也会通过Port交给ovs处理。当ovs创建一个新的Bridge时，会自动创建一个与网桥同名的Internal Port，同时也会创建一个与网桥同名的Interface。另外，Internal Port可配置IP地址，然后将其up，即可实现ovs三层网络。: z8 ^* F5 r, S1 i% C' c6 H; ^6 w
Patch: 与veth pair功能类似，常用于连接两个Bridge。veth pair：两个网络虚拟端口（设备）4 q; K" f& o# ?. [" L5 ?
Tunnel: 实现overlay网络，支持GRE、vxlan、STT、Geneve和IPSec等隧道协议。Tunnel：隧道，三层
$ D' e6 Z/ s7 @2 B9 D0 z9 K+ m3、Interface：网卡，虚拟的（TUN/TAP）或物理的都可以。TAP：单个网络虚拟端口（设备），基于二层；TUN：单个网络虚拟端口（设备），基于三层。veth pair：两个网络虚拟端口（设备），常用于连接两个Bridge。
( k$ T. y4 e$ G) J7 @3 s2 a4、Controller：控制器，ovs可以接收一个或多个OpenFlow控制器的管理，主要功能为下发流表来控制转发规则。
( o# r- \7 O1 d5、FlowTable：流表，ovs进行数据转发的核心功能，定义了端口之间的转发数据规则。每条流表规则可以分为匹配和动作两部分，“匹配”决定哪些数据将被处理，“动作”则决定了这些数据将被如何处理。
: K; D [& O1 s- Q在这里插入图片描述
- y+ E, R. B* B1 x( P& pens160的ip地址没有了，用的是br-ex的ip地址出去的。, c( G* P/ d! ^8 E6 |
在这里插入图片描述2 \1 ]/ l3 }/ A
ovs安装6 }6 s' o; E: w5 I5 ]* @5 X) P+ h0 x
1.开启一台新的linux
3 a' Z p w: B! V2.配置在线yum源（openstack那个在线yum源）# j* ?# G2 l9 [
9 M0 A4 x( m4 P: r
配置yum源（先把原有的备份后清空）5 q% W" b; b7 \- ~
# cd /etc/yum.repos.d/ # rm -rf *. G2 y* |6 j r9 Q$ _5 y
# cat cloud.repo
" Y9 W: h. G0 l% r8 |# j& s2 R6 [5 g- l- y4 O6 d
[highavailability]# @* G: |' M, L' J+ W! {# g
name=CentOS Stream 8 - HighAvailability
! S) @4 l T: F5 L, W7 Rbaseurl=https://mirrors.aliyun.com/centos/8-stream/HighAvailability/x86_64/os/
3 k5 H# M; f9 V9 wgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
9 k9 ^3 \, B9 a, Q# agpgcheck=1* C% k- p. R. ?- `
repo_gpgcheck=0
& A4 ^: r! U- \$ l9 ^( ?% _8 Umetadata_expire=6h
7 ^/ I/ B8 O0 }4 A( P# A7 G0 T8 ~countme=1
2 K" b( u9 ?) T* h8 Y8 Aenabled=1
' D7 h& Z6 w* q: Z5 V3 _& d3 w; B4 r
[nfv]
; { I% s4 u) }name=CentOS Stream 8 - NFV3 L4 n* T" q9 d7 h( s
baseurl=https://mirrors.aliyun.com/centos/8-stream/NFV/x86_64/os/
* [3 Z7 ]2 ^9 i( j9 _0 p' T/ \gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
, ?' }: K0 X$ ^! X$ H; W& }- F5 pgpgcheck=1
J1 T1 N- {. Y! @. q; `2 Y. @repo_gpgcheck=0
7 W5 s, i) D4 {metadata_expire=6h4 ]9 E8 G1 G. \
countme=1" \; n* y R8 Y
enabled=18 A+ [8 e6 l* i/ p0 y I& v
7 I) V8 J& G7 q, m$ s
[rt]. ?+ v- \3 g3 R# X" K% C
name=CentOS Stream 8 - RT. N ^) x2 `8 ?" Z* Y& V/ o7 l
baseurl=https://mirrors.aliyun.com/centos/8-stream/RT/x86_64/os/# k: ]$ q1 V; J, `
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial, B" L3 i# Q, X" a3 D8 C3 [
gpgcheck=1
. ?+ S8 q( p% A. w grepo_gpgcheck=0
5 j7 B4 Z% y9 G% p- L" ?$ n' gmetadata_expire=6h& L, x) i1 G/ Z
countme=19 R- ? l( a8 I: \7 x" i' j |9 _
enabled=1& f0 P% M" s% q9 N: \, z
/ ]. \5 l! k9 N
[resilientstorage]# s& \/ v9 n2 U0 l' u& Q* V4 E
name=CentOS Stream 8 - ResilientStorage* ?# `8 }; Y2 D, `( @
baseurl=https://mirrors.aliyun.com/centos/8-stream/ResilientStorage/x86_64/os// y% s9 x6 v; u
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
& _( V* ?/ x5 X2 W, Pgpgcheck=15 Q3 o: Z t4 e' P/ {
repo_gpgcheck=0
' p" S3 b& j, }metadata_expire=6h' U4 W" y2 [) B" S# o
countme=1! S) s1 z1 e- \9 a
enabled=1
Y9 _3 X+ h$ P6 \! c' D, ]* q
4 u3 q9 R. a, t5 z5 Y; n" a5 Z[extras-common]% U8 b2 i/ {; D$ ]! l' D
name=CentOS Stream 8 - Extras packages
! d$ {8 W# [- E- kbaseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/extras-common/
% A) Z2 c1 H- }' g* Ggpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512
& U! o+ e3 @8 |8 J! a+ Z+ {+ Fgpgcheck=1: o! O4 [+ K. m3 o
repo_gpgcheck=0! J% s6 X( n( A4 t
metadata_expire=6h6 C- r: I- u2 _) S/ O
countme=1" i& R, f# Y0 g/ m
enabled=1
' H4 I: a* s% ?% n2 O) D& V2 T1 k% C1 a( ^
[extras]
/ y. M& b- }0 D- f* v% Uname=CentOS Stream - Extras% @6 g; b i" R, X7 W* U
mirrorlist=http://mirrorlist.centos.org/?release=&arch=&repo=extras&infra=
2 L" Q1 u/ m" R! E* {2 f#baseurl=http://mirror.centos.org///extras//os/
* n3 ]: F7 Z) U8 Q1 N/ ~6 Jbaseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/os/
& E3 v0 d4 \+ m* n% o: V: G- a9 Dgpgcheck=1
/ M" t3 s. ]4 G3 X; r2 |enabled=1
( i4 }3 }6 i4 J( Ngpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial7 O& o$ K& E1 z; w" Y9 e
) V; V- c% V$ B" i- U8 ~
[centos-ceph-pacific]- b- @8 x, |: X+ \& a1 E0 W" ?
name=CentOS - Ceph Pacific0 X( o$ z# J; f* s4 u) L5 E
baseurl=https://mirrors.aliyun.com/centos/8-stream/storage/x86_64/ceph-pacific/8 l! o! G' v1 V) H* e4 ?$ T% Z9 }
gpgcheck=0
+ A( |6 k- \) Jenabled=1
4 A# Q w- U' N+ V) qgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage' w& i+ n- \- [* c! D! `0 ^
! f9 q4 Z1 s+ V4 ~9 K[centos-rabbitmq-38]/ Y% Z8 I3 @/ Q8 r: A
name=CentOS-8 - RabbitMQ 38# O- M7 Q9 N- b# q2 C/ T! ~
baseurl=https://mirrors.aliyun.com/centos/8-stream/messaging/x86_64/rabbitmq-38/
8 p# F. n1 n* S; hgpgcheck=1, ]* B+ a- U" Q
enabled=1
# |2 ], [) h* C+ L+ m, r. tgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging" |: x5 q! y& }" b! o; o3 Z8 j8 v6 M
4 G) d6 b e% u[centos-nfv-openvswitch]$ n5 b+ ?; h, a" D7 r% y! N
name=CentOS Stream 8 - NFV OpenvSwitch
- I0 [7 R2 h5 ?1 X) b/ q( Tbaseurl=https://mirrors.aliyun.com/centos/8-stream/nfv/x86_64/openvswitch-2/0 r; t. [+ o# s0 J9 h
gpgcheck=1
$ H/ z( f' H3 B* |: Qenabled=1
. X/ H' k# A4 \; i" bgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV
* y+ K' x! e- E8 [! p+ xmodule_hotfixes=1
6 ~- j) R: I1 J2 T$ s3 O$ J& g( F4 f E3 S1 X6 n4 E1 K; q
[baseos]6 i( H3 u9 `% |+ p5 f2 i5 x
name=CentOS Stream 8 - BaseOS+ Y# `' d' F; e8 \1 u
baseurl=https://mirrors.aliyun.com/centos/8-stream/BaseOS/x86_64/os/! h- S* t- T/ c8 ]2 F ^8 C+ ] K
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial; M, B( U. k F4 o# H
gpgcheck=1/ H, Q" w, {9 h0 ^2 b2 V$ m1 W
repo_gpgcheck=0
]4 u. o; Y6 o$ Q# cmetadata_expire=6h
0 e% i- e/ j$ O/ o* \& \; ?countme=1
- H, z0 P- g' m/ }enabled=1
. y5 g! o% B Z: a: n6 S) Z
3 w. K: f0 [- O1 ?+ z! t0 V( f[appstream] ?7 C4 b2 S9 ]0 l: Y: E5 q6 Y/ o( q
name=CentOS Stream 8 - AppStream
+ T" Y; T4 w+ y7 jbaseurl=https://mirrors.aliyun.com/centos/8-stream/AppStream/x86_64/os/
4 _3 ]& u$ V1 f% c# ?5 zgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial9 k! s! K- D6 }4 H8 L3 M9 b( }
gpgcheck=1
4 X6 x. n+ I$ ^( x% ]& brepo_gpgcheck=0
+ K) ^4 }( D4 t/ [metadata_expire=6h
) b+ G- f/ h% ?6 T: z7 h, i. ^countme=1; R2 [7 V# @1 v& {
enabled=10 p* Q, X2 K3 @! }( q
* i5 U1 N$ E+ M7 D[centos-openstack-victoria]
+ b3 a" c. i1 I$ Sname=CentOS 8 - OpenStack victoria* | v t C$ Z) T9 g
baseurl=https://mirrors.aliyun.com/centos/8-stream/cloud/x86_64/openstack-victoria/
! W( T$ g6 k& x* L: B#baseurl=https://repo.huaweicloud.com/centos/8-stream/cloud/x86_64/openstack-yoga/, ?* Z' \" U& z/ ~1 f
gpgcheck=1
) O$ k, X) v. Kenabled=1& j. O) ]% R1 v. M
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud
0 V# y% B; {+ F+ Lmodule_hotfixes=1/ T, s9 B0 r) h7 T O* i
( ?& G* D. ?* v, G5 L7 K[powertools]" u2 ^+ b) Q) Z4 n$ L; ~
name=CentOS Stream 8 - PowerTools
+ l0 y, Q( x6 Y$ G h( t9 ~#mirrorlist=http://mirrorlist.centos.org/?release=&arch=&repo=PowerTools&infra=
" ]( {+ g+ t; p( P# k. J$ Zbaseurl=https://mirrors.aliyun.com/centos/8-stream/PowerTools/x86_64/os/1 c" x: s- K& h+ S+ L) |
gpgcheck=1
" t2 b1 Y, K3 ~+ Renabled=1
$ w. `7 o. ^3 z' v @gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
9 z; A' X" y- D/ e, f, g( W
6 }4 V$ ^( L7 i( ~% }5 @# yum clean all 清理缓存2 c$ [ }/ C, e) [' [2 a9 }- p8 [
# yum makecache 重新建立缓存3 B: H" Q7 |1 \
# yum repolist all 列出yum仓库（13个）
; e# p$ c2 q) `2 R8 k6 o6 [3.安装基础包及ovs（Tab补全命令，安装bash-completion包后执行bash就行）
5 M3 K8 b6 c3 \* W( Y- Q/ s/ s: b安装openvswitch3.1过程报错说找不到gpgkey文件就禁用gpgcheck=0再次安装就行了
- A- B1 }! `1 Iyum install -y vim net-tools bash-completion centos-release-openstack-victoria.noarch tcpdump openvswitch3.1
, K, |5 S+ R! ~" s; X& i# O或再单独安装yum install -y openvswitch3.1*5 T4 j6 C& {- j
查看安装版本：[root@ovs ~]# ovs-vsctl --version
( R! F/ H0 G9 Q) l8 A: W3 }" j4.启动ovs服务
: }0 o' U6 _2 L+ N! F# Y2 m[root@ovs ~]# systemctl start openvswitch
7 H* H) F8 `5 G' m1 S& g8 Z[root@ovs ~]# systemctl enable openvswitch
) a7 R* v& S1 p[root@ovs ~]# ps -ef | grep openvswitch
o; W. C5 |/ V3 s[root@ovs ~]# ovs-vsctl show 查看ovs虚拟交换机信息
$ k# _; } {5 Y3 n1 H$ B[root@ovs ~]# ovs-vsctl --help 求帮助 或[root@ovs ~]# man ovs-vsctl# \6 `1 T+ a' p. K8 K% I
5、创建ovs虚拟交换机
( c7 v8 i* a4 W% g% r9 ^当创建一个虚拟交换机会生成一个和虚拟交换机同名的Port 和Interface，type为internal（内部的）1 K: e, w5 G S% v) C
4 o+ N4 ^1 o0 v* A[root@ovs ~]# ovs-vsctl add-br br-int
% o; t3 t+ U; x4 N[root@ovs ~]# ovs-vsctl add-br br-memeda 添加
" v- P8 E4 p* c" Z% q; I[root@ovs ~]# ovs-vsctl del-br br-memeda 删除& y* \- y @* D
[root@ovs ~]# ovs-vsctl list-br 查看
( s6 E- \2 | k lbr-int
4 t7 F4 w8 v( ~! L2 Obr-memeda
1 J' ~3 Y: e6 B" V7 {# n6 a[root@ovs ~]# ovs-vsctl show 查询ovs虚拟交换机信息，Bridge桥指的是虚拟交换机, o% H4 g x' G: A) ~( V
54c67146-9a9f-40be-8cb7-e8792879aafa( `% ]. K& Z5 {
Bridge br-memeda3 O- o! `, @# N( |5 r
Port br-memeda" Q+ [: `+ w# ?1 p$ p3 e
Interface br-memeda, C! N) z ^# ~ E
type: internal4 d% M. ?+ G6 K: Q0 t: Y
Bridge br-int
& v" j- j/ M* V( }; A: `4 N; ] Port br-int
. ]$ K) c3 P" w( | Interface br-int# U; K/ B& q; C' b& k |; a
type: internal
. O; X+ ]6 P$ Q$ T ovs_version: "3.1.3"
# l4 G7 P9 ]6 _0 J用轻量级namespace网络命名空间模拟虚拟机
' n9 |9 Z( @ e6 a: R! P在这里插入图片描述1 g1 }8 R5 j- N- v" {" i3 G
/ k$ W+ Y' l3 ^& t7 y
[root@ovs ~]# ip netns 查看网络命名空间
# ]2 C% V1 u2 A+ l* M/ N3 _. O& c[root@ovs ~]# ip netns add ns1 添加网络命名空间9 G2 T" ?" d/ t( r
[root@ovs ~]# ip netns add ns2# Y- M' z8 i7 E/ ^9 F
[root@ovs ~]# ip netns
( Z, e! o" L( M6 n0 b7 n2 Lns2
; U5 E2 ?% B% `5 f, Kns19 B4 j3 B6 }* s2 C$ l, q
创建两个veth pair（一个veth pair有两个网络虚拟接口，veth可理解为网卡端口） 并将一端虚拟接口（veth1和veth2）连接到两个网络命名空间里面。veth pair：两个网络虚拟端口（设备）。# p# m2 p' _* g, }) U+ y
在这里插入图片描述! |' P w" t1 w. t: P* y' a
- Y$ |1 u; p$ V e& X! S0 W5 c
创建两个veth pair，并分别把这两个veth pair的一端放到上述两个网络命名空间
: j }4 T( b6 `* c# ip link help 或# man ip link 求帮助
: l/ U$ D/ U* T0 K9 y第一个网络命名空间配置# V+ V* t% M( L, x
[root@ovs ~]# ip link add veth11 type veth peer name veth1
8 ~; T1 W7 |* _. g0 `& A[root@ovs ~]# ip link set veth1 netns ns10 B0 x2 y1 {) Q% [7 Q l
[root@ovs ~]# ip netns exec ns1 ip link set veth1 up
2 [+ H6 |7 o9 V+ M第二个网络命名空间配置9 D% `: ]3 x5 F$ q9 n1 b2 D% {- o
[root@ovs ~]# ip link add veth22 type veth peer name veth2
5 O: z/ B8 _+ `# w9 u[root@ovs ~]# ip link set veth2 netns ns2
0 I/ g i% O* c: h[root@ovs ~]# ip netns exec ns2 ip link set veth2 up
3 y4 h/ A d: L% x; m: f将另外一端虚拟接口（veth11和veth22）连接到ovs虚拟交换机上
( ] i( Z5 Z0 R+ _/ C3 m6 H U$ y在这里插入图片描述1 }1 W0 J, G4 U, G* j/ Y
" K% x( s5 L# J. n[root@ovs ~]# ip link set veth11 up
3 T8 d" E" i* j3 K( N7 c" L[root@ovs ~]# ip link set veth22 up3 _ r! ?# W i6 _" ?! l
[root@ovs ~]# ovs-vsctl add-port br-memeda veth11
$ R! c$ J! C0 c$ j: T7 ?[root@ovs ~]# ovs-vsctl add-port br-memeda veth22
8 i; b* [1 G. f[root@ovs ~]# ovs-vsctl show 发现br-memeda虚拟交换机多了2个Port（Port veth22、Port veth11）" }+ ^' P6 a2 [( O. v
3b79f2e1-f433-4015-905e-8945dcada530
z/ U1 L$ s, ]. i3 X j Bridge br-memeda
7 |3 b" y8 A2 m+ t+ Z4 |* c0 H+ D Port br-memeda2 O0 h: I$ v* b( J
Interface br-memeda
' u. u- J6 o( V V) a7 Y' A type: internal1 _! b8 I; y& w* {; \
Port veth22/ x( n: Y8 a0 N9 H) g8 T- [
Interface veth22
, i6 [3 `# _$ V. _ N* E. n Port veth11
4 S' j/ {( c+ `8 R1 h w Interface veth11$ Q: `# m( S3 l1 }8 {
Bridge br-int
& B% R2 h1 o& B1 h6 |8 ] Port br-int) S3 \7 l3 U2 v# _3 b6 Y0 T* C7 H2 y3 H
Interface br-int8 a; \* f- p4 {: B; i9 t4 I$ K) H
type: internal
5 s/ [* S" a) f1 ^+ l% M ovs_version: "3.1.3"
$ \2 ]) m; q* m为两个网络命名空间手动设置ip地址
7 a7 [9 [! S3 A w在这里插入图片描述
' Q# H2 O5 t- Y- f/ `
' a7 [/ b- W1 V[root@ovs ~]# ip netns exec ns1 ip addr add 1.1.1.1/24 dev veth1- T7 f4 H" Y6 R$ I5 i
[root@ovs ~]# ip netns exec ns1 ip a- q1 \( C& I3 J: p
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
: M3 o, ]) F5 z& C link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00% Q/ n. b" `8 [! M4 W+ i
7: veth1@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group d efault qlen 1000: M9 V1 z8 K% u' u
link/ether fe:f9:3b:cb:9b:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0# }" n: C2 _' [5 ?* g P9 B; |3 N
inet 1.1.1.1/24 scope global veth1
3 m# p' l; ^' F' ~& O7 `0 f valid_lft forever preferred_lft forever9 I* A/ G( \; b7 ~
inet6 fe80::fcf9:3bff:fecb:9bc5/64 scope link4 f5 d- T" U6 [: W
valid_lft forever preferred_lft forever$ s4 ]1 @; c- a! l/ y
[root@ovs ~]# ip netns exec ns2 ip addr add 1.1.1.2/24 dev veth2
8 v& F! s7 {+ x* a" D) Z& X[root@ovs ~]# ip netns exec ns2 ip a
! {) k" P( }( B* `1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000 R7 F J5 J# B. \' ^) { D
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00- Z% \( l: k4 n: }. U. b' V: Q+ H
9: veth2@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
3 `# S* H) A5 C7 J link/ether 0a:e3:ac:a8:f3:bc brd ff:ff:ff:ff:ff:ff link-netnsid 0
8 z9 _ b/ I* h& O! {" Z& [ inet 1.1.1.2/24 scope global veth26 v" ] G: X# g
valid_lft forever preferred_lft forever
- _ y: B: m$ r+ C/ [& T# |( K inet6 fe80::8e3:acff:fea8:f3bc/64 scope link. L0 H. u4 X7 J. U; x& a
valid_lft forever preferred_lft forever
- I* u* m, M+ a& M2 C' C, ~两个网络命名空间测试连通性
: ~" N# O. I5 Z$ U+ L: ?[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2
# }6 G7 i% I# k8 f U$ QPING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.% z( u2 \8 I4 q+ H* M: H; L0 p
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=2.98 ms; S c9 Y( l3 F+ b L5 t* _
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.167 ms- h3 V \# w$ L- R2 @* D! ^
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.081 ms- h+ I' l' \% T/ ]7 V* ^9 F
$ h; R! s3 C. R, [( B* q0 E1 w0 S7 J% @
--- 1.1.1.2 ping statistics ---
n/ z, B, A4 S0 v9 I$ |( ?. b3 packets transmitted, 3 received, 0% packet loss, time 2065ms: z5 n! a0 ]' u8 j2 Y$ M# A( |. ?1 p
rtt min/avg/max/mdev = 0.081/1.075/2.979/1.346 ms) E) O2 ?; F! @9 j
[root@ovs ~]# ip netns exec ns2 ping -c 3 1.1.1.1* L0 Z9 W6 _0 ?
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
?8 u9 g0 [8 M4 }. o. g6 u5 F64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.923 ms
* a+ U9 z+ ^( z- Q( n64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.084 ms% v( ~7 @1 x7 P- t/ N9 K
64 bytes from 1.1.1.1: icmp_seq=3 ttl=64 time=0.091 ms
$ n1 }# ?" ~6 a n9 t
) H& p. O0 D# N8 w$ ?1 \- Q% G--- 1.1.1.1 ping statistics ---9 x" B5 {+ U3 q
3 packets transmitted, 3 received, 0% packet loss, time 2007ms% d, z7 z/ l9 l! d9 n0 t
rtt min/avg/max/mdev = 0.084/0.366/0.923/0.393 ms0 h4 ~4 `, `* F0 a6 d& h0 s) O' [
vlan虚拟的本地局域网，vlan隔离为了减少网络阻塞和数据包安全
$ c1 R7 j, r( K4 n$ Covs虚拟交换机能和物理交换机一样定义vlan，一个vlan10（tag10）,一个vlan20（tag20），把插在ovs交换机上的两个虚拟网络设备对端口分别打上不同的tag（默认是0），也就是配置到不同的vlan里，再验证网络连通性。3 W2 y1 f: E. G6 N
在这里插入图片描述
4 N0 J# _0 q, T6 K; j F( o$ H/ e
: C6 G, s8 }! ~4 H, x( T, [[root@ovs ~]# ovs-vsctl set port veth11 tag=10
: [. T4 Y( ?. l" e+ b6 p2 a: i[root@ovs ~]# ovs-vsctl set port veth22 tag=20% L! ^1 x" m4 v* I4 ^. n5 F
[root@ovs ~]# ovs-vsctl show 发现br-memeda虚拟交换机的Port veth22和Port veth11下面多了tag标签
! K) ?8 O4 M: B7 ?- P2 _5 v0 ^ S3b79f2e1-f433-4015-905e-8945dcada530! V+ E& Q4 F: a# }, ^
Bridge br-memeda
8 w9 A/ U1 e S! z Port br-memeda
' C/ P( ] j# v Interface br-memeda
3 A2 |' e( ~2 @+ L4 t" t2 n/ N/ e type: internal4 l! A& ]+ ^( q
Port veth22
% h! e; W% S" t( F' d tag: 20
$ r) j. V& t' Z% }; q8 |( c' w" @ Interface veth22: d/ T! `/ j8 b! ]8 V( y+ @
Port veth11
9 J% a$ ?4 \4 B, g8 f tag: 10
0 `: ?8 b* H/ A8 @ Interface veth111 z* @; q. c" l2 T, b
Bridge br-int, ?2 B3 Y; X! x) Z8 Z7 R. T
Port br-int
, I1 R- N1 a( u W- }; j- h* K: s8 ] Interface br-int
" S( Y5 c, G8 p) ^$ \/ G3 f type: internal; v9 ]' q6 a$ u& y
ovs_version: "3.1.3"
8 }, |$ H# y0 g9 C6 e添加不同vlan（tag标签）后ping不通，需借助路由或物理三层交换机
! q! L7 M) W# f4 e7 g# T
: j0 d$ I; E ^. i# B; F[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2
3 {4 H' i# @! k# S/ L# `8 HPING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
) w2 E2 G8 _+ \' w2 h' g, k- d# J* J+ A) E6 c" ~+ t2 S2 _- x) M
--- 1.1.1.2 ping statistics ---+ ~% j t; b5 t) O" t
3 packets transmitted, 0 received, 100% packet loss, time 2064ms
* k, K8 l( d; q9 ^. W在这里插入图片描述
' y0 U9 ?# i, ?' ?0 j* i( d& }) J6 T4 A" Z' a3 h- r
[root@ovs ~]# ovs-vsctl set port veth22 tag=10 把veth22也改成tag=10就相当于同一个vlan二层互通了
: k$ n/ d( G" t9 g[root@ovs ~]# ovs-vsctl show7 ^4 Y! d3 j3 M2 M
3b79f2e1-f433-4015-905e-8945dcada530
a; M4 _' M# b4 p1 ` Bridge br-memeda# c; b9 u( S, h( C$ w" t
Port br-memeda
- z7 c0 \7 g1 w3 k9 f Interface br-memeda3 C5 C% X6 u: e0 `* ?: d
type: internal; f8 q& N7 v! C4 m
Port veth22
; B2 A7 K, ], ]) F X$ d% I tag: 10 T( c& M2 T0 B% _: ^
Interface veth22; u1 M4 d/ Z4 K( [) F4 w% @
Port veth11
_7 d7 f. Q Z- { tag: 10
; p6 Z# N' c/ g# J Interface veth11: D6 B6 Z+ |( M0 r6 x
Bridge br-int* R0 ~+ Y0 I1 y
Port br-int- N) Z; ?# a0 m; ?( ~% f" H
Interface br-int
4 h0 T" m; I- A' m: R type: internal
; R6 v' D. t5 ?* m) @* P9 C; D ovs_version: "3.1.3"
F6 H1 d% ?4 R# f3 k[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2 同一个vlan（tag标签）能ping通进行二层通信
( h% J6 S7 u" q' UPING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.0 Q O8 p# a; [2 L# ~, g) @( B
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=1.43 ms. _# G1 G; U4 |& _) F4 [7 S
64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.093 ms1 d P' \# @4 f4 H/ s0 w* T$ S
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.086 ms
5 Z) F T0 C9 y* E) U2 y6 D: z! k- m; R+ U0 @. q: k
--- 1.1.1.2 ping statistics ---
3 w; G }7 v$ ?3 g+ j3 packets transmitted, 3 received, 0% packet loss, time 2051ms8 J; @- k l( R! V# V- U: d
rtt min/avg/max/mdev = 0.086/0.535/1.426/0.630 ms! b, }! P" N% c- y
FlowTable：流表，ovs进行数据转发的核心功能，定义了端口之间的转发数据规则。每条流表规则可以分为匹配和动作两部分，“匹配”决定哪些数据将被处理，“动作”则决定了这些数据将被如何处理。
* D' _( D# Z0 J/ [8 Q流量走向，添加流表，针对流量进口添加规则。
$ h7 |2 n& ~- f4 y; S# \" O# \7 p在这里插入图片描述8 o1 b" X0 U8 _# ~& `2 P1 k/ T9 X
在这里插入图片描述, E1 Y3 |: |$ p8 `0 x* c1 y& H
; Q- Z" ]' Y, J7 g& L: u8 `查看ovs默认的流表
1 G( S/ e/ k$ ^) C' n! n9 ?! i[root@ovs ~]# ovs-ofctl dump-flows br-memeda 查看虚拟交换机的流规则
7 Y$ ^( n# a2 z. Y6 e cookie=0x0, duration=2161.884s, table=0, n_packets=49, n_bytes=3682, priority=0 action s=NORMAL
7 B2 V2 w* B8 ?: C) I' Y此时ovs就类似于传统交换机，我们给ovs交换机添加一条优先级为2（数字越大优先级越高，高于默认表项的0优先级）的流表项，把veth11进来的请求都drop掉，发现ns1不能ping通ns2。3 M, z8 P7 s9 n, a" v
[root@ovs ~]# ovs-ofctl add-flow br-memeda "priority=2,in_port=veth11,actions=drop" 添加流规则
6 T. K5 V/ ~; ~* i4 g[root@ovs ~]# ovs-ofctl dump-flows br-memeda3 y+ z& e7 h2 \1 b! U
cookie=0x0, duration=2.578s, table=0, n_packets=0, n_bytes=0, priority=2,in_port=veth11 actions=drop. k5 g7 g8 S$ s
cookie=0x0, duration=2217.329s, table=0, n_packets=49, n_bytes=3682, priority=0 actions=NORMAL: X2 t. J& d! C" E0 O& `) i2 r* L
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2- I' c( D. A. ]# q5 Z) I9 Y$ n& ^
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data." S+ p( b" B. l. j( @2 E
) Z G! ^( H+ {6 X
--- 1.1.1.2 ping statistics ---
6 ]0 T% Q# H( H" c- A# G+ o' \8 B2 \7 l3 packets transmitted, 0 received, 100% packet loss, time 2076ms2 v+ \7 [1 `7 R- ?. e) m
删除刚添加的表项，ns1与ns2又能正常通信2 u( r; T9 {2 t* I- F
[root@ovs ~]# ovs-ofctl del-flows br-memeda "in_port=veth11" 删除刚添加的流规则就互通了' h' _% z1 X& s( \/ b
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2
3 E; e' D: f5 k7 N5 GPING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
: p" K2 p; t8 V3 i3 ^64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.766 ms
; N' ~% |" E3 d8 m64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.096 ms- T6 m6 o V( B5 }, x6 P6 b. s
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.088 ms
3 H% Q" n- y% T9 R8 ?( F
2 F% e, v1 u, S4 o--- 1.1.1.2 ping statistics ---
9 `4 g& U G* }. X3 packets transmitted, 3 received, 0% packet loss, time 2043ms5 S4 Y9 G1 o) i# I' _: o
rtt min/avg/max/mdev = 0.088/0.316/0.766/0.318 ms
; c5 N- S& {' ^; r" |[root@ovs ~]# ovs-ofctl dump-flows br-memeda: g8 ]! v1 V% V6 ~
cookie=0x0, duration=2315.744s, table=0, n_packets=59, n_bytes=4438, priority=0 action s=NORMAL
: \* u. I; n5 ~* g: s' i4、OVN
6 s/ A$ o2 V @* W, ZOVN建立在OVS之上的，遵循SDN(Software Defined Network，软件定义网络)架构来管理的，用软件将控制面和转发面分离，OVN做控制面，OVS做转发面。
3 f& N7 c( c3 b% c7 jovn是建立在ovs之上的，ovn必须有底层的ovs，ovs可理解为二层交换机，ovn可理解为三层交换机。5 E6 `# K t9 O. v( Q
OVS介绍参考：https://mp.weixin.qq.com/s?__biz ... 189#wechat_redirect, j7 Z% M9 A2 E1 C2 k
单纯的ovs在云计算领域还存在着一些问题，例如：2 _- V [' W1 b2 w( w/ L. P
1、ovs只能做二层转发，没有三层的能力，无法在ovs上进行路由配置等操作；7 c- I# u+ I. I0 l5 X
2、ovs没有高可用配置；
+ w- \1 @: Y) o/ ?! g3、在虚拟化领域vm从一台物理机迁移到另一台物理机，以及容器领域container从一个节点迁移到另一个节点都是非常常见的场景，而单纯的ovs的配置只适用于当前节点。当发生上述迁移过程时，新的节点因对应的ovs没有相关配置，会导致迁移过来的vm或者container无法正常运作。
% W/ ]+ p) l7 a7 C r- S. A( D针对这些问题，出现了ovn(Open Virtual Network)，ovn提供的功能包括：0 R! N* W! e5 ^0 d
1、分布式虚拟路由器（distributed virtual routers）3 N7 k0 }" e9 s- A
2、分布式虚拟交换机（distributed logical switches）
" h+ j: N. J) ?3、访问控制列表（ACL）/ I7 d W9 U" n/ @
4、DHCP# R: L0 f- ]% ?! ^- }8 d7 M5 [
5、DNS server
* z9 `7 ?! {5 J9 h6 C5 f1 x; B在openstack里面，创建一个网络，就相当于创建了一个逻辑虚拟交换机，这个逻辑交换机（网络）信息会被保存到北向数据库里面。openstack创建一个网络，会以逻辑交换机（switch）的形式保存到北向数据库。2 R/ t* o {. X! j4 K. u3 U- A
在这里插入图片描述* @7 p% I0 ]; d& M. g4 ]/ l- v
在这里插入图片描述
' ?4 S3 z+ g* w" k/ B" Novn官网对ovn的逻辑架构如下所示：. G% |& j/ }4 h) `* `
; N. \3 h7 ]. @/ c: Y# m
CMS: O# r& D3 Q: T4 D+ _8 o+ j
|
1 A' X+ |/ `7 G. T) o |
8 a; \/ w6 V: E6 W +-----------|-----------++ n4 @. ]4 S' z; X+ u- v, P
| | |
& C% A) h: a, _ | OVN/CMS Plugin |2 \2 w7 c7 q; C/ `
| | |
2 }# O# c K5 F, p3 J | | |
, l' z- a T" u; q+ a) V | OVN Northbound DB |
5 K2 U9 T; |& P+ f, R( t | | |
( L# J; S2 }) R | | |% ?! o" c9 t: y) R' b9 L0 ?
| ovn-northd |9 i6 C9 K& R' ^1 k2 @
| | |
' c' }* b' Z" \2 p& v$ F0 J- m +-----------|-----------+2 U0 {! [1 s# P3 @1 j. g9 F
|
. X) Y$ Z2 `( M, p! c3 Y |
F5 b$ D& {9 o4 [. k +-------------------+% l3 r, d+ X5 w+ \
| OVN Southbound DB |
4 t5 A1 y) F# U1 H0 J +-------------------+
6 W h% A8 n! v: h- `+ ]4 [ |+ z: [( L+ r8 t/ ]
|
2 [# i: K( S1 F6 u +------------------+------------------+3 ] |7 G1 e; x' \3 }$ e
| | |$ z p2 Z- ^4 [$ s9 l9 t; s
HV 1 | | HV n |, r6 j& c5 P6 V& t6 n# K
+---------------|---------------+ . +---------------|---------------+2 T& t; w2 U* l/ q- b
| | | . | | |
3 M: ]& S1 k s& k" U6 T, O6 d | ovn-controller | . | ovn-controller |9 @" W$ O6 z+ r7 K& {
| | | | . | | | |
' w; x& Q) j( N* z | | | | | | | |7 r9 G: L- ?# R9 V7 J( S- G1 [
| ovs-vswitchd ovsdb-server | | ovs-vswitchd ovsdb-server |
4 p4 M+ ]' ~9 w* [( v6 I* [ | | | |5 e( f2 e$ p# `% g: t' K" e
+-------------------------------+ +-------------------------------+; o, i6 D1 g8 r; u
ovn根据功能可以把节点分为两类：* c: x9 i) i! m* m2 I# w. M/ H
central: 可以看做中心节点，central节点组件包括OVN/CMS plugin、OVN Northbound DB、ovn-northd、OVN Southbound DB。0 u) W% ~- V. ~1 Y* B
hypervisor（hv）: 可以看做工作节点，hypervisor节点组件包括ovn-controller、ovs-vswitchd、ovsdb-server。, K- _, f9 M& Q: j7 u& J
central节点相关组件和hypervisor组件运行在同一个物理节点上。
" `( h6 [. g T# G9 J相关组件的功能如下：; U5 @6 \; ~/ v- a) b( A
1、CMS: 云管软件（Cloud Management Software），例如openstack（ovn最初就是设计给openstack用的）。/ x! c6 a" s3 Z# N# A5 _" ]
2、OVN/CMS plugin: 云管软件插件，例如openstack的neutron plugin。它的作用是将逻辑网络配置转换成OVN理解的数据，并写到北向数据库（OVN Northbound DB）中。
1 N+ K- i' k3 e0 ~& D& ^9 Q3、OVN Northbound DB: ovn北向数据库，保存CMS plugin下发的配置，它有两个客户端CMS plugin和ovn-northd。通过ovn-nbctl命令直接操作它。北向数据库保存逻辑网络信息（交换机和路由器等）
" o7 l9 o: ?$ C+ V" H) ]% k4、ovn-northd: 北向进程将OVN Northbound DB中的数据进行转换并保存到OVN Southbound DB。所有信息经过北向数据库通过ovn-northd北向进程和南向数据库互通。
& s$ `) O" @! |" J" k5、OVN Southbound DB: ovn南向数据库，它也有两个客户端: 上面的ovn-northd和下面的运行在每个hypervisor上的ovn-controller。通过ovn-sbctl命令直接操作它。南向数据库保存各个节点的物理网络信息。. c( N5 U. |+ a1 Y+ l. }
6、ovn-controller: 相当于OVN在每个hypervisor上的agent（代理）。北向它连接到OVN Southbound Database学习最新的配置转换成openflow流表，南向它连接到ovs-vswitchd下发转换后的流表，同时也连接到ovsdb-server获取它需要的配置信息。! S1 h- W* M- j, J& Y
7、ovs-vswitchd和ovs-dbserver: ovs用户态的两个进程。3 O! S: e4 p3 L
每个节点都有个ovn-controller控制器，这个ovn-controller控制器是管理ovs（ovs-vswitchd、ovsdb-server）的，ovn-controller对接到南向数据库，经过ovn-northd北向进程和北向数据库互通，之后和openstack互通。
9 I2 t$ I. c' Z+ N, @$ N南向数据库保存物理网络状态信息，北向数据库保存逻辑网络状态信息。; [9 P& x# l& W6 x0 ^
在这里插入图片描述- `/ x: T/ f* @& K, W$ O6 T
克隆出两台虚拟机，安装ovs、ovn5 f2 k/ W) `, L0 i6 D8 s9 g$ w
* r: L, L, h- }' T
CentOS Stream 8 版本) Z* ^% d. g* }2 G& D9 w( m! a
8 V8 w5 N6 x4 E4 G! z: X* j5 K% dsystemctl stop firewalld.service , P- |! S6 U8 w: g$ }2 a6 w
systemctl disable firewalld.service
- _- M: D8 b/ Z; s4 C' N& qsetenforce 0
4 { u, q) g$ I0 E2 vsed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
8 Q1 y H& |$ w0 X" `* D7 ?mkdir /etc/yum.repos.d/bak8 B5 o. y1 X" A% b4 I) F- ^$ k
mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/7 ^6 R$ p, C) `& z+ O0 | O" Q- J
6 \, f0 N E S( e1 i7 g6 \cat <<EOF > /etc/yum.repos.d/cloudcs.repo# L q: R0 H. b) R9 _7 U, f
[ceph]
m8 v1 v6 H$ Fname=ceph
& h1 R/ Z; E" t6 d% ?5 Obaseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/x86_64/+ ?: r% G% b6 A
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc r$ v( O5 r. r- a- M8 j: F7 ~7 C
gpgcheck=1; n! ^% l8 r! s* P* ~+ V+ h
enabled=1* i- l {) x' J8 [
7 R% l8 d& d+ C2 Q' h/ u[ceph-noarch]
7 X0 t8 {8 m7 l. Q. i3 q) g. h- wname=ceph-noarch! \1 W- Q7 m2 T1 C4 r6 D+ o/ e
baseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/noarch/
6 p6 F6 h; n: f5 h" [gpgcheck=1
8 p4 u& z# E% p9 P/ Jgpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc3 m# B7 \. W! |6 |/ ~* @: q# K1 X
enabled=1
' \0 z4 y" ~$ F- X3 E7 P# `. [
@& o( }0 Y S[ceph-SRPMS]
: U6 p. H1 s s" B! C- Fname=SRPMS' r' S; F7 v$ m6 n* c
baseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/SRPMS/+ p& n3 z! d. R4 g9 ]
gpgcheck=19 z7 f7 Z; M0 @; `+ V, |- B
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc! T4 V! P: b2 t* x) w; \* R
enabled=1
4 x5 N3 \4 _: e/ {
w. c1 ^* A7 x/ Z[highavailability]3 a, R" z1 o) I; a. H: z2 {
name=CentOS Stream 8 - HighAvailability
$ W; {; E' j5 P2 V) p& sbaseurl=https://mirrors.aliyun.com/centos/8-stream/HighAvailability/x86_64/os/
, q+ c: F8 U! U1 t" I8 j, ^) L/ b" Kgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
, v8 B' r U- o4 |5 |8 Q& ogpgcheck=17 @! ~' t3 d4 j2 e
repo_gpgcheck=0
9 L* _9 o( P- r) z) n# e" g; \# ~% Jmetadata_expire=6h
* z& E+ m9 Q, A7 P; N+ f+ Z/ ?countme=1! w4 C2 X7 \' n1 y
enabled=1
! l; p0 D! r6 x5 S6 \& { J9 W; p0 q8 x* S5 u% _
[nfv]
, U+ O! ]/ G8 Z; N# A1 Sname=CentOS Stream 8 - NFV1 S! Q: `0 d# O1 U7 a
baseurl=https://mirrors.aliyun.com/centos/8-stream/NFV/x86_64/os/
+ ]: p1 w% o4 R/ Qgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial0 r* o3 F L6 A& M
gpgcheck=1
0 T" S% |! `; H: R6 o7 k6 jrepo_gpgcheck=0
0 d/ T1 w6 r) a2 P1 h( ?. g; X' [metadata_expire=6h
: O4 r0 d0 `0 O8 `countme=1
) @0 ~- ^# J. l, k$ I: L% Tenabled=1
7 L5 F2 [9 [# D8 y# b) X
% {8 J2 U( c% s6 C9 ~[rt]+ P6 }; ?' J7 }* B; {& m
name=CentOS Stream 8 - RT/ v9 y5 [3 \& S' I5 V# ?8 I5 N7 m- v
baseurl=https://mirrors.aliyun.com/centos/8-stream/RT/x86_64/os/& g) k2 [0 }* p A& R
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial ?. E! k4 U# q% F
gpgcheck=1
! M' \3 V$ R5 t0 q1 B& F6 Jrepo_gpgcheck=0
, I7 H$ D% }: ]7 ]metadata_expire=6h0 o, z" \: n3 E, a8 {! h/ V
countme=1
: W5 b1 J. L8 i+ K# W% ?5 M" Lenabled=1- W9 ?3 L/ Q3 k# N- n
X! K. ~- F2 |# Z5 l
[resilientstorage]" G* n& |$ P" `# W5 x
name=CentOS Stream 8 - ResilientStorage6 V: ]3 x! C. k& e; k
baseurl=https://mirrors.aliyun.com/centos/8-stream/ResilientStorage/x86_64/os/ [$ K; i+ @. {- S2 C p
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
1 q9 ^! t% b; k, r% \8 ugpgcheck=12 o9 h' M9 k% ~2 j# D' \1 @
repo_gpgcheck=0
7 s% t8 F6 u0 I$ C7 ?) ^; K/ Qmetadata_expire=6h- p3 n; N+ S* r' t* c
countme=1
( B$ W3 ~. m N! c9 N2 [( zenabled=1( h' B% j6 r# c/ ?& l2 j2 W- R
' Q3 J5 j4 i- R. }" k, E
[extras-common]- }; @7 A- \" `1 U, x2 m; ^
name=CentOS Stream 8 - Extras packages
, Y0 d( k: o1 }baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/extras-common/
. G5 V" R. k* j2 tgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512& q- v4 ?) ?! H) G |8 v
gpgcheck=1& ~ N: C, b. |7 L! U# z6 N6 y& K
repo_gpgcheck=0 Q3 M r/ O9 ~/ j" q
metadata_expire=6h
# S, a) v8 v# c" a$ K- S- G' }countme=1& f B( d& I) y' z ]
enabled=12 H4 m' \" m1 P
$ ?6 Y1 Q4 v7 J4 _# j0 Y- k[extras]
" X$ e$ X1 W' j; Rname=CentOS Stream $releasever - Extras7 v: v8 n3 d$ L8 W, ]& h2 U
mirrorlist=http://mirrorlist.centos.org/?release=$stream&arch=$basearch&repo=extras&infra=$infra. W6 S# z5 C' |- |
#baseurl=http://mirror.centos.org/$contentdir/$stream/extras/$basearch/os/ W8 Q) m7 O# h" @/ i: B& V
baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/os/# w# s- r( W$ c4 z V8 H; s# i
gpgcheck=1
! ?7 @$ z3 A5 i' U$ x/ renabled=1
. |& s7 Q5 h1 N$ v, `6 ?4 sgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
( }* ]: k9 K2 m8 S" f( Z/ K; [9 ^! w' [, N" D
[centos-ceph-pacific]5 s$ n1 m( N* h. F/ [0 n% G
name=CentOS - Ceph Pacific+ C3 {( C& y6 V' b o4 n
baseurl=https://mirrors.aliyun.com/centos/8-stream/storage/x86_64/ceph-pacific/% N. S$ {5 @- V ~9 f) V) g; X' ~
gpgcheck=0# V) Y: i+ C; f0 z$ D
enabled=1
9 m u: |( p! G/ k4 k* Agpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
, ~% d0 h: J" k
) {! |$ c& w. `' h: o[centos-rabbitmq-38]
+ j0 K6 ~( y- o. W6 O Ename=CentOS-8 - RabbitMQ 38/ ^5 q3 {; Y: E
baseurl=https://mirrors.aliyun.com/centos/8-stream/messaging/x86_64/rabbitmq-38/
5 @. S7 E4 D* U9 U" mgpgcheck=14 R J% P* p9 r/ B \8 }
enabled=16 \6 E0 ?/ p6 @: w
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging
' X4 E3 ~& l' d* y7 g. I2 m
- S7 ~/ V" j7 L[centos-nfv-openvswitch]
8 A3 U2 k$ ]- k1 i9 |$ E. U# f$ s6 gname=CentOS Stream 8 - NFV OpenvSwitch
# z/ Z9 B( b" H& Q# M; mbaseurl=https://mirrors.aliyun.com/centos/8-stream/nfv/x86_64/openvswitch-2/
. v; U* m/ _0 [$ ~5 L4 vgpgcheck=1
3 w( k4 I3 h" Penabled=1' r. K U* q# B/ Q" I0 S0 L' U+ l5 d
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV8 `3 D: l2 K8 P0 ?6 O
module_hotfixes=1" ], [# ^# A$ e
" |% O; a" @1 r* I) {
[baseos]
; Z9 A) _: A' K0 N0 Vname=CentOS Stream 8 - BaseOS( W( x# g3 G% Z4 h+ P. \7 I
baseurl=https://mirrors.aliyun.com/centos/8-stream/BaseOS/x86_64/os/- s# F; x6 D. Z7 b
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
/ z" O' B" o7 j3 a; egpgcheck=1
8 ?: ?1 k% I3 @( d5 Y& Mrepo_gpgcheck=0
* t P2 G7 L i4 t Umetadata_expire=6h2 [, T) Q* C) P) K# m7 E1 W5 E
countme=1
/ m4 w; v4 P) `+ f9 N Z) W0 Jenabled=1
1 J: o% U k( Y U2 c7 y6 N' ]- B& _- m8 v. n/ ]4 e% ~# i
[appstream]& F. N+ x: X- |
name=CentOS Stream 8 - AppStream
6 H( j- u) I* b" {/ E/ [9 dbaseurl=https://mirrors.aliyun.com/centos/8-stream/AppStream/x86_64/os/
, D# Y( V- I. a5 v/ `- fgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
: t/ V7 }5 @5 E' I k$ A& J! s" Igpgcheck=1
4 Y3 F4 a& \( Arepo_gpgcheck=0" p# x. B$ q& y* v" p- e
metadata_expire=6h+ f! y; E$ Q2 R* k7 @5 W; P5 N
countme=1
+ O; t) }6 P. O" Penabled=17 v' ?# W- }$ v4 l3 g& o
3 ]# ]$ G: ]" u7 F
[centos-openstack-victoria]
; l0 k5 e. g% l8 vname=CentOS 8 - OpenStack victoria
- w# H1 m1 i$ zbaseurl=https://mirrors.aliyun.com/centos/8-stream/cloud/x86_64/openstack-victoria/
+ x- ~$ i- `( {* V$ t9 ]( _/ \#baseurl=https://repo.huaweicloud.com/centos/8-stream/cloud/x86_64/openstack-yoga/
" H4 P% G- s- K4 Igpgcheck=16 M' R$ ^6 V$ q/ _9 T0 u# g
enabled=1
" d8 W# \8 V/ Mgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud: r/ z1 s& X2 z' ]
module_hotfixes=1
9 y8 d1 c' S. p
# V+ k: |: a% L; C7 D1 T& z[powertools]6 f7 G& U4 s; J. z- F
name=CentOS Stream 8 - PowerTools) k0 H$ t8 j) U d
#mirrorlist=http://mirrorlist.centos.org/?release=$stream&arch=$basearch&repo=PowerTools&infra=$infra
$ P5 d6 {% t0 N: ? ~4 |. hbaseurl=https://mirrors.aliyun.com/centos/8-stream/PowerTools/x86_64/os/! D& W0 F% f* Z) G( n9 d
gpgcheck=1
; c: S9 d; j1 ?1 Yenabled=13 p) g: x4 ~; i! W4 P( _8 D
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial# f1 k/ M$ f5 k5 r
EOF! w; p! U4 _. d/ v
2 k- C9 @$ ?* X2 }4 k0 Dyum install -y vim net-tools bash-completion git tcpdump autoconf automake libtool make python3 centos-release-openstack-victoria.noarch
3 {$ A: E. _2 ] ^3 cyum install -y openvswitch3.1*" _, E1 \- F9 T: b# I( r
yum install -y ovn22.12*! [) j8 Q$ y( Z4 u6 Z
查看安装版本来检查ovn是否安装成功，# ovn-appctl --version Z( t9 g2 |& S) X+ U& w" j
echo 'export PATH=$PATH:/usr/share/ovn/scripts:/usr/share/openvswitch/scripts' >> /etc/profile: o2 j7 A, ^' I/ k: x1 K9 m
source /etc/profile 重新读取配置文件让配置文件立即生效, F, E/ t4 j7 k0 c" J
在这里插入图片描述
0 Z$ Y, H) L5 L; bcentral相关组件启动：把node1作为central节点，安装central必需的三个组件：OVN Northbound DB、ovn-northd、OVN Southbound DB。
1 N3 o a9 W P* D+ Q0 i在控制节点启动central，只用在一个控制节点上启动即可（node1或node2上开启都行，这里是在node1开启），central只需要一套即可。
. _2 X& A E8 g1 {
% U. C1 s5 U; _- |; W$ w% T. Vovn-ctl start_northd命令会自动启动北桥数据库、ovn-northd、南桥数据库三个服务" G# Z7 c% C) T z
[root@node1 ~]# ovn-ctl start_northd
4 }" V2 C' \" H. ^- W4 Q/etc/ovn/ovnnb_db.db does not exist ... (warning).; ~ c: t9 b& k4 t2 a( v3 _$ C
Creating empty database /etc/ovn/ovnnb_db.db [ OK ]1 A7 y- J4 j- I- ^, e- S6 A% c
Starting ovsdb-nb [ OK ]- E8 V4 n0 y- C) A
/etc/ovn/ovnsb_db.db does not exist ... (warning).
# P4 T7 C# D" ?8 O6 [& RCreating empty database /etc/ovn/ovnsb_db.db [ OK ]% N: k5 o* W; u- v
Starting ovsdb-sb [ OK ]: O' L( U( |* b- ^9 Y, Z
Starting ovn-northd [ OK ]
6 |7 {7 s/ a9 l2 Z
# V; F2 T* D" U1 ^4 |[root@node1 ~]# ps -ef | grep ovn
! D) O& m6 Y5 N: e$ ]6 yroot 34102 34101 0 21:02 ? 00:00:00 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --detach --monitor --remote=db:OVN_Northbound,NB_Global,connections --private-key=db:OVN_Northbound,SSL,private_key --certificate=db:OVN_Northbound,SSL,certificate --ca-cert=db:OVN_Northbound,SSL,ca_cert --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /etc/ovn/ovnnb_db.db
7 a/ {: Y* h" O2 z5 j" @ Hroot 34118 34117 0 21:02 ? 00:00:00 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-sb.log --remote=punix:/var/run ovn/ovnsb_db.sock --pidfile=/var/run/ovn/ovnsb_db.pid --unixctl=/var/run/ovn/ovnsb_db.ctl --detach --monitor --remote=db:OVN_Southbound,SB_Global,connections --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers /etc/ovn/ovnsb_db.db
+ d& I. |2 @/ X1 _( z: p, n' Groot 34128 1 0 21:02 ? 00:00:00 ovn-northd: monitoring pid 34129 (healthy)
) g* U S1 ]+ n/ a9 m+ i# zroot 34129 34128 0 21:02 ? 00:00:00 ovn-northd -vconsole:emer -vsyslog:err -vfile:info --ovnnb-db=unix:/var/run/ovn/ovnnb_db.sock --ovnsb-db=unix:/var/run/ovn/ovnsb_db.sock --no-chdir --log-file=/var/log/ovn/ovn-northd.log --pidfile=/var/run/ovn/ovn-northd.pid --detach --monitor
4 i% s! ~7 t6 d2 e& }7 |# T ~root 34302 34259 0 21:07 pts/0 00:00:00 grep --color=auto ovn' T6 M0 Y. [& Y$ ^& M# u {
在这里插入图片描述9 C9 n1 @1 t3 U
hypervisor相关组件启动：hypervisor节点包含三个组件：ovn-controller、ovs-vswitchd和ovsdb-server。+ S0 A) M. B* @9 s* G
启动hypervisor（hv）相关组件：node1和node2两台节点上都要启动，首先启动两个节点上的 ovs-vswitchd 和 ovsdb-server1 b5 n) q) P, v
% n4 X, a6 ^9 \7 g. E' a" P
[root@node1 ~]# ovs-ctl start --system-id=random
/ k. d9 Z, h' i4 [8 B `1 X Z/etc/openvswitch/conf.db does not exist ... (warning).! \6 T1 F p+ ^& X7 O" e
Creating empty database /etc/openvswitch/conf.db [ OK ]/ N" N$ ~% c7 B
Starting ovsdb-server [ OK ]* r+ U) a7 a# }8 p% C, a2 G! f3 {
Configuring Open vSwitch system IDs [ OK ]
! }4 Z) s% @, ~$ }( K) sInserting openvswitch module [ OK ]1 b2 V* a C9 E9 Z+ e
Starting ovs-vswitchd [ OK ], q2 _$ V# Y1 T. F8 U# X6 D# j: \1 b k
Enabling remote OVSDB managers [ OK ]1 ~5 h' o1 b0 U ~* a, K! a2 Y9 _; ~
( L& d% ?' n& \8 X: R0 |! j[root@node2 ~]# ovs-ctl start --system-id=random
3 ~1 h; s9 M2 ^4 G/etc/openvswitch/conf.db does not exist ... (warning).
( G; P0 J9 `4 }. [7 d: J* KCreating empty database /etc/openvswitch/conf.db [ OK ]5 e1 N( a$ W$ L+ k; O
Starting ovsdb-server [ OK ]1 z2 j. i0 }0 K$ h6 ], `- Q {
Configuring Open vSwitch system IDs [ OK ]
" l! E0 C9 M7 l' d- V# AInserting openvswitch module [ OK ]
' ]; }- Q9 O. ]9 \8 a, HStarting ovs-vswitchd [ OK ]
, J0 F) j. G" I: mEnabling remote OVSDB managers [ OK ]
' l& k9 `3 Q, ^% n7 m& g在这里插入图片描述
# {4 R4 ?: Q- y* `: m, N" a# j7 }两个节点分别启动ovn-controller
1 Z3 d5 m& o4 h8 K6 o9 C/ ?; u6 |- w# a5 ?) j% D
[root@node1 ~]# ovn-ctl start_controller
& T0 } U& c: l$ O& Q0 H$ YStarting ovn-controller [ OK ]
: [ t8 D+ c2 ?5 L[root@node1 ~]# ovs-vsctl show ovn-controler启动后会自动创建br-int网桥
- M3 s/ e7 s% Sed157e0c-cac3-46b9-830c-f2d710b475d5
2 S2 h8 A) ]' D. s7 K; {/ B6 m; @ Bridge br-int
! M/ z/ a* O, | U# ` fail_mode: secure: i9 P* T& V8 Q8 O1 d
datapath_type: system
2 F B9 |) f, D, O. q. } Port br-int% ?/ T' z0 @( w: _
Interface br-int
}+ L9 o% T2 @, A! e1 `9 S2 q% I type: internal2 V% g9 ~9 _% U/ w l% z6 q* G
ovs_version: "3.1.3"7 c% ~: H1 l* I
$ A) C0 t9 n5 R" s3 T[root@node2 ~]# ovn-ctl start_controller
; _1 D2 Y" z$ L p; R ?+ tStarting ovn-controller [ OK ]
0 D+ N: P( [) x6 P9 y[root@node2 ~]# ovs-vsctl show ovn-controler启动后会自动创建br-int网桥9 `+ j L, L) J& |" f$ m
f6669675-b42d-47de-be95-b26bf6d1e0695 i2 @% V, g: S* C5 e( b& L
Bridge br-int
* n+ C* E9 T6 `) b: W0 U, Z, } fail_mode: secure' Z3 v) t7 }+ `
datapath_type: system
( C6 d7 ]/ u6 Y: N. ]( J T9 ? Port br-int) }0 C# Y% N8 F' O* \
Interface br-int3 j; f: X- R1 u
type: internal1 p! `) K3 t7 h T/ X8 S
ovs_version: "3.1.3"* L& X2 H* T c) K
在这里插入图片描述) J" @% U, a4 O; p* `
可以看出此时hypervisor并没有和central关联起来（也就是ovn-controller没有和南向数据库连接）。可以在node1上验证：[root@node1 ~]# ovn-nbctl show
3 N- p. U/ `' x; Y( d) x" Ohypervisor连接central，开放南北数据库端口：
( X' S: j0 Y7 Y/ A' a
) k+ s% }" o" |+ Covn-northd之所以能连上南向数据和北向数据库，是因为它们部署在同一台机器上，通过unix sock连接
; o. ?, C9 a8 W# n! Y' `% }central节点开放北向数据库端口6441，该端口主要给CMS plugins连接使用
+ o* K1 _' g* ]4 Ncentral节点开放南向数据库端口6442，该端口给ovn-controller连接
. J/ \% t1 v2 G0 g l* r[root@node1 ~]# ovn-nbctl set-connection ptcp:6641:10.1.1.41& e5 x1 a8 I2 J3 `1 s3 ^
[root@node1 ~]# ovn-sbctl set-connection ptcp:6642:10.1.1.41
# [6 y! Y( s* |% X" l[root@node1 ~]# netstat -tulnp |grep 664
- U' A! L; Q2 m- ytcp 0 0 10.1.1.41:6641 0.0.0.0:* LISTEN 34102/ovsdb-server
% j& J! m7 o, qtcp 0 0 10.1.1.41:6642 0.0.0.0:* LISTEN 34118/ovsdb-server
7 N2 T& p1 ]0 U: i0 O o1 inode1上ovn-controller连接南向数据库8 g% |, E* a6 Y
ovn-remote：指定南向数据库连接地址/ T. s! S3 H$ S2 r
ovn-encap-ip：指定ovs/controller本地ip
( E4 T+ C7 e1 l( G govn-encap-type：指定隧道协议，这里用的是geneve
+ u9 ^7 S' A! O8 v' }: V5 Esystem-id：节点标识9 F& I/ m! u' Y- z$ Q4 a3 @
[root@node1 ~]# ovs-vsctl set Open_vSwitch . external-ids:ovn-remote="tcp:10.1.1.41:6642" external-ids:ovn-encap-ip="10.1.1.41" external-ids:ovn-encap-type=geneve external-ids:system-id=node11 c9 p0 _5 s: ~, K' z6 b
3 ~' L4 r! F( B5 _node2上ovn-controller连接南向数据库
3 A l0 h: e( a$ s0 ~1 A& y8 P5 }' @7 N[root@node1 ~]# ovs-vsctl set Open_vSwitch . external-ids:ovn-remote="tcp:10.1.1.41:6642" external-ids:ovn-encap-ip="10.1.1.42" external-ids:ovn-encap-type=geneve external-ids:system-id=node2+ I. W. i$ y% m6 \1 ?
. \6 d0 L, J: J
在node1查看南向数据库信息2 Q4 V @, f$ }4 K" j
[root@node1 ~]# ovn-sbctl show
% i) V/ z( w8 Q4 W" ^4 QChassis node2
4 F& l. v @* H# ~ hostname: node2
( R8 C+ p, ?4 l* C Encap geneve
) U1 Y9 t" g$ L' ~+ W ip: "10.1.1.42"
2 z$ T( f4 n2 [' X6 F options: {csum="true"}
* H0 B6 C/ P0 u3 r l, w, |8 nChassis node1
7 ^0 L/ q) M: \, G7 V l1 u' h hostname: node1# s9 l5 K# H/ s& Y" J& C. y
Encap geneve
9 z% I* D& f8 F! b' a ~* d ip: "10.1.1.41"! j( J9 Q$ Z7 x
options: {csum="true"}
% e H$ I# J! Q* f$ U在这里插入图片描述
( `1 {! b1 R3 q以上的逻辑架构是站在底层组件和服务的角度来看的。
. D8 H5 z- ]0 z9 ^2 J9 _0 x接下来换一种角度，站在逻辑网络的角度来看。" ]5 n- r# r* A }7 z4 D1 i: z/ {8 B3 m
在这里插入图片描述
9 c# E# b; z0 Y; g0 i! F2 ^geneve隧道：ovn-controller连接南向数据库时，指定了external-ids:ovn-encap-type=geneve参数，此时看看两个节点上的ovs信息如下，会发现两个节点上都有一个ovn创建的ovs交换机br-int，而且br-int交换机上添加的节点port/interface类型都为geneve/ M# t% M. j2 |( p' o
% R$ N. {! W; ?3 L5 P9 V* T9 N[root@node1 ~]# ovs-vsctl show node1上查看ovs信息% V: P) S4 n- Z5 s' d
ed157e0c-cac3-46b9-830c-f2d710b475d5& j) X" z8 S4 g n/ i! P
Bridge br-int
$ ~6 n' B2 N. h' i fail_mode: secure7 j% D% V1 x7 v6 @) K+ I
datapath_type: system2 h) f) O j, G4 y& r3 ^
Port br-int
( q/ K# j8 s4 E Interface br-int. Z/ G# s4 Q" a8 d5 |6 m, [, b6 J
type: internal3 F( s' P/ V7 F8 d
Port ovn-node2-0" X# q; T5 \' |7 V3 Q; |+ e+ L
Interface ovn-node2-0% l& U, ^! ~' A' s* S( F) y' o
type: geneve
/ _) z5 _9 L# { options: {csum="true", key=flow, remote_ip="10.1.1.42"}' p; K4 ]4 Y- V( `- B4 |' M. O' d, s
ovs_version: "3.1.3"
9 K0 |; y# P% i, F r# @& ]& ]+ f# b* z5 W# M. p, o
[root@node2 ~]# ovs-vsctl show node2上查看ovs信息 G2 [! G# C8 e4 D6 r/ j7 [# Z. `" N" J
f6669675-b42d-47de-be95-b26bf6d1e069# K" z5 k) i$ }9 ?
Bridge br-int
4 D2 X) y( X. B$ \ fail_mode: secure9 \, g- g' a, P6 U* p- v# p
datapath_type: system2 }1 B; a( w, D8 |! `
Port ovn-node1-0
& S' ^4 F: O9 b& h6 h$ h Interface ovn-node1-0+ G$ b% F% I- P" ~( q% v5 V* y9 [$ U
type: geneve1 {4 O& L! s( u
options: {csum="true", key=flow, remote_ip="10.1.1.41"}
/ T2 ?& j: \1 Y- D7 ]( P Port br-int" T- _' R! L) V
Interface br-int$ \3 Z( T; b/ ?4 b3 F
type: internal
# p& t7 O b! a" `1 z# A' c ovs_version: "3.1.3"( |- Y7 b5 [, u3 T9 @
[root@node1 ~]# ip link | grep gene 查看geneve隧道link
+ f& p7 e: H# ~& T; B2 S: N! H5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
$ I! Z' t9 J3 O7 l3 T O8 ^查看geneve隧道link详情，从dstport 6081可以看出geneve隧道udp端口是6081
8 n ^2 W1 I/ _4 O) L[root@node1 ~]# ip -d link show genev_sys_6081
$ {9 }, t4 ^7 {) D- o5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
) t4 O: O, I7 `$ f% I& {9 P# H link/ether 6a:e3:ff:a5:cc:d6 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65465
0 ~, e; {* U/ ]# ?# t! l, _& | geneve external id 0 ttl auto dstport 6081 udp6zerocsumrx7 @& _* \/ {- u' {
openvswitch_slave addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
' W) L6 G* U9 }' s5 }7 E6 _查看geneve隧道udp端口，最后一列为“-”表示这个端口是内核态程序监听7 t6 F6 v) v5 [
[root@node1 ~]# netstat -nulp|grep 6081. T$ ]2 R) m7 y7 I8 i, O
udp 0 0 0.0.0.0:6081 0.0.0.0:* -
* Y' g6 |8 I* g3 F' D4 G9 mudp6 0 0 :::6081 :::* -5 _" ?/ E, K8 x- R
. Z2 |0 ] F. f5 l1 @+ O[root@node2 ~]# ip link | grep gene+ U$ j4 Z0 v7 j) u% [
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
: h+ p/ Y! Q# I5 v4 o[root@node2 ~]# ip -d link show genev_sys_6081+ n1 U$ R2 } l+ r! V& @: n3 a
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000' U$ X3 h6 I7 `, @6 q% [# J
link/ether 4e:db:f1:e4:43:94 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65465
. p8 o& E) e$ F/ j7 d7 B geneve external id 0 ttl auto dstport 6081 udp6zerocsumrx
" y) t8 F7 _& l' T/ L" V openvswitch_slave addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
1 d: j" p. X* E6 P" |[root@node2 ~]# netstat -nulp|grep 6081% F/ X2 V N0 i! R5 u* { P/ b
udp 0 0 0.0.0.0:6081 0.0.0.0:* -
6 s4 A/ m/ C2 Y! J7 K+ _8 Fudp6 0 0 :::6081 :::* -
3 [. B" V8 G; Z0 l d! c$ L0 Y在这里插入图片描述
; ]/ }7 f5 c% {5 t& D! W# m, L$ T7 M- c
在做以下实验验证时需要注意MAC地址的合法性，不要误配置。MAC地址分为三类：
! B3 [9 k6 F% \, D) c广播地址（全F）
! T" ^! P7 W; I; VFF:FF:FF:FF:FF:FF
* u1 z5 X8 |5 ?4 w主播地址（第一个字节为奇数）
" y/ @7 l D, C9 i8 BX1:XX:XX:XX:XX:XX' c5 O3 T0 t/ Z. c' o
X3:XX:XX:XX:XX:XX
5 _4 w- H9 D0 j' p1 \X5:XX:XX:XX:XX:XX" M* [& r% g) K$ v: u9 f
X7:XX:XX:XX:XX:XX# G# q4 E4 [* H% N+ W: I, |0 `9 R
X9:XX:XX:XX:XX:XX5 |: s* s( o* M6 W
XB:XX:XX:XX:XX:XX
3 t# }8 V3 U( M3 b* ]6 B4 yXD:XX:XX:XX:XX:XX
+ S8 I+ V" x& u5 aXF:XX:XX:XX:XX:XX
$ D5 r& a" |! Y可用MAC地址（第一个字节为偶数）
, N( c9 G+ X1 t+ A8 p; d2 LX0:XX:XX:XX:XX:XX3 [7 {/ Q4 R& X& j" H+ C' i
X2:XX:XX:XX:XX:XX$ S, Z- B* ?- t( B$ u
X4:XX:XX:XX:XX:XX
# l: |( e" F0 ~8 p$ YX6:XX:XX:XX:XX:XX: V- g/ t% h9 e8 u
X8:XX:XX:XX:XX:XX
( n" `- A' n& ]8 N/ CXA:XX:XX:XX:XX:XX
$ z7 a% I- ~# B8 @' V0 M) HXC:XX:XX:XX:XX:XX
/ h9 A: u/ E' NXE:XX:XX:XX:XX:XX
, u. E' E# D. Q在每个节点上创建一个网络命名空间ns1（因为在两个节点上所以同名ns1不会冲突），网络命名空间可理解为虚拟机，并且在ovs交换机上创建一组port和interfacce，然后把interface放到网络命名空间下。veth pair：两个网络虚拟端口（设备），veth可理解为网卡端口，一个端口在虚拟机上，一个端口在br-int虚拟交换机上。
) K+ w. \2 A/ {1 V0 M8 N6 I8 x2 L* x: ?+ b+ u3 ^, H& f
node1上执行5 d# h( p6 L F& S. T/ k
[root@node1 ~]# ip netns add ns1# O5 e% Z/ e0 S
[root@node1 ~]# ip link add veth11 type veth peer name veth12' K! y: ~: l# B& k) t7 q" s
[root@node1 ~]# ip link set veth12 netns ns1
' W8 h) \" ^8 I8 o[root@node1 ~]# ip link set veth11 up, F! X d4 S8 y" n T+ Y
[root@node1 ~]# ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:01) [- e/ Z! q1 Q) {( a6 K- U
[root@node1 ~]# ip netns exec ns1 ip link set veth12 up
' N# |9 X/ v r% x[root@node1 ~]# ovs-vsctl add-port br-int veth11
7 ~7 D3 E u- b5 d! _, U- i[root@node1 ~]# ip netns exec ns1 ip addr add 192.168.1.10/24 dev veth12
' w! k4 V. V4 r3 p- B9 { S/ M. c% r. u, l% [
node2上执行，注意veth12的ip和和node1上veth12 ip在同一个子网 S; ~# x) Z/ z5 r C: x2 J
[root@node2 ~]# ip netns add ns14 b( J- s+ A( c. o, i$ ]
[root@node2 ~]# ip link add veth11 type veth peer name veth12
2 h1 N8 n( D* l) h[root@node2 ~]# ip link set veth12 netns ns1; g* s/ |- _8 {) O Q
[root@node2 ~]# ip link set veth11 up, R- t+ p3 o! U. I" u6 _2 i
[root@node2 ~]# ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:02
- E' V, a* J8 v* i8 P4 s[root@node2 ~]# ip netns exec ns1 ip link set veth12 up
3 @# k& i7 g$ B4 D) A[root@node2 ~]# ovs-vsctl add-port br-int veth11 w. x5 }$ p2 J* A
[root@node2 ~]# ip netns exec ns1 ip addr add 192.168.1.20/24 dev veth12
6 [8 R$ ^% ^/ \ k5 k0 j% d" @- a7 m+ U7 g9 R) @% W
查看node1上br-int交换机信息& U: {& E' P; {+ Z+ Q2 Y
[root@node1 ~]# ovs-vsctl show
, _' b" s! Y9 j/ f- }* }2 ned157e0c-cac3-46b9-830c-f2d710b475d5
0 j, z% G. u) `" D* t1 Y Bridge br-int
2 v3 t/ X. i* L fail_mode: secure4 k9 D3 r3 e7 E" o
datapath_type: system% Z5 W4 M8 Q5 b
Port br-int
5 B, R( a& S3 ~, Z6 Q Interface br-int# V- b6 y) y+ H0 d4 N
type: internal# Z( K0 T- u) R7 d6 Z) C
Port veth11/ H/ \" N9 W) I9 g
Interface veth11
7 v" x, [" R# x4 c2 [ Port ovn-node2-0: E. |) Y4 C D, l3 U4 v6 C
Interface ovn-node2-0+ d5 f% b& Z$ l4 \7 W! K# t
type: geneve3 R) E9 q( h$ T* i! a! r+ _
options: {csum="true", key=flow, remote_ip="10.1.1.42"}% T" N% Y- f2 n# `* L9 c7 V! U3 e
ovs_version: "3.1.3"
: c" [& ~$ F4 U- L* t7 ]查看node2上br-int交换机信息
# z5 H' L' n# _) y" |3 ~. L) @[root@node2 ~]# ovs-vsctl show, x% W- l! C" V* \6 L) h6 A1 @
f6669675-b42d-47de-be95-b26bf6d1e069( u9 }6 ?* w0 M" _7 F \, x
Bridge br-int9 v% w5 `# V" K4 Q7 y& s6 S
fail_mode: secure* j; L q k# |4 r5 b2 R( Z; U& s
datapath_type: system
; S9 ^( z9 t+ l! Z( H Port veth11) N, N. G) G* e' X: U+ b- k. O
Interface veth11
& U7 Q3 \" @3 V Port ovn-node1-0
# G) C( b0 c5 l2 g3 U o# t+ K Interface ovn-node1-0/ ^9 A% \' n4 u5 _2 J
type: geneve
" a: J( r' }8 k1 p( W. ^8 \+ v0 C7 |. [ options: {csum="true", key=flow, remote_ip="10.1.1.41"}
8 ?/ \% v1 D- F6 V/ Z: L$ m M ? Port br-int
+ J. s8 B; w6 l; g/ w& [0 C Interface br-int. C* o# i" Z L* \$ p% F4 {
type: internal
5 L8 S3 k# I$ g# W" K7 t7 m2 Q3 B ovs_version: "3.1.3"' H. a/ {! {/ [ D: r
5 m! Z J) `/ E3 s" o
现在从node1上的ns1 ping node2上的ns1是不通的，因为它们是不同主机上的网络，二/三层广播域暂时还不可达。
/ W: V. m8 B4 J6 H! m3 y5 u[root@node1 ~]# ip netns exec ns1 ping -c 3 192.168.1.20
( `$ R* v# {) z% LPING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
7 O* a7 {; @" Z2 Z; x; i" D% F ?% [) A4 ~( o3 n- ]
--- 192.168.1.20 ping statistics ---
0 f: A2 a& ~# b2 r# c3 packets transmitted, 0 received, 100% packet loss, time 2047ms
- u! H0 G) H% j7 n在这里插入图片描述
( k5 q, K: k9 h2 S查看openstack的控制节点发现，ovn的北向数据库中有逻辑交换机信息。
" \) R% t3 m T) ~0 [3 L) M在openstack里面，创建一个网络，就相当于创建了一个逻辑虚拟交换机，这个逻辑交换机（网络）信息会被保存到北向数据库里面。一个网络就是一个逻辑交换机。
, `5 M2 B" \* ]+ F在这里插入图片描述
3 Q0 S; {; C% V, v在node1中查看发现，ovn的北向数据库中没有逻辑交换机信息" T; T2 d! N1 {* `
在这里插入图片描述
2 N* L. W: i% u% v% w; F在openstack不同节点的虚拟机ip互通，这两个虚拟机ip连的是同一个网络，是同一个逻辑交换机上的同一个子网不同ip所以互通。6 f- i+ ^& ?5 S n
这两个节点的虚拟机ns1的ip是手工配置的独立的、不互通，这两个虚拟机ip没有连到逻辑交换机上，加个逻辑交换机就能互通。
& M0 c @& _) p" s在这里插入图片描述. K6 g! H* f: Z' {
逻辑交换机（Logical Switch）：为了使node1和node2上两个连接到ovs交换机的ns能正常通信，需借助ovn的逻辑交换机，注意逻辑交换机是北向数据库概念。
5 E8 ?' w: c1 W2 o$ E& {, {# z# b+ `3 H" E7 |1 I
在node1上创建逻辑交换机
% R, R7 o- s5 N ][root@node1 ~]# ovn-nbctl ls-add ls1* p+ L( [4 A/ z* T2 t
[root@node1 ~]# ovn-nbctl show g" m7 i# v g' H) M& l- o/ q
switch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)" w; Y6 g9 i" X6 I
在逻辑交换机上添加端口3 _' u% I3 s* O" s3 D% x( Q* t
添加并设置用于连接node1的端口，注意mac地址要和veth pair网络命名空间内的那端匹配起来
) q- X N* v# G' Q8 j" L/ g[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-node1-ns1! Y0 H t* ~& Q3 T, |" C9 K
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node1-ns1 00:00:00:00:00:01
2 D* y1 J( d5 _; S+ [ q0 V[root@node1 ~]# ovn-nbctl lsp-set-port-security ls1-node1-ns1 00:00:00:00:00:01
! W0 n$ x; S5 \7 u$ N% L添加并设置用于连接node2的端口，注意mac地址要匹配起来
2 v! n/ s: y- P1 t[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-node2-ns1
" G$ o0 V: I; s% y" j6 X[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node2-ns1 00:00:00:00:00:02" c0 B9 _: C2 X7 v) Y6 T% q
[root@node1 ~]# ovn-nbctl lsp-set-port-security ls1-node2-ns1 00:00:00:00:00:02. R/ L2 d L; ?7 z0 x5 R: m
查看逻辑交换机信息
t! W: m I* p5 C2 M[root@node1 ~]# ovn-nbctl show
( J/ z: B$ Q, N$ P( `1 qswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)) T0 L) V# b- v. t
port ls1-node1-ns12 p7 W; Y. i5 G2 l
addresses: ["00:00:00:00:00:01"]1 @9 ^+ v! i5 `) n% D
port ls1-node2-ns1: r7 ?0 s" P( k2 H2 H' n3 q
addresses: ["00:00:00:00:00:02"]
6 |. h$ P( [9 s4 v" @
. ]5 u8 G2 d4 v$ \- V5 u% `node1上执行，veth11端口连接逻辑交换机端口
4 U! ~- p* S7 F3 h. W' M7 m* B, N3 J1 \4 L[root@node1 ~]# ovs-vsctl set interface veth11 external-ids:iface-id=ls1-node1-ns1
: ]9 @$ T' E$ m5 l9 ?# J+ nnode2上执行，veth11端口连接逻辑交换机端口- I" |+ j3 s' r$ `. s
[root@node2 ~]# ovs-vsctl set interface veth11 external-ids:iface-id=ls1-node2-ns1
; F$ A( P" U6 h! o$ Z* q! _再次查看南向数据库信息，发现端口已连接
7 P& R" d/ {- r1 s; N/ X[root@node1 ~]# ovn-sbctl show
, S( d1 `- i3 OChassis node2
( f$ A# Q3 W# ?2 N& G' u Z hostname: node2
1 g& E ?% h4 v8 u( S$ k: V Encap geneve
7 j& y6 q7 s+ J. k! Y3 }! r ip: "10.1.1.42"# O+ l. U7 _$ B
options: {csum="true"}
% I( S( `$ g0 F2 @" X8 w Port_Binding ls1-node2-ns1' Z4 H5 p6 E4 q8 {# p! M) F+ q
Chassis node16 p5 K8 H% }, [+ z$ J
hostname: node1
Z* y' O0 a& n Encap geneve
* @+ D9 D* A6 B! ]1 L ip: "10.1.1.41"7 B9 F! n) i- M
options: {csum="true"}
3 r% f5 D8 p4 S" I& M4 z" I Port_Binding ls1-node1-ns1+ }4 K" o' |1 u1 ~7 a
node1上验证网络连通性1 Y g0 O# I4 C
[root@node1 ~]# ip netns exec ns1 ping -c 3 192.168.1.20) r1 I) M9 r/ \1 ~/ [3 x
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
' O6 `5 k2 M$ @: I" o! M7 R64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=4.68 ms# m8 ^9 \' m0 D
64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.908 ms
O% m$ ?( `4 R/ G% @: \& V64 bytes from 192.168.1.20: icmp_seq=3 ttl=64 time=0.756 ms
% A# t3 c: J9 i1 H. G" r
" i9 Z; s" Q; l) Y--- 192.168.1.20 ping statistics ---
. \7 j8 _( T3 H4 V. z' f6 |" o3 packets transmitted, 3 received, 0% packet loss, time 2004ms
9 n, A$ M& B* i* A$ k+ ortt min/avg/max/mdev = 0.756/2.115/4.682/1.816 ms
& |! U8 S5 P! T% i. Y) O* u Cnode2上验证网络连通性" ]! E4 d; m! y8 Q5 H, ~
[root@node2 ~]# ip netns exec ns1 ping -c 3 192.168.1.10
! |9 ^* A J. ?9 e: W' b0 IPING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.( K( V. j2 C, ?
64 bytes from 192.168.1.10: icmp_seq=1 ttl=64 time=3.34 ms
1 C% P" I+ R7 A0 B( \' Q: k64 bytes from 192.168.1.10: icmp_seq=2 ttl=64 time=0.863 ms
' o/ a1 n8 ]* ^4 @8 _2 ~2 a/ h64 bytes from 192.168.1.10: icmp_seq=3 ttl=64 time=0.372 ms- Y! f- D) E6 k+ D
0 z& |* u+ }* z
--- 192.168.1.10 ping statistics ---* F: w5 z0 ]2 a" a' X5 F1 q
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
?/ S4 o* {1 \2 ~8 R) m" @, yrtt min/avg/max/mdev = 0.372/1.525/3.342/1.300 ms& q2 c2 |; S" l1 n6 H, K# ?
现在node1和node2的ns1互通了，相当于创建了两个实例，这两个实例ip用的子网是连在同一个逻辑交换机上的，是同一个逻辑交换机上的同一个子网不同ip所以互通。
9 {; U2 G$ G, y! X( B& K在这里插入图片描述
" W* H+ M. x5 S+ y) T! j在这里插入图片描述. i3 Q% G. k n( h) }+ G0 A w
geneve隧道验证：从node1上的ns1 ping node2上的ns1的例子，抓包看看各个相关组件报文，验证geneve隧道封解包。通过抓包分析，可以看出geneve隧道在ovn/ovs跨主机通信的重要作用，同时也能看到ovn逻辑交换机可以把不同宿主机上的二层网络打通，或者说ovn逻辑交换机可以把ovs二层广播域扩展到跨主机。( O$ v+ M% K( e6 u; P! m) ~
* @! x, T! C0 H* K! |// node1上ns1 ping node2上ns1
# i& c1 N* d/ A# ip netns exec ns1 ping -c 1 192.168.1.20
0 {- X6 v, y4 |3 @5 x' D& PPING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.. {0 C2 g/ ~7 S
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=1.00 ms
0 q( M& v! g+ ]& Z--- 192.168.1.20 ping statistics ---! X$ Q2 N9 V3 o4 ]" S7 J
1 packets transmitted, 1 received, 0% packet loss, time 0ms
9 p; ]* o% C' }) S. V+ zrtt min/avg/max/mdev = 1.009/1.009/1.009/0.000 ms; B6 W+ {, Q+ y+ o9 _. Z
+ o6 t: N( i3 O1 f// node1上ns1中的veth12抓包
1 Z- a% u* `$ Y: y( j; V+ A# ip netns exec ns1 tcpdump -i veth12 -n4 |; H* ]$ } n3 {
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
0 l3 q' X# L$ g% g: Z# P( U I: S/ w, Clistening on veth12, link-type EN10MB (Ethernet), capture size 262144 bytes: M/ {# L" j: \/ |3 p
22:23:11.364011 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 24275, seq 1, length 64. c; M( K! h0 _& B
22:23:11.365000 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 24275, seq 1, length 64
! Y5 s6 h8 I% a x% x22:23:16.364932 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 287 C3 f: j% a& c% t$ o; N3 B
22:23:16.365826 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28, r0 |% A4 Y: L8 y8 g8 r# l
* t# t" g. X/ j3 O' i+ q# [
// node1上veth12的另一端veth11抓包& d$ |) o: o) f3 i) }0 N
# tcpdump -i veth11 -n
+ I. j$ c4 a. r! x" Z1 G6 Itcpdump: verbose output suppressed, use -v or -vv for full protocol decode G8 q" D# V" o4 ^7 i* Z
listening on veth11, link-type EN10MB (Ethernet), capture size 262144 bytes: P' E0 I: `# W) F
22:25:11.225987 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 25166, seq 1, length 64
i0 [8 ~9 j+ A @ s' m22:25:11.226914 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 25166, seq 1, length 64* L7 c- f, l2 [" E4 ^% ?/ J$ U& {. A
22:25:16.236933 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28
, J. U) J8 O s9 D+ P+ g* H& \22:25:16.237563 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
) r& |( Y, L+ \% z3 B4 e( R22:25:16.237627 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 285 ~4 s" ^* b" P3 a) }
22:25:16.237649 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
9 |% ?' {% N+ j! M6 d
8 |% o: C) p: E& A5 r# ^" @, H- ~// node1上genev_sys_6081网卡抓包) K1 X0 V0 p; u
# tcpdump -i genev_sys_6081 -n6 O; R0 C: l6 ]- X, J7 x0 P
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode8 J+ @4 o9 C' H9 { P1 J! s
listening on genev_sys_6081, link-type EN10MB (Ethernet), capture size 262144 bytes7 j2 Y0 Z) P& P* Y- R' U: B9 d0 M
22:28:15.872064 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 26492, seq 1, length 64
$ Z& N5 f6 R+ O `) F22:28:15.872717 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 26492, seq 1, length 64+ m `6 |7 I3 U7 H' |9 a* j
22:28:20.877100 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28$ M1 t& Y6 _' [* O' y2 d4 @
22:28:20.877640 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
1 R) Q$ [) `% E22:28:20.877654 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 284 D' z3 A0 t$ f2 O8 w
22:28:20.877737 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 282 Q/ c( p6 @6 K% @& E
- K5 Y1 j, T& G// node1上eth0抓包，可以看出数据包经过genev_sys_6081后做了geneve封装) K R6 z6 z% j4 F8 x" R9 u; _
# tcpdump -i eth0 port 6081 -n1 d" r. i/ {, Q; M+ E: Z
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
' s" ?0 u- m; u# ulistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes" O7 M, x' B3 x. M) P
22:30:23.446147 IP 10.0.12.7.51123 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 27458, seq 1, length 64
' } p3 ~( Q. ]# p22:30:23.446659 IP 10.0.12.11.50319 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 27458, seq 1, length 64
2 N8 m% K d) G22:30:28.461137 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28; J4 k- D" f) J) B5 b# Z, y
22:30:28.461554 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
' d3 N4 v) w! h. F1 t22:30:28.461571 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28& f; g' f; H$ |" m* ]% H& F
22:30:28.461669 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
. x" f- ?# |: [; q' z/ X' `
: Y0 Q! @* {+ Y" Q0 {===================跨主机===================
* `4 X# r1 t2 Z
' o2 w* @% H( s7 R// node2上eth0抓包) e$ m3 M# B5 X6 q: E1 C5 h
# tcpdump -i eth0 port 6081 -n1 ]5 L* r5 }1 f3 @) B
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
]1 |1 S) w! u0 q3 y8 P# A# h) Elistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
: Q; E/ h+ I0 R6 y" J22:23:11.364189 IP 10.0.12.7.51123 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 24275, seq 1, length 64. U# ?4 |4 |' U3 ~% Z* Z
22:23:11.364662 IP 10.0.12.11.50319 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 24275, seq 1, length 64
! L% x) ^* ^6 p& L8 e' `4 y1 y22:23:16.365086 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28! i6 X! k P, v( R |
22:23:16.365487 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28" R3 t- q4 e6 z4 }
+ ^* w& D3 @5 V$ e0 y1 b
// node2上genev_sys_6081网卡抓包，可以看到数据包从genev_sys_6081出来后做了geneve解封
0 ?0 m. X+ m d# y2 c. j" I# tcpdump -i genev_sys_6081 -n
& L# P2 m1 ^, u5 p/ c1 a. B. Xtcpdump: verbose output suppressed, use -v or -vv for full protocol decode
6 z+ a% d2 L# m* ?, ]; Z$ nlistening on genev_sys_6081, link-type EN10MB (Ethernet), capture size 262144 bytes7 X" Z# f: [, h* E# [7 J
22:25:11.226186 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 25166, seq 1, length 641 E+ C% m! o. ?5 y7 g& q6 q
22:25:11.226553 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 25166, seq 1, length 64
0 }; B2 Y3 G( Q; R: X6 g22:25:16.237070 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28, Q% h+ E+ \3 w
22:25:16.237162 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28( ?' q. Z# ~% d& p' C
22:25:16.237203 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28! Q$ v" w( T* m/ Q
22:25:16.237523 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
8 ^( p$ e* [9 R0 T! U
4 ]/ i% t. z# l0 @2 b+ A// node2上veth11抓包3 A9 O* S) }/ W, O9 s1 p1 E
# tcpdump -i veth11 -n
8 a$ y/ X2 z% P& o. }tcpdump: verbose output suppressed, use -v or -vv for full protocol decode" J1 Z/ |) e6 J# }9 [! K
listening on veth11, link-type EN10MB (Ethernet), capture size 262144 bytes
% U4 {0 z: n0 m! |22:28:15.872198 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 26492, seq 1, length 64% a0 r! o I4 J: A8 P6 _4 A6 |- D
22:28:15.872235 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 26492, seq 1, length 64
* m+ j6 G0 b( Z- B22:28:20.876913 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 280 y5 |" ]" t6 n& J; o1 H: f
22:28:20.877274 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28' ?3 h( C& f/ p- e0 @$ U1 a8 r: Y
22:28:20.877287 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 282 Y( Y o2 s; M2 M* ?: r& J
22:28:20.877613 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28, O6 u/ C) a/ n5 ~, _
0 M0 k6 Q. ^ i i/ g( }
// node2上ns1中的veth12抓包
/ h0 u) v( w1 u2 u! C# ip netns exec ns1 tcpdump -i veth12 -n7 F# A( h- f3 ^
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode" I' r1 h, A" Q) e |5 x
listening on veth12, link-type EN10MB (Ethernet), capture size 262144 bytes7 d$ g1 b) n( ]9 S( C4 h8 P
22:30:23.446212 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 27458, seq 1, length 64
: O9 W, }* X' U* h22:30:23.446242 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 27458, seq 1, length 64 S! C8 [8 E( d$ K' Q) B" d! @
22:30:28.460912 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28$ j; B( m' H1 O6 Z/ m+ U
22:30:28.461260 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 286 H" W6 O$ ?; a- \7 [+ r9 {& d8 v1 ]
22:30:28.461272 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 281 F$ C3 i) K; w( T6 Z
22:30:28.461530 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
. ~0 @/ P: G* y: V! s# k* e; y逻辑路由器（Logical Router）：
! W& g5 S5 a' A6 j) G前面验证了ovn逻辑交换机跨主机同子网的通信，那不同子网间又该如何通信呢？这就要用到ovn的逻辑路由器了。
4 o6 i. X ^2 v: i& F" t# N; j) j先在node2上再创建个网络命名空间ns2，ip设置为另外一个子网192.168.2.30/24，并且再增加一个逻辑交换机。
+ p5 B: a, \, e7 k* Y2 k k在这里插入图片描述1 C, {) D* J3 b
, ]; e0 x d( H8 h* J% Dnode2上执行/ E6 U5 p. F! c! H0 ?" y3 K
[root@node2 ~]# ip netns 查看网络命名空间, p( m8 `' W S/ h7 S& P" Y
ns1 (id: 0)
! |0 N, n% C1 _1 u3 e0 w[root@node2 ~]# ip netns add ns22 C5 t3 |+ \6 x/ T; p
[root@node2 ~]# ip link add veth21 type veth peer name veth22 x0 N5 u# j# A: s) `% c8 o5 C
[root@node2 ~]# ip link set veth22 netns ns2
4 X" M9 o; T# N0 S, i( Z[root@node2 ~]# ip link set veth21 up
* J, [8 P/ H& r; N1 y" J' m[root@node2 ~]# ip netns exec ns2 ip link set veth22 address 00:00:00:00:00:03" k0 A# _2 m/ [% Z. T
[root@node2 ~]# ip netns exec ns2 ip link set veth22 up1 |; i% I- F) L0 s/ t
[root@node2 ~]# ovs-vsctl add-port br-int veth21
9 P! x: q- Z' y( p# m3 h[root@node2 ~]# ip netns exec ns2 ip addr add 192.168.2.30/24 dev veth225 |0 m* g) p/ T
[root@node2 ~]# ip netns' c6 O( A* z8 l' \6 x
ns2 (id: 1)
8 y8 k D; d( C0 F: Mns1 (id: 0)0 F; h( Z1 e/ [( Z+ g, x6 N
: Q6 B6 \' V% e9 H+ R8 Lnode1上用ovn命令新增一个逻辑交换机，并配置好端口
' S( [* K( b3 I; _[root@node1 ~]# ovn-nbctl ls-add ls2" u8 k. T2 S' m5 _. h. S; w# o
[root@node1 ~]# ovn-nbctl lsp-add ls2 ls2-node2-ns2+ k7 m+ Y J1 }3 Q
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-node2-ns2 00:00:00:00:00:03
0 v% n: e6 r+ ?+ y[root@node1 ~]# ovn-nbctl lsp-set-port-security ls2-node2-ns2 00:00:00:00:00:03
+ A% r* w! m; l1 }+ o
1 {& I! Y- A; l onode2上ovs交换机端口和ovn逻辑交换机端口匹配起来
/ H# q7 N) j/ }0 j' e[root@node2 ~]# ovs-vsctl set interface veth21 external-ids:iface-id=ls2-node2-ns2
& q$ ?$ k8 V9 R F+ A
; Q" s9 y( s' Y/ Y0 w查看北向数据库和南向数据库信息
. q( M$ ^0 S8 L: R" K: o[root@node1 ~]# ovn-nbctl show
! p) E5 E% h( I' Nswitch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)
! ? \- O$ C I$ \6 C: L; } port ls2-node2-ns2
. h+ S& g+ \9 k7 h- @ addresses: ["00:00:00:00:00:03"]
& _0 Q( V& J" B' u: q& {- vswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
7 i" h4 c. j: u port ls1-node1-ns1$ B% Q3 F+ i) g; z$ P/ w
addresses: ["00:00:00:00:00:01"]7 C+ y5 U0 L2 \( x" {
port ls1-node2-ns1
- }" X4 Y0 E1 }! s7 B addresses: ["00:00:00:00:00:02"]8 f% e# B5 p9 ^) |* `
[root@node1 ~]# ovn-sbctl show
! ^& p1 v7 P5 g9 G" _Chassis node2
* [$ V, z) u+ k7 o/ m. v% k# c( O hostname: node2
6 i3 y" y+ x5 t+ A Encap geneve+ C5 @' `+ _* _5 {
ip: "10.1.1.42". [" D3 l$ I' F3 u+ x: v9 C
options: {csum="true"}, l1 ]7 B) F3 P$ M( ?. d3 ?
Port_Binding ls2-node2-ns2
" G/ Z3 S# Q j/ |$ R% U Port_Binding ls1-node2-ns12 L4 {, ~/ g3 C E: i
Chassis node1
- _7 p% F$ A0 W V hostname: node1( y$ G6 }7 l2 l* w6 l% b8 G R5 h. y
Encap geneve
" }$ N4 X, h8 x5 K0 Q$ |4 m+ ] ip: "10.1.1.41"
# s' y K+ F4 h/ f) i% q: N options: {csum="true"}
) w: n" F6 M0 L% ?2 q* j Port_Binding ls1-node1-ns1
- B8 Q6 K9 ~# ^2 h# j: f创建ovn逻辑路由器连接两个逻辑交换机! ^ F) @' t# W
1 S' V) p. |) w+ H; o& [0 W添加逻辑路由器，路由信息保存在北向数据库
; a$ n4 U$ M, N' H8 b7 Y[root@node1 ~]# ovn-nbctl lr-add lr13 k- B8 C3 T( o2 j& h9 [& f
逻辑路由器添加连接交换机ls1的端口( Y$ C0 |8 e! j- H2 K3 ~
[root@node1 ~]# ovn-nbctl lrp-add lr1 lr1-ls1 00:00:00:00:11:00 192.168.1.1/248 T# I3 k( O% p, N0 T' p
逻辑路由器添加连接交换机ls2的端口
( T4 P5 G; x+ M" V[root@node1 ~]# ovn-nbctl lrp-add lr1 lr1-ls2 00:00:00:00:12:00 192.168.2.1/24" c& Q% G3 U% {) D
5 t' v8 K& c- T* @
逻辑路由器连接逻辑交换机ls14 z3 h. V4 D6 }7 E1 U) s
[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-lr1
9 B7 i$ D8 G( T# P2 L% ^! q# x: f$ M[root@node1 ~]# ovn-nbctl lsp-set-type ls1-lr1 router; K! o' E+ X7 s! Q
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-lr1 00:00:00:00:11:006 w7 q! v) S. h. s1 S
[root@node1 ~]# ovn-nbctl lsp-set-options ls1-lr1 router-port=lr1-ls1
0 J; P# Y5 X8 x6 F& |" X
7 w, Q$ t' Q* x逻辑路由器连接逻辑交换机ls2: {' z1 s% M( [* C" B
[root@node1 ~]# ovn-nbctl lsp-add ls2 ls2-lr1$ U$ B9 F. N) X8 e2 |$ ~& E
[root@node1 ~]# ovn-nbctl lsp-set-type ls2-lr1 router3 c1 y1 Q) E( \, [( C
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-lr1 00:00:00:00:12:007 D( H* [/ \0 C6 t) G% ~
[root@node1 ~]# ovn-nbctl lsp-set-options ls2-lr1 router-port=lr1-ls2
K" a& m& a+ |. q% ^0 F! @
3 b# R2 H2 v# | k' x( n. j查看北向数据库和南向数据库信息$ g1 b8 H: \% Z8 {' j
[root@node1 ~]# ovn-nbctl show, ]4 g3 c) K1 u9 X7 l" R
switch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)' G" j p( M* k3 C. z
port ls2-node2-ns2
$ K" j, P1 e4 T addresses: ["00:00:00:00:00:03"]
8 k3 C( x/ O4 K0 z- _1 n1 S port ls2-lr1
6 J) x& T5 Y! o2 R$ y' x. {$ F8 Q type: router5 L" J! ]; j' l9 F2 Z# [( \
addresses: ["00:00:00:00:12:00"]
- y5 P) c2 g9 X; u( B4 S0 S0 h router-port: lr1-ls2
2 j1 ~% p4 w0 O' K! Qswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
; m" q g+ e$ ?- r$ ~5 V port ls1-node1-ns1: g! m. k0 u; p6 l' Q8 h$ y, B
addresses: ["00:00:00:00:00:01"]
) F2 V" s* L' k/ @( N port ls1-node2-ns1+ p Y4 X1 }/ [3 j6 y
addresses: ["00:00:00:00:00:02"]
! p5 A7 T1 G" f# a port ls1-lr1) R8 I( F" C5 W8 J
type: router
5 u# B( d* ?# [, j$ n! P' @ addresses: ["00:00:00:00:11:00"]
4 K: p5 a2 U: F+ B7 @ router-port: lr1-ls10 S1 w; g4 B7 V! z8 p9 x! M
router e9c151a0-5db7-4af6-91bd-89049c4bbf9f (lr1)
8 m$ Y- D0 ^# U) ~7 H3 ] port lr1-ls2
& Z+ ^- p8 P5 D0 i) Q- ` mac: "00:00:00:00:12:00"
% e' E: A; }0 U1 L. A; } networks: ["192.168.2.1/24"]
% \) ^3 w* D. a4 O s. z port lr1-ls1
1 c% n$ h2 [5 I+ i. b1 Q' ?( N mac: "00:00:00:00:11:00"
# T& ^9 Z* f+ t& T3 C/ N networks: ["192.168.1.1/24"]
9 d A1 O. ]# b0 U1 W* b. e4 `[root@node1 ~]# ovn-sbctl show; l: f# s9 O* j8 N' I. [
Chassis node2
7 a) b% W3 c, f2 e hostname: node23 W+ b# i. j7 `* v1 C/ Q+ n5 u
Encap geneve
% Q4 Y& W1 t/ \7 Q. b ip: "10.1.1.42"$ o( L$ B3 b. ^4 N* J) f
options: {csum="true"}
$ o& b( a I; V7 { Port_Binding ls2-node2-ns2
8 g3 s; k8 p4 Q. L$ ]# d Port_Binding ls1-node2-ns15 ^: S) c2 c% p- {# }
Chassis node1
$ H" }. Z- E2 y" d6 J7 u/ t0 N. m hostname: node1
) t& K- }' p9 z1 ?1 o: t Encap geneve, v* `0 C3 _: U. k" z
ip: "10.1.1.41"
3 R w# q9 p1 X0 e options: {csum="true"}
: z% O$ D' r! y* e% w0 T Port_Binding ls1-node1-ns1- s7 b% M' X l# y
在这里插入图片描述( a" R/ U- U; `
从node1的ns1(192.168.1.10/24) ping node2的ns2(192.168.2.30)，验证跨节点不同子网的连通性。9 U# f h6 h* Q: {3 W; e$ c
+ X! u J6 h. G, {[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.2.309 C! b+ a% m7 c5 j# W7 g
connect: Network is unreachable connect: 网络不可达
/ M: Y. F7 t0 q' O查看ns1上的路由配置，显然此时没有到192.168.2.0/24网段的路由9 a5 l& F! Z3 b9 c% i
[root@node1 ~]# ip netns exec ns1 ip route show- s* q* V7 _- G
192.168.1.0/24 dev veth12 proto kernel scope link src 192.168.1.10$ u3 Y) }; W+ {% g6 `6 s! h2 w
[root@node1 ~]# ip netns exec ns1 route -n% a. J! J; X. e) a7 q
Kernel IP routing table4 ~0 c( K8 x0 H1 c7 Y; \
Destination Gateway Genmask Flags Metric Ref Use Iface, {4 P9 j* ^$ W" s! @
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 veth12. F7 _0 C* k, H1 q
因为路由器是三层概念，要先给ovs的相关port配置上ip3 o' d9 i7 I0 z! q$ n) c
( c( R: k) S( M) V
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node1-ns1 00:00:00:00:00:01
" B" ]3 q5 N/ I3 N# {) {" ]1 n[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node2-ns1 00:00:00:00:00:02+ o. ~/ B1 t0 o7 ?* e) p! ^
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-node2-ns2 00:00:00:00:00:03' t) m/ j+ T) M! f1 T& z
再给三个网络命名空间添加默认路由，网关为ovn逻辑路由器对应的port ip
, E. ]; F% R2 s- \
: N8 Z% A$ i) J! f; Y+ ^( R node1上ns1' ~3 g. i% \# O
[root@node1 ~]# ip netns exec ns1 ip route add default via 192.168.1.1 dev veth123 A0 C" L! D7 ?: \8 u# c( O4 A
node2上ns1
! {; X# t5 y7 V. f [root@node2 ~]# ip netns exec ns1 ip route add default via 192.168.1.1 dev veth12
1 p% t) i6 Q, L/ q! `& P node2上ns25 h& |- ~" }- r$ t- s, w0 I1 j; D
[root@node2 ~]# ip netns exec ns2 ip route add default via 192.168.2.1 dev veth22 V* g" x9 ^1 O
再次查看下南北向数据库信息* ]: p! s, [4 p% H
( S. I- ~0 }4 e$ U) w+ O) W: n
[root@node1 ~]# ovn-nbctl show
7 V( d6 x6 {7 C1 fswitch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)
- v0 j8 }9 M8 B7 | W4 b port ls2-node2-ns2
! p0 b3 V3 ~7 S5 J addresses: ["00:00:00:00:00:03"]
# \5 X+ S) S8 {$ q port ls2-lr1
`5 H: \. O0 i% }8 F) k type: router2 }( w2 u$ l4 Y$ J0 i i
addresses: ["00:00:00:00:12:00"]
' h2 R% J! \- c) D7 g& Q3 J! z. Z router-port: lr1-ls2
m2 S" |0 l% p. Hswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1). `% V& G. o5 k. x; f: L
port ls1-node1-ns1
: c$ z r: Y9 e addresses: ["00:00:00:00:00:01"]6 J$ J- s0 u5 g5 ?7 ^0 @, [
port ls1-node2-ns19 k- b: Q3 I2 {% q ^
addresses: ["00:00:00:00:00:02"]! F- e" C7 I8 @
port ls1-lr1! Y F6 v1 @ d" h: n
type: router1 m/ Y/ b- D. z/ b: z2 v1 r3 v
addresses: ["00:00:00:00:11:00"]7 I1 t' ^# C! E. L
router-port: lr1-ls1
B& O% \6 Y+ r. J8 jrouter e9c151a0-5db7-4af6-91bd-89049c4bbf9f (lr1)! v/ j2 d* x# X1 @1 z4 c8 r* T
port lr1-ls2+ c, ^4 R5 R# T) T
mac: "00:00:00:00:12:00"5 y6 Z) o z$ o5 e" x
networks: ["192.168.2.1/24"]/ i& k$ i3 o7 x) `$ W" d c p
port lr1-ls1
, L( p' j' J/ D/ { N2 D8 |/ h" w mac: "00:00:00:00:11:00"
. B% [! C* A( C& J( K5 p4 o. ]3 M; { networks: ["192.168.1.1/24"]* h7 [1 k& X7 M4 M- S+ l+ J
[root@node1 ~]# ovn-sbctl show
5 F/ G$ e( U$ wChassis node2% E6 U- J& z, y9 v) N4 o' H" a
hostname: node2
+ ]" Y5 Q6 ? R9 a! \2 r9 Z Encap geneve8 [2 _9 ^& k' C% ] p
ip: "10.1.1.42": z. B2 U; u3 L8 C, \5 _& N
options: {csum="true"}
; m, i3 _# b( [, ?% B1 y( w, W$ X Port_Binding ls2-node2-ns2% W. g0 K$ t" z8 w8 X! g/ W
Port_Binding ls1-node2-ns1
- z. s( W9 U( [Chassis node1
- ~1 y6 X9 ]% y& m5 Q" T6 Q hostname: node1
; Y% e' {7 O! e* y! z- t! ^/ x9 ] Encap geneve A4 T: ]7 C0 `* K7 I: O: g9 }
ip: "10.1.1.41"4 g9 P4 ^0 G6 _ y7 b' Y& n& o
options: {csum="true"}# c% z3 z1 g" _0 F& N- l9 E G4 o; R
Port_Binding ls1-node1-ns1, l5 N; D9 Y; y6 P% j
在这里插入图片描述$ x0 D8 n; _ m3 h
验证网络连通性. ]6 c1 ~$ U9 Y* J- q* \
9 M# {% I8 T1 \6 {' C. w$ J
node1上ns1连通网关% }7 i* E4 h; r4 e g _" w: I2 l
[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.1.1) f0 A1 v) B& v
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
- j8 V5 F$ P- t8 e) ~7 g64 bytes from 192.168.1.1: icmp_seq=1 ttl=254 time=20.10 ms v; W$ g) @0 I; O
% |, ^# b3 [2 W, b. P--- 192.168.1.1 ping statistics ---6 _" v# E) r1 \ s. o
1 packets transmitted, 1 received, 0% packet loss, time 0ms
' Y( I' C }+ [3 h9 d: O2 Jrtt min/avg/max/mdev = 20.950/20.950/20.950/0.000 ms
6 C4 ~, }& K9 f& A
5 D; Z0 i5 [3 ~* x' g8 Anode2上ns2连通网关
" a7 r* ?5 B) @* }0 x[root@node2 ~]# ip netns exec ns2 ping -c 1 192.168.2.1
3 k m1 Q8 h ^" q( X! zPING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
+ _# t7 K4 H0 Z \64 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=38.5 ms
5 }% s/ J7 Z- h! B. g% o# ]5 S
' z) I5 e# @4 U7 a- K+ q$ P--- 192.168.2.1 ping statistics ---
0 L) k! ~( b4 n1 packets transmitted, 1 received, 0% packet loss, time 0ms5 O o1 I: `7 R
rtt min/avg/max/mdev = 38.477/38.477/38.477/0.000 ms7 ?$ Y. z8 [ }3 N* w# R9 Z
9 r4 g) M( G! t% u
node1上ns1 ping node2上ns2+ e. W+ c) w/ \/ m) m) Y
[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.2.308 y+ |% s* L5 O- A3 c/ h3 h& ]& G
PING 192.168.2.30 (192.168.2.30) 56(84) bytes of data.
! i$ ]5 d' ]2 c3 \. L, I64 bytes from 192.168.2.30: icmp_seq=1 ttl=63 time=1.23 ms
( }' R" I2 Y& m8 q. B( X- |& g2 K8 }
8 ~: d) t4 I7 M2 q4 I0 h& ~--- 192.168.2.30 ping statistics ---
0 ~' M9 q0 M* @7 @+ o( {1 packets transmitted, 1 received, 0% packet loss, time 0ms- y8 j- U3 z# l# O
rtt min/avg/max/mdev = 1.225/1.225/1.225/0.000 ms9 ]' c3 \0 [$ q( ~( H, f
复制 G8 s- H. \6 h2 k
注意：ovn逻辑交换机/逻辑路由器是北向数据库概念，这两个逻辑概念经过ovn-northd“翻译”到了南向数据库中，再通过hypervisor上的ovn-controller同步到ovs/ovsdb-server，最终形成ovs的port和流表等数据。
7 ?3 C4 r! f+ A- ~, K' i" \ ^8 sovn逻辑交换机通过geneve隧道，把二层广播域扩展到了不同主机上的ovs；而ovn逻辑路由器则是把三层广播域扩展到了不同主机上的ovs，从而实现跨主机的网络通信。
3 ~8 b. m" i! I& s. h8 T: Novn逻辑交换机和逻辑路由器都会在所有的hypervisor中生成对应的流表配置，这也是ovn网络高可用以及解决实例迁移等问题的原理。 |
|
发表于 2025-12-18 08:50:57
* U6 J, w$ `% ?8 Q7 J
" x9 O& p9 M7 L' n. k) X
" ~* y- I4 X0 [9 r
1 S7 Q) B! `! @% S( |& P
1 l' ^6 I; f4 g4 ~/ Z! m! O
. r' W' m3 L6 n" D! g
, Y- k% v8 K, R
+ D$ w: R' L' c. S
" u5 k0 O5 D' i/ k
& K0 I: _: Z" S0 ? k* T
, B6 w& O) e8 T7 g p: i+ A0 j
6 g1 P/ }% P: I, i" R5 v8 T
/ d" N( Z4 h+ p/ w" E
1 x5 {' G, P0 x7 A
& v: _, p0 n v7 |5 M