|
|
楼主 |
发表于 2025-12-18 08:51:30
|
显示全部楼层
2、网络服务Neutron# N, r: \. M4 u: o0 s7 l
Neutron基于软件定义网络的思想，实现了网络虚拟化下的资源管理。Neutron的设计目标是实现网络即服务（NaaS），在设计上遵循SDN(Software Defined Network，软件定义网络)架构来管理的。
7 m; r& _% q, V5 A( M$ INeutron主要包含Neutron server、Plugin和Agent等组件。Neutron server对外提供 OpenStack网络 API，接收请求，并调用Plugin处理请求；Plugin处理 Neutron Server发来的请求，维护OpenStack逻辑网络的状态， 并调用 Agent 处理请求；Agent处理Plugin的请求，负责在network provider上真正实现各种网络功能；此外还有database，用来存放OpenStack的网络状态信息，包括Network、Subnet、Port、Router等。. U( p) t; B7 y
q. T& V8 H# J0 `" R5 k9 m: c
3、OVS. Q& S9 M' I! I6 G+ e8 O
OVS(Open vSwitch)是虚拟交换机，遵循SDN(Software Defined Network，软件定义网络)架构来管理的。
/ Y4 K* z+ _8 ^2 \, D) iOVS介绍参考：https://mp.weixin.qq.com/s?__biz ... 189#wechat_redirect
9 g/ m5 A9 h- {9 i在这里插入图片描述
$ W+ O5 W; B! B! {" j; G2 u0 {ovs由三个组件组成：dataPath、vswitchd和ovsdb。* M$ J" M6 u- M, E3 s
dataPath(opevswitch.ko)：openvswitch.ko是ovs的内核模块，当openvswitch.ko模块被加载到内核时，会在网卡上注册一个钩子函数，每当网络包到达网卡时这个钩子函数就会被调用。openvswitch.ko模块在处理网络包时，会先匹配内核中能不能匹配到策略（内核流表）来处理，如果匹配到了策略，则直接在内核态根据该策略做网络包转发，这个过程全程在内核中完成，处理速度非常快，也称之为fast path（快速通道）；如果内核中没有匹配到相应策略，则把数据包交给用户态的vswitchd进程处理，此时叫作slow path（慢通道）。dataPath模块可以通过ovs-dpctl命令来配置。* G# G- k1 \8 t* V) T
vswitchd：vswitchd是ovs的核心模块，它工作在用户空间(user space)，负责与OpenFlow控制器、第三方软件通信。vswitchd接收到数据包时，会去匹配用户态流表，如果匹配成功则根据相关规则转发；如果匹配不成功，则会根据OpenFlow协议规范处理，把数据包上报给控制器（如果有）或者丢弃。$ x, U5 ]% v* s/ C7 H4 h+ ~
ovsdb：ovs数据库，存储整个ovs的配置信息，包括接口、交换内容、vlan、虚拟交换机信息等。# D7 l. `- `2 |9 k. T; Q2 [5 J
ovs相关术语解释：2 `% Z! d: G* N* t8 _
1、Bridge：网桥，也就是交换机（不过是虚拟的，即vSwitch），一台主机中可以创建多个网桥。当数据包从网桥的某个端口进来后，网桥会根据一定的规则把该数据包转发到另外的端口，也可以修改或者丢弃报文。Bridge桥指的是虚拟交换机。1 U6 s1 p) M2 P: \- [
2、Port：交换机的端口，有以下几种类型：
! E% h7 \. y" d3 v; I4 iNormal: 将物理网卡添加到bridge时它们会成为Port，类型为Normal。此时物理网卡配置ip已没有意义，它已经“退化成一根网线”只负责数据报文的进出。Normal类型的Port常用于vlan模式下多台物理主机相连的那个口，交换机的一端属于Trunk模式。
( q3 ?$ P3 H$ Y5 EInternal: 此类型的Port，ovs会自动创建一个虚拟网卡接口(Interface)，此端口收到数据都会转发给这块网卡，从网卡发出的数据也会通过Port交给ovs处理。当ovs创建一个新的Bridge时，会自动创建一个与网桥同名的Internal Port，同时也会创建一个与网桥同名的Interface。另外，Internal Port可配置IP地址，然后将其up，即可实现ovs三层网络。
3 ?+ n' e; y( J4 b3 ?1 r; ]# z( v. RPatch: 与veth pair功能类似，常用于连接两个Bridge。veth pair：两个网络虚拟端口（设备）4 T+ j2 @$ j1 J
Tunnel: 实现overlay网络，支持GRE、vxlan、STT、Geneve和IPSec等隧道协议。Tunnel：隧道，三层+ l# \" k0 x6 h# d. j
3、Interface：网卡，虚拟的（TUN/TAP）或物理的都可以。TAP：单个网络虚拟端口（设备），基于二层；TUN：单个网络虚拟端口（设备），基于三层。veth pair：两个网络虚拟端口（设备），常用于连接两个Bridge。; A1 ?: `' ?( S8 e f/ ]4 D8 x6 e
4、Controller：控制器，ovs可以接收一个或多个OpenFlow控制器的管理，主要功能为下发流表来控制转发规则。: i$ s/ L5 s/ n }5 Q
5、FlowTable：流表，ovs进行数据转发的核心功能，定义了端口之间的转发数据规则。每条流表规则可以分为匹配和动作两部分，“匹配”决定哪些数据将被处理，“动作”则决定了这些数据将被如何处理。
0 _, Q+ Q# x/ I' J) x0 R* u1 q在这里插入图片描述
8 \7 B5 p3 h: Y7 j9 gens160的ip地址没有了，用的是br-ex的ip地址出去的。
' W3 M, }" t2 L- v0 G( o9 t5 K在这里插入图片描述
! w$ `- _; \9 K# hovs安装
' {! q7 T$ \9 I7 x, b, I4 r S. | `1.开启一台新的linux5 ` v4 ?) ~8 D! u3 ?( v
2.配置在线yum源（openstack那个在线yum源）8 I4 B$ m- V2 n
( {7 I- u9 m [0 J+ e c& O: M
配置yum源（先把原有的备份后清空）
2 F7 P5 o; I9 c# T$ _# cd /etc/yum.repos.d/ # rm -rf *
6 o3 @2 U$ A# @) H8 G# cat cloud.repo
. j& J! U( V( S6 q5 }* ]
8 D p3 R5 y6 r9 O& X4 B[highavailability]& M& }: b2 R4 j/ l7 j7 e
name=CentOS Stream 8 - HighAvailability, f( {4 N& w) j
baseurl=https://mirrors.aliyun.com/centos/8-stream/HighAvailability/x86_64/os/+ A7 N3 K2 E8 E$ E
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
5 U1 ~6 W- J9 V _. O2 Bgpgcheck=1
# x1 C: K ?& }7 \1 zrepo_gpgcheck=0
* { c1 N+ p N0 Q0 t& R8 ^) W& ometadata_expire=6h; f# u7 N; ^' l- ]$ i% P, R
countme=1( |5 w; p4 |) y' Y/ P; G Y9 b
enabled=14 M. T' e, P+ G" j
q! I! Y6 q" f2 {[nfv]
+ Z2 j( I2 N" B5 k3 o) ~* fname=CentOS Stream 8 - NFV
+ ~9 T" ]! g7 |; ]9 rbaseurl=https://mirrors.aliyun.com/centos/8-stream/NFV/x86_64/os/0 r7 L% }/ f( W$ B- \" |% v
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial# n2 R: p* ~# t$ G
gpgcheck=10 S+ M1 g$ s" v) D1 F8 z6 Y
repo_gpgcheck=0) t+ E+ e4 u& x
metadata_expire=6h3 i$ k$ s1 @% y& v6 B k$ Y n
countme=1
, ^3 g' D! c* Q; ?% s, t: venabled=13 z( n9 R6 X' X7 ]7 t" y
- k, N" U& p1 m% ^2 R# T[rt]
( H4 `4 A. L' Q7 Z6 Zname=CentOS Stream 8 - RT! v2 i9 W4 Q9 m2 s3 |: M) X$ Y
baseurl=https://mirrors.aliyun.com/centos/8-stream/RT/x86_64/os/0 y+ U; f( V& @3 e- G- b( b
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial& o5 p. K2 W1 C- l, D7 B
gpgcheck=15 Z# m' o& g6 [8 E
repo_gpgcheck=08 {) ?7 M& a, W4 N; F
metadata_expire=6h
8 V x% z$ ~ q) `* }countme=1; E# R' U7 d# |7 l$ |
enabled=1' Y5 U4 ]+ D8 p( y
/ }" E1 M( a8 o- @. Y% s2 b8 W
[resilientstorage]
2 b' S0 _4 J7 d- i. Z1 Vname=CentOS Stream 8 - ResilientStorage% E S" t5 Z' f% I$ M1 F
baseurl=https://mirrors.aliyun.com/centos/8-stream/ResilientStorage/x86_64/os/
, Z$ H2 ?1 y8 i: K6 @gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
% v0 G% P4 U4 R# }- ~& r. Q! _$ ggpgcheck=1
6 G: f0 ^8 O. q |9 J- o! b6 l0 |repo_gpgcheck=0
& f& d% s3 b8 _# c7 X; }metadata_expire=6h
2 `# E1 z& Q k9 e0 B9 Tcountme=10 L3 P9 K4 R I# @
enabled=1
* H0 i; Q4 E, ^ t1 F/ \) s+ x( z( j! n3 X2 v$ T; `- N
[extras-common]! U: o, W C! E, `; f0 C
name=CentOS Stream 8 - Extras packages3 Q# [0 U# f+ J+ M9 Y
baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/extras-common/5 \7 u4 Z: C- b# Y) K1 Q$ {
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA5128 L9 |5 W8 t$ D7 c* u$ e7 z, D
gpgcheck=1
9 A: t* T" ~0 k7 \! Wrepo_gpgcheck=0
/ d# z( L7 S, \$ y; X& {metadata_expire=6h9 x" h+ G- H/ i/ _0 A2 f: d% L
countme=1
$ T0 I1 v, n7 w2 Nenabled=1# |, _: c. g) n* r1 k4 A
0 M. P" I% | T3 ~ j' V
[extras]
9 @5 k0 J/ }7 gname=CentOS Stream - Extras
l5 y9 L$ K$ U: r; x; x( r6 J. umirrorlist=http://mirrorlist.centos.org/?release=&arch=&repo=extras&infra=" o; v: d: b8 m
#baseurl=http://mirror.centos.org///extras//os/
, _0 c' I; k$ D1 ^baseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/os/
# H( ]# u8 I3 m0 _! W" Y. cgpgcheck=1
8 L( p" J, O, l/ m% y' Q9 wenabled=1
$ \3 O# T8 C, u, F* v* Qgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
; P: u6 f' f, ~" u& w9 |. U( p8 n1 T$ {3 V% Y' I) s* ]1 i% x1 u
[centos-ceph-pacific]
4 `7 K& I7 W* K: u8 pname=CentOS - Ceph Pacific
+ z# [( b3 W7 |$ ?2 Dbaseurl=https://mirrors.aliyun.com/centos/8-stream/storage/x86_64/ceph-pacific/9 D# ]+ w( A w% R
gpgcheck=0
: r0 D6 s, i2 E* L7 Nenabled=1
* A" }8 r! A4 {3 s J, F7 M% G7 xgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage) O+ W2 w5 J+ ]! b. P# P
+ [: R0 x* _# C9 X0 p2 X& N
[centos-rabbitmq-38]1 f6 \$ O) i7 v9 I; j
name=CentOS-8 - RabbitMQ 38
6 I! N+ q9 n @+ mbaseurl=https://mirrors.aliyun.com/centos/8-stream/messaging/x86_64/rabbitmq-38/
9 E7 e, z5 K' E: Cgpgcheck=1
- L) {, v6 p' a. g8 [enabled=1& m4 d% l# z: {% D
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging0 b0 T! S5 B3 q, U$ R' [2 v
0 k8 }- w0 t/ e[centos-nfv-openvswitch]) f8 |$ W' W/ K5 W" _
name=CentOS Stream 8 - NFV OpenvSwitch
: u1 E1 o. b, V% X8 S2 Xbaseurl=https://mirrors.aliyun.com/centos/8-stream/nfv/x86_64/openvswitch-2/9 [; E7 R! W6 T# X
gpgcheck=1
* T6 E3 c b+ I. [enabled=14 o& h3 d& |8 ~) _+ g
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV
1 F9 w. I7 r; D( q7 _+ pmodule_hotfixes=1
% S& h% x" F% u1 v* e, L! s& a, X/ J( K1 H1 D0 y2 z
[baseos]
$ U7 ~$ Y) \6 Sname=CentOS Stream 8 - BaseOS* E1 Q' D3 P; H7 Y- o) I
baseurl=https://mirrors.aliyun.com/centos/8-stream/BaseOS/x86_64/os/
0 [7 c# U6 G+ ]2 S3 f& b% Vgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial) q9 |$ ?2 [9 x* E# ]
gpgcheck=1
R* K8 p' Y, N0 I [6 grepo_gpgcheck=0
6 @' w( t8 c! C8 h0 F' t3 Umetadata_expire=6h6 b, u. f# U8 w
countme=1- E0 }+ o6 h2 a, j! p. L, q0 w
enabled=16 T* D% B: K! H& P; u
: w1 i3 K1 E; R/ i
[appstream]8 ^4 S9 ?! K$ g4 I, S
name=CentOS Stream 8 - AppStream
% ?& t8 z4 P8 \baseurl=https://mirrors.aliyun.com/centos/8-stream/AppStream/x86_64/os/
5 U& ?: Z2 S/ U! ogpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial) p7 _ Q$ ~7 T$ w& }
gpgcheck=1; n& [6 |0 Z! `! J% ^" P& M. Y
repo_gpgcheck=0' @3 h, ^5 ]7 x4 ~# T
metadata_expire=6h
7 x6 H3 {! h$ {( b0 s! e4 ]countme=1/ ]5 |% O0 h; u% _1 D6 i+ q% W
enabled=1
9 K: E+ y- e/ i& V5 ]7 {% k
o0 b3 C, a& M1 i+ x2 ]% m[centos-openstack-victoria]$ ?# R7 ` P. Z4 E
name=CentOS 8 - OpenStack victoria" W$ D6 S" H3 F' u
baseurl=https://mirrors.aliyun.com/centos/8-stream/cloud/x86_64/openstack-victoria/3 {! n2 e# H. h( p7 g: Q3 j* S
#baseurl=https://repo.huaweicloud.com/centos/8-stream/cloud/x86_64/openstack-yoga/
% E: U B H9 k4 C/ o9 c0 m% Ogpgcheck=1
6 B" D) \+ C- n) W6 _8 Fenabled=1' B9 _+ z! [* ]) b+ K2 w: k- |
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud
, W6 ~; p2 o- z( h7 s, S: Omodule_hotfixes=1
$ p2 W9 b: V* U0 `
, R9 S- ^% U5 X1 N( |[powertools]% t, Y' Y& R- D+ O: ]0 w
name=CentOS Stream 8 - PowerTools
; D, ]! U3 ^' `7 `. z* L! x( H0 {#mirrorlist=http://mirrorlist.centos.org/?release=&arch=&repo=PowerTools&infra=5 [3 C: v1 E F
baseurl=https://mirrors.aliyun.com/centos/8-stream/PowerTools/x86_64/os/4 u; [( x( r, r' b
gpgcheck=15 ] i" x4 ]& T; \7 Q* s
enabled=1& y, W2 B/ N! Y) H
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
! }" _% I5 y* B3 H% R2 F
! T, N9 `/ c1 R' W# yum clean all 清理缓存
6 A- Q U$ `" ~6 o: j# yum makecache 重新建立缓存
- t+ n# N' n6 p* ]# yum repolist all 列出yum仓库（13个）. u0 q3 g! a# C0 G m# y, |
3.安装基础包及ovs（Tab补全命令，安装bash-completion包后执行bash就行）5 `% Q6 p8 H: X
安装openvswitch3.1过程报错说找不到gpgkey文件就禁用gpgcheck=0再次安装就行了
. i( G% n6 v5 t' N% R, Y" Yyum install -y vim net-tools bash-completion centos-release-openstack-victoria.noarch tcpdump openvswitch3.1; j$ x1 l- r- h% B4 a2 W
或再单独安装yum install -y openvswitch3.1*
! f* l( N3 V1 |3 J) Y查看安装版本：[root@ovs ~]# ovs-vsctl --version+ O" Q4 }3 Q; a* K
4.启动ovs服务7 L3 p' h/ j1 ^4 [, ^- M0 s& p4 ^
[root@ovs ~]# systemctl start openvswitch
. O% X& [+ C8 D$ E[root@ovs ~]# systemctl enable openvswitch& a( }1 N. [) u3 G% k
[root@ovs ~]# ps -ef | grep openvswitch
( O6 e4 @4 J8 q. h: _5 N[root@ovs ~]# ovs-vsctl show 查看ovs虚拟交换机信息, I* c; @, P: n8 i0 X
[root@ovs ~]# ovs-vsctl --help 求帮助 或[root@ovs ~]# man ovs-vsctl9 m2 A2 O3 i0 Y+ Q8 d; c/ Y
5、创建ovs虚拟交换机# J* s$ \- K' c+ [
当创建一个虚拟交换机会生成一个和虚拟交换机同名的Port 和Interface，type为internal（内部的）& S3 t5 c5 @( t+ @' q9 V
' ?* a) W: ^+ ]1 G0 ]
[root@ovs ~]# ovs-vsctl add-br br-int
8 b# M- g: H6 T( n% K[root@ovs ~]# ovs-vsctl add-br br-memeda 添加
9 K5 q. }9 m$ s) `. V! X! V$ H[root@ovs ~]# ovs-vsctl del-br br-memeda 删除
: B) ^/ B# ^5 z! L% x[root@ovs ~]# ovs-vsctl list-br 查看
5 L, m5 @3 L6 p/ p" kbr-int
z: e+ @) x9 a: O- f- kbr-memeda% L% e% k0 F" F5 W; y2 q+ B
[root@ovs ~]# ovs-vsctl show 查询ovs虚拟交换机信息，Bridge桥指的是虚拟交换机7 `% L( | u0 S; E+ c0 `# p8 N
54c67146-9a9f-40be-8cb7-e8792879aafa* A4 U) W: W$ e
Bridge br-memeda) G' A/ M9 c- F/ w; ~
Port br-memeda) Z9 }% _' L: [! m8 D
Interface br-memeda
; F- _; [2 ~0 o3 o type: internal- v. ^. Z; s9 X& P4 D0 A
Bridge br-int1 _* L# f* e5 t# p" q# c
Port br-int$ f5 ]" J" U$ G g. f* J+ V8 n8 D
Interface br-int
" n) l9 V( A1 V' N } type: internal
9 @5 l, t9 s5 o1 D& G7 ^ ovs_version: "3.1.3"' f5 B! j. B" z2 b9 |" D
用轻量级namespace网络命名空间模拟虚拟机
* C5 u9 x+ Y* b在这里插入图片描述8 p+ o5 Q. E7 g7 w0 i+ R. W
8 k+ f8 d4 N& F' i2 ?. `" x[root@ovs ~]# ip netns 查看网络命名空间9 z4 |6 W, G7 N5 q: |( g7 T
[root@ovs ~]# ip netns add ns1 添加网络命名空间
" N" {8 _6 n+ K) R[root@ovs ~]# ip netns add ns2
1 L6 K$ w- V% v+ c* i- K- X[root@ovs ~]# ip netns
- p/ M9 f" h9 t% Y( m$ Hns2
/ D2 p1 y5 c: l }& p# qns1
5 v8 F4 H) e5 N* e- N( c创建两个veth pair（一个veth pair有两个网络虚拟接口，veth可理解为网卡端口） 并将一端虚拟接口（veth1和veth2）连接到两个网络命名空间里面。veth pair：两个网络虚拟端口（设备）。9 w3 H5 O* F8 A6 j0 {3 o
在这里插入图片描述
+ M- j4 H+ I" x6 h! f& }) V/ Q) m2 E- k6 O' ?
创建两个veth pair，并分别把这两个veth pair的一端放到上述两个网络命名空间8 I. C; W; ?7 t( D9 Z* b
# ip link help 或# man ip link 求帮助% X/ s0 @ L' L8 b. O' m( ~. V8 G" W
第一个网络命名空间配置
2 g" k3 w$ y) v, v4 a" g1 z. ?[root@ovs ~]# ip link add veth11 type veth peer name veth1
) R5 S- {0 R' f) ?+ Z[root@ovs ~]# ip link set veth1 netns ns1. ^" e1 \! W- N* S1 r) E3 e
[root@ovs ~]# ip netns exec ns1 ip link set veth1 up+ d7 l' u5 k& D
第二个网络命名空间配置/ I8 }" \; G- N
[root@ovs ~]# ip link add veth22 type veth peer name veth2
0 w+ `4 b! a0 O9 S8 J[root@ovs ~]# ip link set veth2 netns ns2) A# V# {, D' g! D+ t* I, h( B/ c2 v5 z
[root@ovs ~]# ip netns exec ns2 ip link set veth2 up
% H; l3 {& Z& y将另外一端虚拟接口（veth11和veth22）连接到ovs虚拟交换机上
F1 ?: [' m: m q在这里插入图片描述
; M/ m7 M" ?% c9 r0 ]7 _+ }2 F2 ^2 r- r, \! i- W( L
[root@ovs ~]# ip link set veth11 up
' n: T. }; P/ t2 u: |4 S[root@ovs ~]# ip link set veth22 up
- u `6 h8 S3 ]" T5 Y# M: Y[root@ovs ~]# ovs-vsctl add-port br-memeda veth11& M5 A6 O7 ~# o5 l4 R5 y, ?
[root@ovs ~]# ovs-vsctl add-port br-memeda veth22
# l. ~& X+ o% o! B1 |[root@ovs ~]# ovs-vsctl show 发现br-memeda虚拟交换机多了2个Port（Port veth22、Port veth11）
$ W) }, e! N) z& A+ v% R' m V3b79f2e1-f433-4015-905e-8945dcada530) z6 n3 W: o% m
Bridge br-memeda
r2 V! A$ |! ~- a/ R! l Port br-memeda, U2 `( y, ~0 _0 G
Interface br-memeda
0 k. M* w+ Z) l2 J. T type: internal
' v' ], A- e9 a6 S6 l' ~ Port veth22
/ {; I4 Q, I* h8 e Interface veth22
, `" Z7 k1 w: B+ A$ A1 |2 T* ] Port veth11
' `7 S2 b5 W' B; B% f, n* R) [- K Interface veth11
2 M p* G$ }8 Q) @, c! z( Z ] Bridge br-int* C0 X1 g% d2 X1 |5 `7 S: F1 j
Port br-int5 r. T6 g }1 [+ n5 ~8 `0 G7 s8 C
Interface br-int
9 Q/ |5 j/ M0 Q1 E3 s type: internal
9 u6 _3 o3 n+ t3 Q" \9 d ovs_version: "3.1.3"
' B' b* `2 c3 S0 E. s7 i1 d: q1 z为两个网络命名空间手动设置ip地址
e8 p4 P. \6 O6 h8 ?+ A: S在这里插入图片描述
. f9 t* s2 w. m! @' b
. Z/ W! a v! m. |' p[root@ovs ~]# ip netns exec ns1 ip addr add 1.1.1.1/24 dev veth13 `& k7 p: B" {# X+ @5 M9 l
[root@ovs ~]# ip netns exec ns1 ip a) v' q2 ^5 k# O& W' j+ j
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
; U, ?) P9 W$ I! h( ?6 [) w link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:002 I; ~3 _& a R/ R; I+ Q
7: veth1@if8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group d efault qlen 1000
7 s0 ^0 w8 }& l: V! y: t link/ether fe:f9:3b:cb:9b:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0' [; b# f, I: }2 k m
inet 1.1.1.1/24 scope global veth14 G; ? j' o6 ]+ `. h1 }
valid_lft forever preferred_lft forever: q$ L0 a& y3 K8 E2 W/ `7 H
inet6 fe80::fcf9:3bff:fecb:9bc5/64 scope link* k+ ]' A& s1 v9 J0 r
valid_lft forever preferred_lft forever* {, w; x4 Y- d! p
[root@ovs ~]# ip netns exec ns2 ip addr add 1.1.1.2/24 dev veth2/ s8 b, f' S8 C+ _
[root@ovs ~]# ip netns exec ns2 ip a
2 z' R0 b# z/ m5 K5 o6 y' U1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
L9 n Z/ N/ S9 L* N" `# V( q3 a5 l link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00: R% E- q6 x7 w) C: C8 ^
9: veth2@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
# a& @0 o) E n6 J! Z& X4 @1 X# k5 | link/ether 0a:e3:ac:a8:f3:bc brd ff:ff:ff:ff:ff:ff link-netnsid 07 J" E. f2 j7 n" m/ ?( @- D
inet 1.1.1.2/24 scope global veth2
6 E) C8 x, W D7 b8 [! [6 F valid_lft forever preferred_lft forever' {% W9 X' {5 C6 p$ {
inet6 fe80::8e3:acff:fea8:f3bc/64 scope link2 n4 {+ r+ n1 r- X( e
valid_lft forever preferred_lft forever% P- R+ v9 J) P5 n+ W0 J
两个网络命名空间测试连通性
9 o9 T$ M5 U" l3 m0 p, N3 r- A[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2
0 f% A+ [; R6 t5 B8 mPING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
; `6 |) a A6 h/ c6 B8 G1 @64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=2.98 ms
: x3 D! K0 s8 v6 h- B$ B5 Y; \64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.167 ms
3 M2 D- |% U, x5 E64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.081 ms
1 ]! v4 j6 u+ @* W9 C5 h/ w. r- H4 C. ]( b* b' ~, [3 W
--- 1.1.1.2 ping statistics ---
# R2 |8 P1 f$ U# N3 packets transmitted, 3 received, 0% packet loss, time 2065ms1 v# o0 m) g/ S+ ]$ e' r
rtt min/avg/max/mdev = 0.081/1.075/2.979/1.346 ms
! P! e1 ]/ o& W$ l. W+ T% U5 d# O[root@ovs ~]# ip netns exec ns2 ping -c 3 1.1.1.1( ?4 N' M, ^- J& M5 }
PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.' X7 a& o* t: e. Q) C+ u: t' q: ^
64 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=0.923 ms
- H/ j+ O( D% \0 H1 |6 V$ E0 ~64 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.084 ms8 s0 O8 @, c+ h( s
64 bytes from 1.1.1.1: icmp_seq=3 ttl=64 time=0.091 ms( j4 [0 q- d, R4 ~+ k
5 S2 W2 X1 j' W; o" A# ]
--- 1.1.1.1 ping statistics ---
6 }+ x4 s$ V9 \! X; j# s3 packets transmitted, 3 received, 0% packet loss, time 2007ms
8 ~2 r9 I) c! m6 i+ L v7 Qrtt min/avg/max/mdev = 0.084/0.366/0.923/0.393 ms+ o; j& B# L5 k; j4 I& w1 l0 K! c
vlan虚拟的本地局域网，vlan隔离为了减少网络阻塞和数据包安全! k' y+ d# u- b: c/ |4 a
ovs虚拟交换机能和物理交换机一样定义vlan，一个vlan10（tag10）,一个vlan20（tag20），把插在ovs交换机上的两个虚拟网络设备对端口分别打上不同的tag（默认是0），也就是配置到不同的vlan里，再验证网络连通性。
+ ]2 M9 t' |7 k; U: P5 b+ p' f( A在这里插入图片描述5 [5 g2 z6 ` t% g$ z
' _/ T. H( z+ V9 [
[root@ovs ~]# ovs-vsctl set port veth11 tag=10
$ e5 k; x: u$ o( { Z, {7 g[root@ovs ~]# ovs-vsctl set port veth22 tag=20
- y( ?5 |: B7 v* k' g[root@ovs ~]# ovs-vsctl show 发现br-memeda虚拟交换机的Port veth22和Port veth11下面多了tag标签4 ^; H) Q) F1 l; R: ^5 ]
3b79f2e1-f433-4015-905e-8945dcada530
! C0 m! E3 [+ a- Z8 m( _- ?: i Bridge br-memeda6 F6 K- i7 B, D8 h% j
Port br-memeda
. w" `1 k8 K" n Interface br-memeda
. l5 u1 U8 m* K' X9 j% @( p' N type: internal
( J& N7 A( K5 h8 \* f) e: R Port veth22( ~8 W4 V. Z$ E+ [' D% K+ r
tag: 20$ E$ j" {9 w3 c8 q( J, I( j
Interface veth22$ `+ i. {* F3 R
Port veth11
& } B; m' u: a e tag: 10. d6 K8 G4 L. z1 h V0 X8 V! J
Interface veth116 P. t& P$ _. K; ?# Q d
Bridge br-int( |8 c: ^5 Q& i$ |/ }; l
Port br-int0 V( X Y" W, U0 m+ {, N" @
Interface br-int2 b9 s( r# O0 h7 L8 a" t1 z- d
type: internal
. C* M z4 m$ C/ W+ z# R ovs_version: "3.1.3"
0 o. W# T7 ~; _7 p! _' E添加不同vlan（tag标签）后ping不通，需借助路由或物理三层交换机$ {, M5 ? Y1 B4 Q* w
& [- P6 [ a/ f; f
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2
( B& W0 D* \+ }' ZPING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
) h9 Q! J" q( L( s9 f" @' E+ u; r1 r$ |
--- 1.1.1.2 ping statistics ---
% o4 E: u7 b0 F) t6 r3 packets transmitted, 0 received, 100% packet loss, time 2064ms# ]. O, ~- P& L9 W( J
在这里插入图片描述& H0 g- _; P, [# @4 P# g
. t* d4 Z" N( u[root@ovs ~]# ovs-vsctl set port veth22 tag=10 把veth22也改成tag=10就相当于同一个vlan二层互通了
0 c* o+ |# b/ `1 z$ |[root@ovs ~]# ovs-vsctl show& m% O) e- b$ O! a
3b79f2e1-f433-4015-905e-8945dcada5302 h A2 @. o1 [$ p& b( E' F" e
Bridge br-memeda# u; f5 k# D$ f' `
Port br-memeda
! e' `' P( @% l6 a Interface br-memeda2 _9 y4 Z$ B4 |+ Z
type: internal( _' R' Q9 {& H3 x! E( N. a, x
Port veth22- b+ |- G4 @* H5 w7 T
tag: 10 @8 M4 D; I- j! b# b, U9 L
Interface veth22# Y+ G! y; P- |- K9 B! r9 o
Port veth118 o @2 b; ^' C3 C- {
tag: 10
- X' d2 {8 ~3 n: C8 @2 A Interface veth11
; H' h+ v" p: a3 t0 X Bridge br-int
0 c3 c6 ]! N0 \ X! z" B$ W+ Y Port br-int7 y- {8 }/ y! g/ `$ V# h* o
Interface br-int
' h6 K7 P0 b4 i) g type: internal7 ]2 \) S) A9 @9 |
ovs_version: "3.1.3"2 I% |9 ?- M$ h2 r5 A0 s
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2 同一个vlan（tag标签）能ping通进行二层通信5 ` k- |4 v, P7 M' x# n9 o
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.
7 m; Z" ]# C, {) Z64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=1.43 ms
4 {6 J: `' G A64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.093 ms
; e6 B: h% v& G. h9 L" D% `64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.086 ms8 g5 z& b4 M( x+ P' G7 l1 q Z
( u2 h8 l/ ?; G! k; b6 M--- 1.1.1.2 ping statistics ---
1 P# Q+ p, x, G& E, J0 V3 packets transmitted, 3 received, 0% packet loss, time 2051ms
. }7 Y2 b* b* a' Irtt min/avg/max/mdev = 0.086/0.535/1.426/0.630 ms, H) r% P! {, M- W* r+ t5 |
FlowTable：流表，ovs进行数据转发的核心功能，定义了端口之间的转发数据规则。每条流表规则可以分为匹配和动作两部分，“匹配”决定哪些数据将被处理，“动作”则决定了这些数据将被如何处理。) s2 e; D8 l6 E' |6 D
流量走向，添加流表，针对流量进口添加规则。
4 }/ I# v1 J% d: ^7 N5 w) t在这里插入图片描述
3 e! ~6 m% }% [6 `" E7 b9 O在这里插入图片描述
5 r8 R0 m9 q# o7 A$ |, O$ B5 |7 A# C
查看ovs默认的流表
" Q9 Y; J8 [# B3 \2 o0 m1 t8 R5 J1 a7 L[root@ovs ~]# ovs-ofctl dump-flows br-memeda 查看虚拟交换机的流规则
- h. R9 q: P: M6 H cookie=0x0, duration=2161.884s, table=0, n_packets=49, n_bytes=3682, priority=0 action s=NORMAL
& \ F, J2 V w/ F t此时ovs就类似于传统交换机，我们给ovs交换机添加一条优先级为2（数字越大优先级越高，高于默认表项的0优先级）的流表项，把veth11进来的请求都drop掉，发现ns1不能ping通ns2。
6 A" Q2 @4 P6 F& Y% {7 B ~[root@ovs ~]# ovs-ofctl add-flow br-memeda "priority=2,in_port=veth11,actions=drop" 添加流规则! L) w* d& i1 o; y( O
[root@ovs ~]# ovs-ofctl dump-flows br-memeda
9 L; N+ y2 m" P8 H cookie=0x0, duration=2.578s, table=0, n_packets=0, n_bytes=0, priority=2,in_port=veth11 actions=drop
2 m8 o/ q! J6 z0 r$ m) n0 H cookie=0x0, duration=2217.329s, table=0, n_packets=49, n_bytes=3682, priority=0 actions=NORMAL- h3 B9 ?( M7 i! s7 K* m# u$ G
[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.23 W1 e2 G% I( b* J
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data.: n5 E5 v6 c! Z, a- ], F, J
7 w5 N/ N4 Q; f& b E--- 1.1.1.2 ping statistics ---4 M( }4 I4 u4 |3 @
3 packets transmitted, 0 received, 100% packet loss, time 2076ms2 l1 S" o$ Q; p9 v% p% R; Q4 S
删除刚添加的表项，ns1与ns2又能正常通信" p) y! e: Z3 S$ `+ l$ T; J% q) M+ P
[root@ovs ~]# ovs-ofctl del-flows br-memeda "in_port=veth11" 删除刚添加的流规则就互通了
5 ~9 F5 L8 ?8 v. n[root@ovs ~]# ip netns exec ns1 ping -c 3 1.1.1.2& J( V4 ]/ v. e2 y% y
PING 1.1.1.2 (1.1.1.2) 56(84) bytes of data." Z/ g7 l/ h- I |" X e( x
64 bytes from 1.1.1.2: icmp_seq=1 ttl=64 time=0.766 ms
6 k* ~: [4 r6 E3 c) L64 bytes from 1.1.1.2: icmp_seq=2 ttl=64 time=0.096 ms Q9 [% `9 M6 P4 ^; n O: I# r
64 bytes from 1.1.1.2: icmp_seq=3 ttl=64 time=0.088 ms% X' b. H' f, I
E' I+ g/ p) ?' U- A--- 1.1.1.2 ping statistics ---
! t2 m' Q1 A+ r/ @/ {( m5 d* I9 y7 p3 packets transmitted, 3 received, 0% packet loss, time 2043ms3 {3 l" X; f% j* h# G- E$ L& e$ @
rtt min/avg/max/mdev = 0.088/0.316/0.766/0.318 ms' X: }" A: W& w- U% ~
[root@ovs ~]# ovs-ofctl dump-flows br-memeda% c9 q& w% v* u& w3 Y' v# @% u. w
cookie=0x0, duration=2315.744s, table=0, n_packets=59, n_bytes=4438, priority=0 action s=NORMAL- ], A* q& b; M4 w' R& }9 {6 i& }/ {
4、OVN+ R' c Q' B$ f9 U; i4 l
OVN建立在OVS之上的，遵循SDN(Software Defined Network，软件定义网络)架构来管理的，用软件将控制面和转发面分离，OVN做控制面，OVS做转发面。3 d* b2 t$ G0 e" M# c9 H
ovn是建立在ovs之上的，ovn必须有底层的ovs，ovs可理解为二层交换机，ovn可理解为三层交换机。9 ?' c% c- G* l: s. l' u5 Q' u- h
OVS介绍参考：https://mp.weixin.qq.com/s?__biz ... 189#wechat_redirect
4 |+ y1 h7 }# ^单纯的ovs在云计算领域还存在着一些问题，例如：- u, u) I5 e/ g
1、ovs只能做二层转发，没有三层的能力，无法在ovs上进行路由配置等操作；& f) _ y/ d0 U5 a3 Y0 y
2、ovs没有高可用配置；
) O9 B0 {$ Y; v+ S, d3、在虚拟化领域vm从一台物理机迁移到另一台物理机，以及容器领域container从一个节点迁移到另一个节点都是非常常见的场景，而单纯的ovs的配置只适用于当前节点。当发生上述迁移过程时，新的节点因对应的ovs没有相关配置，会导致迁移过来的vm或者container无法正常运作。
# r* y% T, b0 V* K针对这些问题，出现了ovn(Open Virtual Network)，ovn提供的功能包括：4 } c1 e3 U/ P' A5 H
1、分布式虚拟路由器（distributed virtual routers）
* U" C4 [* w0 k; x2、分布式虚拟交换机（distributed logical switches）6 d6 H5 B# y/ \) f( C8 ` |" V& U, {
3、访问控制列表（ACL）
& y1 x, n$ W2 m+ f4 o3 }4、DHCP
+ b% X* F7 Y L" q5、DNS server3 I/ X) R/ Y2 y% d/ y' T
在openstack里面，创建一个网络，就相当于创建了一个逻辑虚拟交换机，这个逻辑交换机（网络）信息会被保存到北向数据库里面。openstack创建一个网络，会以逻辑交换机（switch）的形式保存到北向数据库。
6 D/ x$ o4 i8 O, D0 a9 y0 [在这里插入图片描述1 k; B6 H2 }$ P% G6 v6 ^2 S3 s& y
在这里插入图片描述
! b7 Y3 }# m4 G$ oovn官网对ovn的逻辑架构如下所示：3 U7 k4 a5 y7 x7 `7 ?3 a+ x
6 E5 o: q }4 G, P" H7 b
CMS$ J) \5 u: P, t0 x
|
. Y% J4 X) K1 ? |
( H* t5 {: S3 x R5 d: q +-----------|-----------+
4 T: ^3 H- k. H- Y; S: g | | |
7 _6 B' I6 u& H, ` | OVN/CMS Plugin |/ E% j3 ~9 C1 U! p5 i
| | |. _: }/ p' z3 v+ M) u! t; X
| | |- e" k2 u h& A* r5 J% W
| OVN Northbound DB |
8 T, Q. @1 n' T4 { | | |) X3 U( }+ C# F2 r; J$ l8 W% }
| | |
( d Z1 R. Y* e, R2 v | ovn-northd |
8 w) X! b& Z% U& [ | | |$ k' a8 ]" }7 V$ Z+ [
+-----------|-----------+
: y1 c8 q( w* b* H |8 d" @+ f8 G; m& s& P
|: w% ~5 c% J; Z. C
+-------------------+2 v" K! n- W7 b: q3 W: w9 A# ]
| OVN Southbound DB |
. a9 h! O2 K/ Q- s6 o4 ` +-------------------+
3 a; ~2 A9 Z5 j- c |
0 L* r* `) \" k: J: Z: _ |+ O4 i- e; Y, Y5 U1 Z: F+ J1 P
+------------------+------------------+
( J2 d$ r) T6 Z: e' f( ^( q- q t | | |; \0 |* S6 ~/ `" S: p
HV 1 | | HV n |; C* n) C' d3 O1 E$ N8 a" P
+---------------|---------------+ . +---------------|---------------+! v; z( f3 l, x, i. |
| | | . | | |$ r7 \9 M' Q8 ]( V# D2 p! M# p
| ovn-controller | . | ovn-controller |
, Y! W- n6 B- ~ | | | | . | | | |
) W" ]+ Q- U0 I/ J% P | | | | | | | |
% I8 V- {; h) O' ]5 [# l0 ^+ ` | ovs-vswitchd ovsdb-server | | ovs-vswitchd ovsdb-server |
9 T* Z( r0 A4 ^$ {0 x | | | |
/ ?3 h# J. I# T2 \ +-------------------------------+ +-------------------------------+6 Z9 W3 C- f0 W1 J
ovn根据功能可以把节点分为两类：- v0 a: k: a0 F* i3 L3 H5 y% o: W
central: 可以看做中心节点，central节点组件包括OVN/CMS plugin、OVN Northbound DB、ovn-northd、OVN Southbound DB。
/ k6 S! j% z2 n5 ?8 @hypervisor（hv）: 可以看做工作节点，hypervisor节点组件包括ovn-controller、ovs-vswitchd、ovsdb-server。, X- y5 l: Q. ^ v% W8 O' [# | {
central节点相关组件和hypervisor组件运行在同一个物理节点上。
6 S4 n3 Q# s K& `" u( n' t相关组件的功能如下：
p8 O( t x+ H2 x1、CMS: 云管软件（Cloud Management Software），例如openstack（ovn最初就是设计给openstack用的）。
6 { b, ^6 X. M- b; F2、OVN/CMS plugin: 云管软件插件，例如openstack的neutron plugin。它的作用是将逻辑网络配置转换成OVN理解的数据，并写到北向数据库（OVN Northbound DB）中。3 j1 b' v. ]- F2 S: D
3、OVN Northbound DB: ovn北向数据库，保存CMS plugin下发的配置，它有两个客户端CMS plugin和ovn-northd。通过ovn-nbctl命令直接操作它。北向数据库保存逻辑网络信息（交换机和路由器等）+ C( H& i; I$ Z+ W8 e' p% A
4、ovn-northd: 北向进程将OVN Northbound DB中的数据进行转换并保存到OVN Southbound DB。所有信息经过北向数据库通过ovn-northd北向进程和南向数据库互通。
7 F' ?' L2 i1 R2 h5、OVN Southbound DB: ovn南向数据库，它也有两个客户端: 上面的ovn-northd和下面的运行在每个hypervisor上的ovn-controller。通过ovn-sbctl命令直接操作它。南向数据库保存各个节点的物理网络信息。
9 A8 G* x, z& ~8 J3 [! K$ G- C6、ovn-controller: 相当于OVN在每个hypervisor上的agent（代理）。北向它连接到OVN Southbound Database学习最新的配置转换成openflow流表，南向它连接到ovs-vswitchd下发转换后的流表，同时也连接到ovsdb-server获取它需要的配置信息。 p1 `8 g9 Z6 @! i; b2 ^0 f3 c9 g- f
7、ovs-vswitchd和ovs-dbserver: ovs用户态的两个进程。" ~4 k0 x: e4 s9 ~- h
每个节点都有个ovn-controller控制器，这个ovn-controller控制器是管理ovs（ovs-vswitchd、ovsdb-server）的，ovn-controller对接到南向数据库，经过ovn-northd北向进程和北向数据库互通，之后和openstack互通。% i2 M* \( x$ O: [. D
南向数据库保存物理网络状态信息，北向数据库保存逻辑网络状态信息。
) x, w- d2 g% f; h5 e在这里插入图片描述- S" _5 D! w2 A+ [/ C8 w1 u
克隆出两台虚拟机，安装ovs、ovn
# G- Z1 i% D9 p8 r
. s4 y W& z0 U1 u! [: X% M% eCentOS Stream 8 版本2 L0 c5 K- Z* z
. R v' O' R# e! [systemctl stop firewalld.service S6 R8 z) }" w+ [7 v
systemctl disable firewalld.service
3 H/ e2 {" X/ hsetenforce 0
1 x: J% P! Y* s! b( }sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
2 z+ }0 J% u9 W. G/ M2 Qmkdir /etc/yum.repos.d/bak
9 E- b6 t6 l1 X5 E) x% wmv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak/* `! O/ I* g: f Z
. T: o2 \! e% N. [! I# G: Q# [' |7 Z
cat <<EOF > /etc/yum.repos.d/cloudcs.repo
8 D4 m0 x. U3 B- H" |: z8 l$ c: F[ceph]6 T `: P& S: k/ N8 K( k$ k
name=ceph
7 `; {, C* I; z8 ebaseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/x86_64/' b# e8 K3 e' f+ U
gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
" u2 X6 k: Q6 ]: F4 Egpgcheck=1! ^0 X. `. N ]
enabled=1, w8 C# v$ X& `
9 {! F% P( V6 N+ _, W4 T
[ceph-noarch]
! V. O; }1 u) ?name=ceph-noarch7 \# L$ A7 W' ?2 x- f; }
baseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/noarch/( x2 R+ v6 D) c# B# L
gpgcheck=1
& [& i) \1 |! a5 B1 `gpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc
J' Z+ Z. h/ h1 l8 ~: a- |- Z: \enabled=1+ \: x' L3 E+ b; a
( Y7 s0 s6 F$ x% v! g! B+ T$ U) N0 \[ceph-SRPMS]. @4 _8 {% ^. b5 e5 o2 }" Z1 f
name=SRPMS
. `# i, l S. s5 |baseurl=https://mirrors.aliyun.com/ceph/rpm-18.1.1/el8/SRPMS/
/ e8 C( M) k) g* u; u0 ggpgcheck=1
( I1 j; o( X) h# mgpgkey=https://mirrors.aliyun.com/ceph/keys/release.asc8 Y8 ?- l$ h; D
enabled=1
' A; T ~! U( ?" t# N# P1 b
. q% S4 y) I8 Y7 ?9 l* G* y5 i[highavailability]
' Z) n! K0 R6 [* }9 f$ q2 _4 Oname=CentOS Stream 8 - HighAvailability8 ]4 `. l) H: j# x+ E$ O
baseurl=https://mirrors.aliyun.com/centos/8-stream/HighAvailability/x86_64/os/
$ p' _# Q0 O; ?' @gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
+ x% x% C2 V# |" i R/ ]/ U0 [gpgcheck=1
7 T6 ?! p4 m6 p0 X6 zrepo_gpgcheck=0
& d0 H$ c) l; Z2 M2 S# mmetadata_expire=6h
9 A; o( U( F! p! ]8 g8 w: Ccountme=1
& H% a' [# t l4 h5 E7 |enabled=1
- W% Q0 B1 n3 Y, {- N& o9 q/ _) L% F6 m H) V+ V" O
[nfv]( I( |; f P5 e$ V# T" q
name=CentOS Stream 8 - NFV0 R1 s! t1 U" R. T, {) ~
baseurl=https://mirrors.aliyun.com/centos/8-stream/NFV/x86_64/os/
- ]0 X0 f! ?0 u+ @5 Z. \0 k8 }gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
3 K( ^# R( x( |! r3 [gpgcheck=1
4 P5 i# ^) A. p9 h; n; Drepo_gpgcheck=04 s5 X" \) i% \0 }3 C
metadata_expire=6h
+ u: @: W8 D7 a2 b0 t) F! d; _% Scountme=1! K1 r2 D/ K" K# D' {- U% j. X
enabled=14 J9 e# k, W3 T1 `' f+ K
: j8 @& ]: J! X6 N/ a[rt], n. f5 I1 t0 z4 u
name=CentOS Stream 8 - RT' ]! @8 W- h/ c' a" b
baseurl=https://mirrors.aliyun.com/centos/8-stream/RT/x86_64/os/, y/ s( H& c$ R6 t# S6 f
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial# G& h; D) A1 K; g$ w) f. v1 t
gpgcheck=14 o" N$ I5 S) E: P& C0 a7 _, \. O
repo_gpgcheck=0" q8 C8 Q% X/ |7 ?% O
metadata_expire=6h g- e$ W2 s& f3 g5 l& |
countme=19 h$ Z; P6 |/ u* K+ \
enabled=1) ^" S# ^$ i% @
6 N6 J4 r2 O* L V7 N% o B[resilientstorage]
) q3 F$ b( p8 pname=CentOS Stream 8 - ResilientStorage
1 [4 \: a% r' m# U% @" Q, {7 Z* N& Fbaseurl=https://mirrors.aliyun.com/centos/8-stream/ResilientStorage/x86_64/os/6 I% ]+ h# }7 h- P+ V3 w5 E0 m
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial/ Q2 c$ g8 l$ B. Q
gpgcheck=1% ?+ d: U! n* l- C8 i) \6 i
repo_gpgcheck=0
& O2 E' q9 u7 d. bmetadata_expire=6h
' [* R. @5 J, Z/ E5 V* z* Rcountme=1
4 [ e/ X1 B9 U$ L% ~enabled=1
. i; j1 X8 R, a# l! K( u! @ f: t. q3 k" Q: c
[extras-common]
, k1 {/ z3 f( ]4 |6 q, Jname=CentOS Stream 8 - Extras packages
2 }7 [, B3 `, Z# Mbaseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/extras-common/
' {& O8 y/ a, [; e# [% D: Bgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512
+ }/ @3 T6 [7 j" P5 X* Igpgcheck=1
7 f, P1 u! O) q! b; Rrepo_gpgcheck=0
8 S. i2 M I$ u+ p- [ a0 fmetadata_expire=6h5 \2 [7 p9 X5 k9 i' C+ N8 T
countme=1. Y ^" u F( C! U
enabled=13 d) R' [1 M) r
; p' [0 L p8 s c o3 b[extras]6 n7 i3 d1 P( |5 ?
name=CentOS Stream $releasever - Extras; f, p' V5 s; [7 H/ p
mirrorlist=http://mirrorlist.centos.org/?release=$stream&arch=$basearch&repo=extras&infra=$infra# W! X& }: W! t" G2 {6 i/ K
#baseurl=http://mirror.centos.org/$contentdir/$stream/extras/$basearch/os/
8 D8 I2 w! p3 c7 [% D4 [0 z9 Ibaseurl=https://mirrors.aliyun.com/centos/8-stream/extras/x86_64/os/# x$ y0 A2 q, O
gpgcheck=1( b* W3 A% g4 z `
enabled=18 \8 z0 F! C, e
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial @3 q8 L1 q% Z5 P0 p% J: J
7 o8 y+ ]; |% e8 _8 g' h" M
[centos-ceph-pacific]
: W6 t. V7 x; Tname=CentOS - Ceph Pacific
, g# {# F- K# `' C! }$ Z, Rbaseurl=https://mirrors.aliyun.com/centos/8-stream/storage/x86_64/ceph-pacific/
! v! p. _! b% ]6 Ygpgcheck=0
5 h. m2 W' v3 K" d: i0 t& B5 F( @1 @2 renabled=1
& ^8 g( I+ W6 o/ n, E5 ~gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Storage
0 i: u5 t- u" l! u" a' k! F8 m5 O: I9 E
[centos-rabbitmq-38]. Z7 ^& r* s9 t: Z' Z
name=CentOS-8 - RabbitMQ 38
; L w) y" s0 Xbaseurl=https://mirrors.aliyun.com/centos/8-stream/messaging/x86_64/rabbitmq-38/, b# i$ m; n/ A5 `. w# D# m
gpgcheck=1
0 c& w7 R3 `' Oenabled=1/ ~: p$ I6 ?' Y% i- i
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Messaging9 i! {6 `: |. E' b
8 z' F& b% b6 ?' Y
[centos-nfv-openvswitch]
# j: G4 ?! t+ [# N4 [* n% b2 ^name=CentOS Stream 8 - NFV OpenvSwitch, O4 }5 U/ {% C) I0 O
baseurl=https://mirrors.aliyun.com/centos/8-stream/nfv/x86_64/openvswitch-2/
F Y1 `3 U, w7 I# k. v6 vgpgcheck=17 j) J+ p7 V/ M) o7 D/ p
enabled=1" g/ K H/ ?+ E6 v! T, E
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-NFV
% W6 @" F, E+ O* S |module_hotfixes=1: I' ^/ ?. m1 A/ i( y/ u3 z* G6 {
- H& k4 _( o. u4 F3 D$ {$ N; P[baseos]3 N) q; v+ a- v; ~2 v! Z8 }
name=CentOS Stream 8 - BaseOS }" u: \. a$ N6 H
baseurl=https://mirrors.aliyun.com/centos/8-stream/BaseOS/x86_64/os/
8 f7 E, x0 p( u" i8 [7 A! B- [gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial, }3 E1 E+ ]0 C" N# }; M- s9 O$ W( E0 }
gpgcheck=1
: }3 z2 X/ B0 a0 o. L1 qrepo_gpgcheck=0
& L+ P. u% E. N" Y8 G- K! rmetadata_expire=6h
( [- o6 n9 B( s9 qcountme=1
( E' P) w/ {( F! c& D7 [1 _ kenabled=1; m: m! ]! ^2 J1 k9 G. p1 u
, A' s* t1 y1 a3 [ |2 h. R
[appstream]" j4 o, L7 b! s' H% k& |
name=CentOS Stream 8 - AppStream5 {2 X5 R- u% @) J, _$ {" M# Q- z% j
baseurl=https://mirrors.aliyun.com/centos/8-stream/AppStream/x86_64/os/
& F; |. t- Q. }. H# h2 lgpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
+ B/ q0 e! _' D, i7 _1 lgpgcheck=1* I' Q0 q, D4 E. z/ `6 I- ?
repo_gpgcheck=0
: h- w; b% Q A7 Y! Tmetadata_expire=6h
9 ?$ G% e1 d, Vcountme=1
% v# f% c- ?1 U$ F' G Wenabled=1
( f9 B$ c l/ C- m- Z8 M- V: \* T$ F* d1 m
[centos-openstack-victoria]
: F0 \8 _3 \* F; Dname=CentOS 8 - OpenStack victoria' _; T' b0 Y9 a! A+ l8 W% D
baseurl=https://mirrors.aliyun.com/centos/8-stream/cloud/x86_64/openstack-victoria/* i& ?6 ^9 m0 E( Z4 S$ |/ y
#baseurl=https://repo.huaweicloud.com/centos/8-stream/cloud/x86_64/openstack-yoga/1 K2 T3 H+ B: e; Q; m- R
gpgcheck=1
: p% ^& I$ s3 H ^% k- Jenabled=19 F3 O( {" {( f5 j) o
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud
( p) U+ g$ f; A$ n6 ]3 pmodule_hotfixes=1
& s1 c X6 y7 o8 p- q% ]2 W7 L" k
& L/ k _# T' p( Y* ^6 C# S8 l[powertools]
8 Q6 b. G6 |0 pname=CentOS Stream 8 - PowerTools
8 Y. \+ x( M2 b3 `#mirrorlist=http://mirrorlist.centos.org/?release=$stream&arch=$basearch&repo=PowerTools&infra=$infra
% \- `7 Q. e1 }5 L1 H* b( Zbaseurl=https://mirrors.aliyun.com/centos/8-stream/PowerTools/x86_64/os/* w) b% l# q% W! w- V+ x2 p& P
gpgcheck=1! V) m) q* K; Y, P: Y
enabled=1. V( g5 S* q& m1 K7 g
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial
2 b4 L+ _' D! S7 p4 aEOF1 `8 F6 |; c* m% B5 L+ l1 @
1 R F, s6 w2 {/ M" N; n! ?7 l0 Yyum install -y vim net-tools bash-completion git tcpdump autoconf automake libtool make python3 centos-release-openstack-victoria.noarch6 ~) U; D7 y& q2 g
yum install -y openvswitch3.1* D$ n& S7 s* N; ], {# l3 v5 v
yum install -y ovn22.12*6 u- W5 x5 o( c2 k) o5 L
查看安装版本来检查ovn是否安装成功，# ovn-appctl --version9 g# \% t% k& ^' E
echo 'export PATH=$PATH:/usr/share/ovn/scripts:/usr/share/openvswitch/scripts' >> /etc/profile9 h# s# `) d; Q9 z T. K* S" v
source /etc/profile 重新读取配置文件让配置文件立即生效
; F$ `+ L+ T9 x y3 w在这里插入图片描述
& ?: v# z5 j" ]# f4 c- ^) M& Tcentral相关组件启动：把node1作为central节点，安装central必需的三个组件：OVN Northbound DB、ovn-northd、OVN Southbound DB。
- ]- [. [5 U5 v' C4 \) Z: m6 }在控制节点启动central，只用在一个控制节点上启动即可（node1或node2上开启都行，这里是在node1开启），central只需要一套即可。( b! V6 i2 {$ C$ m" Q
- p: R2 p/ A$ F( `# z" |4 r
ovn-ctl start_northd命令会自动启动北桥数据库、ovn-northd、南桥数据库三个服务, Q' h1 _/ M) I N2 U7 {
[root@node1 ~]# ovn-ctl start_northd
) c8 @4 p$ S1 J; U- @, E( A/etc/ovn/ovnnb_db.db does not exist ... (warning).7 s/ {% i" R( [+ N& o
Creating empty database /etc/ovn/ovnnb_db.db [ OK ]
+ B1 `% l, I( j% m( P; s. h& }Starting ovsdb-nb [ OK ]
, @* R; t# |: ]0 Z- w/etc/ovn/ovnsb_db.db does not exist ... (warning).$ W$ j; Z- z# W( n# @4 y
Creating empty database /etc/ovn/ovnsb_db.db [ OK ]
2 |; ]+ n* w( c w2 G8 Q- H7 C: k3 YStarting ovsdb-sb [ OK ] K: r1 ]1 N4 Y5 y0 {1 w, M
Starting ovn-northd [ OK ]
' t9 N$ U8 B. |" X: ~0 d. p0 w g* L; {3 f
[root@node1 ~]# ps -ef | grep ovn
/ `* B* A8 m# f% r, x/ }0 ~+ Droot 34102 34101 0 21:02 ? 00:00:00 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-nb.log --remote=punix:/var/run ovn/ovnnb_db.sock --pidfile=/var/run/ovn/ovnnb_db.pid --unixctl=/var/run/ovn/ovnnb_db.ctl --detach --monitor --remote=db:OVN_Northbound,NB_Global,connections --private-key=db:OVN_Northbound,SSL,private_key --certificate=db:OVN_Northbound,SSL,certificate --ca-cert=db:OVN_Northbound,SSL,ca_cert --ssl-protocols=db:OVN_Northbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Northbound,SSL,ssl_ciphers /etc/ovn/ovnnb_db.db
5 t% e# L8 F% kroot 34118 34117 0 21:02 ? 00:00:00 ovsdb-server -vconsole:off -vfile:info --log-file=/var/log/ovn/ovsdb-server-sb.log --remote=punix:/var/run ovn/ovnsb_db.sock --pidfile=/var/run/ovn/ovnsb_db.pid --unixctl=/var/run/ovn/ovnsb_db.ctl --detach --monitor --remote=db:OVN_Southbound,SB_Global,connections --private-key=db:OVN_Southbound,SSL,private_key --certificate=db:OVN_Southbound,SSL,certificate --ca-cert=db:OVN_Southbound,SSL,ca_cert --ssl-protocols=db:OVN_Southbound,SSL,ssl_protocols --ssl-ciphers=db:OVN_Southbound,SSL,ssl_ciphers /etc/ovn/ovnsb_db.db
% C' M3 v- O0 Z+ h8 C2 Q2 ]( zroot 34128 1 0 21:02 ? 00:00:00 ovn-northd: monitoring pid 34129 (healthy)
* p+ k5 X2 s# v0 oroot 34129 34128 0 21:02 ? 00:00:00 ovn-northd -vconsole:emer -vsyslog:err -vfile:info --ovnnb-db=unix:/var/run/ovn/ovnnb_db.sock --ovnsb-db=unix:/var/run/ovn/ovnsb_db.sock --no-chdir --log-file=/var/log/ovn/ovn-northd.log --pidfile=/var/run/ovn/ovn-northd.pid --detach --monitor! k' k) j( w+ h
root 34302 34259 0 21:07 pts/0 00:00:00 grep --color=auto ovn, T/ ]5 E# f& v0 K* Z' e
在这里插入图片描述
! b w2 n, ^' R( T) @. c, qhypervisor相关组件启动：hypervisor节点包含三个组件：ovn-controller、ovs-vswitchd和ovsdb-server。0 P2 n9 j4 h G3 M( V
启动hypervisor（hv）相关组件：node1和node2两台节点上都要启动，首先启动两个节点上的 ovs-vswitchd 和 ovsdb-server
$ }0 b/ n, i2 J3 [ c5 ?( Y
% z" r' ?3 ?% |& `+ o[root@node1 ~]# ovs-ctl start --system-id=random
) c8 h- e, M- a/ \# ^/etc/openvswitch/conf.db does not exist ... (warning).
- a" F- j, G' y: F3 e0 k$ PCreating empty database /etc/openvswitch/conf.db [ OK ]
8 I" U9 o) G. ]0 A# A; n. YStarting ovsdb-server [ OK ]. _' \ G2 i% Z. }$ R- c" [3 T- K$ f
Configuring Open vSwitch system IDs [ OK ]7 G5 `# F" p4 V4 r
Inserting openvswitch module [ OK ]
7 d. h: X' Y3 n8 ]+ XStarting ovs-vswitchd [ OK ]
% c4 `+ M5 |& g, S% P9 f$ qEnabling remote OVSDB managers [ OK ]3 L2 ]* @: D+ [+ }
# i2 @7 q: P" n' H
[root@node2 ~]# ovs-ctl start --system-id=random; ]& h7 c* I: \& O- M) m" J
/etc/openvswitch/conf.db does not exist ... (warning)., }% ]( X, N7 T2 W
Creating empty database /etc/openvswitch/conf.db [ OK ]
3 i4 f% \( x; G) H) p4 k4 t8 |" j& }Starting ovsdb-server [ OK ]2 x* Q( F: P J( K' a7 C, j! N0 U
Configuring Open vSwitch system IDs [ OK ]
$ U2 F% u! R: [" s" yInserting openvswitch module [ OK ]
/ }( G% E' K8 v" ]Starting ovs-vswitchd [ OK ]$ H; } P. |$ w8 }: I; \9 S0 T
Enabling remote OVSDB managers [ OK ]4 r" Y2 ^, r( I* p7 g: T$ V
在这里插入图片描述0 c; H/ W+ @6 [2 A, l
两个节点分别启动ovn-controller
; [+ e5 }9 Z5 r% V- w" G
; G g; M0 `: g[root@node1 ~]# ovn-ctl start_controller# c2 \) ~" Y( _& Z3 l7 H/ `
Starting ovn-controller [ OK ]3 d. A$ |" v- m/ }# c/ V
[root@node1 ~]# ovs-vsctl show ovn-controler启动后会自动创建br-int网桥3 h* d4 b: Y7 A% Z. d- A
ed157e0c-cac3-46b9-830c-f2d710b475d5
8 e% {% _4 w% Y; D; f Bridge br-int+ Y' x+ W. k3 K& c
fail_mode: secure
|) B# c% b. d; p9 N( b datapath_type: system5 o7 o7 r2 r+ B0 S
Port br-int
' z0 w' M; z& k Interface br-int
( H1 ~# i5 v: F Q) N' [) ]8 `* S type: internal
/ t* O7 m# a/ x0 V0 I4 v% V ovs_version: "3.1.3"
% H# X& n+ H$ g' m# u9 |3 d4 G6 x$ l. C6 B
[root@node2 ~]# ovn-ctl start_controller' i# C) {) _# }& `3 w
Starting ovn-controller [ OK ]( C7 ^! r2 N8 H+ ?% g3 }- P8 a
[root@node2 ~]# ovs-vsctl show ovn-controler启动后会自动创建br-int网桥+ v* J {: d1 Q3 {
f6669675-b42d-47de-be95-b26bf6d1e069
! w0 f7 R- T7 [( Z# ~ Bridge br-int. b4 N' K( j& c# r2 W
fail_mode: secure, A3 z) g8 X/ J( }: U, a& c0 R
datapath_type: system
' a; f& p9 S2 q8 A, C Port br-int
: S* x7 P0 t3 _ Interface br-int4 m- v- e$ v5 o9 C0 |3 w; B" Q
type: internal3 r6 y& K E: Z0 s5 c
ovs_version: "3.1.3"
+ {4 V3 w9 {- X7 O) _# w, P) O在这里插入图片描述- ~* B: s4 m X0 u: B0 L
可以看出此时hypervisor并没有和central关联起来（也就是ovn-controller没有和南向数据库连接）。可以在node1上验证：[root@node1 ~]# ovn-nbctl show
8 s: W+ B9 G) r! ?' e5 Xhypervisor连接central，开放南北数据库端口：
4 v1 x4 B- ~3 m6 v) T) } {1 u* i. }) i9 ?( c
ovn-northd之所以能连上南向数据和北向数据库，是因为它们部署在同一台机器上，通过unix sock连接6 S, a7 d+ N, T" t5 h9 c: W
central节点开放北向数据库端口6441，该端口主要给CMS plugins连接使用' N' W! ^/ g$ ~
central节点开放南向数据库端口6442，该端口给ovn-controller连接& B* _: _# y. P0 X9 O. C
[root@node1 ~]# ovn-nbctl set-connection ptcp:6641:10.1.1.41& `3 Y. Z- |/ G
[root@node1 ~]# ovn-sbctl set-connection ptcp:6642:10.1.1.41
! @! }8 Z( K' D8 ~# L[root@node1 ~]# netstat -tulnp |grep 664% o% L$ A9 P% }# a+ e
tcp 0 0 10.1.1.41:6641 0.0.0.0:* LISTEN 34102/ovsdb-server( {( p! q5 X9 q% O( v! i5 V# q, {
tcp 0 0 10.1.1.41:6642 0.0.0.0:* LISTEN 34118/ovsdb-server
# K" Q/ m/ R! U4 _0 S& B$ x: tnode1上ovn-controller连接南向数据库
* D. @; }; H' d" I$ ?3 D5 h# j1 Z+ S tovn-remote：指定南向数据库连接地址! r4 S) ?9 p; p# Z; p
ovn-encap-ip：指定ovs/controller本地ip
1 o( P Q, j4 j% ^ovn-encap-type：指定隧道协议，这里用的是geneve
& |, p+ F8 X! G1 w. P6 m4 w9 X3 lsystem-id：节点标识- a7 D% @! A6 T" @5 l- ^4 v
[root@node1 ~]# ovs-vsctl set Open_vSwitch . external-ids:ovn-remote="tcp:10.1.1.41:6642" external-ids:ovn-encap-ip="10.1.1.41" external-ids:ovn-encap-type=geneve external-ids:system-id=node13 n; U6 e/ g2 ~7 W: G
0 i6 z* w8 R7 t) t1 d" k
node2上ovn-controller连接南向数据库
% H3 D0 b. {/ I0 y- d6 d[root@node1 ~]# ovs-vsctl set Open_vSwitch . external-ids:ovn-remote="tcp:10.1.1.41:6642" external-ids:ovn-encap-ip="10.1.1.42" external-ids:ovn-encap-type=geneve external-ids:system-id=node2
3 Y1 j: n" |( |4 L7 W# {
5 \$ @. e* }' w0 r3 F# r3 b3 ~在node1查看南向数据库信息; b# B% i3 G& U8 |2 Z
[root@node1 ~]# ovn-sbctl show
. i' ]% F, n* ]5 x' \Chassis node2
, U; p* O6 O" n9 t n1 A hostname: node2
9 _6 R4 I2 e; z Encap geneve+ b; n; o1 \' O( i
ip: "10.1.1.42"
8 K5 E; A' {0 L" K& G* O options: {csum="true"}8 y9 \ a, \3 b& q- }& N& }( [
Chassis node17 C- H6 J0 r5 n2 I3 n9 Y* W& Q$ F( Q' Z
hostname: node1
6 i0 \/ @* H' F p% Y+ _ Encap geneve
' K' @2 w: j6 N' T) a8 b' ? ip: "10.1.1.41"
7 M9 l# ~8 q: ]) B0 B* a( n options: {csum="true"}
1 l0 M$ T; j5 X, M在这里插入图片描述
4 l) Y/ B2 F: Q& m4 d以上的逻辑架构是站在底层组件和服务的角度来看的。
) y# ?4 v& g9 U0 }- r接下来换一种角度，站在逻辑网络的角度来看。5 z5 C$ ^: `6 C1 W2 c. I
在这里插入图片描述 J. f' O3 T# F' _% F
geneve隧道：ovn-controller连接南向数据库时，指定了external-ids:ovn-encap-type=geneve参数，此时看看两个节点上的ovs信息如下，会发现两个节点上都有一个ovn创建的ovs交换机br-int，而且br-int交换机上添加的节点port/interface类型都为geneve7 |& r. G2 k* j
$ M. t; ?) d Q[root@node1 ~]# ovs-vsctl show node1上查看ovs信息
5 i& ?$ t! \% c- X# V: U. Ted157e0c-cac3-46b9-830c-f2d710b475d5
% @5 }& O3 H: M/ O5 v0 x, n Bridge br-int& u8 k. `) P( d# J& I4 B0 \
fail_mode: secure
: K, l' b, O7 I$ A datapath_type: system) }$ O. l! B X: o9 x
Port br-int: ?8 V* b4 J' f4 O5 @! e
Interface br-int$ T# _1 x& P7 ^! `. R4 X4 D- T
type: internal
Y j l' W! d4 o. s1 @0 q Port ovn-node2-0
. `8 [) B/ a4 M* j F; m: r; y Interface ovn-node2-0
' h& _( y* [. k# o type: geneve5 ~# _' T9 p/ \- @9 G$ o* G- g
options: {csum="true", key=flow, remote_ip="10.1.1.42"}
8 W" Z) W* C+ d) ?0 @. U7 X3 Z ovs_version: "3.1.3"2 ~9 J& Z9 ` D' G4 |1 L- G
& V3 _% s4 }0 P: n: n[root@node2 ~]# ovs-vsctl show node2上查看ovs信息2 M% v, ]& ^4 R
f6669675-b42d-47de-be95-b26bf6d1e069; e- n7 z) _) M8 \
Bridge br-int
; S% v" A# Y- Q$ n fail_mode: secure
* p" ]; l! I1 C datapath_type: system) l% K! l) T& L, P& W$ X
Port ovn-node1-0
5 x( a- W, }1 t Interface ovn-node1-0$ g5 q, G8 M, w, w% P
type: geneve. v* K( l; |$ |- ?
options: {csum="true", key=flow, remote_ip="10.1.1.41"}! x3 X" n! O# A0 a S- _
Port br-int* d F% N$ l- ~- L# t
Interface br-int
& C3 g# \2 O7 R/ t* l5 P# J% s- b type: internal0 D8 i q b' \3 j% Y5 X+ V
ovs_version: "3.1.3"& a! f) Q+ }! j. c
[root@node1 ~]# ip link | grep gene 查看geneve隧道link! Q+ a5 ?. B3 D
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
% d* P' v. T) t* [" S查看geneve隧道link详情，从dstport 6081可以看出geneve隧道udp端口是6081
* j' V1 W, l" Y; x[root@node1 ~]# ip -d link show genev_sys_6081
/ R, a. ~) M P2 |; j+ a, @5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
& V* d+ L* @( o% h7 K) k link/ether 6a:e3:ff:a5:cc:d6 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 654654 C9 v& p. p( N; V
geneve external id 0 ttl auto dstport 6081 udp6zerocsumrx* n& h; ~2 V' e4 o+ p
openvswitch_slave addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
2 e4 R- ]- ?% G2 r2 p1 ^8 o查看geneve隧道udp端口，最后一列为“-”表示这个端口是内核态程序监听
0 v, P4 c ~# U5 B- T) x3 F$ T[root@node1 ~]# netstat -nulp|grep 6081
( k8 K9 @$ \! W, a4 K9 a1 \udp 0 0 0.0.0.0:6081 0.0.0.0:* -
- W6 m. a' @4 u) budp6 0 0 :::6081 :::* - O* N* c# M1 }, N6 B3 |, o& \
# s+ d/ |3 `+ }+ ^[root@node2 ~]# ip link | grep gene) o: S: u, B' f% P) R! Q& |" u
5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000
0 O+ M' N$ }7 @* L[root@node2 ~]# ip -d link show genev_sys_6081
5 V* V% `8 g' q. q6 D5: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN mode DEFAULT group default qlen 1000" _5 j" V( |: H r$ O( @+ W
link/ether 4e:db:f1:e4:43:94 brd ff:ff:ff:ff:ff:ff promiscuity 1 minmtu 68 maxmtu 65465! V. P: |/ g$ [5 |/ [6 }
geneve external id 0 ttl auto dstport 6081 udp6zerocsumrx4 V7 K1 \% f% ?, d2 P9 t5 C
openvswitch_slave addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535# x, q) L2 `! r8 ~7 T% L
[root@node2 ~]# netstat -nulp|grep 60815 w' t# T9 S0 I, R) V. x
udp 0 0 0.0.0.0:6081 0.0.0.0:* -
, C8 v n2 G5 \# M8 gudp6 0 0 :::6081 :::* -# t% v. Y' x8 P
在这里插入图片描述/ T9 H2 O# v3 U5 I. }5 z4 K1 {
- @/ k/ p( O* U6 d2 z/ d% q在做以下实验验证时需要注意MAC地址的合法性，不要误配置。MAC地址分为三类：+ Q/ b4 X, N* |+ n
广播地址（全F）: p0 R+ G" \) H0 G: o, F( r
FF:FF:FF:FF:FF:FF
" i4 l+ Q1 q: O- L! T( o4 }& g3 q }主播地址（第一个字节为奇数）( R! I3 G' X' O1 {( P
X1:XX:XX:XX:XX:XX
; a/ s' ~8 M8 {$ y/ _4 q/ I _X3:XX:XX:XX:XX:XX
# _# p- n5 ^. P" b" |4 U* P' O5 ^. }! q! {X5:XX:XX:XX:XX:XX
9 Z/ X& P, g6 L. ~X7:XX:XX:XX:XX:XX- H9 G) b$ ~/ L% D
X9:XX:XX:XX:XX:XX- ?( ^6 l) E$ D/ ~( u+ j5 H9 S
XB:XX:XX:XX:XX:XX$ t0 a& T6 Q: V
XD:XX:XX:XX:XX:XX
3 x6 T6 q' }; I; }" mXF:XX:XX:XX:XX:XX
. i* E! ^& d' C8 ]: m可用MAC地址（第一个字节为偶数）4 `6 h9 y) \2 ?, t2 Y. \* ~( R
X0:XX:XX:XX:XX:XX
o* n4 b0 U, f2 {X2:XX:XX:XX:XX:XX
: E) J( ^( y! s( ^* BX4:XX:XX:XX:XX:XX
# {! d9 J7 ~3 i3 o0 ~X6:XX:XX:XX:XX:XX: |2 c- j% _% [
X8:XX:XX:XX:XX:XX: P, N' W6 }' g" I# l
XA:XX:XX:XX:XX:XX) @1 ~, s# m0 \* l3 E0 z
XC:XX:XX:XX:XX:XX/ X. S8 c1 D3 }- d3 A, c2 k2 R
XE:XX:XX:XX:XX:XX
! S2 q3 l' q) f# O8 A3 M4 T在每个节点上创建一个网络命名空间ns1（因为在两个节点上所以同名ns1不会冲突），网络命名空间可理解为虚拟机，并且在ovs交换机上创建一组port和interfacce，然后把interface放到网络命名空间下。veth pair：两个网络虚拟端口（设备），veth可理解为网卡端口，一个端口在虚拟机上，一个端口在br-int虚拟交换机上。
# O& O9 K! u# Y
* A3 @0 o! v* ]0 h' w, q& vnode1上执行
; `: y/ H( V) L[root@node1 ~]# ip netns add ns1
* J$ ^% F1 j" K0 M[root@node1 ~]# ip link add veth11 type veth peer name veth12$ D( T% D6 m! t! _ h7 L! E
[root@node1 ~]# ip link set veth12 netns ns1
8 p# E' X Y' [[root@node1 ~]# ip link set veth11 up. `( P" I) h" y4 M; S8 L
[root@node1 ~]# ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:01
# U5 C1 y+ H! e, B! A. e5 G0 E[root@node1 ~]# ip netns exec ns1 ip link set veth12 up
" _4 x! z, [, k8 B, ]- @0 Y, o/ A[root@node1 ~]# ovs-vsctl add-port br-int veth11: ~9 S4 C( h# t& c& d% v) ^, ^
[root@node1 ~]# ip netns exec ns1 ip addr add 192.168.1.10/24 dev veth124 ]2 b. r4 e8 j1 @
7 P6 j+ u" d6 P5 t& I$ ^* Q d
node2上执行，注意veth12的ip和和node1上veth12 ip在同一个子网
: w# F4 ?' R0 ?0 l& k: l. Q[root@node2 ~]# ip netns add ns1: h* ?5 D% M6 e6 ]
[root@node2 ~]# ip link add veth11 type veth peer name veth12+ |+ ~6 K* V h, v
[root@node2 ~]# ip link set veth12 netns ns1
# |% Z" |7 S3 y/ \1 G[root@node2 ~]# ip link set veth11 up
' i7 o2 b# X% Z* `[root@node2 ~]# ip netns exec ns1 ip link set veth12 address 00:00:00:00:00:026 p& g- s! P% u+ ~; T4 M) @( D
[root@node2 ~]# ip netns exec ns1 ip link set veth12 up
# O( s6 R. T7 X: L5 y8 k: a[root@node2 ~]# ovs-vsctl add-port br-int veth11( V) e' f# P+ x" ]+ a
[root@node2 ~]# ip netns exec ns1 ip addr add 192.168.1.20/24 dev veth12# a% t- u. m' f
7 \1 k2 A2 C% I) ?4 s
查看node1上br-int交换机信息
+ n( n2 \+ |) M7 x- \: q% u[root@node1 ~]# ovs-vsctl show
* @7 o; ?% q. x bed157e0c-cac3-46b9-830c-f2d710b475d59 \% Z8 o$ D, k& o
Bridge br-int
0 x! H5 L: p! g# D0 R7 H7 e. W! \ fail_mode: secure
$ Z7 G( j) m( ^. L- h, j' _ datapath_type: system
- j M" _4 a, Y Port br-int
3 Q: P' P7 W4 l* @! f Interface br-int) A* V/ a6 h/ m. k; @
type: internal
! |/ s( E+ a1 @2 f! _% n* a Port veth11* X$ p. t4 }1 p0 V8 j
Interface veth11
5 j; X9 Q( r: v* q Port ovn-node2-0$ w" t% \% J2 U' r* m
Interface ovn-node2-0
5 o! ]6 j0 l: J type: geneve
) f9 y0 ]5 o+ P q4 Z options: {csum="true", key=flow, remote_ip="10.1.1.42"}
0 E1 E/ r) p9 o6 o5 r, d: c2 p ovs_version: "3.1.3"+ d" X! P D" v+ f2 F0 s
查看node2上br-int交换机信息. ]: d+ r5 H O+ g1 P# u# D Y
[root@node2 ~]# ovs-vsctl show7 s- l' |% M# X& z0 |+ D) k
f6669675-b42d-47de-be95-b26bf6d1e069; t( H% c% s p# b* E
Bridge br-int) \4 M) [' ?: \4 e( \% q
fail_mode: secure+ f5 t' `% @9 [, T
datapath_type: system
9 |2 _) ?% w" @( I& e Port veth11
4 ^5 Z. f- d5 R- [1 n Interface veth11
' A2 @5 E9 c3 R! v Port ovn-node1-0
2 p+ G5 x+ b M# O, ~ Interface ovn-node1-0, A" |2 c# f2 p" h
type: geneve3 y/ H& e4 c# N) s( k+ o
options: {csum="true", key=flow, remote_ip="10.1.1.41"}0 `2 n( |' g7 G9 F1 b
Port br-int
, y! ~. D9 {! A Interface br-int9 O' j# m$ t a0 Y+ u# n& b
type: internal" M$ ?3 ? D$ W2 G W, T
ovs_version: "3.1.3"
# q% Y4 k ~' N! q' M9 F
9 t5 e2 D* d* X现在从node1上的ns1 ping node2上的ns1是不通的，因为它们是不同主机上的网络，二/三层广播域暂时还不可达。, x, a! g. j2 h/ M6 A
[root@node1 ~]# ip netns exec ns1 ping -c 3 192.168.1.201 `# i6 G1 n! A' X+ ?
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
( e L' o! A& x& O" E: \0 e9 Z" w5 M: t. [. B
--- 192.168.1.20 ping statistics ---
" u2 j0 ^' e# ^& G4 B$ n9 X3 packets transmitted, 0 received, 100% packet loss, time 2047ms% W1 T# x A2 {' X, z
在这里插入图片描述# a$ h4 Y' V$ n4 t
查看openstack的控制节点发现，ovn的北向数据库中有逻辑交换机信息。
2 H- z! `$ ^% z7 s+ S$ z在openstack里面，创建一个网络，就相当于创建了一个逻辑虚拟交换机，这个逻辑交换机（网络）信息会被保存到北向数据库里面。一个网络就是一个逻辑交换机。& n9 Q# e6 l. M3 _' I6 H* e% s) Z: p
在这里插入图片描述
+ ~5 U% J; S$ q3 r$ C% I在node1中查看发现，ovn的北向数据库中没有逻辑交换机信息
/ n& x2 k- Q% S( x R* x在这里插入图片描述; ?5 n4 C! e, h) ?3 ]
在openstack不同节点的虚拟机ip互通，这两个虚拟机ip连的是同一个网络，是同一个逻辑交换机上的同一个子网不同ip所以互通。- \2 |3 [1 v6 \: z
这两个节点的虚拟机ns1的ip是手工配置的独立的、不互通，这两个虚拟机ip没有连到逻辑交换机上，加个逻辑交换机就能互通。9 h1 j0 C$ y( J2 T
在这里插入图片描述' N$ P5 X1 ]5 v5 y9 E' s0 |
逻辑交换机（Logical Switch）：为了使node1和node2上两个连接到ovs交换机的ns能正常通信，需借助ovn的逻辑交换机，注意逻辑交换机是北向数据库概念。! r, \) p( T0 {5 f: m9 N
6 ?% D2 \; E' ?, M, r6 e C
在node1上创建逻辑交换机' |& \3 {% R0 i6 b' ` S
[root@node1 ~]# ovn-nbctl ls-add ls1
4 d; G+ C0 C9 b) R[root@node1 ~]# ovn-nbctl show
' e. I m$ L& D3 Q% t: ~switch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)! ]& ~; }# D5 X0 R/ ?9 @: [
在逻辑交换机上添加端口
" ~. h/ ^8 x+ h$ L7 A- u添加并设置用于连接node1的端口，注意mac地址要和veth pair网络命名空间内的那端匹配起来
$ H5 \2 V) z6 d0 ][root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-node1-ns12 b% o% R5 n% K* E
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node1-ns1 00:00:00:00:00:01
3 w4 M6 z* s5 b! }1 ~[root@node1 ~]# ovn-nbctl lsp-set-port-security ls1-node1-ns1 00:00:00:00:00:01
& A2 l, W: s$ {& ^/ M添加并设置用于连接node2的端口，注意mac地址要匹配起来% v% O% {( L! w0 N3 N G% ` ^/ ^
[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-node2-ns1
) @8 C( O5 ?9 o$ I9 S7 A0 k6 T[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node2-ns1 00:00:00:00:00:02
, T4 ^, m' a3 X[root@node1 ~]# ovn-nbctl lsp-set-port-security ls1-node2-ns1 00:00:00:00:00:02
], h+ [. D( z: t查看逻辑交换机信息. h J O& R$ O9 W% ]8 R2 K k( j
[root@node1 ~]# ovn-nbctl show5 W3 y& X* q- J
switch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
1 M5 B% n: N. [ port ls1-node1-ns1
2 z3 E( U7 c: { w( S addresses: ["00:00:00:00:00:01"]
8 n, Y* @+ K; V T% J. ]4 O0 @. q; E/ E port ls1-node2-ns19 h2 F! ?; `4 S) u. P8 z- }
addresses: ["00:00:00:00:00:02"]2 p% H* N" e1 m" X
# s" ]& t8 H. h1 p& {- m9 hnode1上执行，veth11端口连接逻辑交换机端口" l! e6 H) z6 R# A, c- G0 o. f
[root@node1 ~]# ovs-vsctl set interface veth11 external-ids:iface-id=ls1-node1-ns1
/ @% c4 i9 X" i2 Unode2上执行，veth11端口连接逻辑交换机端口
% E" y) H. p* h0 x0 [4 U6 j[root@node2 ~]# ovs-vsctl set interface veth11 external-ids:iface-id=ls1-node2-ns1
; x; u9 o* P3 Z+ S再次查看南向数据库信息，发现端口已连接, q- L9 v& K& Y; R$ l/ Q: L" m" V
[root@node1 ~]# ovn-sbctl show: x# d( F5 J/ I2 h2 z! _
Chassis node2
H1 K, c4 R" M2 L% I hostname: node25 |( j. R: M( v! C$ Z2 Y
Encap geneve
: v) ?0 [ }' P8 e* U) e9 D; ] ip: "10.1.1.42"
! r; \3 H! z1 b: M, T* g. q options: {csum="true"}
4 ~0 K2 R& B) b) r9 M: S Port_Binding ls1-node2-ns1( X2 |* B, Q; O9 A$ p) T5 K
Chassis node1
$ B" P" s6 P/ n hostname: node1 g7 i& E" e2 U
Encap geneve
5 G( V* y, Y8 I' D0 V1 M ip: "10.1.1.41"
& F; }4 O# u9 P- [4 W& w: l$ r9 V options: {csum="true"}1 E& \5 c2 B/ X$ M: [
Port_Binding ls1-node1-ns1 N. J3 K! }9 x1 W5 H4 K
node1上验证网络连通性5 G" g! u# ~0 h6 V1 }2 M$ l
[root@node1 ~]# ip netns exec ns1 ping -c 3 192.168.1.20
+ _) j- w2 q* n$ h& \, J& HPING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
% `$ f! [% T) D) f0 m8 @64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=4.68 ms
5 d1 W8 k) v* b) F% {( M64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.908 ms% h+ h1 D' S G' Y S$ d
64 bytes from 192.168.1.20: icmp_seq=3 ttl=64 time=0.756 ms
0 l4 W* E' e1 G/ h+ V2 V+ I+ A* o8 _7 j% x9 x! |& d8 M
--- 192.168.1.20 ping statistics ---0 H g) H/ f' d3 g9 u+ d, [- X$ b
3 packets transmitted, 3 received, 0% packet loss, time 2004ms4 P" V: B$ P* y: g4 N3 \% A
rtt min/avg/max/mdev = 0.756/2.115/4.682/1.816 ms
+ I+ @2 M$ b( X; Xnode2上验证网络连通性0 _" m5 ?4 J0 q
[root@node2 ~]# ip netns exec ns1 ping -c 3 192.168.1.10
) ?! j" I7 @' c2 DPING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.
$ `" ]1 P* {9 K4 z64 bytes from 192.168.1.10: icmp_seq=1 ttl=64 time=3.34 ms
- }8 V7 V& p) s& N7 Y& ]% ?9 c64 bytes from 192.168.1.10: icmp_seq=2 ttl=64 time=0.863 ms6 u6 N9 ` p$ p9 x0 |7 s
64 bytes from 192.168.1.10: icmp_seq=3 ttl=64 time=0.372 ms
$ H+ \1 `# O; C' n! r, N( T& M! e1 k6 X2 C
--- 192.168.1.10 ping statistics ---
: Q7 v; o y! I! P3 packets transmitted, 3 received, 0% packet loss, time 2003ms" p' q3 R& H. }7 s9 k9 O
rtt min/avg/max/mdev = 0.372/1.525/3.342/1.300 ms. w( L: `/ E2 {- F
现在node1和node2的ns1互通了，相当于创建了两个实例，这两个实例ip用的子网是连在同一个逻辑交换机上的，是同一个逻辑交换机上的同一个子网不同ip所以互通。
4 G+ V1 x1 T6 @4 r在这里插入图片描述* V- N, G* o, |9 O8 ^
在这里插入图片描述
" |) @ I& [5 Kgeneve隧道验证：从node1上的ns1 ping node2上的ns1的例子，抓包看看各个相关组件报文，验证geneve隧道封解包。通过抓包分析，可以看出geneve隧道在ovn/ovs跨主机通信的重要作用，同时也能看到ovn逻辑交换机可以把不同宿主机上的二层网络打通，或者说ovn逻辑交换机可以把ovs二层广播域扩展到跨主机。
- E: k. L/ @. X. [* z
( B/ x8 a, h* o% m+ C// node1上ns1 ping node2上ns1
# O9 F1 S% U6 Q& W( g% B# ip netns exec ns1 ping -c 1 192.168.1.20% N6 @! M1 C$ W% L+ ?% m
PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.+ l, C, E# K& s# z9 ^
64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=1.00 ms
1 y( e! u% h6 Z5 p- C--- 192.168.1.20 ping statistics ---
8 G" z) L& M E. N1 packets transmitted, 1 received, 0% packet loss, time 0ms
2 [& n: S, } G- V4 zrtt min/avg/max/mdev = 1.009/1.009/1.009/0.000 ms1 T6 a$ K7 K" a; v
( U! ~8 S4 G* i3 j5 `1 B// node1上ns1中的veth12抓包% a% Z7 h& a6 G7 R5 d
# ip netns exec ns1 tcpdump -i veth12 -n
; n+ \3 b: m3 E8 D F, r7 `tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
% P7 k7 @* [, j' `listening on veth12, link-type EN10MB (Ethernet), capture size 262144 bytes
6 ?1 f6 a" ]" r/ ~22:23:11.364011 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 24275, seq 1, length 64) R6 M6 Q* J# H6 T$ A3 r$ G
22:23:11.365000 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 24275, seq 1, length 643 R% H$ u$ ]. s% X {/ W5 L
22:23:16.364932 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28- B* E' Z! N ~8 L
22:23:16.365826 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28 O6 A8 i" E6 p0 f
5 r8 `; U; I2 K; K; O' k+ S4 `
// node1上veth12的另一端veth11抓包5 C9 `9 P$ M9 v
# tcpdump -i veth11 -n$ K$ S6 _7 R- y5 Z7 m2 [3 C
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
/ N; t9 f. y; i$ p! Y/ rlistening on veth11, link-type EN10MB (Ethernet), capture size 262144 bytes
0 C# Q% u$ o; T5 p22:25:11.225987 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 25166, seq 1, length 64
' \5 l7 D7 H; a; [22:25:11.226914 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 25166, seq 1, length 64
5 k5 w' H* u3 Z8 l1 Y$ [+ T- L22:25:16.236933 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 284 Y3 w+ S" f; [! e8 t! x
22:25:16.237563 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
. a* o* f; Z2 `- L" Y1 V22:25:16.237627 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 287 |. L& ]0 s* T' ?( K8 S
22:25:16.237649 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
# U( g: n) _1 {3 @! a9 P! }2 ]' r9 S) ] u' U
// node1上genev_sys_6081网卡抓包
* i% j9 `- [) j- [# tcpdump -i genev_sys_6081 -n" M7 X5 I6 [0 `5 M5 z
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode0 O! h# ]3 y' z! X7 ]! T- o* e+ h8 X
listening on genev_sys_6081, link-type EN10MB (Ethernet), capture size 262144 bytes" k! x! r; k' Y$ X+ P* ?. ]5 Z
22:28:15.872064 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 26492, seq 1, length 64
' g3 ^ a" c1 @; x C% T22:28:15.872717 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 26492, seq 1, length 645 F0 y/ {" M( l' Q0 A: `% M3 ^
22:28:20.877100 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28
" P( K' ^& b$ F& ?22:28:20.877640 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
u+ y+ J9 k. p1 V0 U. O+ B" e22:28:20.877654 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
# a# f, J- {: R& u( l; _22:28:20.877737 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28$ p3 R) L$ X4 }' Q! t- M
% w' Y! s5 J; X; A4 W// node1上eth0抓包，可以看出数据包经过genev_sys_6081后做了geneve封装; y* e& d9 U, F( G( B
# tcpdump -i eth0 port 6081 -n/ X' \+ V2 {- G0 a
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
: @3 v" A; D3 K( h6 _$ f# F* c, alistening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
# j7 C3 @$ L& e) {' q0 Y; ]22:30:23.446147 IP 10.0.12.7.51123 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 27458, seq 1, length 640 c7 P- b& N1 g; j$ O% A" @
22:30:23.446659 IP 10.0.12.11.50319 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 27458, seq 1, length 64) I* ^" }& m+ K# g$ L) k* w* Q
22:30:28.461137 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28
* |7 \+ z R! C! M* U9 ]5 X22:30:28.461554 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
8 r9 L$ u0 h X" p22:30:28.461571 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
_! W" j" J1 p& i22:30:28.461669 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28, ^7 V2 p( F9 ~( p* j- ~( ^, p P
; w5 Z9 _: W/ q4 z$ ]! }8 R; v8 C
===================跨主机===================
" k& j+ z8 P) ]0 S' i$ x
1 {2 N1 k! _' z& C* z3 g$ Q5 O// node2上eth0抓包
" Y2 {* N6 j) W. ~0 l# tcpdump -i eth0 port 6081 -n( M" ~3 L( t( P, E
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode7 u, Z. H5 a' S* e) m
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
2 t) A- d; }0 g& z: X/ T8 ^8 y22:23:11.364189 IP 10.0.12.7.51123 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 24275, seq 1, length 64
6 A0 T7 o! U5 ] C! T9 t O* A22:23:11.364662 IP 10.0.12.11.50319 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 24275, seq 1, length 648 \& b4 x' E4 C
22:23:16.365086 IP 10.0.12.7.49958 > 10.0.12.11.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 282 c, I, l7 v% r# K. u
22:23:16.365487 IP 10.0.12.11.61016 > 10.0.12.7.6081: Geneve, Flags [C], vni 0x1, options [8 bytes]: ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28# K3 b6 ^/ l0 R% @$ \& D9 T" K$ Q
) H0 K- A' ~; q( x6 t// node2上genev_sys_6081网卡抓包，可以看到数据包从genev_sys_6081出来后做了geneve解封0 U# c0 B4 d" }$ o3 i5 p- _ ^, U
# tcpdump -i genev_sys_6081 -n/ J& n6 O, j3 w* k4 g$ w3 o# A/ k
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
) a- m) l/ x8 R$ |listening on genev_sys_6081, link-type EN10MB (Ethernet), capture size 262144 bytes/ ~) m: o( Z* A6 o4 s, c: |
22:25:11.226186 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 25166, seq 1, length 64 U; ?$ q R5 m/ s' W6 \
22:25:11.226553 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 25166, seq 1, length 64/ i. E& r f5 t: G7 N: b
22:25:16.237070 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28
+ V( I3 E5 v/ l3 { u" |22:25:16.237162 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
* |6 N( l+ [; P# Z$ ]: Z& Q, e9 m/ J22:25:16.237203 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
: J3 q/ }: H& @22:25:16.237523 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 281 O' T& p$ \6 x" e2 N
; A! W# n- \, ~* B& Q5 h, F// node2上veth11抓包+ y) o% u* d) j: J
# tcpdump -i veth11 -n
- `% L3 G- [9 Y- f2 K; V& qtcpdump: verbose output suppressed, use -v or -vv for full protocol decode
" r, S/ v' f1 u, K4 [listening on veth11, link-type EN10MB (Ethernet), capture size 262144 bytes
1 f2 k$ c6 |: q22:28:15.872198 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 26492, seq 1, length 64
5 @1 [& E! C6 E( |0 p' @22:28:15.872235 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 26492, seq 1, length 64
9 y4 A1 ^" M3 |: T22:28:20.876913 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28! N# J' O3 h4 i' Q- b
22:28:20.877274 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 28' p' z) e& F! V
22:28:20.877287 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28, T) S! n* S) l* T: N/ r5 Q
22:28:20.877613 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 283 m, l3 F2 i L
3 ^* a1 r- m) e1 K3 w" i
// node2上ns1中的veth12抓包! ~ \4 n' V+ K; x. k
# ip netns exec ns1 tcpdump -i veth12 -n* o7 T" M- t: E5 N4 D! f5 L
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode" M4 ?9 W& ~/ w1 @
listening on veth12, link-type EN10MB (Ethernet), capture size 262144 bytes! h0 a% v: T+ h4 m' ^) v8 e& T
22:30:23.446212 IP 192.168.1.10 > 192.168.1.20: ICMP echo request, id 27458, seq 1, length 64) K% E/ c& y% u5 w* x
22:30:23.446242 IP 192.168.1.20 > 192.168.1.10: ICMP echo reply, id 27458, seq 1, length 64
, k) s1 j5 {, [# o22:30:28.460912 ARP, Request who-has 192.168.1.10 tell 192.168.1.20, length 28
' ]# X7 E; P G$ _) E1 y22:30:28.461260 ARP, Request who-has 192.168.1.20 tell 192.168.1.10, length 281 v7 {. A* {$ [5 u' N2 l7 l' C
22:30:28.461272 ARP, Reply 192.168.1.20 is-at 00:00:00:00:00:02, length 28
" O- p& r; C% L8 {8 D1 R/ D$ u0 D8 p22:30:28.461530 ARP, Reply 192.168.1.10 is-at 00:00:00:00:00:01, length 28
! } l# F, ?8 J, u! r- Y3 D6 t逻辑路由器（Logical Router）：
" E5 Q; W# l- i1 I前面验证了ovn逻辑交换机跨主机同子网的通信，那不同子网间又该如何通信呢？这就要用到ovn的逻辑路由器了。( B0 i8 h+ o4 I" }: \
先在node2上再创建个网络命名空间ns2，ip设置为另外一个子网192.168.2.30/24，并且再增加一个逻辑交换机。
' U! g/ M5 ~4 E1 o- @在这里插入图片描述
- r0 `9 E1 N3 C8 z+ r3 p* h, y5 k/ D
node2上执行6 w, x; r: [* H7 x
[root@node2 ~]# ip netns 查看网络命名空间
" o, F( x( j5 e( F( {ns1 (id: 0)
- J- o$ l9 w. I. P6 t[root@node2 ~]# ip netns add ns2
# t7 }- s% J2 j7 Z+ V8 q* b( Z[root@node2 ~]# ip link add veth21 type veth peer name veth22
3 u; w0 u* Z' w7 X[root@node2 ~]# ip link set veth22 netns ns27 x, M, @( `+ [ y5 P. V$ I
[root@node2 ~]# ip link set veth21 up
) z! @2 ]% V# p[root@node2 ~]# ip netns exec ns2 ip link set veth22 address 00:00:00:00:00:03
6 [$ C5 E2 e! ~9 O% q; q[root@node2 ~]# ip netns exec ns2 ip link set veth22 up- ], ~6 \, D/ Q4 b+ o0 D
[root@node2 ~]# ovs-vsctl add-port br-int veth21
% K8 J' G0 _* N8 B9 J6 T[root@node2 ~]# ip netns exec ns2 ip addr add 192.168.2.30/24 dev veth22
! \/ x% Z n' J7 S* w; u/ [[root@node2 ~]# ip netns
: a9 O3 `" J9 g7 x; Lns2 (id: 1)
" k7 [. D) y+ u' S/ ]+ M8 g# @ns1 (id: 0)- Y8 R+ x+ K2 M0 v. _$ p' e
) B. M$ H' i- N# w
node1上用ovn命令新增一个逻辑交换机，并配置好端口
, z1 o0 y( y# Q# O1 a h[root@node1 ~]# ovn-nbctl ls-add ls2
, h; s& m% [6 V+ b! D- |[root@node1 ~]# ovn-nbctl lsp-add ls2 ls2-node2-ns2. Z) N9 a4 x* J
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-node2-ns2 00:00:00:00:00:03* N- g8 f% c. e* I c
[root@node1 ~]# ovn-nbctl lsp-set-port-security ls2-node2-ns2 00:00:00:00:00:03" k$ t5 G/ j7 A) u9 z$ S
" A, y" c! c" h0 m( E3 W# anode2上ovs交换机端口和ovn逻辑交换机端口匹配起来
_. r% Y7 y8 l$ V& P$ s. F" u[root@node2 ~]# ovs-vsctl set interface veth21 external-ids:iface-id=ls2-node2-ns2
2 h+ p0 l8 J4 c. F4 ~+ C0 K& ]; x; E
查看北向数据库和南向数据库信息
, D. D0 Z- ~! Q3 s0 A- {4 L* |6 ^+ Y[root@node1 ~]# ovn-nbctl show
0 H5 N% G3 C) g3 |switch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2): @( V8 ^3 U$ i3 N
port ls2-node2-ns2
0 I4 g0 u! w8 V) @( O addresses: ["00:00:00:00:00:03"]
0 o9 G7 Y/ F- g+ I& Wswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)
5 G. E$ j3 W/ e4 Q port ls1-node1-ns1
$ K- G& T' K0 x; U addresses: ["00:00:00:00:00:01"]
7 S3 M7 x" S2 W# b; D0 l- m port ls1-node2-ns1
. ?. ]0 H' `8 s! L* c9 m4 t4 V addresses: ["00:00:00:00:00:02"]+ H' j' [2 J, N
[root@node1 ~]# ovn-sbctl show6 m( P( V& J+ S) S) F: n& p
Chassis node2: M0 ~. G1 @7 d7 b
hostname: node2
8 d$ [0 I4 F/ B% h, P3 F, m Encap geneve
, N: a1 C* J% w ip: "10.1.1.42"0 A: u% _) H( |3 Q6 ~# b* V& Y
options: {csum="true"}
4 I3 s1 @7 m( X6 d: R! a Port_Binding ls2-node2-ns2
' M) g p4 Z4 J E- s Port_Binding ls1-node2-ns1! k2 b5 K3 Y) E0 n1 R: t
Chassis node1
8 w7 ~) o* r4 l2 E hostname: node1+ e' b4 U. r2 a j* M+ i
Encap geneve
) A: v& i) Y- o2 d: B ip: "10.1.1.41"
% ]# |; U, Y# r2 z: \ options: {csum="true"}0 A. l7 L4 m2 d% F0 O: [
Port_Binding ls1-node1-ns1
4 I! p3 I+ L: r0 i; d7 l创建ovn逻辑路由器连接两个逻辑交换机$ m4 \, x0 V: f( s0 p* e5 t. o
+ \( a: v) ~2 \( w @. F添加逻辑路由器，路由信息保存在北向数据库
1 G# {# ~' ?* {! W1 _3 ?: O[root@node1 ~]# ovn-nbctl lr-add lr1
6 L" E" @' T7 p. _逻辑路由器添加连接交换机ls1的端口& J* d) a; g7 l8 w8 y5 |
[root@node1 ~]# ovn-nbctl lrp-add lr1 lr1-ls1 00:00:00:00:11:00 192.168.1.1/24
4 s, \: i6 ` G- S% S逻辑路由器添加连接交换机ls2的端口3 w2 n j" Q3 a0 e+ l1 }
[root@node1 ~]# ovn-nbctl lrp-add lr1 lr1-ls2 00:00:00:00:12:00 192.168.2.1/24
! h2 ]5 Y( o& Y* @7 N' u/ {( b
7 q& c& O# H7 P& { s2 M8 [逻辑路由器连接逻辑交换机ls1
" U6 Y$ S; h& K! ?, O[root@node1 ~]# ovn-nbctl lsp-add ls1 ls1-lr1" G5 ]7 z/ d8 f7 I( d% x
[root@node1 ~]# ovn-nbctl lsp-set-type ls1-lr1 router
; r3 G" J6 J; I1 Z[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-lr1 00:00:00:00:11:00
/ J4 k+ a' ]5 Y[root@node1 ~]# ovn-nbctl lsp-set-options ls1-lr1 router-port=lr1-ls1
k# b* e6 j' j: L: M7 r( G2 L; K( _; x; q! `0 _
逻辑路由器连接逻辑交换机ls29 ^7 F# J0 L) |
[root@node1 ~]# ovn-nbctl lsp-add ls2 ls2-lr16 \* L3 n; n3 @, z
[root@node1 ~]# ovn-nbctl lsp-set-type ls2-lr1 router
7 A4 e& d, g" o/ D& r2 y2 R[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-lr1 00:00:00:00:12:003 @. O& t" P1 S) M; Y, [4 e9 v
[root@node1 ~]# ovn-nbctl lsp-set-options ls2-lr1 router-port=lr1-ls2: W/ p8 z' U, q# L2 |. z3 v0 j
0 s; d, b ~; H \6 D: a6 H# H
查看北向数据库和南向数据库信息0 C/ H! K5 _/ W2 _8 H
[root@node1 ~]# ovn-nbctl show* a a) J: }& W* G
switch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2). U' ^* h4 J( \6 P B- ^/ G$ z% S
port ls2-node2-ns2
6 j: L# P+ B; I. w5 v, i addresses: ["00:00:00:00:00:03"]
, s8 t& ?, q3 Q2 d, B# J( S6 B port ls2-lr1
' Y1 q) G/ O, ~& B3 b% R+ r$ P type: router9 V6 a. D7 V! o& x
addresses: ["00:00:00:00:12:00"]. c6 u5 w& p, c4 G8 p
router-port: lr1-ls2
: L& \! U- s2 M7 L( K7 a' Aswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1) f- R0 @6 m! q% b; w9 ]) `/ V
port ls1-node1-ns1 ]: p0 d0 \" @6 l8 h
addresses: ["00:00:00:00:00:01"]
1 r( [* ?) u* V6 u port ls1-node2-ns1
9 ^" |- |: E- s$ T0 ~ t addresses: ["00:00:00:00:00:02"]
7 N7 X: e' [$ E. w2 J, R port ls1-lr1% D% R1 h- o6 z- ^* q
type: router
5 g+ E' X$ h: i: Q6 A addresses: ["00:00:00:00:11:00"]
; @4 @* M3 |: p% s1 m router-port: lr1-ls1
2 l) k H1 e7 S2 w P8 zrouter e9c151a0-5db7-4af6-91bd-89049c4bbf9f (lr1)" e4 Q9 F3 O8 `$ y
port lr1-ls2/ p8 ^5 y a; @+ l/ ]# E
mac: "00:00:00:00:12:00"
! e1 Y$ K U- ?5 t networks: ["192.168.2.1/24"]
! }7 e, i1 `7 B/ ~0 M Q port lr1-ls1- j% e8 h9 m/ I6 x, A$ }4 I1 w
mac: "00:00:00:00:11:00"; _' H* b& m/ k; l
networks: ["192.168.1.1/24"]
1 n0 x% l% M q/ r# I( j4 h[root@node1 ~]# ovn-sbctl show
6 T; z- C p4 M2 gChassis node2
5 h% C1 m7 G# H" R hostname: node22 B+ W7 g5 v0 C& P) |7 X# w6 O$ |
Encap geneve1 }) ^0 b, }" u1 R
ip: "10.1.1.42"' w& c( D# }7 H) ~; T6 `
options: {csum="true"}. }7 E0 F* H5 `! n
Port_Binding ls2-node2-ns2
# A$ ] r) S* U Port_Binding ls1-node2-ns1$ r% c! s {( q6 w. \* I6 j1 ]
Chassis node1 f( e0 `9 D9 I' U5 t2 B
hostname: node15 {/ {8 Y% u! e( c4 @* ~! t2 W
Encap geneve2 x% s( u: X5 K8 v9 k3 z
ip: "10.1.1.41"
" A& e: D' x, m options: {csum="true"}
0 h- Z! [+ v( i- Q+ c, P$ a8 b Port_Binding ls1-node1-ns1# K4 e/ w2 ^: b# ~8 L, X
在这里插入图片描述
+ r$ Y( {, ~. y6 y: G1 c6 i8 Y从node1的ns1(192.168.1.10/24) ping node2的ns2(192.168.2.30)，验证跨节点不同子网的连通性。
' r. l' ~" a' b) u# b5 J3 W9 e( j7 i) Y, Q8 ^4 X
[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.2.30/ K: i m+ o2 G) p8 z: l
connect: Network is unreachable connect: 网络不可达 u, t5 q0 ] U4 Q9 A3 G0 t
查看ns1上的路由配置，显然此时没有到192.168.2.0/24网段的路由' p0 W; u9 B+ u
[root@node1 ~]# ip netns exec ns1 ip route show6 R& q* O; D" P- H# e/ _
192.168.1.0/24 dev veth12 proto kernel scope link src 192.168.1.100 B# n; o8 `* V1 l9 U9 X4 |, d
[root@node1 ~]# ip netns exec ns1 route -n
8 A& \4 i* p9 HKernel IP routing table
3 p, i7 Z4 f6 ?( B# A7 B. UDestination Gateway Genmask Flags Metric Ref Use Iface8 p3 H. M4 N2 |: q4 n
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 veth127 X4 y/ d4 v# b2 `7 n' l/ [$ ]' L
因为路由器是三层概念，要先给ovs的相关port配置上ip
$ u- z. W" H+ G, P% H! f2 v9 V6 |. z. Q* R9 S
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node1-ns1 00:00:00:00:00:01
- |* K' F) d5 s# {9 Q3 b V2 Z- ?[root@node1 ~]# ovn-nbctl lsp-set-addresses ls1-node2-ns1 00:00:00:00:00:02$ i9 J. d, Z; T4 I9 w k5 Y. S6 J
[root@node1 ~]# ovn-nbctl lsp-set-addresses ls2-node2-ns2 00:00:00:00:00:03/ A( I" T# {; Q+ S
再给三个网络命名空间添加默认路由，网关为ovn逻辑路由器对应的port ip. f- `- G W, {- k
" I, b: R( d& a node1上ns1 C' I* n" O% `& l% I# g
[root@node1 ~]# ip netns exec ns1 ip route add default via 192.168.1.1 dev veth123 r7 b8 F% {; ^; U1 P
node2上ns12 @! y+ `% Q9 h7 {% J
[root@node2 ~]# ip netns exec ns1 ip route add default via 192.168.1.1 dev veth12
- B" e/ d* z, ~9 k2 z node2上ns2
$ b. x: C0 R' i! R# G2 J7 R! V g [root@node2 ~]# ip netns exec ns2 ip route add default via 192.168.2.1 dev veth22
5 H' e' a% _) D0 i/ ] [ ]4 l再次查看下南北向数据库信息
3 T- D: C! E+ e5 a) ?: w7 T0 ~. s/ Q+ g9 e1 `1 ]! t4 a
[root@node1 ~]# ovn-nbctl show
4 q( K$ g5 B, S* X# ^3 Tswitch 484606e0-944d-4c6b-9807-502f05bebb18 (ls2)
/ U% t( m0 h' X3 V/ h& w. S$ p8 P port ls2-node2-ns2
. Z: C9 ]1 @# x) W% c addresses: ["00:00:00:00:00:03"]
/ e. G- J' b- k9 H" F5 P5 T port ls2-lr1
2 d2 \$ ~* G/ C" h3 ?' g/ |$ { type: router; W; b- V! G% i+ K
addresses: ["00:00:00:00:12:00"]
- Z4 C/ a1 F5 L router-port: lr1-ls2
3 M3 y/ S" V" n5 B* ^# K9 Sswitch 86349e35-cdb4-42f7-a702-4b4a9d5653ef (ls1)) I+ K5 n! A- Y0 T
port ls1-node1-ns1
" ?) X X' m, v addresses: ["00:00:00:00:00:01"]. @ G6 a( s( G j+ c3 F- P2 c
port ls1-node2-ns15 K" @, S8 B5 P2 i
addresses: ["00:00:00:00:00:02"]
" o* J4 G& I7 x# y: S port ls1-lr1
) X% P/ S; o: c9 z type: router: }% y0 @( f' R( K {" H7 Q3 i
addresses: ["00:00:00:00:11:00"]
1 K: ]2 i1 S0 d9 p router-port: lr1-ls1* D& d4 `: o- f- C- s+ {: v
router e9c151a0-5db7-4af6-91bd-89049c4bbf9f (lr1)
! B$ v, s0 }& z2 z! w8 _) Q port lr1-ls25 \$ v& e7 _2 X
mac: "00:00:00:00:12:00": c9 ?: c4 \; p2 Q3 f" i
networks: ["192.168.2.1/24"]
- f9 p k# M+ Y5 @: @5 i" e port lr1-ls1
( J1 P/ m. v9 j i) C. b+ K mac: "00:00:00:00:11:00"
9 J/ G/ ?8 q- C# I3 {1 _8 k/ B0 D networks: ["192.168.1.1/24"]
4 ?# r$ Q+ y1 h4 l$ \! i9 K[root@node1 ~]# ovn-sbctl show3 |, D* P; r% J( A* o
Chassis node2
/ H* y4 \4 e3 \: i D hostname: node2
; x: W m8 N6 P8 g6 G Encap geneve
9 I8 G G. U$ H- g0 u! Z ip: "10.1.1.42"4 k/ H& o) o4 ~
options: {csum="true"}* K# V E& K: T# m5 N/ c; z
Port_Binding ls2-node2-ns2" Z) @; @$ [. d `* g5 `! g+ Y2 O* C
Port_Binding ls1-node2-ns1
) {5 D, p' g' ~( L d% R8 L2 l) XChassis node1
* `9 M/ E$ x' Y4 [8 a hostname: node1: n# Q" N3 C3 q4 R
Encap geneve2 }' y: @/ _8 Z2 {: Q& d
ip: "10.1.1.41"2 O1 ?0 O3 ^+ X% e- M
options: {csum="true"}
0 ]2 D. |" S/ Z0 H! @ Port_Binding ls1-node1-ns1& \9 Z! [! `8 n& A
在这里插入图片描述) y5 Z% y6 n! J- {9 ~
验证网络连通性
2 M6 h9 W( m/ E- \; R K, Q; B. ~# R6 S! i
node1上ns1连通网关
! {5 P5 ~4 F0 b) \1 e$ z[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.1.1' }- D- ~, O; _6 Z4 d2 K" F5 O
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.3 \6 n- C6 E0 [: p% M
64 bytes from 192.168.1.1: icmp_seq=1 ttl=254 time=20.10 ms
6 D- e8 N) ]5 N$ D$ G, N/ L- Q& c- [7 i3 Y& f; G" w5 s* x5 i, v! x" x
--- 192.168.1.1 ping statistics ---
2 } A8 C: p; ~. S1 packets transmitted, 1 received, 0% packet loss, time 0ms) t8 x: z$ p6 G% w
rtt min/avg/max/mdev = 20.950/20.950/20.950/0.000 ms6 _' x5 o7 ^; V
6 G4 ~3 x' f0 g
node2上ns2连通网关3 Z- m* I; C8 l- c2 Y/ j
[root@node2 ~]# ip netns exec ns2 ping -c 1 192.168.2.1/ H" I& O& \! a4 I. G' P
PING 192.168.2.1 (192.168.2.1) 56(84) bytes of data.
' S. d: w8 x. B2 t64 bytes from 192.168.2.1: icmp_seq=1 ttl=254 time=38.5 ms
4 e& j8 h. V; y3 r. f5 i
- J4 {# ?, N. F+ s3 u) {--- 192.168.2.1 ping statistics ---
+ ?3 Z3 i+ z8 F# H1 D1 W7 H1 packets transmitted, 1 received, 0% packet loss, time 0ms
' K: V' q' P8 v5 [5 a) C& brtt min/avg/max/mdev = 38.477/38.477/38.477/0.000 ms
( l2 F8 U0 I( a* V6 w; _
( \2 C9 d4 x4 M- H7 ~0 P/ onode1上ns1 ping node2上ns2
8 w5 t! } L% p S" W. q! P( {- I[root@node1 ~]# ip netns exec ns1 ping -c 1 192.168.2.30! x' N* F7 ~9 n6 k
PING 192.168.2.30 (192.168.2.30) 56(84) bytes of data.' u& `+ I# k; a, i: X$ X
64 bytes from 192.168.2.30: icmp_seq=1 ttl=63 time=1.23 ms
7 y1 X; x' V3 N+ p5 y9 e4 a
8 O) {0 A) w. n- }) l" W4 B--- 192.168.2.30 ping statistics ---
# Q0 J( n* u3 d. E. w+ e1 packets transmitted, 1 received, 0% packet loss, time 0ms$ A; \5 ^7 u V* z1 ]; c
rtt min/avg/max/mdev = 1.225/1.225/1.225/0.000 ms
* `8 T" {8 S, }( k* j复制
; _) D- Z- i5 `2 O# ~5 @注意：ovn逻辑交换机/逻辑路由器是北向数据库概念，这两个逻辑概念经过ovn-northd“翻译”到了南向数据库中，再通过hypervisor上的ovn-controller同步到ovs/ovsdb-server，最终形成ovs的port和流表等数据。- h$ ~# |7 m0 _2 r! S9 A# h& p
ovn逻辑交换机通过geneve隧道，把二层广播域扩展到了不同主机上的ovs；而ovn逻辑路由器则是把三层广播域扩展到了不同主机上的ovs，从而实现跨主机的网络通信。3 q5 W) u! Q" Z% U, }& |, G
ovn逻辑交换机和逻辑路由器都会在所有的hypervisor中生成对应的流表配置，这也是ovn网络高可用以及解决实例迁移等问题的原理。 |
|
发表于 2025-12-18 08:50:57
3 e+ c ^5 S4 F
: i% L8 z3 \2 Z$ s
2 X4 N2 J" S+ l+ e0 h' V
0 T7 J% h: d; M; G' J
0 C [- A/ n; n! a" l+ f: y
- v8 V" e8 ~$ N; m; o
& e% B0 z/ f6 s% g8 f+ M
# h1 F" q- G, h
( f: C# r; ~; T$ U, g9 h
' L+ O8 n/ V0 n
! L9 w3 J( |4 J/ f6 N
) }# f( v9 F8 N; v
2 [' @) o# g9 X; G7 x
+ u1 j) G: h- w" Q) H
0 c, s7 _1 ?( F, o" c) L- v" f1 J& j) R