|
|
27 Nov 2016 8 min read 9 Comments SDN
7 v+ z2 c8 g- D/ n! w5 MVanilla OpenStack networking has many functional, performance and scaling limitations. Projects like L2 population, local ARP responder, L2 Gateway and DVR were conceived to address those issues. However good a job these projects do, they still remain a collection of separate projects, each with its own limitations, configuration options and sets of dependencies. That led to an effort outside of OpenStack to develop a special-purpose OVS-only SDN controller that would address those issues in a centralised and consistent manner. This post will be about one such SDN controller, coming directly from the people responsible for OpenvSwitch, Open Virtual Network (OVN).
2 g( p5 k) ?2 Q) A7 B$ o% O4 m
( M. S: c" c3 v) X! F/ l" H9 ]OVN quick introduction: E' z7 C) q0 o3 H; I3 a
OVN is a distributed SDN controller implementing virtual networks with the help OVS. Even though it is positioned as a CMS-independent controller, the main use case is still OpenStack. OVN was designed to address the following limitations of vanilla OpenStack networking:+ y3 {# `8 u8 w _& _
- L& |: y5 W8 A* c) [0 w8 \+ PSecurity groups could not be implemented directly on OVS ports and, therefore, required a dedicated Linux bridge between the VM and the OVS integration bridge.
" l" v/ q3 Y/ WRouting and DHCP agents required dedicated network namespaces.6 { u1 Y! x3 Q& e0 o
NAT was implemented using a combination of network namespaces, iptables and proxy-ARP.0 g& F8 h% C# d1 G2 E2 }) {* \
OVN implements security groups, distributed virtual routing, NAT and distributed DHCP server all inside a single OVS bridge. This dramatically improves performance by reducing the number of inter-process packet handling and ensures that all flows can benefit from kernel fast-path switching.4 f/ d2 U1 _9 \& M1 v2 I3 o
# _# d/ |' {! u# z8 F1 \4 }
At a high level, OVN consists of 3 main components:
3 b% ^8 K# W9 z) Q/ f& k! T% \. U& |8 q; F3 ?5 z
OVN ML2 Plugin - performs translation between Neutron data model and OVN logical data model stored in Northbound DB./ `; u9 c% }; s; W$ m) L
OVN northd - the brains of OVN, translates the high level networking abstractions (logical switches, routers and ports) into logical flows. These logical flows are not yet OpenFlow flows but similar in concept and a very powerful abstraction. All translated information is stored in Southbound DB.
+ [5 q" ?1 N: `OVN controllers - located on each compute node, receive identical copies of logical flows (centralised network view) and exchange logical port to overlay IP binding information via the central Southbound DB. This information is used to perform logical flow translation into OpenFlow which are then programmed into the local OVS instance., R( [: N' `, W7 G
/ D" G$ o' y6 H# k- l; B3 K
$ Z& ]/ Z8 V& H$ {If you want to learn more about OVN architecture and use cases, OpenStack OVN page has an excellent collection of resources for further reading." B8 ^' J3 X, [* S0 x
5 \7 R1 d8 g4 Q f7 N& h- aOpenStack installation
: e7 |5 [4 a/ GI’ll use RDO packstack to help me build a 1 controller and 2 compute nodes OpenStack lab on CentOS7. I’ll use the master trunk to deploy the latest OpenStack Ocata packages. This is required since at the time of writing (Nov 2016) some of the OVN features were not available in OpenStack Newton." y8 T& j; n% B, o& _ g5 ?
- {/ f+ g; ?# I* T
cd /etc/yum.repos.d/' n: y. Y' e& m0 |2 L+ s4 k
wget http://trunk.rdoproject.org/centos7/delorean-deps.repo0 P8 ^$ ?5 ?$ n3 ^2 r" r
wget https://trunk.rdoproject.org/centos7-master/current/delorean.repo. k: f6 O+ p: ?3 J0 ~- N
On the controller node, generate a sample answer file and modify settings to match the IPs of individual nodes. Optionally, you can disable some of the unused components like Nagios and Ceilometer similar to how I did it in my earlier post.
: w7 v# `5 U; D9 l9 Z' a" m6 _1 m, ^, z
9 j% j7 q1 `2 O& Z% ryum install -y openstack-packstack crudini( i2 K, n- c( e) ^1 f
packstack --gen-answer-file=/root/packstack.answer
/ V7 }, Z$ ]. ?* f U$ I. ~: \8 R% ycrudini --set --existing defautl CONFIG_COMPUTE_HOSTS 169.254.0.12,169.254.0.13
: `$ W2 ?9 A( ?; pcrudini --set --existing defautl CONFIG_CONTROLLER_HOST 169.254.0.11
9 d1 g( O c( B( i2 Tcrudini --set --existing defautl CONFIG_NETWORK_HOSTS 169.254.0.119 S1 g c/ p2 T; P" w& z1 b- v
packstack --answer-file=/root/packstack.answer2 E" u- ~+ q9 ]
After the last step we should have a working 3-node OpenStack lab, similar to the one depicted below. If you want to learn about how to automate this process, refer to my older posts about OpenStack and underlay Leaf-Spine fabric build using Chef.$ h* z5 f U3 ]
) @7 H5 Z+ _6 q! u# @
& S+ }: E$ t9 x7 |# u8 M6 K0 k! m6 ?1 A- B8 L, U# c
OVN Build
8 z& Z: k: E! [* ?: `OVN can be built directly from OVS source code. Instead of building and installing OVS on each of the OpenStack nodes individually, I’ll build a set of RPM’s on the Controller and will use them to install and upgrade OVS/OVN components on the remaining nodes.
# q+ Z! d2 a- s4 U" O8 x- d& p: ^
" @. c* W5 Y; hPart of OVN build process includes building an OVS kernel module. In order to be able to use kmod RPM on all nodes we need to make sure all nodes use the same version of Linux kernel. The easiest way would be to fetch the latest updates from CentOS repos and reboot the nodes. This step should result in same kernel version on all nodes, which can be checked with uname -r command.$ E: M6 X! E& @* L+ C
N$ P* W3 v. D0 u5 ?" h$ V
yum -y update kernel
, G: D3 {; N) d. a! [reboot
* M$ E" }4 S$ l0 Q5 i4 q* O: a" qThe official OVS installation procedure for CentOS7 is pretty accurate and requires only a few modifications to account for the packages missing in the minimal CentOS image I’ve used as a base OS.0 W! @) J; s* P( |: g% ?/ i- C+ i
+ t/ r( t6 D: R) g5 O9 Gyum install rpm-build autoconf automake libtool systemd-units openssl openssl-devel python python-twisted-core python-zope-interface python-six desktop-file-utils groff graphviz procps-ng libcap-ng libcap-ng-devel
6 n' T; @3 S$ ~; `- X3 e: E# H* d3 ?0 t' r T
yum install selinux-policy-devel kernel-devel-`uname -r` git
! O) e% L3 @. q7 v6 V w
1 R% y( [! d y! K& [git clone https://github.com/openvswitch/ovs.git && cd ovs+ E- U" D: D% A5 s. c O' e: \
./boot.sh
. s1 J8 _6 p# B. C4 V2 w1 V. o./configure
y; P @% G: }, A3 \; {* ^4 y3 n+ Mmake rpm-fedora RPMBUILD_OPT="--without check"( c4 o* q; R7 S7 S
make rpm-fedora-kmod" D% f+ q7 \) L7 n( X/ Z' z- _
At the end of the process we should have a set of rpms inside the ovs/rpm/rpmbuild/RPMS/ directory.+ K7 D7 s1 y4 f |
4 M9 o1 Q! u' d7 U' g/ e! NOVN Install; c$ a4 |- z3 ~0 d
Before we can begin installing OVN, we need to prepare the existing OpenStack environment by disabling and removing legacy Neutron OpenvSwitch agents. Since OVN natively implements L2 and L3 forwarding, DHCP and NAT, we won’t need L3 and DHCP agents on any of the Compute nodes. Network node that used to provide North-South connectivity will no longer be needed.
, F8 h( |. c4 `+ |6 U) [- m1 t+ x1 L: p! w
OpenStack preparation4 ]) {3 g/ {: }8 K% B3 k, S
First, we need to make sure all Compute nodes have a bridge that would provide access to external provider networks. In my case, I’ll move the eth1 interface under the OVS br-ex on all Compute nodes.
& z! Z5 g* Q5 s; X/ A8 u
" J7 H! b+ ~6 ?$ c* @1 Z) qDEVICE=eth1
0 i6 T5 ]" W: fNAME=eth1* q" n- D- N Y3 x
DEVICETYPE=ovs
! A" c1 m6 s( r: a dTYPE=OVSPort
2 X! t* T3 q0 Z% S+ `2 tOVS_BRIDGE=br-ex
' i h% ~4 i( _ONBOOT=yes
& z& v# O; K! ?! U5 k# fBOOTPROTO=none
' d! ]5 N8 p2 ?$ R6 ]2 W7 s& w; XIP address needs to be moved to br-ex interface. Below example is for Compute node #2:
) P* W9 n9 X: j! V- S3 o; G5 N5 J3 q* R. m8 S
ONBOOT=yes
7 @% D! f" Q8 s0 s- [DEFROUTE=yes
8 |' N1 d% |' x2 [, {. E) OIPADDR=169.254.0.124 X; C+ l3 P" m) f' h2 D
PREFIX=24
) _5 Y! Z+ M3 E0 r) ^GATEWAY=169.254.0.1
! H$ k9 `* Q# ]. x2 ]+ g( lDNS1=8.8.8.8$ n6 g% q% r6 K5 a7 t$ V& w
DEVICE=br-ex
& ]4 z/ N" P: n/ @NAME=br-ex
9 b) x' X% t8 j8 f6 ^ iDEVICETYPE=ovs
+ S) f5 e, Q" I0 J* hOVSBOOTPROTO=none
! @6 T+ W& y7 |" lTYPE=OVSBridge, V5 F( s( o( A7 h1 k0 v8 c
At the same time OVS configuration on Network/Controller node will need to be completely wiped out. Once that’s done, we can remove the Neutron OVS package from all nodes.
- j6 ~' J7 }6 f, D4 s6 j% ~( U+ T; {1 }
yum remove openstack-neutron-openvswitch" ^# r9 `" _8 i7 V
OVS packages installation2 l& Z L3 i, O' F6 F* H7 N
Now everything is ready for OVN installation. First step is to install the kernel module and upgrade the existing OVS package. Reboot may be needed in order for the correct kernel module to be loaded.% M2 c }9 O x. e5 ~. y
3 e" p9 @, b4 d6 I8 r: L" }; ?$ Z
rpm -i openvswitch-kmod-2.6.90-1.el7.centos.x86_64.rpm
. D' z. p. N# N1 arpm -U openvswitch-2.6.90-1.el7.centos.x86_64.rpm# V' k; G: T0 ~7 Q+ ^' O
reboot
& k. F/ p q5 V: r- l/ ?7 z& `Now we can install OVN. Controllers will be running the ovn-northd process which can be installed as follows:
% d* J) f- \$ B8 f
! n- ]3 G8 i2 t S9 Drpm -i openvswitch-ovn-common-*.x86_64.rpm8 d: w f% [; E4 g/ Z* l
rpm -i openvswitch-ovn-central-*.x86_64.rpm9 T4 S5 e1 _% v) X/ }5 u
systemctl start ovn-northd8 K1 h& R6 y* Z
The following packages install the ovn-controller on all Compute nodes:
7 L1 w. p" o3 V2 k4 M# K, m5 O9 a- G
rpm -i openvswitch-ovn-common-*.x86_64.rpm1 o v- J+ A5 X4 A; T) o& c
rpm -i openvswitch-ovn-host-*.x86_64.rpm% U9 \$ U2 t3 B5 n+ g! \) Q
systemctl start ovn-controller: w) J* |7 f. M4 P6 q& X: k
The last thing is to install the OVN ML2 plugin, a python library that allows Neutron server to talk to OVN Northbound database.) F7 B6 H9 D' p! P( O, L, ?
7 |' s5 S3 V3 r4 R6 U& i6 S
yum install python-networking-ovn
$ M9 ]$ Y, p+ O* [. {! EOVN Configuration
( a( N6 {7 C& } wNow that we have all the required packages in place, it’s time to reconfigure Neutron to start using OVN instead of a default openvswitch plugin. The installation procedure is described in the official Neutron integration guide. At the end, once we’ve restarted ovn-northd on the controller and ovn-controller on the compute nodes, we should see the following output on the controller node:" d- v! C% `. x: k8 [
" C1 q0 l$ U/ |$ ovs-sbctl show
' a% R6 J! `9 _, f; e+ m% r% G% |Chassis "d03bdd51-e687-4078-aa54-0ff8007db0b5"+ @6 w0 Y$ F! \% J& @( v
hostname: "compute-3"
; k3 `6 t! _3 x6 t* l Encap geneve
5 q: Z. t- b0 D% ` ip: "10.0.0.4"
& S' e" f. `. V" ]1 H options: {csum="true"}* K% K7 q4 W6 Y% Q
Encap vxlan& j3 s, K( t$ g1 f( L1 H! f' Q
ip: "10.0.0.4"9 L6 z& g, W( {
options: {csum="true"}' u: S. J% X! j- m D- V V
Chassis "b89b8683-7c74-43df-8ac6-1d57ddefec77"8 d+ {1 _; m+ T, v8 D" M! @7 m" V: p
hostname: "compute-2"5 v2 z) U$ ?) ?! q$ M/ L+ w) o2 N
Encap vxlan
$ Q# q, V5 y+ ?8 x+ }/ Z f ip: "10.0.0.2"! z" m& J; O8 O x( c
options: {csum="true"}
: D, F# o' p. X1 O7 N1 ` G Encap geneve
/ F. v1 M* ?# y ip: "10.0.0.2"
7 u" @6 O: w, }( C- Y, H: L j$ Q0 W options: {csum="true"}' I, V! d$ V; Q# F
This means that all instances of a distributed OVN controller located on each compute node have successfully registered with Southbound OVSDB and provided information about their physical overlay addresses and supported encapsulation types. w! \& p6 A, ^& r" P
. o/ Q% O }8 \2 p: A: h
(Optional) Automating everything with Chef4 e2 K9 \: ]) O5 {
At this point of time there’s no way to automate OVN deployment with Packstack (TripleO already has OVN integration templates). For those who want to bypass the manual build process I have created a new Chef cookbook, automating all steps described above. This Chef playbook assumes that OpenStack environment has been built as described in my earlier post. Optionally, you can automate the build of underlay network as well by following my other post. Once you’ve got both OpenStack and underlay built, you can use the following scripts to build, install and configure OVN:
" d& ^* J S8 O ]0 \ W1 r& s/ C5 D9 E$ R% a+ u+ \9 `
git clone https://github.com/networkop/chef-unl-os.git9 s- D; j( M, n8 w- V0 ~
cd chef-unl-os
* C3 ^- }7 |) [ X( I7 `chef-client -z -E lab ovn.rb
, A" D( w p9 g) W, lTest topology setup0 X$ `# R0 z7 a" C6 z
Now we should be able to create a test topology with two tenant subnets and an external network interconnected by a virtual router.+ V1 o) X5 h* O4 e: S3 A
# C7 H( ~' @6 [- N6 Mneutron net-create NET-RED
6 o: J1 q6 U& I, c8 D3 ]. Cneutron net-create NET-BLUE
# ~- ?. G1 D0 a+ }neutron subnet-create --name SUB-BLUE NET-BLUE 10.0.0.0/24! P8 C) O! s+ _
neutron subnet-create --name SUB-RED NET-RED 20.0.0.0/247 E3 y; i) L# J8 x2 v
neutron net-create NET-EXT --provider:network_type flat \
% g: E& `# n9 x1 G: V- l --provider:physical_network extnet \% s* B( @. x& ?
--router:external --shared7 z5 n: z9 c3 J7 t. H R" ]8 ^5 V( P
neutron subnet-create --name SUB-EXT --enable_dhcp=False \4 {) u3 Q4 W8 N; D7 ?' w
--allocation-pool=start=169.254.0.50,end=169.254.0.99 \
4 c# A0 ]# A1 M7 z3 X' |, e5 i4 p7 F --gateway=169.254.0.1 NET-EXT 169.254.0.0/24
5 `$ |. I: M |* Oneutron router-create R1
* j) g1 e7 ?0 H$ X* T+ m/ g/ y1 {3 N- g- Sneutron router-interface-add R1 SUB-BLUE1 A/ Y+ f1 f3 I
neutron router-interface-add R1 SUB-RED2 \, b0 E3 j, M( N
neutron router-gateway-set R1 NET-EXT5 _7 |$ {# Y- D0 a# H/ x. r% i
When we attach a few test VMs to each subnet we should be able to successfully ping between the VMs, assuming the security groups are setup to allow ICMP/ND.
- l2 A E: s& u% O `, }& n! |+ G+ l- l0 [- p
curl http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img | glance \
- O& `! M+ A* _! `5 fimage-create --name='IMG-CIRROS' \& |4 T* Z2 O5 k3 x* d
--visibility=public \" Z4 f7 r$ l" t5 y: R4 t; {
--container-format=bare \
8 S2 `& q0 a7 y8 C c5 J --disk-format=qcow2) H: A7 y4 t* |4 t
nova aggregate-create AGG-RED AZ-RED% M& N0 s0 Y0 a0 Y+ B' ~. V0 Q
nova aggregate-create AGG-BLUE AZ-BLUE9 J3 ]) _% [( y* Q! V+ T3 Z6 E: j
nova aggregate-add-host AGG-BLUE compute-21 e; ]) I! A5 n6 O* A
nova aggregate-add-host AGG-RED compute-3
& d Z5 ^+ P h3 P" s) z$ Ynova boot --flavor m1.tiny --image 'IMG-CIRROS' \# W% t2 p9 H8 q" K0 J( a
--nic net-name=NET-BLUE \0 W7 O; x2 K7 F# N
--availability-zone AZ-BLUE \
8 @% D& W1 w; C6 X) M# L VM12 e0 E: C H$ {9 i b) W0 M* r
- ]$ J, X2 m# T7 T# {- p9 _5 knova boot --flavor m1.tiny --image 'IMG-CIRROS' \
8 A" Z7 L( I) g" j --nic net-name=NET-RED \
+ U+ N$ x& O$ O; P i8 y' W5 V --availability-zone AZ-RED \' i$ O: g1 W9 _2 F4 k
VM2
! O I6 Y: y( A7 u6 i/ D' g0 fnova boot --flavor m1.tiny --image 'IMG-CIRROS' \2 n1 [7 N' e* q% s: a p& g
--nic net-name=NET-BLUE \( e$ j' r( O9 ?7 }# F: _
--availability-zone AZ-RED \& K6 i" k+ ^& S; E; K& a0 R% }
VM3
1 m2 c: ?) g/ X- y& zopenstack floating ip create NET-EXT
- O% h; w2 `$ M& w% p6 gopenstack server add floating ip VM3 169.254.0.53
) p! a" s! R( ]3 J$ ^
& @1 O% a4 I# }& Q! z/ P6 _1 y
$ M! O6 n$ i0 ~0 Z* P3 \ jIn the next post we will use the above virtual topology to explore the dataplane packet flow inside an OVN-managed OpenvSwitch and how it uses the new encapsulation protocol GENEVE to optimise egress forwarding lookups on remote compute nodes.
; l R3 J1 h5 @' p: N, v6 ?8 Z$ m# T' s0 _
OpenStack-SDN OVN
: f+ j- w6 n8 l. ?5 ^: p( F$ E, Y% LRelated' f! `1 r2 s7 O; c) B& D
OpenStack SDN - Distributed Virtual Routing
- |5 _# p$ l' h; O4 b3 T2 aAutomating the Build of OpenStack Lab (Part 2)
$ y$ s6 ]& @% } H! gAutomating the Build of OpenStack Lab (Part 1)
( Q. X0 }3 I3 K9 y) mOpenStack SDN - Interconnecting VMs and Physical Devices With Cumulus VX L2 Gateway# P' b% l0 l/ Z& G% P
OpenStack SDN - Extending a L2 Provider Network Over a L3 Fabric |
|
发表于 2018-12-20 01:59:11