问题情况
( P7 R0 O7 c, e' {3 |openstack xina版本创建虚机后,虚机在dashboard上获取到ip地址了,但打开虚机控制台之后,使用ip add 检查网络状态时,虚机内部并未获取到ip地址:
: B& D; ~! }/ r% x" Y t T
/ T* m% R) I0 E8 ^ N[td][tr][/tr]| 正在显示 1 项 |
( x( P7 L Z3 u& r( \# T | Instance Name | Image Name | IP Address | Flavor | Key Pair | Status | 4 F( z# I) Z6 a7 F {3 ? x
| Availability Zone | Task | Power State | Age | Actions | ; W" Q2 C4 D c+ c% I
| m2 | CentOS-7.9 |
0 n; A" i/ U+ z: W f( ?
8 J" l7 \4 B$ I+ H) r+ \, E- B3 ~" w1 Y5 v
3 ?6 p/ b3 t5 V5 {( {: W* d5 f3 P/ h' [, m B' F7 a
172.168.10.101
| m2 | - | 运行 | | nova | 无 | 运行中 | 12 小时,14 分钟 | 7 ^( Z% ^1 M% u8 K
|
* o0 K! F( z, ~4 T9 A+ e
$ c/ a3 ]0 ~( V7 s/ N. C* n5 _) g" v, a" E7 ?
分析排查思路:+ Q, @& f! C3 ]6 I2 C. M; G S
% z+ E) C' m5 m* n9 a(1)检查neutron服务状态,确保dhcp服务正常运行:
8 b/ c) V+ N+ B' H7 r$ W! E+ D4 D& I! _+ a
[root@controller ~]# neutron agent-list * j ?; ~' G( k7 j+ S' }( b
neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.: n3 q% ^2 |: G2 Q
+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+
- v5 S- i% t( [8 I( Q& A8 H$ i5 C| id | agent_type | host | availability_zone | alive | admin_state_up | binary |+ r6 d8 E0 u0 _: t7 N
+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+4 k) R8 `5 @9 f( @+ V* U2 O( P
| 133d6414-7d3c-42f5-8422-90ab1c7f3721 | L3 agent | controller | nova | :-) | True | neutron-l3-agent |
0 k" ^4 N7 w: i/ N# I6 y4 C| 2bfc7c83-94aa-4fdc-b7e2-055bb8db0f10 | Open vSwitch agent | compute01 | | :-) | True | neutron-openvswitch-agent |
% U& o- J& o/ `9 k| 4164d4b2-04f8-4d78-b514-351b1205d3ce | Metadata agent | controller | | :-) | True | neutron-metadata-agent |
/ {1 P/ l" H/ A9 a% X| 53fa495d-8039-4580-b1cc-20414ef1303d | Open vSwitch agent | controller | | :-) | True | neutron-openvswitch-agent |* t4 }& [! M4 E& D" s
| ef59abb4-35d0-48c6-876e-983ed713e2d4 | DHCP agent | controller | nova | :-) | True | neutron-dhcp-agent |4 P$ N* I0 _, M* h4 {7 H* f
+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+, ~5 n: y' K4 z" s
" `0 E! L0 E4 Z3 o. t- ~: f' N5 G; Y, K- Y
(2)查看dnsmsp进程:1 j! ?$ B4 t1 H9 G3 u
5 w2 E- m3 Q, ^' T4 q2 s, j
[root@controller ~]# ps -ef |grep dnsmasq
& C* X5 h, ^* n0 ]% } Sdnsmasq 3548 1 0 07:52 ? 00:00:00 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/host --addn-hosts=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/opts --dhcp-leasefile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/leases --dhcp-match=set:ipxe,175 --dhcp-userclass=set:ipxe6,iPXE --local-service --bind-dynamic --dhcp-range=set:subnet-ab92c638-b52e-4c32-8675-38b24f608b55,172.168.16.0,static,255.255.252.0,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=1024 --conf-file=/dev/null --domain=openstacklocal: H. B3 j4 f% z6 x8 w* Y
dnsmasq 3553 1 0 07:52 ? 00:00:00 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/host --addn-hosts=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/opts --dhcp-leasefile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/leases --dhcp-match=set:ipxe,175 --dhcp-userclass=set:ipxe6,iPXE --local-service --bind-dynamic --dhcp-range=set:subnet-e7722a92-a4ab-439c-b7af-129133c310b2,172.168.8.0,static,255.255.248.0,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=2048 --conf-file=/dev/null --domain=openstacklocal0 c1 j2 G) B4 m" l$ p3 p8 n
root 5024 2518 0 08:15 pts/0 00:00:00 grep --color=auto dnsmasq% e: P8 D7 A2 o/ F" d9 R
F0 S/ o& l2 t# i3 S, I) U
(3)检查ovs网桥中的 br-int 集成网桥是否有 tap口设备 连接到了dchp-agent 的 namesapce上
* `0 f' j3 x: i4 H0 w+ r) k
4 I M# }6 l, Y6 E% v, q. n c U1 m1 P5 {5 P! w
[root@controller ~]# ovs-vsctl show
$ a) u( p' z4 H$ n/ }* r0 Y& z04659b20-7658-4782-abe5-84ee5f33282f% Z3 C/ D4 D3 u' ~
Manager "ptcp:6640:0.0.0.0"! ~0 S+ Z. Z0 i1 `
is_connected: true
! I9 F* Z2 r5 }# r Manager "ptcp:6640:127.0.0.1"
9 z; K4 Q1 m B# d' J7 Q3 { Bridge br-tun9 O' X. a( p/ A) `! J
Controller "tcp:127.0.0.1:6633"
( J6 {2 T! Z6 q% e: X% V$ z9 v is_connected: true
/ F# f; ?6 H2 o4 T& I" t fail_mode: secure/ E8 C/ r( M* N& C/ K! r* J
datapath_type: system, X, d. y3 s- x5 U! [6 Y
Port br-tun
- V& ~4 X) f x4 b# B# B Interface br-tun
9 }: }9 j* h+ a, o/ V% }+ a& G type: internal
8 Y$ k: R1 o+ j Port patch-int
; g3 [% {3 G, I. ~; d8 X Interface patch-int& D7 l1 |! V" r! g \
type: patch/ n) y6 N1 F4 V( y1 ^
options: {peer=patch-tun}+ i$ o5 U8 }( ~3 a6 z) g6 ^4 J" ^5 [4 G
Bridge br-int
! `4 w# @% @7 H L0 Z- x Controller "tcp:127.0.0.1:6633"
! a' ]0 i+ J7 ~* I+ P! b5 F% P8 C: ^ is_connected: true
4 J: a: ^% p/ K) Y fail_mode: secure# n- s9 j5 s8 g. w' l" c8 m3 X9 L# M# C
datapath_type: system
# r/ ^. p9 t* D+ a9 Z& l& |, e Port patch-tun- K& v( W; S# L6 l4 u
Interface patch-tun
# d: V7 g4 W0 d3 I$ _, Z/ u- Q type: patch0 v- R" q9 a: L& [. |/ Z
options: {peer=patch-int}1 w* y6 |4 X0 a) L
Port tapd2a5f73d-5b0 U) f: r! U3 J* l( ]1 G
tag: 2" I- g. C' [2 W" O# i' u& V
Interface tapd2a5f73d-5b; r: b4 f! B0 p( s$ O3 I- b4 [
type: internal
e( S4 q: ?5 ^* `% ]1 W6 A Port tapcee79ebe-a59 g+ g/ ?0 u' C7 j, J8 T+ V
tag: 1
9 m2 r3 k& h) l" e Interface tapcee79ebe-a5
0 {8 T# u7 M$ e9 h$ X" Y type: internal
9 J* N# M. W8 ^6 ` Port br-int& V1 G! r% O. ]5 b: R
Interface br-int
3 Y2 g/ _1 X+ U* r' w) { type: internal
" F$ B' [2 ~. D/ D# A* | Port int-br-ex
6 X- `* W+ Z: g Interface int-br-ex4 G u; R. M& [6 ]7 j( K1 {; J' k
type: patch4 P* v* M$ v9 F) \/ P
options: {peer=phy-br-ex}& o ]' e* _) _! c0 y
Bridge br-ex
+ o N+ m9 r- B) y3 O Controller "tcp:127.0.0.1:6633"
' Q2 G0 B$ l0 C7 ]# f( P is_connected: true
7 A) i# u1 |, u( } fail_mode: secure
7 f2 ?1 @( b) B. G datapath_type: system
3 |0 z. S' w1 a Port phy-br-ex
+ {5 d3 ?+ m$ h, I P4 K G7 h9 d Interface phy-br-ex( g3 `3 _- B, x+ O. |8 r7 o
type: patch$ m2 h$ [0 S3 L1 z% \5 e8 l
options: {peer=int-br-ex}
, S$ \4 m8 H( L/ S7 t Port enp7s0f05 g& y$ F) n5 t( y% _8 e: k7 A
Interface enp7s0f09 L- v" B7 n- r, x( I; c
Port br-ex& n/ Q4 P6 M( n [2 L9 m! E3 U# F! G
Interface br-ex
, R1 w) a# @. c0 L' o8 X$ x- { type: internal% e" Z- T2 `! ^ p- O" J v
ovs_version: "2.15.4"
' o8 g) Q1 A4 J8 R( k
7 k- d' T+ w9 R) ~3 F' I& G2 `+ n9 c" G4 m
在dhcp命名空间中找到对应网络的 namespace 中找到 br-int 网桥上对应的 tap 设备,然后查看 ip 配置: * m; t: n* W* K( T
* \3 C7 N8 U& c' k9 O+ O3 R
: R) |' }! d# Q* [7 h& X' U/ o4 [
[root@controller ~]# ip netns show" D# K. s9 k( {" Y& u
qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 (id: 1)% c. E/ s3 N+ m
qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa (id: 0)) z/ P% R1 ~+ {
1 s$ t6 O9 l# Y# a1 w& ?$ |
[root@controller ~]# ip netns exec qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa ip a( n- ]9 y8 F7 r- `2 k& ?
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
2 d! {; ? I# ]" v link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00' N, \/ h# o" _/ }; k$ h) D, L9 ^+ v
inet 127.0.0.1/8 scope host lo8 G- e9 u {7 {0 y% ?
valid_lft forever preferred_lft forever
: T# j, Q+ P/ Q, S; }* H3 X" `) s inet6 ::1/128 scope host 0 n) h7 M0 n% _3 s
valid_lft forever preferred_lft forever# L' o5 g! C/ v7 `6 M3 u7 B7 p4 C
14: tapcee79ebe-a5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000# B) c+ G5 z* M
link/ether fa:16:3e:0e:1b:80 brd ff:ff:ff:ff:ff:ff
) v2 L- y" l* `9 W6 g, [% a5 G7 D inet 172.168.9.2/21 brd 172.168.15.255 scope global tapcee79ebe-a5) q9 B, x+ Q- B, ?. B+ _8 ^$ _
valid_lft forever preferred_lft forever
+ T& s: r i0 @8 ]. s3 T: `7 v1 K inet 169.254.169.254/32 brd 169.254.169.254 scope global tapcee79ebe-a5
* R- {7 I3 I# u6 ?/ T$ d valid_lft forever preferred_lft forever
1 Z$ w' k& @3 @- t inet6 fe80::a9fe:a9fe/64 scope link ; R2 N8 l# C# G! U. C9 ^
valid_lft forever preferred_lft forever1 b" w y% k4 q- k/ a
inet6 fe80::f816:3eff:fe0e:1b80/64 scope link & i# [0 H4 c6 q9 X; F5 H$ f
valid_lft forever preferred_lft forever' C2 q2 t7 L X# W. f: p
8 Q* Y- S7 A Z
; T1 {) _3 A0 l7 A
定位问题:% Y8 c3 ?0 V: a* G& a
通过上面排查,发现br-int 上是有tap口设备的,也已经连接到dhcp-namespace中,暂时没有找到问题的原因2 j2 w9 k6 r/ h) n
% c& d1 \' N5 j* l! {9 S4 X
* U2 [ d$ i s( G
" n5 F! ~- H5 T- h6 E[root@controller ~]# ip netns exec qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 ip a
/ F, r: R8 q2 ` Z- f1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
0 @& G2 z& P2 q9 \: A& n link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:009 ?* A, u4 s) ]$ U
inet 127.0.0.1/8 scope host lo
/ p% j$ g* m" r' A4 E! E valid_lft forever preferred_lft forever
/ _4 T: V' J; o7 \, J' S inet6 ::1/128 scope host 1 ^. u" F1 @9 g% S g4 s/ y% J
valid_lft forever preferred_lft forever9 N% b7 B6 B) Y! @5 {, X
15: tapd2a5f73d-5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
4 }3 E. p! Z0 O link/ether fa:16:3e:22:dc:dd brd ff:ff:ff:ff:ff:ff
, O0 L1 b5 l: ?, k2 p* } inet 172.168.16.1/22 brd 172.168.19.255 scope global tapd2a5f73d-5b
$ J/ \* x+ c. [ valid_lft forever preferred_lft forever
, R& K0 I, {7 V/ f7 i( X8 L# r- ~ inet 169.254.169.254/32 brd 169.254.169.254 scope global tapd2a5f73d-5b2 C. h" }9 K- c7 ]; L
valid_lft forever preferred_lft forever7 U: G- o, j' W! z% E
inet6 fe80::a9fe:a9fe/64 scope link
5 J" _0 f* N" r valid_lft forever preferred_lft forever
, s; P+ {: h, b0 |; Z' x9 h inet6 fe80::f816:3eff:fe22:dcdd/64 scope link
! P7 D- O5 M. l valid_lft forever preferred_lft forever, L# D W: v1 k. z4 V
[root@controller ~]# ip netns exec qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 ip a
2 g: D( w& @2 k# G8 S$ q8 B1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
3 E/ J. s9 a! u; D link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
4 |# f6 y" F6 w/ L& `$ g inet 127.0.0.1/8 scope host lo' _; s, b0 _3 \) d
valid_lft forever preferred_lft forever
9 H. X0 H& i2 M. _% U inet6 ::1/128 scope host
- M4 S( E$ k6 }; a4 T valid_lft forever preferred_lft forever- `7 H1 m0 z5 Z# }8 L: c
15: tapd2a5f73d-5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000' J- z& i/ u; m$ d* ]2 k5 Q
link/ether fa:16:3e:22:dc:dd brd ff:ff:ff:ff:ff:ff
- Z8 B" A5 v: [5 s" x& W4 j inet 172.168.16.1/22 brd 172.168.19.255 scope global tapd2a5f73d-5b% f* d& H4 F1 R, P+ |: }- @
valid_lft forever preferred_lft forever
- S' q w, f8 I) j- K9 v9 e inet 169.254.169.254/32 brd 169.254.169.254 scope global tapd2a5f73d-5b% E9 V- Z- S- \; D( }0 ?
valid_lft forever preferred_lft forever
+ u2 D0 X' r- [6 x" @6 J& v inet6 fe80::a9fe:a9fe/64 scope link
3 g' i u9 X: a a8 o valid_lft forever preferred_lft forever5 i, O( ?, b! Z
inet6 fe80::f816:3eff:fe22:dcdd/64 scope link
`' E; u" c% Q `6 D valid_lft forever preferred_lft forever5 J+ s2 H# ?% \. ?- C# T0 w
[root@controller ~]# ip netns show" a+ f% Y! a+ O9 ~; f) w, j
qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa (id: 0)2 t# h" }0 p" \3 l% p
qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 (id: 1)5 H, G' G% o( q, j. a/ y+ U
[root@controller ~]# ip netns exec qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa ip a
+ E& w8 q) k& J- Y) Q, b1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 10002 e7 {% P/ y3 d% Q+ N( y5 {
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
/ K0 `; r+ Q5 j1 n inet 127.0.0.1/8 scope host lo+ ?# \; g& {( a- Y- m
valid_lft forever preferred_lft forever0 M- ]- x; }: c$ d
inet6 ::1/128 scope host
% `4 Z9 D' r X valid_lft forever preferred_lft forever
9 q- h4 L: q+ d! O' R+ ~( i# `1 C16: tapca61a844-c4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
! s: ^' f4 U6 o$ F link/ether fa:16:3e:3f:e4:a4 brd ff:ff:ff:ff:ff:ff; q# Z% b+ K1 V, ?( \: D. A4 G
inet 169.254.169.254/32 brd 169.254.169.254 scope global tapca61a844-c4
6 k7 V* c: y e valid_lft forever preferred_lft forever5 w: ?$ L, p7 X
inet 172.168.8.1/21 brd 172.168.15.255 scope global tapca61a844-c4
* f, d" ]6 z f/ a1 B: _/ o valid_lft forever preferred_lft forever/ g2 c9 J9 K, D' i" g
inet6 fe80::a9fe:a9fe/64 scope link . B0 A7 ^& l$ n u/ F
valid_lft forever preferred_lft forever
" _8 d- N. N8 t inet6 fe80::f816:3eff:fe3f:e4a4/64 scope link
7 ]: }. o. K/ s5 I6 W1 a5 h7 I valid_lft forever preferred_lft forever4 q( q% Q# f/ h' a+ m* @7 M
5 L% X* p& d1 [) {) p0 { m
% v6 @, C b+ J6 D' o: g- }
4 B0 E# h8 I/ P7 O, M/ k, t6 K1 \+ W$ U2 m+ Z% N2 a6 S9 l* a4 [9 k+ ]
重启虚机,之后依然没有办法获取到IP地址。: i; V% c3 U+ g9 E
$ ~; q( s* H( f1 ]9 T+ d+ c2 o# q) t/ F
! G) a$ b% M/ Y8 O9 S" m) t3 @/ t' \6 M& }4 F1 S
在创建虚拟机下发请求后,dnsmasq进程会给虚拟机分配好mac地址和ip地址,并写入到/var/lib/neutron/dhcp/network-id 目录下的host文件中。虚拟机在内网中发送广播来获取ip的过程中,dnsmasq 会监听到然后将host文件中的对应ip通过dchp-namespace分配给虚拟机。 所以,在虚拟机获取ip过程中,必须虚拟机发出的包可以到达dhcp-namespace 经过的虚拟网络设备都存在且正常工作。 如果没有在subnet中开启上述的dhcp功能,那就少了一个对应网络的name-sapce dhcp服务了,所以虚拟机获取不到 ip。 5 n1 w0 k% G: k$ [; e- m8 H
& p% V, f J4 {. P2 q8 F |