问题情况
( \ S j8 l: Y$ ?- G, Kopenstack xina版本创建虚机后，虚机在dashboard上获取到ip地址了，但打开虚机控制台之后，使用ip add 检查网络状态时，虚机内部并未获取到ip地址：/ G) ]$ X& z1 ]# Z* B6 _
; B% r$ b) C s% W# C6 W+ m
[td][tr][/tr]| 正在显示 1 项 |
4 \$ U; k3 x8 ?3 `1 u: [+ D/ I3 U | Instance Name | Image Name | IP Address | Flavor | Key Pair | Status | k" n2 n" g0 T. }% B
| Availability Zone | Task | Power State | Age | Actions | * z: Q w/ g4 R( h
| m2 | CentOS-7.9 | * n: D: M! V3 r% L4 N/ I5 p
6 p# L1 D/ F4 y; d: F! l& ~% v, H
7 C2 ?! n1 U6 o5 A
9 z6 v" R' I9 P6 H
/ x" T. X4 y, Q% ^0 V! \" \172.168.10.101 | m2 | - | 运行 | | nova | 无 | 运行中 | 12 小时，14 分钟 | " l- O G: b! S* T A- w# X
|
9 c: `% T# A1 t1 }) ~ c. Q
 . w6 x% d# z7 }, i2 `
! W% {8 W; B* ]" F8 n
分析排查思路：
$ F4 H& } p7 b/ ?
* g) g5 \/ m' w: c5 e' M+ B（1）检查neutron服务状态，确保dhcp服务正常运行：
, r$ w- t9 C, F' X" { h! u$ P. N$ w6 U: A }' U% G
[root@controller ~]# neutron agent-list
/ M! ?5 O) B/ I8 ~neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.
$ j8 t' k+ h) Z5 S+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+
' _# K8 W! n& _- ~( J| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
/ G- k, a P: x9 O5 V+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+
% x: D+ z3 j- a* w! v| 133d6414-7d3c-42f5-8422-90ab1c7f3721 | L3 agent | controller | nova | :-) | True | neutron-l3-agent |1 o8 a- C* l4 z* i
| 2bfc7c83-94aa-4fdc-b7e2-055bb8db0f10 | Open vSwitch agent | compute01 | | :-) | True | neutron-openvswitch-agent |# m" `8 s; X$ } ~( G3 S% x' {: G
| 4164d4b2-04f8-4d78-b514-351b1205d3ce | Metadata agent | controller | | :-) | True | neutron-metadata-agent |3 c# n5 O1 n% Z3 d
| 53fa495d-8039-4580-b1cc-20414ef1303d | Open vSwitch agent | controller | | :-) | True | neutron-openvswitch-agent |6 w3 \" v+ \9 h& e2 n9 v, ?
| ef59abb4-35d0-48c6-876e-983ed713e2d4 | DHCP agent | controller | nova | :-) | True | neutron-dhcp-agent |
: \3 M; v7 \8 w9 C+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+
) M: l& ~- o, ]9 q' i5 D' e% o$ v) L5 V4 R# F9 @. A6 ~) @
* n, p! e) }: N! m( w! w
（2）查看dnsmsp进程：
( M$ a q# t0 q1 M: L6 K. O" P, X4 m
[root@controller ~]# ps -ef |grep dnsmasq9 l- }( n6 K; K# K4 G' @# b
dnsmasq 3548 1 0 07:52 ? 00:00:00 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/host --addn-hosts=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/opts --dhcp-leasefile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/leases --dhcp-match=set:ipxe,175 --dhcp-userclass=set:ipxe6,iPXE --local-service --bind-dynamic --dhcp-range=set:subnet-ab92c638-b52e-4c32-8675-38b24f608b55,172.168.16.0,static,255.255.252.0,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=1024 --conf-file=/dev/null --domain=openstacklocal
) m1 @3 F( B, x. E' {& Hdnsmasq 3553 1 0 07:52 ? 00:00:00 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/host --addn-hosts=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/opts --dhcp-leasefile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/leases --dhcp-match=set:ipxe,175 --dhcp-userclass=set:ipxe6,iPXE --local-service --bind-dynamic --dhcp-range=set:subnet-e7722a92-a4ab-439c-b7af-129133c310b2,172.168.8.0,static,255.255.248.0,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=2048 --conf-file=/dev/null --domain=openstacklocal. y9 K$ t5 j8 w' a0 ?* [- X
root 5024 2518 0 08:15 pts/0 00:00:00 grep --color=auto dnsmasq, i" P' u8 s3 z) w/ W2 X0 Y1 y
, J5 c2 X/ a; p, x* f
(3)检查ovs网桥中的 br-int 集成网桥是否有 tap口设备 连接到了dchp-agent 的 namesapce上 8 s+ V; m' e/ J9 {% Z% N7 ^
: z; q G- m+ z( k" i
0 t4 a2 M1 p" v/ N! Q
[root@controller ~]# ovs-vsctl show
/ ~: G+ n- o! B D7 O# q04659b20-7658-4782-abe5-84ee5f33282f
# V! w5 [ h! o' O Manager "ptcp:6640:0.0.0.0"
8 T' N6 D+ v( h3 o is_connected: true4 B& }! m5 C: V' f- e5 n
Manager "ptcp:6640:127.0.0.1"" ]3 _! ^3 H3 _6 j/ |
Bridge br-tun
- B: A' Z; Y I/ _& p- f Controller "tcp:127.0.0.1:6633"% R" E; l3 v Z/ ?* q( k: U F
is_connected: true
# B) D- d% T: ]' v4 r fail_mode: secure
# O% U, q. p5 [4 Y" e* ^9 a datapath_type: system
. s- h4 Y/ B; s8 ?" f Port br-tun
, `' x8 W/ K3 w5 a Interface br-tun; O& ^+ h) U) `+ r/ l: r* N
type: internal" R/ C! u/ ?* t
Port patch-int
1 i0 r& f! \0 E- z Interface patch-int
6 f* J8 `- N; m2 v2 K+ b type: patch
) j/ b+ s1 u& u' u) Y0 c options: {peer=patch-tun}$ `3 P/ `* D+ q2 u
Bridge br-int
% X: s2 [; J3 D% V$ i. { Controller "tcp:127.0.0.1:6633"
4 N* k7 h" [9 e3 I f. l is_connected: true+ n# j1 h2 v3 ^. P' a" P
fail_mode: secure
) W' h8 c4 H% ? datapath_type: system. H t2 R, ~) W* U2 `* Z
Port patch-tun
2 o8 [0 s0 b& r. N* @ B Interface patch-tun: U d7 U& f. y4 t
type: patch
4 |; J, m0 I6 {! d6 d1 @; N5 t options: {peer=patch-int}) Y$ |, J3 C4 W) T
Port tapd2a5f73d-5b% Z( o; a5 Y" C! [* H
tag: 2
# Z; x2 _6 I" g' W3 N2 } Interface tapd2a5f73d-5b. I3 N+ e8 j2 P f& |
type: internal4 k% w; T- y% U, L2 ]# W+ f. P! }
Port tapcee79ebe-a57 j. x$ U( |# u# R( x2 ?8 U3 J/ L
tag: 1
/ `4 u. p+ D" ~- M9 Y" ]+ c Interface tapcee79ebe-a5
* b2 {5 t3 l3 {0 c" m, r type: internal
. N* r* g% ]6 q6 C+ ^. c6 c+ r$ s Port br-int2 W5 q2 | j1 \
Interface br-int
1 x. g& O e- T! E6 Q type: internal
: z! p* ^7 K/ @0 ^5 h Port int-br-ex
8 q/ |8 `% N3 G( p4 Z) C Interface int-br-ex* m# T5 q8 E& ~/ m9 k. @, c
type: patch
/ F+ J, X$ S- S/ U0 C4 Q options: {peer=phy-br-ex}; b( e p* h6 u2 J
Bridge br-ex
# R1 d" |# L6 f& O4 m& W% K Controller "tcp:127.0.0.1:6633"
. t9 l2 p, c# K1 \7 k9 L( W is_connected: true o3 @+ X- C8 h! Q
fail_mode: secure
6 s% n8 g! {2 v- ^ datapath_type: system% _7 F5 h9 b4 f @+ j9 D8 y* n
Port phy-br-ex- i8 `' o+ @3 V$ [+ j) ]* U
Interface phy-br-ex
4 \# [9 }3 x G( ?6 d0 ~. H+ i type: patch
( ]# _ O5 ?/ l5 m! R4 E6 g& J options: {peer=int-br-ex}
- j' @3 r- Q8 E+ S( [0 M" \$ x Port enp7s0f0
: c% E$ h% }/ ]+ Z( E Interface enp7s0f02 [% S, z( |, W& F' N7 j; S: Z
Port br-ex
, q. E+ _6 {+ `! [# T* ]# M, n8 h$ u Interface br-ex
& G9 H! B: E8 m: H type: internal0 X1 v7 V b+ Z* w* B/ Y% ~
ovs_version: "2.15.4"
1 Y, I; L* ^/ ^/ e. t6 Y' D S' ~! s6 Z' }, H
' ?9 ^6 R2 N: G# j
在dhcp命名空间中找到对应网络的 namespace 中找到 br-int 网桥上对应的 tap 设备，然后查看 ip 配置： 5 L& l! o2 z. U' F. e) _# Z* h
7 J, q6 w; @; v( f: b( c
8 j( m2 d# P. m. l0 x" I[root@controller ~]# ip netns show' s& _- u6 L& O
qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 (id: 1)( e1 E2 ^. R' W# M7 \5 E/ ~
qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa (id: 0)% r' @( o* Q* _( j6 ^/ ~7 h
8 N( e1 I* p% Q7 l* r8 Q
[root@controller ~]# ip netns exec qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa ip a
) V: i* D9 l# H1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000+ S4 n* n% ?9 a8 z. H1 g
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00, n. A3 u9 J' L1 b9 }4 \- x, c3 T
inet 127.0.0.1/8 scope host lo( I2 |+ O* z1 }% r1 a
valid_lft forever preferred_lft forever
, E, I" [# V( x6 P- n, |2 Q inet6 ::1/128 scope host + O% S+ y* K& I
valid_lft forever preferred_lft forever X) G- S2 I& J7 Y; U C
14: tapcee79ebe-a5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000) z4 D" B, e" `( h4 o# l
link/ether fa:16:3e:0e:1b:80 brd ff:ff:ff:ff:ff:ff/ F/ o. h! K1 c( R0 ~# M* n2 N
inet 172.168.9.2/21 brd 172.168.15.255 scope global tapcee79ebe-a5
! }/ U3 H" t; r valid_lft forever preferred_lft forever" W- E9 f9 f8 n7 Z( |
inet 169.254.169.254/32 brd 169.254.169.254 scope global tapcee79ebe-a5
8 j5 J/ Q" o1 [9 [+ ~ valid_lft forever preferred_lft forever, i. }! j' r, x, J, y! n* _( C
inet6 fe80::a9fe:a9fe/64 scope link + f0 S& `7 }7 N; \: _3 U* |0 C4 C
valid_lft forever preferred_lft forever( i4 p# K% z/ y- h- i0 y
inet6 fe80::f816:3eff:fe0e:1b80/64 scope link
! e3 ]$ j$ _- M valid_lft forever preferred_lft forever3 r y. u/ A! K$ T. K6 a& g, U
/ P+ B( Q# _/ s8 t9 X
. b& ?3 J8 `/ |4 v% e. F定位问题：& m, q9 c F/ n# w
通过上面排查，发现br-int 上是有tap口设备的，也已经连接到dhcp-namespace中，暂时没有找到问题的原因4 L/ t7 m2 y7 @
4 C) \; p5 ~- U" \- [
* v# u( S, V' o8 ~+ _" o2 C3 N
& N' H; C( y! L2 W1 {
[root@controller ~]# ip netns exec qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 ip a) N0 w/ B6 q% k i& d2 e) g; X
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
$ f7 E) @2 e4 D- R$ e# p, d8 H0 w link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
+ I& ]: S; c. r& A' @$ k$ H inet 127.0.0.1/8 scope host lo# X' f- p& S- @
valid_lft forever preferred_lft forever
) \% {2 a) p7 |: d; @' W inet6 ::1/128 scope host " W: H# }% ~: m$ ~$ U
valid_lft forever preferred_lft forever
* E) J5 f! C) E( J# {7 D( X15: tapd2a5f73d-5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
' S* z3 P' O" A link/ether fa:16:3e:22:dc:dd brd ff:ff:ff:ff:ff:ff
# s; R9 U! a! t, G. S) ] inet 172.168.16.1/22 brd 172.168.19.255 scope global tapd2a5f73d-5b; T, `6 G0 c* T4 {7 @* v
valid_lft forever preferred_lft forever
# C2 }/ E. o) b+ l" h$ i9 C inet 169.254.169.254/32 brd 169.254.169.254 scope global tapd2a5f73d-5b/ }2 s; p1 R7 J) K! e( ^
valid_lft forever preferred_lft forever
2 L, V/ \3 ]! z N' s0 Y1 `% | inet6 fe80::a9fe:a9fe/64 scope link 5 `- Y3 r9 n3 b$ p0 M. E1 ~
valid_lft forever preferred_lft forever( \+ h$ p" X5 J+ R( {8 c* ]
inet6 fe80::f816:3eff:fe22:dcdd/64 scope link
$ q/ ]1 z# S! Q3 l1 \, W5 ~3 M valid_lft forever preferred_lft forever
' [6 J# q0 M8 L* I[root@controller ~]# ip netns exec qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 ip a
) w1 C% n8 Z" `1 p1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000# R G* Z7 \) a3 H3 [, o4 ~6 o3 ]0 L
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
/ Z3 o9 r3 N, G- v inet 127.0.0.1/8 scope host lo
6 _0 Y7 c/ T" m7 d1 }" F4 S valid_lft forever preferred_lft forever0 U) V7 e/ [% }8 m/ R
inet6 ::1/128 scope host
`8 h' `: i5 m; \; w+ ~ valid_lft forever preferred_lft forever3 |- u+ M! C1 \* }6 ~' W
15: tapd2a5f73d-5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000+ j. [) B8 i4 B4 V! G* f
link/ether fa:16:3e:22:dc:dd brd ff:ff:ff:ff:ff:ff3 P8 T4 w( ]% t2 |5 \: G) h
inet 172.168.16.1/22 brd 172.168.19.255 scope global tapd2a5f73d-5b
2 @' I+ C% N0 W: v- q valid_lft forever preferred_lft forever+ ^4 D9 ?! u& q+ ]! B
inet 169.254.169.254/32 brd 169.254.169.254 scope global tapd2a5f73d-5b
, g0 q4 A8 [$ z5 c% ] valid_lft forever preferred_lft forever$ A- j! T+ x, Q# Y2 I7 ` s3 A
inet6 fe80::a9fe:a9fe/64 scope link / b" S9 e2 x+ r8 S9 O) n* n4 K
valid_lft forever preferred_lft forever. I+ j' r; z( [5 |* Q, N# ^8 F
inet6 fe80::f816:3eff:fe22:dcdd/64 scope link q8 V: T6 j2 \
valid_lft forever preferred_lft forever
# {; @ f# ?/ k0 W/ [* ~[root@controller ~]# ip netns show
) w" y- `) I2 @qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa (id: 0)" q" A3 [! u8 o8 G* b" H0 e7 s
qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 (id: 1)# E7 T8 B! u. X
[root@controller ~]# ip netns exec qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa ip a
$ M) r+ E+ R0 D1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 10005 [. W# z8 C0 r: c* g/ o6 r
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
9 X7 O3 U( B$ l9 W$ p% L inet 127.0.0.1/8 scope host lo, B' U+ z9 m, c; Y/ @1 B# x2 B6 ] A
valid_lft forever preferred_lft forever
- w1 \ @0 i9 M" S& _: x: v inet6 ::1/128 scope host " j/ N/ @: B3 W' W
valid_lft forever preferred_lft forever' u4 e4 K; w; a/ G* x
16: tapca61a844-c4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
" c& x$ C0 T5 Q link/ether fa:16:3e:3f:e4:a4 brd ff:ff:ff:ff:ff:ff
, r: p/ z# ^, a5 Z% R* ] inet 169.254.169.254/32 brd 169.254.169.254 scope global tapca61a844-c4
( @6 \0 Y _; ~ M9 H* {4 D& d% V valid_lft forever preferred_lft forever+ C# r/ _ l6 w
inet 172.168.8.1/21 brd 172.168.15.255 scope global tapca61a844-c4
% P) S/ {( w5 o& ? valid_lft forever preferred_lft forever
! N6 J$ t+ J, _7 N5 r& `# ] inet6 fe80::a9fe:a9fe/64 scope link
7 O0 e; B8 V/ [0 y* S valid_lft forever preferred_lft forever1 ]" c- W( V j; h6 o
inet6 fe80::f816:3eff:fe3f:e4a4/64 scope link
$ J- ]0 s+ b r/ v valid_lft forever preferred_lft forever
9 X2 n2 b5 m4 I& V! v. a. e. I: J) `1 O, @
8 C7 A. G9 r) b% _0 G4 |1 _, g" y# w7 h6 o0 q- d
; {6 J! x$ y* [重启虚机，之后依然没有办法获取到IP地址。
! i% e) `9 |2 l; m$ L8 S 5 z. n d: {& d
/ T0 ~$ }# G, [ t3 K {& e4 `$ B$ q
在创建虚拟机下发请求后，dnsmasq进程会给虚拟机分配好mac地址和ip地址，并写入到/var/lib/neutron/dhcp/network-id 目录下的host文件中。虚拟机在内网中发送广播来获取ip的过程中，dnsmasq 会监听到然后将host文件中的对应ip通过dchp-namespace分配给虚拟机。 所以，在虚拟机获取ip过程中，必须虚拟机发出的包可以到达dhcp-namespace 经过的虚拟网络设备都存在且正常工作。 如果没有在subnet中开启上述的dhcp功能，那就少了一个对应网络的name-sapce dhcp服务了，所以虚拟机获取不到 ip。 ) ~+ m# u0 q7 X, C: u- C0 o6 g
1 ~: e- B! d% ?- V* B. E' U/ u7 `/ o |
发表于 2022-5-23 08:10:18