马上注册,结交更多好友,享用更多功能,让你轻松玩转社区。
您需要 登录 才可以下载或查看,没有账号?开始注册
x
问题情况! c; H. q* R' N6 \9 u- a
openstack xina版本创建虚机后,虚机在dashboard上获取到ip地址了,但打开虚机控制台之后,使用ip add 检查网络状态时,虚机内部并未获取到ip地址:
; E; ?4 Q- T; i# _. b3 G- s
4 N" z) m$ [! ~, n0 o4 g! x: Y) X, H: ][td][tr][/tr]| 正在显示 1 项 |
. `/ L& H' p v6 G5 a | Instance Name | Image Name | IP Address | Flavor | Key Pair | Status | " |: o8 B. w8 C5 o, \5 _, W/ U
| Availability Zone | Task | Power State | Age | Actions |
' I7 e! J" @: R5 P+ p( V( E | m2 | CentOS-7.9 | - C. @# z+ e X5 C! x
5 x) a+ l. ?* O0 Q. O
6 }$ \* ~6 i7 g- Q0 I) ]* }6 u& i$ n
/ `2 D& [2 X+ ^4 y( C8 T+ m
172.168.10.101 | m2 | - | 运行 | | nova | 无 | 运行中 | 12 小时,14 分钟 |
* i7 g; v, m* P# l( f1 O# C2 q | 2 K! P# v. V1 I& q& ]
+ ?+ n9 V Q: n; y4 V
- K7 d' W( [* |: Y4 u6 @3 d1 U
分析排查思路:0 `# I9 D5 a/ k1 R% j: y, x" @: R
: O7 L4 Z' _) \5 Q2 p: i6 m" k(1)检查neutron服务状态,确保dhcp服务正常运行:( t1 b, ?; o5 A9 A
( T3 h2 b; j- v d
[root@controller ~]# neutron agent-list ) N2 H& E' w+ X, K1 t
neutron CLI is deprecated and will be removed in the Z cycle. Use openstack CLI instead.
/ M$ k) L. U, c# J/ N, Y+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+ Z, K4 Z0 P6 e* { _2 ?, y& N
| id | agent_type | host | availability_zone | alive | admin_state_up | binary |
( V& n7 K+ K: D7 e( A* ~7 J+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+
+ s1 G* s5 n7 C| 133d6414-7d3c-42f5-8422-90ab1c7f3721 | L3 agent | controller | nova | :-) | True | neutron-l3-agent |
p" K9 \- {7 k. ~8 B% o7 l' `/ H$ C| 2bfc7c83-94aa-4fdc-b7e2-055bb8db0f10 | Open vSwitch agent | compute01 | | :-) | True | neutron-openvswitch-agent |) G! G% ^5 ~. ]: D
| 4164d4b2-04f8-4d78-b514-351b1205d3ce | Metadata agent | controller | | :-) | True | neutron-metadata-agent |# V t) u# E- X. _) `
| 53fa495d-8039-4580-b1cc-20414ef1303d | Open vSwitch agent | controller | | :-) | True | neutron-openvswitch-agent |
, k, }0 T! L; t& f| ef59abb4-35d0-48c6-876e-983ed713e2d4 | DHCP agent | controller | nova | :-) | True | neutron-dhcp-agent |
1 z; I& t. o3 k0 \* [+--------------------------------------+--------------------+------------+-------------------+-------+----------------+---------------------------+9 I1 K" U' a. w. W
) `1 l/ U6 N {: z6 D
5 ^9 `# o; A# ]5 ^& d& Z(2)查看dnsmsp进程:
. ]* @$ h b4 B+ e/ ~% Y6 M+ k) x; e
[root@controller ~]# ps -ef |grep dnsmasq8 @/ s+ @0 R* V) \
dnsmasq 3548 1 0 07:52 ? 00:00:00 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/host --addn-hosts=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/opts --dhcp-leasefile=/var/lib/neutron/dhcp/ef99d400-71e0-468f-a969-e5d63fd79dc3/leases --dhcp-match=set:ipxe,175 --dhcp-userclass=set:ipxe6,iPXE --local-service --bind-dynamic --dhcp-range=set:subnet-ab92c638-b52e-4c32-8675-38b24f608b55,172.168.16.0,static,255.255.252.0,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=1024 --conf-file=/dev/null --domain=openstacklocal$ o9 X0 Y2 N" A0 x. D) m
dnsmasq 3553 1 0 07:52 ? 00:00:00 dnsmasq --no-hosts --no-resolv --pid-file=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/pid --dhcp-hostsfile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/host --addn-hosts=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/addn_hosts --dhcp-optsfile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/opts --dhcp-leasefile=/var/lib/neutron/dhcp/b3fdf316-0089-4ef3-9674-bd8fd8d6edaa/leases --dhcp-match=set:ipxe,175 --dhcp-userclass=set:ipxe6,iPXE --local-service --bind-dynamic --dhcp-range=set:subnet-e7722a92-a4ab-439c-b7af-129133c310b2,172.168.8.0,static,255.255.248.0,86400s --dhcp-option-force=option:mtu,1500 --dhcp-lease-max=2048 --conf-file=/dev/null --domain=openstacklocal
- A5 H" R6 _6 T5 U1 P' lroot 5024 2518 0 08:15 pts/0 00:00:00 grep --color=auto dnsmasq& ] H" g6 r! D& N
! a0 S" w! d8 @! }7 q. ]0 ~
(3)检查ovs网桥中的 br-int 集成网桥是否有 tap口设备 连接到了dchp-agent 的 namesapce上 4 D" C# n# a8 k; v. g9 [
6 y, @, a# k; w) L" a. O
9 b/ {0 A4 X2 w' C* u/ ~" s w[root@controller ~]# ovs-vsctl show + s- s9 h% [* u- X: i5 e
04659b20-7658-4782-abe5-84ee5f33282f
: c$ P$ s; h* s4 a' }4 O9 \ Manager "ptcp:6640:0.0.0.0"
% W2 f: k9 ^4 x6 E is_connected: true
! r& t2 o. d: R' E7 d! p- C Manager "ptcp:6640:127.0.0.1"
/ A- ?. Z0 B q Bridge br-tun
N5 M- z: K+ n# x! Y' m2 l) d1 ] Controller "tcp:127.0.0.1:6633"- P! G; T \" X. c3 W4 U
is_connected: true2 }( i. Y! e# {) L7 C+ J
fail_mode: secure! ~! A& ?1 B/ q4 v6 a
datapath_type: system
3 b; X5 p8 n- J" |: ^* u Port br-tun
; t' A1 Z$ \" j$ c4 f( z% F Interface br-tun
/ ~) p1 Q# A! ?2 o: @: i type: internal
$ _( H. w2 I$ e8 ~- y" Z Port patch-int
% {1 f0 c/ t- O' \ Interface patch-int
& J6 H/ X6 P# ~0 ]" K type: patch9 `3 K& _# n; d
options: {peer=patch-tun}
. C8 P7 d) P& ?1 p2 O* G w Bridge br-int! U$ X* z1 f7 I4 h' G- P2 Y/ B
Controller "tcp:127.0.0.1:6633"- N3 A$ T- X$ c* [* l0 @: ^. Y
is_connected: true, Y W# S m3 K+ ^% g8 g
fail_mode: secure
) M' R% g8 {% i4 Q' U datapath_type: system
9 T- H. B k, ] Port patch-tun# v% x3 S4 Y+ K! C* q
Interface patch-tun0 A6 p+ T* v8 D+ D, K5 \3 P
type: patch: u$ Y7 q) `* ` S! J5 x
options: {peer=patch-int}
9 o) ~8 o: m% Y2 H7 R$ H/ U- `# s Port tapd2a5f73d-5b
+ C: I/ U5 ~) r9 s) v tag: 2& O9 G" H; ~- Q" F+ Z R
Interface tapd2a5f73d-5b4 D' I y0 N0 ^! l2 j
type: internal5 Q. N. G& [7 Q6 W0 O
Port tapcee79ebe-a5
" f a6 [1 i9 N( d% r; [ tag: 14 i, Y5 F# N1 C
Interface tapcee79ebe-a5
* K. R' s: q5 }) i- g type: internal8 a( J0 O, s: t% E9 ?9 W
Port br-int. ] w" L: z9 O. j2 P- v
Interface br-int) p1 J- M# \; ?" R
type: internal4 p5 B @- P9 }
Port int-br-ex8 S. R* i M: ]5 C9 S" w
Interface int-br-ex6 a; p* H; z/ s" l% `2 V8 T
type: patch
4 k/ O' `) c' m; M- n* b0 B$ S options: {peer=phy-br-ex}# Z5 t: B+ Z8 G' n; {) L# z
Bridge br-ex
2 i: d0 _9 u6 v8 d+ s) J& D4 S Controller "tcp:127.0.0.1:6633"
, F, z6 E3 f7 E$ L4 H2 J/ y is_connected: true3 G! h/ R. ^, z8 _. Q2 ]- q9 y# }6 U
fail_mode: secure
& Y) o4 d% N8 A$ f* v) i5 a datapath_type: system$ G8 H) q: u0 M& s$ _4 w$ g$ B* f
Port phy-br-ex& Q0 k4 a/ y: D9 `5 p
Interface phy-br-ex
4 I3 f: q' \# ?: L# q$ ? type: patch6 k" ]- X4 a9 ^' Z/ M
options: {peer=int-br-ex}) h n$ u5 p, E8 A; w( K
Port enp7s0f0
1 T! A% x1 a8 r6 Z. |7 b* K Interface enp7s0f02 g' y0 [0 a) Q' W6 Q5 f& w
Port br-ex9 \; i C5 {: `8 Z" U8 D& M% a
Interface br-ex
3 {# P# A4 H" B type: internal; j0 s6 o, M D' G8 _& O
ovs_version: "2.15.4" o0 T% M. |5 _# Y u' a
- `; z: j) m4 |+ ]
+ r% {, | I R1 @7 u在dhcp命名空间中找到对应网络的 namespace 中找到 br-int 网桥上对应的 tap 设备,然后查看 ip 配置:
) C& _- h* G3 ~4 A# |' i/ m8 ?2 U1 C: T
7 a& ], K4 {9 x[root@controller ~]# ip netns show
/ Q5 \- C4 \5 `+ H: kqdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 (id: 1)
0 |9 l U9 v9 e, e$ |3 r) Hqdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa (id: 0)
6 |- C- Z$ n. S/ m8 Y) s! a
& ?0 z0 }3 Q! G5 S[root@controller ~]# ip netns exec qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa ip a
- Q2 M, n. L' w' P/ d1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
A! |9 \/ t- {8 k1 U# x link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
4 _6 a4 y% y* d. { inet 127.0.0.1/8 scope host lo, q9 a. B2 Z+ E( G
valid_lft forever preferred_lft forever
4 j' D& m1 A% \% E, V. A inet6 ::1/128 scope host
0 Y: J( m$ ]) t% f M valid_lft forever preferred_lft forever
& n0 H8 u4 J C- H$ {" t& U6 L14: tapcee79ebe-a5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
4 @3 K$ p2 ?+ m7 Z; ]) a9 I9 d! I link/ether fa:16:3e:0e:1b:80 brd ff:ff:ff:ff:ff:ff
2 Z2 i6 ?* X5 u* Q: ]3 b inet 172.168.9.2/21 brd 172.168.15.255 scope global tapcee79ebe-a5
4 I0 C( [, ~" g# E y valid_lft forever preferred_lft forever2 [5 R' Z" [4 L d/ N
inet 169.254.169.254/32 brd 169.254.169.254 scope global tapcee79ebe-a50 ~& ~# m6 ]5 O. [5 u5 B+ E" m7 g
valid_lft forever preferred_lft forever
! b2 I; ~9 G, p E3 _ | inet6 fe80::a9fe:a9fe/64 scope link
: b7 V6 e7 S5 _, \: F valid_lft forever preferred_lft forever- ^: ]* W& a+ }$ ?+ K0 u1 D, ] l, J6 N
inet6 fe80::f816:3eff:fe0e:1b80/64 scope link : G; F" M G5 v2 ~/ `6 h6 I# R
valid_lft forever preferred_lft forever
: t$ t w6 m+ | `) |0 ^' |
3 l; c: z5 e% `* H
, ]5 r! z+ e0 A2 _7 O. j定位问题:
, N( V0 p/ G5 B- Z U( B% Y通过上面排查,发现br-int 上是有tap口设备的,也已经连接到dhcp-namespace中,暂时没有找到问题的原因
% \+ |2 Y4 d& k e- e u* R+ \. n1 i4 @: c" O
( N. b( h: u: r+ ]: i& X
6 X% C% [, Y* m7 B: u$ |
[root@controller ~]# ip netns exec qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 ip a
$ D1 t% G% c" \6 q1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000+ k: g- \ k& H9 b5 q/ _
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
0 E; f" _2 \7 B inet 127.0.0.1/8 scope host lo6 T! h: z% U; F7 B& N
valid_lft forever preferred_lft forever2 x' S6 Q! u( s2 ^3 g! Z
inet6 ::1/128 scope host 2 S8 a7 D( G; [* [# z+ `" P. M
valid_lft forever preferred_lft forever
' Q0 V) m$ L' E: {15: tapd2a5f73d-5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
# U* D) Y# E' C/ D5 k9 C4 h) { link/ether fa:16:3e:22:dc:dd brd ff:ff:ff:ff:ff:ff
' |2 T# j+ S6 k$ ~1 H8 ~ inet 172.168.16.1/22 brd 172.168.19.255 scope global tapd2a5f73d-5b4 \& A# G9 d! F! P. H7 |
valid_lft forever preferred_lft forever
+ c. G3 \0 h7 J" g inet 169.254.169.254/32 brd 169.254.169.254 scope global tapd2a5f73d-5b
3 I( b& c. A/ X: q$ u% L valid_lft forever preferred_lft forever3 @5 p3 a, D5 w$ i# X# m. w
inet6 fe80::a9fe:a9fe/64 scope link
4 ~6 `8 \: N& x: N$ T T$ f valid_lft forever preferred_lft forever7 R) Y7 p& c; _& i, w
inet6 fe80::f816:3eff:fe22:dcdd/64 scope link
3 r9 ]& o' k0 l5 b0 ]- m0 Q valid_lft forever preferred_lft forever# {/ I5 N, i9 u2 l- E; g7 Q
[root@controller ~]# ip netns exec qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 ip a
% B5 k% h! h/ {5 p7 I1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000: U Z# o2 R! V( |- u; G
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
/ a+ @9 q8 a* @! {- l inet 127.0.0.1/8 scope host lo
/ r) x+ E- q9 B8 n# x0 E+ J5 x) C$ P valid_lft forever preferred_lft forever4 F) \! U0 Y- X" y
inet6 ::1/128 scope host
2 J/ [! `% ]2 y4 v, H valid_lft forever preferred_lft forever4 {; O8 K# M4 i+ t% @. Q; y
15: tapd2a5f73d-5b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
( ^' H* u3 ^* q7 D- m5 v! R4 M link/ether fa:16:3e:22:dc:dd brd ff:ff:ff:ff:ff:ff
+ F4 e& S% D9 V, ] inet 172.168.16.1/22 brd 172.168.19.255 scope global tapd2a5f73d-5b; B, M( s+ H2 R! k" Y; l- ]" o
valid_lft forever preferred_lft forever. H) o. c6 ?; A+ v! O5 F0 f
inet 169.254.169.254/32 brd 169.254.169.254 scope global tapd2a5f73d-5b" J* w: S' `9 |
valid_lft forever preferred_lft forever5 ?, s" o9 y+ [4 F+ h+ T
inet6 fe80::a9fe:a9fe/64 scope link # b% K0 e; e4 i# B6 l
valid_lft forever preferred_lft forever+ Z# b! D" ]' a
inet6 fe80::f816:3eff:fe22:dcdd/64 scope link
% y+ B2 c! O( t: W) i valid_lft forever preferred_lft forever
) u) Q& t! H1 |3 o) t[root@controller ~]# ip netns show5 ]8 z0 ^9 }. i! w
qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa (id: 0)
* \1 _& @. n0 g( a- m9 e* S/ `qdhcp-ef99d400-71e0-468f-a969-e5d63fd79dc3 (id: 1)/ J* `5 o" o' `7 ]
[root@controller ~]# ip netns exec qdhcp-b3fdf316-0089-4ef3-9674-bd8fd8d6edaa ip a0 U L$ ~! w8 ~$ {$ I2 H, Q7 \
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000( B& |0 ~ [# p. B7 C
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:004 M6 O9 p0 {5 L/ F
inet 127.0.0.1/8 scope host lo
$ x. u( m/ {% S! y! ~4 @0 P# ~ valid_lft forever preferred_lft forever
, Q9 M5 q0 w, Y4 ^7 M+ f inet6 ::1/128 scope host ' S. E7 {+ ^* r+ ^" D7 V
valid_lft forever preferred_lft forever
+ F, l5 w ?4 z' K% x& X16: tapca61a844-c4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
, k" U/ A0 N! f7 J6 M7 i link/ether fa:16:3e:3f:e4:a4 brd ff:ff:ff:ff:ff:ff
; l9 E$ A+ @/ y; i+ o inet 169.254.169.254/32 brd 169.254.169.254 scope global tapca61a844-c4
L" O! }- x1 } valid_lft forever preferred_lft forever3 ?9 D" j/ @8 @5 v5 }. S) _/ R
inet 172.168.8.1/21 brd 172.168.15.255 scope global tapca61a844-c4
* x& s3 o* G& G+ P. V' l valid_lft forever preferred_lft forever5 l4 A* R8 o, c" G7 [
inet6 fe80::a9fe:a9fe/64 scope link
1 X L ^) G+ F. d0 e6 z* j valid_lft forever preferred_lft forever
" D' f' P9 q" z& S inet6 fe80::f816:3eff:fe3f:e4a4/64 scope link
# u6 d, P& X: L- t9 ]# w valid_lft forever preferred_lft forever. z a, V/ t$ S4 |, C6 h; c# R0 u7 {& u
4 r0 [: S- j3 z0 [/ ?, E1 @) n- _
' K6 V( ~. c, [% S# v
, [2 V' l0 w3 B$ D
8 a" F( S$ J' x& G6 v! V' E重启虚机,之后依然没有办法获取到IP地址。
6 ^* u3 l) g! l( r: f$ b2 H: m
) U* M0 D& f. K# _6 ^: l
) S3 X: ~& I" ?3 W. p' d1 {& H
/ L% Y n/ g: @% s8 n0 N在创建虚拟机下发请求后,dnsmasq进程会给虚拟机分配好mac地址和ip地址,并写入到/var/lib/neutron/dhcp/network-id 目录下的host文件中。虚拟机在内网中发送广播来获取ip的过程中,dnsmasq 会监听到然后将host文件中的对应ip通过dchp-namespace分配给虚拟机。 所以,在虚拟机获取ip过程中,必须虚拟机发出的包可以到达dhcp-namespace 经过的虚拟网络设备都存在且正常工作。 如果没有在subnet中开启上述的dhcp功能,那就少了一个对应网络的name-sapce dhcp服务了,所以虚拟机获取不到 ip。
# P% R5 ^$ r2 t* B) f9 I) {
9 F, e6 |( v# N2 F9 T5 a1 {+ D |