找回密码
 注册
查看: 4094|回复: 0

Ironic

[复制链接]

40

主题

29

回帖

471

积分

新手上路

积分
471
发表于 2017-6-30 22:58:25 | 显示全部楼层 |阅读模式
Ironic对接原生的Neutron6 j( o; o, Y. z# m

部署、配置相关:5 l+ n2 \7 A; o6 S& z8 |/ y# ^- R; @
  • Ironic自己有一个dhcp-server,在inspect过程使用
  • neutron-dhcp,在provision过程使用
  • inspect和provision过程使用的tftp server可能不同! v$ I! G0 L6 e. W0 b3 ^5 s; {4 b

2 O( ~% m3 A' [1 oRegister过程1 |/ p6 ^) t; n' b* d
用户录入ironic node,包含ipmi等信息9 o2 K! v, f7 R; r# `" [+ F5 F
: k6 ]- T* t* H
Inspect过程
9 M8 j7 n7 I- p3 C- Q5 B, y/ W
这个过程中使用Inspect Network,要求:8 `0 n) w1 N8 X/ O& y
  • Ironic dhcp-server能收到BM节点的DHCP请求。
  • BM节点拿到IP后,能和tftp-server-1互通(三层可达)& @% `4 n1 P* l& U2 F
用户获取BM节点信息
/ N5 ?6 ?* A! k0 `6 l; |. ^
  • Ironic通过IPMI设置BM节点PXE启动
  • Ironic通过IPMI启动BM节点,做PXE启动
  • BM节点从Ironic dhcp-server获取IP。此时BM节点的请求报文不带vlan tag,使用上联接入交换机的native vlan(默认tag=1)
  • 拿到IP之后,BM节点从tftp-server-1下载小镜像(ramdisk,内含Ironic Python Agent)
  • 执行某些操作,获取BM节点的详细信息
  • 将BM节点关机。ramdisk运行在内存中,关机后丢失。
    ' W9 S+ `( M9 K; d2 Z2 i; {
( C' u5 ~- w# x
Provision过程" W  {" p) E. D* s
这个过程中使用Provisioning Network(由neutron创建),要求:; x7 Z+ ~2 }; F% B% ]
  • BM, glance-api, ironic-api, ironic-conductor, neutron-dhcp-agent需要保证PROVISION NETWORK连通性- P( H% Z/ o6 h3 F# l: ^
用户申请物理机,安装操作系统,配置业务网卡等  T1 u( U' }5 M) X
  • 从nova入口
  • Ironic IPMI启动BM节点,做PXE启动
  • 此时,要求BM节点从neutron-dhcp-server获取IP(通过native vlan)。但由于Ironic-dhcp-server也允许native vlan过来的请求,所以必须保证DHCP请求被Neutron-dhcp-server处理。
  • 拿到IP之后,BM节点从tftp-server-2(可以和Inspect过程中的tftp server不同)下载小镜像(ramdisk,内含Ironic Python Agent)
  • (这一步怎么控制的?)从glance下载用户要求的镜像,做安装(要求拿到的IP和glance-api能互通)
  • 安装完成之后,通过cloud-init在BM操作系统内部打上对应的vlan tag(必须保证该vlan tag在接入交换机上预先做了配置)7 G6 u0 ^; j2 h. k3 {5 \
  a" b- O/ p9 p" D2 n! u6 q
关键问题:" a/ C( r# X3 l
  • Ironic-dhcp-server和Neutron-dhcp-server都允许native vlan过来的DHCP请求,如果有两个BM节点同时做Inspect和Provision操作,可能引起冲突。
    / w# z6 {( ]# B5 s* D* I
  • 两个DHCP server合并。但是Neutron-dhcp-server是白名单方式,而在Inspect节点,dhcp-server还不知道BM节点的信息,没法配置白名单。
  • 严格将Inspect和Provision过程分开。在机房初始化过程中,开启Ironic-dhcp-server,做完Inspect之后将其关闭;或者在EPC上强制Inspect过程中,disable Provision操作。* |0 u% C. u" U7 J& H, g' `

    5 q5 S( w( M: {" Y3 h3 A* 一级私有云中兴方案,将两个DHCP合并了,运行在ToR交换机上。" L  M2 m. Z' }+ S! x; Q
  • BM节点的租户vlan一定要在接入交换机上预先配置,如果做不到,则需要动态地配置交换机
  • Neutron-dhcp-agent需要在业务网上
    " t4 c8 U! I2 [" V$ P& ?
4 O' S0 K4 [' T  T! L: t5 u

苏州Ironic环境
- N+ u1 K! M5 S) r
10.142.24.12 root/@IDC_host4321
( ?" U, ]" A/ R

% B% ~7 D7 T5 q! u$ e$ }6 K+ c9 R; y6 ?

浙江Ironic测试环境1 k$ O1 w# T( ^
! V: l4 l3 M, G* C! e# F  K* i
Ironic DHCP
/ q0 I1 U0 j" [$ X/ v
[root@csv-yglcs17 ~]# cat /etc/dhcp/dhc
  }% M9 C  j' T) H( kdhclient.d/  dhcpd6.conf  dhcpd.conf
  q- z' f8 k6 }2 _[root@csv-yglcs17 ~]# cat /etc/dhcp/dhcpd.conf
- M6 T# G9 |8 r# {8 F( xoption domain-name "test.com";) n: g5 V. i8 j/ q" z
option domain-name-servers 8.8.8.8, 61.88.88.88;/ [  Z, r" [$ R3 ^
default-lease-time 60000;8 G$ i( Y' k" {3 b
max-lease-time 720000;) [: U# x( f3 w! E6 p0 p( u3 \8 l
subnet 20.26.34.0 netmask 255.255.255.0 {0 Z) R: U* t' ?
    range 20.26.34.10 20.26.34.100;       <== DHCP段- ?1 K! I& w0 S) `3 `' e
    option routers 20.26.34.1;
0 C  ]7 M6 M/ v    next-server 20.26.33.26;          <== tftp server' G' Q7 `9 d  t  W. g) I$ s& n
    filename "pxelinux.0";2 o2 n; Q, X( ]: d0 _% d5 E
}
. ?$ x! [0 p8 {0 e5 D5 l* Fsubnet 20.26.33.0 netmask 255.255.255.0 {      <== conductor节点只有33.0网段IP,如果不配置这个subnet,则dhcp启动时会报下面这个错误
1 G3 o1 x" O& H8 L$ D}* c" E  o' y& N9 S$ v2 N1 |

, v" N) q8 {+ A8 |) x. e, l问题:: l6 ?2 j* m' X0 S/ T' A/ \0 {. C% }
Apr 19 14:30:21 csv-yglcs17 systemd: Starting DHCPv4 Server Daemon...& U9 H+ Q9 \' N) {# W2 w2 d4 F& F' Y
Apr 19 14:30:21 csv-yglcs17 dhcpd: Internet Systems Consortium DHCP Server 4.2.5
2 J7 J1 ~. S4 MApr 19 14:30:21 csv-yglcs17 dhcpd: Copyright 2004-2013 Internet Systems Consortium.
0 o& N) g) a) [" ]. R% FApr 19 14:30:21 csv-yglcs17 dhcpd: All rights reserved.
; ~! w- O3 H! e& e: X3 _Apr 19 14:30:21 csv-yglcs17 dhcpd: For info, please visit https://www.isc.org/software/dhcp/
0 `5 w8 w: P0 @' \2 [5 K$ lApr 19 14:30:21 csv-yglcs17 dhcpd: Not searching LDAP since ldap-server, ldap-port and ldap-base-dn were not specified in the config file' T" C* p  h; F0 E
Apr 19 14:30:21 csv-yglcs17 dhcpd: Wrote 15 leases to leases file.8 Q+ Y" L6 L9 y
Apr 19 14:30:21 csv-yglcs17 dhcpd:
0 m4 z) i8 [7 @' X2 yApr 19 14:30:21 csv-yglcs17 dhcpd: No subnet declaration for eno33557248 (no IPv4 addresses).% V5 L5 l, }" d2 Y' W, C
Apr 19 14:30:21 csv-yglcs17 dhcpd: ** Ignoring requests on eno33557248.  If this is not what( h. i' d! g% o$ R7 a) H1 m9 l
Apr 19 14:30:21 csv-yglcs17 dhcpd:   you want, please write a subnet declaration) ?7 q" U% e% |8 T
Apr 19 14:30:21 csv-yglcs17 dhcpd:   in your dhcpd.conf file for the network segment
3 i3 B9 T# Y4 E% [* T$ qApr 19 14:30:21 csv-yglcs17 dhcpd:   to which interface eno33557248 is attached. **. p1 A' O1 p0 g) x/ C+ O
Apr 19 14:30:21 csv-yglcs17 dhcpd:
7 B' g) \9 a% Y9 m3 J/ qApr 19 14:30:21 csv-yglcs17 dhcpd:
3 A' W- |5 A9 f! v- u3 J: ?  l0 SApr 19 14:30:21 csv-yglcs17 dhcpd: No subnet declaration for virbr0 (192.168.122.1).
/ W- s; K4 T8 A: _  n3 ~Apr 19 14:30:21 csv-yglcs17 dhcpd: ** Ignoring requests on virbr0.  If this is not what. o9 G5 Z$ N/ b/ I
Apr 19 14:30:21 csv-yglcs17 dhcpd:   you want, please write a subnet declaration2 F# s* r; S- K; }0 @) h; j
Apr 19 14:30:21 csv-yglcs17 dhcpd:   in your dhcpd.conf file for the network segment
( U1 V: J' X5 Y5 \3 vApr 19 14:30:21 csv-yglcs17 dhcpd:   to which interface virbr0 is attached. *** z9 q  v! g2 E" P' d
Apr 19 14:30:21 csv-yglcs17 dhcpd:  ?: V* @% q  y6 i/ a% b9 `+ m, K; {8 |
Apr 19 14:30:21 csv-yglcs17 dhcpd:
, f6 W, z% m+ R# ~. BApr 19 14:30:21 csv-yglcs17 dhcpd: No subnet declaration for eno16777984 (20.26.33.26).
1 `- V% D* r1 x9 g: C+ g, [Apr 19 14:30:21 csv-yglcs17 dhcpd: ** Ignoring requests on eno16777984.  If this is not what
" i( S& S( x- Z: n8 t) tApr 19 14:30:21 csv-yglcs17 dhcpd:   you want, please write a subnet declaration
8 Q5 j5 j: e7 E# l- L: QApr 19 14:30:21 csv-yglcs17 dhcpd:   in your dhcpd.conf file for the network segment! M( w& Y, u+ ?, z4 q6 f. X5 J
Apr 19 14:30:21 csv-yglcs17 dhcpd:   to which interface eno16777984 is attached. **
2 S5 f) }2 o% |Apr 19 14:30:21 csv-yglcs17 dhcpd:6 z( U5 t% ?3 w
Apr 19 14:30:21 csv-yglcs17 dhcpd:
& B( r; L( C; a) ]% QApr 19 14:30:21 csv-yglcs17 dhcpd: Not configured to listen on any interfaces!
# ?( R) X- p+ p+ tApr 19 14:30:21 csv-yglcs17 dhcpd:
7 b" s+ q& v$ a4 AApr 19 14:30:21 csv-yglcs17 dhcpd: This version of ISC DHCP is based on the release available5 T/ \6 J! `5 R3 H+ b6 s4 m$ x
Apr 19 14:30:21 csv-yglcs17 dhcpd: on ftp.isc.org.  Features have been added and other changes, A# A2 y! t  d7 @8 Z. @& G
Apr 19 14:30:21 csv-yglcs17 dhcpd: have been made to the base software release in order to make7 z; u! H1 U8 [8 g4 e. ?
Apr 19 14:30:21 csv-yglcs17 dhcpd: it work better with this distribution.0 {( t/ ]8 G7 ?% `1 U
Apr 19 14:30:21 csv-yglcs17 dhcpd:- s6 {; E8 R' b( C' O$ m. M
Apr 19 14:30:21 csv-yglcs17 dhcpd: Please report for this software via the CentOS Bugs Database:/ j4 [( e0 n  Y8 R8 e+ X
Apr 19 14:30:21 csv-yglcs17 dhcpd:    http://bugs.centos.org/4 C- V6 `" y$ r" M3 V" n
Apr 19 14:30:21 csv-yglcs17 dhcpd:
0 s0 \2 y2 Z% ?" M4 L" ?Apr 19 14:30:21 csv-yglcs17 dhcpd: exiting.) \. [6 J1 c1 u' t
Apr 19 14:30:21 csv-yglcs17 systemd: dhcpd.service: main process exited, code=exited, status=1/FAILURE- o6 c0 j1 ~. Q: @- ^0 o
Apr 19 14:30:21 csv-yglcs17 systemd: Failed to start DHCPv4 Server Daemon.: e* A8 G) e3 i8 a; f
Apr 19 14:30:21 csv-yglcs17 systemd: Unit dhcpd.service entered failed state.
9 \1 L' B4 M& U4 R7 U$ T  t* rApr 19 14:30:21 csv-yglcs17 systemd: dhcpd.service failed.
. I4 N& z& q" |+ Y! _! y; l1 |7 h9 C! X# b- k
! G: G0 @0 b5 p) V( C( U/ N6 ^  U2 [
Ironic Inspector
. [" }0 n: V! |1 G) B$ c; v5 x3 r: b
[root@csv-yglcs17 pxelinux.cfg]# pwd
4 [# l5 x8 ~# k. K7 Q. x& ~/tftpboot/pxelinux.cfg: ?- n% K$ }' \. y! i+ v6 M
[root@csv-yglcs17 pxelinux.cfg]# cat default
) A1 }. |9 g. e  wdefault introspect2 B5 j- U1 M5 `0 }7 A$ T' n/ d
label introspect' `+ [* t/ \3 I( X$ ^* s# L/ d
kernel /tftpboot/ironic-inspector/inspector-kernel6 {  p. {! s; E' j
append initrd=/tftpboot/ironic-inspector/inspector-ramdisk  ipa-inspection-callback-url=http://20.26.33.26:5050/v1/continue systemd.journald.forward_to_console=yes ipa-collect-lldp=True
. w/ }1 x% S5 C2 Lipappend 3) v, [" x( `1 E& E
- S9 Z1 D. v% U0 {( ]
inspector在20.26.33.26上
( H. f. @' _* C1 ~4 h7 k
' E7 ?9 E( F- p! u! ?% Q
Ironic Provisioning
- u, ^; {4 o7 M% M: k
ironic.conf中的provisioning_network还没配置。还有cleaning_network。
( F( \! P( o' e

' `8 s+ Y6 X- b检查IPMI/ a5 h* b" G) I6 W* ~/ n% Q
[root@csv-yglcs17 ~]# ipmitool -P Huawei12#$ -H 50.1.65.245 -U root -p 623 -I lanplus power status* t+ n. ]* |& G" J7 l% f( @
Chassis Power is on0 ~& Z  n, j) P/ q/ m

* v* W% A2 @8 q' H" j! [
- f5 k) V" o1 A/ u. r% g$ t/ C* q6 l* J' d  B
== 操作 ==
  B& G+ R& @) K, g
$ a/ s$ P7 u- P1 Q+ K1 Q
. ^) a5 o' u1 m2 g; x* ~* U
& B: B/ P& t9 j, p
0 G. Q1 t0 [* `, M$ e- e  a( l2 f: C- g1 Y

4 [% Q! _( }+ K$ {' o" X. g  {$ `
" F- }* |; v' {& \# i. \5 F6 q" Q+ o
& X" D) T/ ?$ ~2 B9 Q
$ X3 {" H* y8 Y4 A
ironic node-create --chassis_uuid dbb588b3-75e8-4028-b851-110671e05e58 \5 e1 R; l( r! I) O
                   --driver agent_ipmitool \  C3 a% p" K# v, u' v
                   --name pc-zjnacthd01 \
6 ~# j1 H# d8 C' p                   -i ipmi_address=50.1.65.245 \
: v: u; B. z. d2 e5 F  n                   -i ipmi_username=root \5 @6 k2 U3 Z6 o9 s9 r
                   -i ipmi_password=Huawei12#$ \6 Z5 x2 E/ \; w$ G3 ]5 g
                   -i ipmi_port=623 \2 y- x3 }5 x* U4 i
                   -i driver_info/deploy_kernel=4c1855e5-9b6b-47e2-89e5-3bc351c2ae2e \  ~0 T& D- X& n( |) \/ T9 H
                   -i driver_info/deploy_ramdisk=2f603c85-de92-44ea-b4d0-1396b91102cc
! Q7 h# z) n0 _; V- C* D2 z9 t
Update 5/25: 正在开发Ironic AZ功能,通过node-update将AZ属性加入node,同步给nova数据库。nova boot时只需要指定AZ创建机器即可。3 s* g- J  q6 X# T
) Q" L, n/ M" R5 Q* v0 \

( M7 V0 x% |& g, m. B
- y9 c* G7 u7 a& t2 C2 L% B
* A  ?; F- S3 X5 E$ ~- W0 P* C% u7 p/ u9 s( W5 o5 q; I

$ c/ H$ ~( t* J+ E  `) mupdate 5/12:/ T1 Y4 n9 Y# J  X* C' A! t* Y3 |+ ]
  T9 V; J2 U- I( S( F

* m/ Z% S2 e8 a0 y% s9 Z- f% X

% O* d* ~5 I6 {& q, T  H* ]. b6 O# Z* Z/ X/ {. Y' I% H6 X
0 {+ B4 Z* F. n, [3 [
inspect成功之后:
8 {" s  |# Q* j2 l+ R& a) I. u1 C; E, ^( n+ H7 ?  U

( W: h1 B! \4 ^/ W+ x7 b" n
2 j5 O' k3 r5 U6 A8 Q4 |0 }
, S2 L# A+ a' e8 W! U
5 K. o' d: m! @/ k0 k& N0 T

. _; b$ m/ l) P; P3 |+ d' I: j: I1 r! ~
inspect失败,原因见“问题2”
6 q" O; Q9 F: u, N) Y3 k) R
2 o; N8 J2 l! L( z5 Y5 `" S7 X( z8 I# b+ w1 w

7 b8 h, R* U* {/ M* k1 \/ s) c4 S  @2 {; r% U1 e( r
3 d# F$ ]2 ?- B4 |

- x8 [8 n3 m8 l) b- U# C6 l配置provisioning_network:/ n5 q# O9 Y! S7 g6 g5 E
2 T+ b! r' T; @- C/ S9 {. j/ }

4 H7 D7 X/ ]) E" H
( R% _( m. q  ~
. g* ]4 Z" |: ?) L- H! b! ^; Z9 M
9 r( a3 }2 H* X! c$ a( q- \' ?7 }# o5 ]- K
7 I5 F# w8 x% V# q' T

) A# o3 j7 w( o7 g1 _  Q' S3 b- H
' `! X8 `) F7 \9 t* R) o! a2 |
. h- \; c! E& X8 }Inspect成功之后:; H3 K; K  A7 k* x

% X# f% c8 d; g
" X+ N6 m7 \% H0 ?
1 D& a& X( @6 w# G: i. [
2 o8 E! d- c6 `. _" h; I) _5 L1 U1 I8 I2 s
/ l! m" n8 ?; C  Q  H6 S
" e' z) H3 S7 P6 B6 k6 r  T% \& I

/ |: B# G" ^. `8 q
1 y) w5 r- y5 i; l: {* n
  |  z% Y& g" o2 |4 H* y上传Ironic使用的镜像:* u) x" c' I0 `7 v  W9 o
glance image-create --name CentOS-7-64bit-ironic.qcow2 --disk-format qcow2 --container-format bare --file CentOS-7-64bit-ironic.qcow2 --is-public True --human-readable —progress3 l1 O: @7 V. Z! v7 X0 {1 m
glance image-update 40928b81-9be1-402a-8684-4e2d2fcf330f --property hypervisor_type=baremetal
' Y  q  S( t6 H7 u
7 j; Z- n' B0 d' F5 o. F4 t9 J" g5 U* A2 R
nova boot --flavor 2 --image 40928b81-9be1-402a-8684-4e2d2fcf330f --nic net-id=3a151049-ff3f-4bc5-88a1-b9084ec24bc9 pc-zjnacthd01
. Z/ B2 T& p+ R6 }8 |6 y( J2 W* R  P; X# `8 l
+ H/ T3 P. n  z, ?* i, R

+ d" D, L5 Q. T$ y5 ^9 t2 u1 l
- }" j- C; \" p' k0 Q# H
5 N; f/ E! H& t8 C== 问题 ==
5 R1 m) F6 N) ^! J$ O5 u) S& l! R
  • node name有限制?2 n+ }& O! [1 ?- O8 r# O
' i6 ^7 l7 G# L! N; {. t

2 m- ?9 Q  @( C8 m4 ?% k* i' K0 {2 o9 h, h

( E! J4 z" q1 T
( j2 K* ~8 M) ~1 j
  • 第一次Inspect失败
    7 K3 \! B7 @$ k3 f
2017-04-20 15:29:16.409 28596 ERROR ironic_inspector.main   File "/usr/lib/python2.7/site-packages/keystoneauth1/access/service_catalog.py", line 228, in url_for4 U* ^3 \  J! s- e9 q9 a2 }
2017-04-20 15:29:16.409 28596 ERROR ironic_inspector.main     raise exceptions.EndpointNotFound(msg)# D" b1 P, _  i' m" |6 _6 w9 t
2017-04-20 15:29:16.409 28596 ERROR ironic_inspector.main EndpointNotFound: public endpoint for baremetal service in RegionFour region not found
4 h1 E7 ?% }# r. Q! ^( l: Q) h8 E/ C3 n0 B) ^; a  q+ ]
重启ironic服务后解决! X, Z+ v7 D/ y  s
% o1 V7 A& H/ Z/ W& W! v" y
  • 第二次inspect失败,BM拿不到IP+ V0 [! D8 A/ r8 c( ?+ @) K
DHCP请求已经发送到dhcp server:/ W; W# }" q" }" L
8 L4 y% Q" v* E  n" g) s% x

( y. v# B. v# G9 j" s) g$ {3 i1 F/ `; L

( b" o- l2 E- [: D: L4 t+ h; N- R/ c+ @. x; c
  • inspect时找不到cleaning_network
    5 U" r2 i0 g8 V/ G8 ^
配置cleaning_network(=provide_network)
" ]: R& r6 ]' {/ N: {7 d( I' r- }- x- ~5 @
  • nova boot失败, conductor.log:
    * n% }1 ^: L6 ^( L2 V
5 ^9 N! I" g1 \  P& q) Z% j

- ~' @$ c, p) R, @  n. c7 q& n- K6 n2 Y2 W5 @1 S' d
- n# i* {$ |; r0 o0 f) ?8 f# ?
更新控制节点的nova代码、ironic节点的ironic代码、计算节点ironicclient代码之后,问题解决: c6 U$ K; y- J" v8 ]! [- k
* J) D" |% O4 ?, S- y0 \7 L9 ]
  • nova boot失败,compute.log
    + I8 z$ E: ?- S
1 X; F0 H3 g9 H( X
3 V0 Y6 R& w" u! y' d# h

$ R& F) r8 N9 V9 v2 N+ n# D$ b$ p* q/ I$ c. i) w+ s2 y
原因是这个ironic node driver_info还没更新:
* S- {- ~8 F. j) m$ |$ X; y) G
% j' ~" b* y1 d1 o
  d& L3 I/ B2 e0 I5 ?9 D
2 m4 l, b9 m4 j, ^5 q8 @/ z/ z
( C/ t, Y5 b7 n更新一下:
3 c9 D& x5 Z* t2 v1 }ironic node-update baa519fc-7c06-40f8-8e5a-5fd3b6e97e01  add driver_info/deploy_kernel=f8205536-070b-4286-8d0c-35e3b8647741, A! B9 B3 z3 W& e8 ?
ironic node-update baa519fc-7c06-40f8-8e5a-5fd3b6e97e01 add driver_info/deploy_ramdisk=302e6438-4d31-429b-8bae-47e225d4ed67" t" g) \9 G* Z  D1 D
update 05/12: ' B  }/ E. }2 a* p; }3 k; U
ironic node-update baa519fc-7c06-40f8-8e5a-5fd3b6e97e01  add driver_info/deploy_kernel=4c1855e5-9b6b-47e2-89e5-3bc351c2ae2e
5 V7 K, G+ ?; _7 I' @, a$ gironic node-update baa519fc-7c06-40f8-8e5a-5fd3b6e97e01 add driver_info/deploy_ramdisk=2f603c85-de92-44ea-b4d0-1396b91102cc' p8 Q8 C7 v* p* @7 X
9 w3 [1 l6 c  {3 }, J. I& m1 R
. P2 m. K" a* T5 Q$ ^5 w& M
; A& u. Y& \; Y/ e2 {' I
: c3 s. R. Q4 E- }% T
4 E8 F: X7 b8 y- ?6 H, t
  • nova boot失败,镜像找不到,compute.log
    2 W  S& f$ ?. X: w
' a4 c, \, @9 t3 Y. I2 T3 U! I/ g
" D' \! ~$ b6 R1 r. Y5 @
  k# W$ D/ ?9 x$ M" K8 s
" G% n1 L) j, A  N
计算节点nova.conf的glance-api配错了:* C6 H% E0 ~3 G- E: t. Z4 j& J& P

. V7 W# m. U9 C3 H% p8 s# W  ]* |

" Q. ]0 U& Z& S% Z2 Z1 R
2 D: `7 `" s% hironic-conductor节点ironic.conf中添加glance api version=1! m  Z- y( F9 {* V$ g% g$ e2 a4 u
5 x% P4 g$ U' ~0 }

; W3 O6 m8 Y4 V5 Q$ }% A
- Y% C: [; x/ }, [7 X/ F9 r3 ^% B/ k8 H2 Q( n5 H
4 W1 J$ A1 Q4 y( K) f& ^2 ^9 n
$ ]7 l5 P# J6 W# w  V
glance_api_version=15 u6 s( q! l: }9 t
+ ?. C- E: p" |  t- X# ]4 h" V
  • nova boot失败,ironic-conductor.log:; P- j8 F7 o2 q! j& M: W
* X7 L/ H4 k! i, }8 Y9 A/ p
: N, y. @' ?: T9 h% X3 W, I
; S- s0 G9 p0 s2 E6 `

. t1 y& I! F! N$ h% _命令行验证,可以在provisioning network d5a284c3-41d3-4eb3-a11f-58a99d3e2eb1上创建port
* w) P+ d/ \) a4 P. A& Q; i0 }0 Z5 C
9 @- G) H* y2 X原因是没有enable LLDP。enable之后:
/ D9 x: J; P7 m
4 L; _8 Z2 n7 D8 ?ironic port-list | awk '{print $2}' | egrep '[0-9]+' | xargs -I 'X' ironic port-delete 'X'
, p/ f8 _- S7 |ironic portgroup-list | awk '{print $2}' | egrep '[0-9]+' | xargs -I 'X' ironic portgroup-delete ‘X'! J3 `( o& c+ z0 u* w) K& q0 p5 V
重新Inspect:
* ?4 }  F0 r) v- F' [2 N7 Q6 \ironic node-set-provision-state baa519fc-7c06-40f8-8e5a-5fd3b6e97e01 manage
% z& ^8 S  ~% ]( d! E/ g. mironic node-set-provision-state baa519fc-7c06-40f8-8e5a-5fd3b6e97e01 inspect
. c) x7 k2 J' L4 k( Y- \8 q# k+ Z
( i4 a# S( s3 @5 n* X- H
1 N1 r$ f6 L0 L

& A0 T9 k: ]* `# ]
4 N/ t2 z( N) |0 X
% ]- E' X& @  Y1 x3 \' q* T
, ]3 g1 H' U+ `0 V
/ n' L: \3 R. J* V5 e
. L: {- u, N& h+ k8 p) b
2 c  \0 F- R0 O1 r( D
  • nova boot失败,找不到用户镜像
    4 b0 [6 N9 g7 W: H, k0 p
原因是glance-registry.conf中的数据库写错了。. b, _+ @2 s# b; A8 Z/ ^9 g1 T9 j

+ a8 f: T- `. N( [3 ~
  • nova boot失败,找不到ramdisk  e( D- ]/ U. N  F7 e
% j% q$ U0 f; J2 _  N$ b( v5 X, S* K

/ Q3 @4 v. ^" S! {* x  ?! Y- w9 R( a; B" r
0 P: G$ K3 Q, c3 F, {4 B% u7 F+ l

3 n  J9 ^! x$ U9 s9 G! J7 Y这个image UUID是配置在ironic node的driver_info里面的,image需要上传到glance
2 N( L% [3 r% v( F# T
& F: r, ^9 U$ H! [3 e上传镜像:
  j' ~) g, f/ S1 Y' i+ r! d2 r
# R/ M- q* n, U; t
! S& j3 @1 L) b
0 p( C% z4 @/ m  G6 a/ t' n- s) j: A+ F& i1 b
4 i' J, e; U) Y& u& r# h* E4 _$ U

# v3 D) D& D" e4 H, o2 P( e
9 r7 ~- z2 N4 j% t  z
1 H5 C+ z' R3 Y7 Z7 ]% W3 e9 T; {" t% [; Z9 E. U7 u* |  c
; p2 ]" f/ O% t) }
$ B$ c/ {- |2 e( V% v( j

4 a* n9 V$ H* e" `+ h' L4 a- T  Q3 [. b$ M" L
更新Ironic node信息:
! a5 `! G2 r5 y% ~8 c' }
7 j1 {: I, B* |  N9 F9 c' f" W) e" E  t, L- A" k

' }1 L% J7 z( G' V( i2 X& Y* x+ A/ s1 {( }
, K' j: T) K+ s$ i2 a) Q9 W& g
  • nova boot失败,访问tftp权限不够% f5 B1 j& _& {2 u- t2 E

/ I' s, v5 n& X5 Q! V# s  z/ y. {4 T+ u. S. n$ o& [9 p7 j

: m# ~: ?* b4 ]2 T: X: w* c6 K6 G" U
! ~. w7 z; G# w4 A+ y) I
6 ?6 B) d, P  J6 m( b# k. Uchown -R ironic:ironic /tftpboot/1 c0 Q& z( E: U% }7 H- I
, f1 P2 \% M* C, ?. ^

( n. v3 ^% X/ a4 V# Y. F
1 K8 {( p0 C7 [% v$ \- B( T* Q: D- U+ F/ }0 Y% O7 z/ F) }; K

; v  u( X6 ~. X
1 z4 Q1 I0 T9 }- F; D9 o# D0 [
  • nova boot失败,物理机DHCP请求被ironic-dhcp捕获了
    ' ~4 r% |5 \: c  Z
关闭ironic-dhcp- z" L6 k& Z* o) ?: S1 }5 b

+ C3 Y3 X. f2 S+ J. |
  • nova boot失败,物理机DHCP时不能从neutron DHCP拿到IP
    0 ?- d+ g% ]8 \8 w# ]
在控制节点上,neutron dhcp在dnsmasq启动的namespace中。relay的目的地址是控制节点管理网IP(eno16777984),dnsmasq的监听设备为namespace的tap口,IP为20.26.34.91,他拿不到dhcp请求。
. h0 u9 B& R% w: I) R4 C; l现在的方法是:在控制节点上手动启动一个dnsmasq,使用neutron dhcp一样的配置
, {4 X* L, ^* n1 P7 x" n; A) w: ^: N4 Y6 O5 H& z
  • 拿到IP之后,进入ramdisk系统,但是重启之后不能进入用户镜像的操作系统
    0 @# Q( @; q* P
查看BIOS的启动设备顺序,发现是- Boot Device Selector : No override; l9 J; d+ H- G, r* t" Q
查看ironic-conductor.log,发现连不上20.26.34.70:9999。这是IPA的地址和监听端口,需要保证ironic-conductor节点能连上,但是的确不通。7 \. O- |1 N* r
0 i3 t' J, U; H5 F1 q
! g7 q# r1 n1 ^

, C( o$ Y  ~1 m8 @9 L
0 k: ?/ c( X- u2 V3 j7 E8 I* O# X) H
姚军说可能是ramdisk启动之后,有两个网口获取到了IP地址,引起路由错乱,建议我们ramdisk启动之后,删除第二个地址。
% _4 z: ~5 P" S! f  K6 W  g8 q6 w' K+ {; Z- x
05/04 update: 在provisioning network上加上静态路由:destination=控制节点网段,nexthop为provisioning network GW& i- H7 t3 r' G; s
05/11 update:neutron subnet-update aca03dd8-3d2a-4c54-99de-7a8a7bac4f53 --host-route destination=20.26.33.0/24,nexthop=20.26.34.18 X9 n( f. p5 o& K; D
Updated subnet: aca03dd8-3d2a-4c54-99de-7a8a7bac4f53) Z, Y/ S) S/ c1 a
: W! n) O& l2 d6 w- u) g8 V4 E( u% y
, ~- H+ R& R1 o) m9 `' j* \

: D; m6 i+ z* I- O  q3 }" k# M4 i1 `6 B! V! o& ~
' p4 g" ^# r3 }) R' j2 u$ k: y2 E$ n
验证可行,能连接这个端口并下载用户镜像: ==> 为啥会有多个网卡获取到IP,如何从代码层面解决?
4 t9 T. K3 M% u  o4 q
6 m0 x: O/ O7 k& b% D5 R# q/ X8 H+ D& Z; j+ ^

7 H& E, a, a1 f+ k0 Q9 F8 q6 i7 H# @+ O; P

1 T1 K, S1 S6 i: Y7 `: H7 }9 p: l: ]* T! `+ m
IPMI查询启动顺序:ipmitool -P Huawei12#$ -H 50.1.65.245 -U root -p 623 -I lanplus chassis bootparam get 5& x5 t6 y# m: ~4 S" [% ~0 M- T
设置硬盘启动:ipmitool -P Huawei12#$ -H 50.1.65.245 -U root -p 623 -I lanplus chassis bootdev disk
4 Q- r! I& D* Z
" A; J, _3 R2 z$ k1 R6 z/ ?+ M: E  D" h1 w1 |, N

: d* F. z6 O7 J* H8 v
$ {/ P2 s1 z4 ~- r1 {+ T0 y
3 |5 V5 Z# k; E0 D5 b* l
  • 用户镜像下载到了/dev/sdl,没有下载到第一个硬盘,并且整个boot过程超时了
    + t; X5 o! z- \

; E3 \, w4 S. N6 J4 V! J
  L8 J  t/ w* v8 v0 M" H
. Q. d# `* n2 T+ d7 ^: K: x' f5 `3 K: n. c: O- Q. O0 e% o  z
a. 姚军修改了ramdisk,固定使用/dev/sda作为写入的硬盘
2 ?+ o5 F/ k  _4 A+ i b. 修改ironic.conf的deploy_callback_timeout=9003 Q3 ~. p: B" [2 y

; U3 _" I- C+ i1 D" MUpdat 05/04: 6 d9 B' B% ^$ _: K; a# _
李灏:ironic node-update 4fae2ae3-0935-4585-8be2-00298015f8f3 replace properties/root_device='{"name": "/dev/sda"}'( N* Z0 ]- [/ O4 K; J2 r
# A4 N% _: l2 p% i0 `
  • 写入了/dev/sda,但是ironic-conductor没有重启机器,导致boot hang死
    + v# @! d$ T. l7 s! B& `. w, k3 ~
journalctl -fu python-ironic-agent查看IPA内的日志
9 w. t+ R+ O' `+ L8 Fjournalctl --no-pager
% Y" ^2 x/ Z1 q4 o9 m! w7 y" S0 L4 A4 I9 r7 ]( g9 T
  • 镜像写入/dev/sda后,IPA执行partprobe /dev/sda失败" D& h9 f/ Y% [. t- u
# W# _3 p; E+ _* m
/ ~' C9 l: K4 F9 F; d9 \" I' m! Y
% g) H( @: ?! s# l
$ u* {' ]- l9 R9 Y4 J( F$ @
ramdisk中的ironic-lib需要打patch:https://review.openstack.org/#/c/444061/+ l5 c4 F6 h: p. f% M" e$ `! ]+ u

. r/ z9 ]( }% J. O4 {5 Q' L; p+ `) P8 ?- w8 A7 P$ j- n
  • ! z5 d7 i' {: ]. i
您需要登录后才可以回帖 登录 | 注册

本版积分规则

返回首页|Archiver|手机版|小黑屋|易陆发现技术论坛 ( 蜀ICP备2026014127号-1 )

GMT+8, 2026-6-12 01:27 , Processed in 0.016365 second(s), 21 queries .

Powered by Discuz! X5.0

© 2001-2026 Discuz! Team.

快速回复 返回顶部 返回列表