ERROR nova.scheduler.utils qemu unexpectedly closed the monitor

admin

2022-10-20 10:45:08.266 37 ERROR nova.scheduler.utils [req-035fc933-de6b-448a-978e-c15ae171daeb a3cfa6f70675429985132daa6d3b906e 0c915a0d7601493cb1ecfd904b09f54c - default default] [instance: 52baa020-426b-4800-9492-0654b834ef2a] Error from last host: compute07 (node compute07): [u'Traceback (most recent call last):\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1863, in _do_build_and_run_instance\n    filter_properties, request_spec)\n', u'  File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2143, in _build_and_run_instance\n    instance_uuid=instance.uuid, reason=six.text_type(e))\n', u'RescheduledException: Build of instance 52baa020-426b-4800-9492-0654b834ef2a was re-scheduled: internal error: qemu unexpectedly closed the monitor: 2022-10-20 10:44:54.063891 7fded43821c0 -1 Errors while parsing config file!\n2022-10-20 10:44:54.063897 7fded43821c0 -1 parse_file: cannot open /etc/ceph/ceph.conf: (13) Permission denied\n2022-10-20T02:44:54.292119Z qemu-kvm: -drive file=rbd:hdd-8T-volumes/volume-41da2d70-d3bd-4187-9f60-8b262286f292:id=cinder:auth_supported=cephx\\;none:mon_host=192.168.0.75\\:6789\\;192.168.0.77\\:6789\\;192.168.0.79\\:6789\\;192.168.0.86\\:6789\\;192.168.0.89\\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,serial=41da2d70-d3bd-4187-9f60-8b262286f292,cache=none,discard=unmap: error reading header from volume-41da2d70-d3bd-4187-9f60-8b262286f292: No such file or directory\n']

; ?6 T* {/ R8 \8 P
8 b" s; x7 Z* B: v; H4 e* ^  ?) f应该是ceph中pool池的权限问题
) t: v1 s; d$ o7 k( j

admin

[root@controller1 src]# ceph auth caps client.nova mon 'allow r' osd 'allow class-read object_prefix rbd_children,allow rwx pool=images,allow rwx pool=ssd-volumes,allow rwx pool=hdd-2.4T-volumes,allow rwx pool=hdd-8T-volumes,allow rwx pool=nova-vms,allow rwx pool=backups'
) e& W# Z* l# j* d5 `, Z0 nupdated caps for client.nova

admin

0
" p4 }' f5 n2 B0 O
* B2 n3 H4 h  B' _8 k* f, I
4 d, z7 F1 j! B( y9 N  l( A7 Q4 mAs noted in Openstack docs [1], rebuild is not supported for volume-based VMs.

Assuming you are trying to rebuild the VM with a fresh image ("factory reset"), a slightly "hacky" solution is to replace the underlying volume with the fresh image while keeping the same pool/path.

For Ceph based backend, it would be something like

9 S( m& k3 u- O7 u5 G" a$ N* popenstack volume show <...> => Get pool/path from rbd backend
2 s" P' X6 h5 Z' ~. F0 ^openstack image show <...> => Get pool/path from rbd backend
openstack server stop <VM> => Stop the VM
4 l# J9 p( ]1 _# ~6 C
. K/ e* T3 x! g: a; B& \$ e. }rbd -p <POOL NAME> mv <VM VOLUME UUID>  <VM VOLUME UUID>.old
( F' H' q: R3 C2 D" ?3 erbd -p <POOL NAME> cp <IMAGE VOLUME UUID> <VM VOLUME UUID>
, b  }; I+ f( d! nOnce the copy finishes, start the VM

8 q7 e( P- E2 v# ^( mopenstack server start <VM>

admin

Nova:
8 W( R/ T( Y4 s, a& a1 K4 {; _List of instances
, I" L9 o8 d% c6 Y/ Z1
( F0 W6 J: b( m1 D" r/ jopenstack server list
# T4 q4 |* k- i$ }8 j5 s2
nova list
Details on a specific instance
; h- s8 n" t' |1 Q2 k- h0 a1
openstack server show <server id>
2
nova show <server id>
$ t: T8 o& W/ k/ p& K2 \Rebuild an instance
9 Y) w, |  m* Q: I0 P+ L' Q1
0 I* w! x4 F) T, b. q+ R0 T2 }: Bopenstack server rebuild --image <image-id> <server-id>
2 m) ]% u. |- v( j2
2 m  e3 L* G* F; Q0 v/ E) z* Cnova rebuild <server-id> <image-id>
9 q% Q' Y% @3 w7 GRebuild an instance with metadata
) Z7 D. V- G: l+ b4 m/ X1
openstack server set --property <meta> <server-id>
3 p( @2 E, F3 C1 @( j( @& ~2
# Q0 v  Q* [  f3 pnova meta <server-id> set <meta>
2 T% Z2 Y0 m1 `3
2 x) p6 ~& s. k5 F" g* cnova rebuild --meta <meta> <server-id> <image-id>
Attach a port to an instance
' ^4 z. t+ `- e) |8 v$ G) I+ e1
nova interface-attach --port-id <port id> <instance id>
* C- j0 I* p1 c* G5 q6 T- O8 ^Create a instance from a availability zone
2 w, k% B6 u& b' J+ z9 _7 s1
: _& m3 G( J1 z9 t1 Qnova boot --availability-zone nova:metal01:<ironic-id> --flavor <flavor-id> --image <img-id> --nic net-id=<network id>,v4-fixed-ip=<ip-address> <name>
3 P. V; a$ k7 q* r9 q5 U9 P  y) J2
openstack server create --availability-zone nova:metal01:<ironic-id> --flavor <flavor-id> --image <img-id> --nic net-id=<network id>,v4-fixed-ip=<ip-address> <name>
Create a instance with meta property, sec group, fixed ip, ssh-key
7 Q1 e4 Y( I2 A0 n1
3 n$ K( |" Y/ k8 a7 D; {openstack server create --image <image id> --flavor <flavor id> --nic net-id=<network id>,v4-fixed-ip=<ip-address> --property <meta-data> --security-group <security group id> --key-name <keypair name> vmname
Create a instance
" D) ~7 g. r( T3 h1 b' P+ m" Q1 d1
. [, U5 d$ d0 j& }openstack server create --image <image id> --flavor <flavor id> vmname
Remove error state from instance
1
nova reset-state --active <server-id>
7 M! b; J- R9 D5 {# ~Change the flavour (resize)
4 l# ]) E4 Z! i9 l" U1
  R& O5 r8 m: c/ H/ ^openstack server resize --flavor <flavor> <ID>
Neutron:
List of subnets and network ID
: h6 l' A2 R. i# q1
+ u9 |% E! |5 K: z1 t6 w& jopenstack network list
2
neutron net-list
4 T2 I4 ~! P% OList of subnets, their IDs and allocation pools:
3 s9 K! F+ o% R* e. m1
openstack subnet list
2
( U" k$ g- r, I9 ]' qneutron subnet-list
List ports
1
openstack port list
5 L5 [# C& O# ]6 I& L8 v2
neutron port-list
5 ~' l: ]. ~1 y* T8 J! nCreate port
1
7 ]5 n( \/ D! Y; V& Yopenstack port create --network <network id> --fixed-ip subnet=<subnet id>,ip-address=<ip-address>
2
neutron port-create --fixed-ip subnet_id=<subnet id>,ip_address=<X.X.X.X> <network id> <name>
Update or Set/Unset port
1
0 }; R" x4 ^3 O" y1 G' s" bneutron port-update --fixed-ip subnet_id=<subnet id>,ip_address=<ip-address> --fixed-ip subnet_id=<subnet id>,ip_address=<X.X.X.X> <port id>
2
$ F5 B0 V0 w' y- q' o! G6 dopenstack port set --fixed-ip subnet=<subnet>,ip-address=<ip-address> <port id>
. M  s8 c: n; ~. X5 y) `, E) r3
6 k" M, Q! ?2 L0 Z+ Xopenstack port unset --fixed-ip subnet=<subnet>,ip-address=<ip-address> <port id>
Get details on a port
1
openstack port show <port id>
2
' R% e# C3 u2 G) [neutron port-show <port id>
5 B0 j' J$ R" g  t6 K$ {/ TAllowed address pairs - (Allows one port to add additional IP/MAC address pairs on that port to allow traffic that matches those specified values.)
1
. ]! Z3 C9 t/ _. i# U0 w( oneutron port-update <port-uuid> --allowed-address-pairs type=dict list=true ip_address='0.0.0.0/0'
0 L) V8 [7 F! E- o& j4 l& W5 l$ oGlance:
! L2 o( G4 ^1 u2 B0 H- yList images
1
7 I: q7 X8 ?9 V/ c' ]openstack image list
0 e4 c5 c7 Q: k8 h: r" Q2 F2
glance image-list
Get details on an image
1
- b# q4 j( ]8 D2 ]glance image-show <image id>
7 l0 a5 j* {8 X: w( n4 JUpload a image
1
openstack image create --disk-format qcow2 --container-format bare --public --file ./centos7.qcow2 centos7-image
$ L" L  l' [% uDelete
  O8 n& k) E# o* U1
openstack image delete <ID>
Download a image
1
" A8 j+ [; E, j8 m0 a4 _glance image-download <img-id> --file img.qcow2
& M+ |- k" H% O: @7 J- YChange state
; ^2 |- n; R& ?5 w$ c- Z1
cinder reset-state --state in-use <id>
Other:
7 K& x' ^6 L2 Z6 D; JList flavors
  ]$ b* P/ I. Z5 O- e. F3 L1
2 p4 a3 ^7 o8 ~1 O4 iopenstack flavor list
6 k; y+ l" K6 F3 y2 I7 ?8 mList keypairs
1
6 v& V0 F& ^) w) y, R" R4 dopenstack keypair list
List Security groups
- P$ A0 {, L1 y. ]8 |* \1
openstack security group list
Cinder
1
openstack volume set --state error <id>
% X: K2 Y9 n+ L$ vStop a migraion
; {  @7 M. ?; ~# i
Remove the active volume from the migration state
2 d7 z  Q, M" _& n, R9 G) E
* W1 q4 _+ `+ _1 b0 K, e1
cinder reset-state <id> --reset-migration-status
* c; Y6 i) D2 n6 d: l% S  N2 rfind the volume in the attaching state and remove it

/ |7 G2 J9 v1 f1
openstack volume list
1
3 H9 `* I3 r8 T( T0 {& j) mcinder reset-state <id> --state error
" L1 p% [  X1 R: n6 |$ G/ X6 ]8 r1
cinder reset-state <id> --reset-migration-status
openstack volume delete <id>
0 S" [- j8 c! P( W1
openstack volume delete <id>