|
|
2025-02-22 12:01:52.616 36892 ERROR cinder.volume.manager [None req-7ef5ffd0-64c7-4245-ae99-a626b4ece2dc - - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: cinder.exception.DriverNotInitialized: Volume driver not ready.1 f' {$ F* ?- J" S; N4 ^( R
5 d( a5 M& |" }/ |$ l
. @8 B! q) l; q/ ~+ I2 [
: L" \) {: W u$ l9 W1 B8 U; B9 P: p$ N3 q0 g
$ F; C! R; @7 I" q' D
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)6 @% e; j7 z( Y: B7 R9 A
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager
' R( G( D) D, `8 v2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager During handling of the above exception, another exception occurred:
1 N& h$ N* R8 D8 k" F+ e2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager
$ `. O. M8 S, b. K$ i& j& A/ T2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager Traceback (most recent call last):" E9 l8 ?# g& M9 I
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/manager.py", line 525, in _init_host
' V: Y; r- M2 }( m5 Y2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager self.driver.check_for_setup_error()
+ s3 {6 }9 ]8 Y2 \8 [2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 545, in check_for_setup_error
/ G" Q3 v5 Q& j U9 m! w5 b2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager with RADOSClient(self):
+ ^( |& j7 o2 |6 b4 B2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 270, in __init__1 c9 P& x) H G. x
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager self.cluster, self.ioctx = driver._connect_to_rados(pool): s* H; ?+ h. D9 M
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 630, in _connect_to_rados
$ ~" f; ~* ?1 I* }2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager return _do_conn(pool, remote, timeout) B6 U4 J' t1 p/ r! h
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/utils.py", line 669, in _wrapper
, t" q8 V# N0 u$ a6 l. Z. T3 L \2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager return r(f, *args, **kwargs)
6 m- q# |0 k; [* P- j4 d2 T2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 423, in __call__, G& K* u4 K& k. d; R1 Z
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager do = self.iter(retry_state=retry_state)5 S$ ^; V! c5 }: E! E/ E& Y$ D
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 372, in iter
7 l9 f4 n6 p8 E8 @" J' C2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager raise retry_exc.reraise()
0 P! o* h' m# {" e2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 189, in reraise8 k4 `( M5 H; b0 I: B! @9 U7 O( k
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager raise self.last_attempt.result()
" U2 _9 x4 {8 v* s% n2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 439, in result+ X' H; U+ E# v
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager return self.__get_result()6 C3 l& [5 @" L
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result
1 \" D2 L( b( e4 e2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager raise self._exception; \( A) S/ h4 ^8 f
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 426, in __call__
8 a; J: {+ A9 z2 q2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager result = fn(*args, **kwargs)( [, T z K) e! ]( [
2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 628, in _do_conn
; Z+ ^; U7 r ]5 R9 }" ]3 l; P" b7 h2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager raise exception.VolumeBackendAPIException(data=msg)
& G3 R4 ?" W; G& q2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
( {% e' U" M% G& E2025-02-22 12:06:53.209 37564 ERROR cinder.volume.manager
, E: T* X9 t5 p- g3 w V, R2025-02-22 12:06:57.213 37564 INFO cinder.volume.manager [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Starting volume driver RBDDriver (1.3.0)1 f5 b6 U) f" H; Q
2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Error connecting to ceph cluster.: rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)$ h+ m; Y% I7 S, f
2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):$ m% U2 r: K( k% |2 K
2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 621, in _do_conn" }' \6 ] f1 z; W6 P/ n5 Y
2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd client.connect()! J1 F0 W1 q8 K& F1 v; a0 E
2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd File "rados.pyx", line 690, in rados.Rados.connect
2 F' G0 Z- f& \' K2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)* p! Y1 I1 u' ]% f3 c$ V; Y0 A
2025-02-22 12:06:57.224 37564 ERROR cinder.volume.drivers.rbd " i* P; h" z& h+ }5 w5 N
2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Error connecting to ceph cluster.: rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)
3 {9 d+ l6 S. P5 G' r2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):$ R9 ?0 U, n4 S9 z* Y; Q4 E
2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 621, in _do_conn- f; v* ?7 x- H! P* M4 X( \
2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd client.connect()5 B- n) a; V- ^; k: l( X# v7 M: _
2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd File "rados.pyx", line 690, in rados.Rados.connect; k5 r# R: m1 ^) Y ] J
2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)5 M; b1 x" D; @, W+ U3 I a
2025-02-22 12:07:02.240 37564 ERROR cinder.volume.drivers.rbd ! k, S, o' q- A: T6 `
2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Error connecting to ceph cluster.: rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)9 O; F- `, |2 N+ I( @
2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd Traceback (most recent call last):+ D% Y X. k, W2 l
2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 621, in _do_conn
! s5 Y& O0 X2 K; j0 ?5 {2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd client.connect()4 X, g+ W* B% G' L1 {7 i( b- e
2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd File "rados.pyx", line 690, in rados.Rados.connect1 G( @. E- _& x8 h8 _2 H
2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)
! o! t# p5 W- e* H! A2025-02-22 12:07:12.255 37564 ERROR cinder.volume.drivers.rbd 7 Q/ l1 m6 j. t$ f+ P K
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Failed to initialize driver.: cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.
4 n; `/ ~! C. N0 [2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager Traceback (most recent call last):& |1 n4 |9 L3 Y" S9 p
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 621, in _do_conn& V$ H' ^# {2 E* A: ]* f$ z, a
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager client.connect()9 M$ V, {- a! w% S) \
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "rados.pyx", line 690, in rados.Rados.connect
' h& L3 u. _6 a: p; Y2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager rados.ObjectNotFound: [errno 2] RADOS object not found (error connecting to the cluster)
) ^) B( M; K- W2 S2 B1 f a2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager - a" J9 \: Q, V: H9 g/ H; U
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager During handling of the above exception, another exception occurred:" n4 t) r9 o" h( L3 G+ L* h7 i+ N
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager
- B% v* J6 u# G6 d' o- R2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager Traceback (most recent call last):' W, s" \. a% x, x/ ^! H6 u( W! @4 F+ x
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/manager.py", line 525, in _init_host
( L9 S7 x0 F8 s# [& ^5 @, l o9 _2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager self.driver.check_for_setup_error()
" r8 i8 P0 p4 I4 ~2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 545, in check_for_setup_error
) j0 O) c% y/ U2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager with RADOSClient(self):2 U4 u7 p; f( e: Y- U% _
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 270, in __init__( Q; | d& l" p; U! u, L; d' b! Q
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager self.cluster, self.ioctx = driver._connect_to_rados(pool); z4 o0 k$ e# g% U& a6 j
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 630, in _connect_to_rados4 f9 d+ y; }$ R
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager return _do_conn(pool, remote, timeout)
$ N. Y: G; E0 }2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/utils.py", line 669, in _wrapper# o) x8 q% O) _2 P
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager return r(f, *args, **kwargs)& v, M6 w# @; C) e: Z/ f+ _
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 423, in __call__
1 |5 l. k' M! x% Q. o2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager do = self.iter(retry_state=retry_state)
}5 t+ d2 g* o" v, K2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 372, in iter
- D( D( y) _% S: P& r$ Q2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager raise retry_exc.reraise()
, H8 \, d, U3 Q0 D( ?8 @0 @* c2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 189, in reraise" y: i3 j8 d. G$ e
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager raise self.last_attempt.result()
( V9 \* x1 s$ T) H2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 439, in result
$ F$ \+ l) o. h0 U i" J/ D; d" g" i2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager return self.__get_result()
9 Q/ s+ B- ]8 c4 r; k2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib64/python3.9/concurrent/futures/_base.py", line 391, in __get_result
" `( p- l% a8 p% t; v2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager raise self._exception9 X' u* j' A+ I
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/tenacity/__init__.py", line 426, in __call__
! a- J; R: ]! q7 D6 B4 ~ g0 q6 s2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager result = fn(*args, **kwargs)
# t R* S& @4 a2 T7 t2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager File "/usr/lib/python3.9/site-packages/cinder/volume/drivers/rbd.py", line 628, in _do_conn
( p6 T0 s# K+ H1 x% H8 d2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager raise exception.VolumeBackendAPIException(data=msg)
8 Q% H" n' ]% P( x1 n6 T0 a8 T, z+ n2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager cinder.exception.VolumeBackendAPIException: Bad or unexpected response from the storage volume backend API: Error connecting to ceph cluster.! H4 r+ t$ g$ O" i7 d) t( R
2025-02-22 12:07:12.258 37564 ERROR cinder.volume.manager . j3 V0 M/ t8 a0 R9 b ~6 _
2025-02-22 12:07:12.331 37564 INFO cinder.volume.manager [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Initializing RPC dependent components of volume driver RBDDriver (1.3.0)
6 j; o7 N; ^9 w$ b0 x2025-02-22 12:07:12.332 37564 ERROR cinder.volume.volume_utils [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Volume driver RBDDriver not initialized
! f! s$ `' V8 ^% L! Z2025-02-22 12:07:12.332 37564 ERROR cinder.volume.manager [None req-6738215e-a136-4b82-a376-75651f4cb572 - - - - - -] Cannot complete RPC initialization because driver isn't initialized properly.: cinder.exception.DriverNotInitialized: Volume driver not ready.: v! M) U$ x4 c" f, D) O, [
2025-02-22 12:07:22.334 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".3 x& h" j/ `' w
2025-02-22 12:07:32.337 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".- b: g9 A: R5 u- y. S2 |7 X- C6 N
2025-02-22 12:07:42.338 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".: E& o* E8 n8 E& u+ q( Y# ~
2025-02-22 12:07:52.339 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".1 J1 C8 Z+ Q {/ {
2025-02-22 12:07:58.336 37564 WARNING cinder.volume.manager [None req-c6c8d5f3-ae9e-4485-9559-a2e528e8b5b9 - - - - - -] Update driver status failed: (config name ceph) is uninitialized.
9 d% }' b. E/ {! N! K0 s2025-02-22 12:08:02.341 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".
9 i3 R* f5 }0 o+ D4 J% e2025-02-22 12:08:12.342 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".
. Y9 N0 M* @0 f T* t2025-02-22 12:08:22.344 37564 ERROR cinder.service [-] Manager for service cinder-volume streamcontroller@ceph is reporting problems, not sending heartbeat. Service will appear "down".# X' O$ w! h8 H% V' f# V! d
9 v' V9 s! s1 p. c$ n
# d! n+ x/ b( o2 H8 C3 h
# T8 r# ? @' T, D5 A
|
|