部署OpenStack-Caracal遇到的问题解决cinder解绑磁盘解决过程
解决办法（不太严谨）根据日志提示
2025-01-27 20:08:29.656 59902 ERROR cinder.volume.api Detected user call to delete in-use attachment. Call must come from the nova service and nova must be configured to send the service token. Bug #2004555
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault Caught error: <class 'cinder.exception.ConflictNovaUsingAttachment'> Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API: cinder.exception.ConflictNovaUsingAttachment: Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault Traceback (most recent call last):
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/middleware/fault.py", line 84, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return req.get_response(self.application)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application(
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/osprofiler/web.py", line 111, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return request.get_response(self.application)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application(
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/keystonemiddleware/auth_token/__init__.py", line 340, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault response = req.get_response(self._app)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1313, in send
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault status, headers, app_iter = self.call_application(
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/request.py", line 1278, in call_application
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault app_iter = application(self.environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/routes/middleware.py", line 153, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault response = self.app(environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 143, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return resp(environ, start_response)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 129, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault resp = self.call_func(req, *args, **kw)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/webob/dec.py", line 193, in call_func
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self.func(req, *args, **kwargs)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 839, in __call__
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return self._process_stack(request, action, action_args,
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 900, in _process_stack
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault action_result = self.dispatch(meth, request, action_args)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 995, in dispatch
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return method(req=request, **action_args)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/openstack/wsgi.py", line 1160, in version_select
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault return func.func(self, *args, **kwargs)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/api/v3/attachments.py", line 282, in delete
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault attachments = self.volume_api.attachment_delete(context, attachment)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/volume/api.py", line 2668, in attachment_delete
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault self.attachment_deletion_allowed(ctxt, attachment)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault File "/usr/local/lib/python3.9/site-packages/cinder/volume/api.py", line 2659, in attachment_deletion_allowed
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault raise exception.ConflictNovaUsingAttachment(instance_id=server_id)
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault cinder.exception.ConflictNovaUsingAttachment: Detach volume from instance 4164eba5-b07e-4546-8d41-c4cdc030c396 using the Compute API
2025-01-27 20:08:29.656 59902 ERROR cinder.api.middleware.fault
提示我using the Compute API
于是我去设置如下的内容
controller节点
vim /etc/nova/nova.conf
send_service_user_token = True #添加
compute节点
vim /etc/nova/nova.conf
send_service_user_token = True #添加
但是没有解决，还是提示同样的问题
于是去查看源代码
在/usr/local/lib/python3.9/site-packages/cinder/volume/api.py
源代码如下
def attachment_deletion_allowed(self,
ctxt: context.RequestContext,
attachment_or_attachment_id,
volume=None):
"""Check if deleting an attachment is allowed (Bug #2004555)
Allowed is based on the REST API policy, the status of the attachment,
where it is used, and who is making the request.
Deleting an attachment on the Cinder side while leaving the volume
connected to the nova host results in leftover devices that can lead to
data leaks/corruption.
OS-Brick may have code to detect it, but in some cases it is detected
after it has already been exposed, so it's better to prevent users from
being able to intentionally triggering the issue.
"""
# It's ok to delete an attachment if the request comes from a service
if self.is_service_request(ctxt):
return
if not attachment_or_attachment_id and volume:
if not volume.volume_attachment:
return
if len(volume.volume_attachment) == 1:
attachment_or_attachment_id = volume.volume_attachment
if isinstance(attachment_or_attachment_id, str):
try:
attachment = objects.VolumeAttachment.get_by_id(
ctxt, attachment_or_attachment_id)
except exception.VolumeAttachmentNotFound:
attachment = None
else:
attachment = attachment_or_attachment_id
if attachment:
if volume:
if volume.id != attachment.volume_id:
raise exception.InvalidInput(
reason='Mismatched volume and attachment')
server_id = attachment.instance_uuid
# It's ok to delete if it's not connected to a vm.
if not server_id or not attachment.connection_info:
return
volume = volume or attachment.volume
nova = compute.API()
LOG.info('Attachment connected to vm %s, checking data on nova',
server_id)
# If nova is down the client raises 503 and we report that
try:
nova_volume = nova.get_server_volume(ctxt, server_id,
volume.id)
except nova.NotFound:
LOG.warning('Instance or volume not found on Nova, deleting '
'attachment locally, which may leave leftover '
'devices on Nova compute')
return
if nova_volume.attachment_id != attachment.id:
LOG.warning('Mismatch! Nova has different attachment id (%s) '
'for the volume, deleting attachment locally. '
'May leave leftover devices in a compute node',
nova_volume.attachment_id)
return
else:
server_id = ''
LOG.error('Detected user call to delete in-use attachment. Call must '
'come from the nova service and nova must be configured to '
'send the service token. Bug #2004555')
raise exception.ConflictNovaUsingAttachment(instance_id=server_id)
修改后的完整的代码如下
def attachment_deletion_allowed(self,
ctxt: context.RequestContext,
attachment_or_attachment_id,
volume=None):
# 服务请求直接允许删除
if self.is_service_request(ctxt):
return True
# 参数验证
if not attachment_or_attachment_id and not volume:
raise exception.InvalidInput(reason='Either attachment or volume must be provided')
# 获取 attachment
if not attachment_or_attachment_id and volume:
if not volume.volume_attachment:
return True
if len(volume.volume_attachment) == 1:
attachment_or_attachment_id = volume.volume_attachment
# 获取 attachment 对象
attachment = None
if isinstance(attachment_or_attachment_id, str):
try:
attachment = objects.VolumeAttachment.get_by_id(
ctxt, attachment_or_attachment_id)
except exception.VolumeAttachmentNotFound:
return True# 如果找不到 attachment，允许删除
else:
attachment = attachment_or_attachment_id
if not attachment:
return True
# 验证 volume 和 attachment 的关系
if volume and volume.id != attachment.volume_id:
raise exception.InvalidInput(
reason='Mismatched volume and attachment')
# 检查是否连接到 VM
server_id = attachment.instance_uuid
if not server_id or not attachment.connection_info:
return True
# 检查 Nova 端的状态
volume = volume or attachment.volume
nova = compute.API()
try:
nova_volume = nova.get_server_volume(ctxt, server_id, volume.id)
except nova.NotFound:
LOG.warning('Instance or volume not found on Nova, deleting '
'attachment locally, which may leave leftover '
'devices on Nova compute')
return True
if nova_volume.attachment_id != attachment.id:
LOG.warning('Mismatch! Nova has different attachment id (%s) '
'for the volume, deleting attachment locally. '
'May leave leftover devices in a compute node',
nova_volume.attachment_id)
return True
# 检查实例的其他附件
try:
attachments = objects.VolumeAttachmentList.get_all_by_instance_uuid(
ctxt, attachment.instance_uuid)
if attachments:
LOG.info("Instance %s has other attachments: %s",
attachment.instance_uuid,
)
return False
else:
LOG.info("No other attachments found for instance %s",
attachment.instance_uuid)
return True
except Exception as e:
LOG.error("Failed to retrieve attachments for instance %s: %s",
attachment.instance_uuid, str(e))
raise exception.VolumeBackendAPIException(reason=str(e))
最终成功解决，但是这个方法不是很严谨
5.1.5）部署nova-conductor
root@openstack-controller1:~# apt install -y nova-api nova-conductor nova-novncproxy nova-scheduler
5.1.6）配置nova-conductor
root@openstack-controller1:~# vim /etc/nova/nova.conf
# 在此模块下面添加下面4行信息
transport_url = rabbit://openstack:openstack123@openstack-vip.stangj.local:5672/
my_ip = 192.168.139.31
# 在此模块下面添加下面一行信息
connection = mysql+pymysql://nova:nova123@openstack-vip.stangj.local/nova_api
# 在此模块下面添加下面一行信息
connection = mysql+pymysql://nova:nova123@openstack-vip.stangj.local/nova
# 在此模块下面添加下面一行信息
auth_strategy = keystone
# 在此模块下面添加下面9行信息
www_authenticate_uri = http://openstack-vip.stangj.local:5000/
auth_url = http://openstack-vip.stangj.local:5000/
memcached_servers = openstack-vip.stangj.local:11211
auth_type = password
project_domain_name = Default
user_domain_name = Default
project_name = service
username = nova
password = nova
# 在此模块下面添加下面3行信息
enabled = true
server_listen = 192.168.139.31
server_proxyclient_address = 192.168.139.31
# 在此模块下面添加下面一行信息
api_servers = http://openstack-vip.stangj.local:9292
# 在此模块下面添加下面一行信息
lock_path = /var/lib/nova/tmp
# 在此模块下面添加下面8行信息
region_name = RegionOne
project_domain_name = Default
project_name = service
auth_type = password
user_domain_name = Default
auth_url = http://openstack-vip.stangj.local:5000/v3
username = placement
password = placement
# 在此模块下面添加下面9行信息
send_service_user_token = true
auth_url = http://openstack-vip.stangj.local:5000/v3
auth_strategy = keystone
auth_type = password
project_domain_name = Default
project_name = service
user_domain_name = Default
username = nova
password = nova 5.1.7）初始化nova数据库
root@openstack-controller1:~# su -s /bin/sh -c "nova-manage api_db sync" nova
root@openstack-controller1:~# mysql -unova -h192.168.139.248 -pnova123 -e "use nova_api ; show tables"
+------------------------------+
| Tables_in_nova_api |
+------------------------------+
| aggregate_hosts |
| aggregate_metadata |
| aggregates |
| allocations |
| build_requests |
................................
................................
| resource_providers |
| traits |
| users |
+------------------------------+
root@openstack-controller1:~# su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
root@openstack-controller1:~# su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova
....
c14b4cfb-a4f6-41a5-8418-a3d3ee04228f
root@openstack-controller1:~# su -s /bin/sh -c "nova-manage db sync" nova
5.1.8）验证 nova cell0 和 cell1 是否正确注册：
root@openstack-controller1:~# su -s /bin/sh -c "nova-manage cell_v2 list_cells" nova
+-------+--------------------------------------+----------------------------------------------------------+-----------------------------------------------------------------+----------+
|Name | UUID | Transport URL | Database Connection | Disabled |
+-------+--------------------------------------+----------------------------------------------------------+-----------------------------------------------------------------+----------+
| cell0 | 00000000-0000-0000-0000-000000000000 | none:/ | mysql+pymysql://nova:****@openstack-vip.stangj.local/nova_cell0 |False |
| cell1 | c14b4cfb-a4f6-41a5-8418-a3d3ee04228f | rabbit://openstack:****@openstack-vip.stangj.local:5672/ | mysql+pymysql://nova:****@openstack-vip.stangj.local/nova |False |
+-------+--------------------------------------+----------------------------------------------------------+-----------------------------------------------------------------+----------+
5.1.9）启动服务
root@openstack-controller1:~# systemctl enable --now \
nova-api \
nova-scheduler \
nova-conductor \
nova-novncproxy
root@openstack-controller1:~# systemctl restart nova-api nova-scheduler nova-conductor nova-novncproxy
页:
[1]