openstack 相关gpu配置

admin · 发表于 2022-6-10 22:02:28

OpenStack的相关配置
1. 配置nova-scheduler （controller节点），编辑文件 /etc/nova/nova.conf:
[DEFAULT]
scheduler_default_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, ComputeFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, PciPassthroughFilter
scheduler_available_filters = nova.scheduler.filters.all_filters
重启nova-scheduler服务
[root@controller ~]# systemctl restart openstack-nova-scheduler.service

[root@controller ~]# systemctl status openstack-nova-scheduler.service
● openstack-nova-scheduler.service - OpenStack Nova Scheduler Server
Loaded: loaded (/usr/lib/systemd/system/openstack-nova-scheduler.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2022-06-10 21:50:56 CST; 22s ago
Main PID: 105509 (nova-scheduler)
Tasks: 9 (limit: 100963)
Memory: 276.0M
CGroup: /system.slice/openstack-nova-scheduler.service
         ├─105509 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105528 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105529 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105530 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105531 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105532 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105533 /usr/bin/python3 /usr/bin/nova-scheduler
         ├─105534 /usr/bin/python3 /usr/bin/nova-scheduler
         └─105535 /usr/bin/python3 /usr/bin/nova-scheduler
Jun 10 21:50:52 controller systemd[1]: Starting OpenStack Nova Scheduler Server...
Jun 10 21:50:56 controller systemd[1]: Started OpenStack Nova Scheduler Server.

2. 配置nova-api （controller节点），编辑文件 /etc/nova/nova.conf:
[pci]
alias = { "name": "nvidia1080", "product_id": "1b06", "vendor_id": "10de", "device_type": "type-PCI" }
[pci]
alias = { "name": "nvidiaGF119","product_id": "104a","vendor_id": "10de","device_type": "type-PCI" }

重启nova-api服务

[root@controller ~]# systemctl restart openstack-nova-api.service

3. 配置nova-compute（compute 节点），编辑文件/etc/nova/nova.conf:
[pci]
passthrough_whitelist = { "vendor_id": "10de", "product_id": "104a" }
alias = {
   "name": "nvidiaGF119",
   "product_id": "104a",
   "vendor_id": "10de",
   "device_type": "type-PCI"
}

[pci]
passthrough_whitelist = { "vendor_id": "10de", "product_id": "104a" }
alias = { "name": "nvidiaGF119", "product_id": "104a", "vendor_id": "104a", "device_type": "type_PCI" }

重启nova-compute服务
[root@compute01 ~]# systemctl restart openstack-nova-compute.service

[root@compute01 ~]# systemctl restart openstack-nova-compute.service
[root@compute01 ~]# tail -f /var/log/nova/
nova-compute.log privsep-helper.log
[root@compute01 ~]# tail -f /var/log/nova/nova-compute.log
2022-06-10 22:10:51.891 12258 INFO oslo.privsep.daemon [req-e600b0bc-1cc4-4e85-a406-4d0c094560ee - - - - -] Spawned new privsep daemon via rootwrap
2022-06-10 22:10:51.796 12299 INFO oslo.privsep.daemon [-] privsep daemon starting
2022-06-10 22:10:51.800 12299 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0
2022-06-10 22:10:51.804 12299 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN/CAP_NET_ADMIN/none
2022-06-10 22:10:51.804 12299 INFO oslo.privsep.daemon [-] privsep daemon running as pid 12299
2022-06-10 22:10:52.437 12258 INFO os_vif [req-e600b0bc-1cc4-4e85-a406-4d0c094560ee - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:bf:2a:4e,bridge_name='qbr24719437-3e',has_traffic_filtering=True,id=24719437-3ee6-469b-af02-c1fcea041be2,network=Network(b83e2ffc-eaad-455f-b299-18e09d58be32),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap24719437-3e')
2022-06-10 22:10:52.459 12258 INFO os_vif [req-e600b0bc-1cc4-4e85-a406-4d0c094560ee - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:fe:4c:d1,bridge_name='qbr58f2e526-38',has_traffic_filtering=True,id=58f2e526-386b-43da-9818-208b6a34b6e8,network=Network(5eb067d8-cd9b-4eec-ac0b-b5982752e75d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap58f2e526-38')
2022-06-10 22:10:52.478 12258 INFO os_vif [req-e600b0bc-1cc4-4e85-a406-4d0c094560ee - - - - -] Successfully plugged vif VIFBridge(active=True,address=fa:16:3e:bd:8b:42,bridge_name='qbr24c6e701-e5',has_traffic_filtering=True,id=24c6e701-e5b4-4277-9895-cc67a4097280,network=Network(5eb067d8-cd9b-4eec-ac0b-b5982752e75d),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap24c6e701-e5')
2022-06-10 22:10:52.481 12258 INFO nova.compute.manager [req-e600b0bc-1cc4-4e85-a406-4d0c094560ee - - - - -] Looking for unclaimed instances stuck in BUILDING status for nodes managed by this host
2022-06-10 22:10:54.740 12258 INFO nova.virt.libvirt.host [req-e600b0bc-1cc4-4e85-a406-4d0c094560ee - - - - -] kernel doesn't support AMD SEV

三验证
1. 创建设置flavor：
openstack flavor create --public --ram 2048 --disk 20 --vcpus 2 m1.large
openstack flavor set m1.large --property pci_passthrough:alias='nvidia1080:2'
nvidia1080 即为alias中的那么， 2为GPU的数量。
2. 创建instance：
openstack server create --flavor m1.large --image cirros-0.3.5-x86_64-uec --wait test-pci
3. 在cirros下查看GPU信息如下：
$ lspci -k
...
00:05.0 Class 0300: 10de:1b06
00:06.0 Class 0300: 10de:1b06
...
四 NVIDIA显卡的问题
因为NIVIDIA显卡的驱动会检测是否跑在虚拟机里，如果在虚拟机里驱动就会出错，所以我们需要对显卡驱动隐藏hypervisor id。在OpenStack的Pile版本中的Glance 镜像引入了img_hide_hypervisor_id=true的property，所以可以对镜像执行如下的命令隐藏hupervisor id：
$ openstack image set IMG-UUID --property img_hide_hypervisor_id=true
通过此镜像安装的instance就会隐藏hypervisor id。
如果是Pike之前的版本，可以参考Consumer-grade GPUs in an OpenStack system (NVIDIA GPUs)这篇文章的做法。
可以通过下边的命令查看hypervisor id是否隐藏：
$ cpuid | grep hypervisor_id
hypervisor_id = "KVMKVMKVM "
hypervisor_id = "KVMKVMKVM "
上边的显示结果说明没有隐藏，下边的显示结果说明已经隐藏：
$ cpuid | grep hypervisor_id
hypervisor_id = "  @  @ "
hypervisor_id = "  @  @ "

admin · 发表于 2022-6-10 22:13:07

admin · 发表于 2022-6-10 22:17:35

[root@controller ~]# openstack flavor set m1.large --property pci_passthrough:alias='nvidiaGF119:1'

这里的值必须和nova.conf中的值一样
否则报错

		自动登录	找回密码
密码			注册

openstack 相关gpu配置

浏览过的版块