Libvirt

啟動實例時權限被拒絕錯誤(“無法打開備份文件”)

  • March 9, 2018

使用 OpenStack,從 Debian 測試 (buster) 上可用的軟體包安裝。當我嘗試啟動一個實例時,它在 nova-compute.log 中使用此消息建構後失敗:

無法打開支持文件:無法打開“/var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e”:權限被拒絕

該文件具有權限:

$ namei -l /var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e
f: /var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e
drwxr-xr-x root         root         /
drwxr-xr-x root         root         var
drwxr-xr-x root         root         lib
drwxr-xr-x nova         nova         nova
drwxr-xr-x nova         nova         instances
drwxr-xr-x nova         nova         _base
-rw-r--r-- libvirt-qemu libvirt-qemu affe96668a4c64ef380ff1c71b4caec17039080e

我發現了一個相關的問題(https://bugs.launchpad.net/openstack-helm/+bug/1714421),但它的更新檔(在 /etc/libvirt/qemu.conf 中設置使用者和組)不起作用。

據我所知,QEMU 程序以 root 身份執行,所以我不確定哪個使用者在哪個程序下實際上被拒絕了權限。

或者:openstack 或 libvirt 的哪個部分創建該文件並將“libvirt-qemu”設置為使用者和組?

nova-compute.log 的相關部分:

2018-03-02T05:15:27.083155Z qemu-system-x86_64: -drive file=/var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Could not open backing file: Could not open '/var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e': Permission denied
2018-03-01 23:15:27.373 2376 ERROR nova.virt.libvirt.driver [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Failed to start libvirt guest: libvirtError: internal error: process exited while connecting to monitor: outputs=1,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on
2018-03-02T05:15:27.083155Z qemu-system-x86_64: -drive file=/var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Could not open backing file: Could not open '/var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e': Permission denied
2018-03-01 23:15:27.374 2376 INFO os_vif [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:b4:ed:53,bridge_name='brq03b5dd02-ac',has_traffic_filtering=True,id=042a5f68-01bb-453e-bb2a-2d798b7691d5,network=Network(03b5dd02-ac2f-49f0-b1ff-fa26059f352c),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap042a5f68-01')
2018-03-01 23:15:27.434 2376 INFO nova.virt.libvirt.driver [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Deleting instance files /var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05_del
2018-03-01 23:15:27.436 2376 INFO nova.virt.libvirt.driver [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Deletion of /var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05_del complete
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Instance failed to spawn: libvirtError: internal error: process exited while connecting to monitor: outputs=1,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on
2018-03-02T05:15:27.083155Z qemu-system-x86_64: -drive file=/var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Could not open backing file: Could not open '/var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e': Permission denied
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Traceback (most recent call last):
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2192, in _build_resources
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     yield resources
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2007, in _build_and_run_instance
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     block_device_info=block_device_info)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2814, in spawn
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     destroy_disks_on_failure=True)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5294, in _create_domain_and_network
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     destroy_disks_on_failure)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     self.force_reraise()
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     six.reraise(self.type_, self.value, self.tb)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5264, in _create_domain_and_network
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     post_xml_callback=post_xml_callback)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5182, in _create_domain
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     guest.launch(pause=pause)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 144, in launch
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     self._encoded_xml, errors='ignore')
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     self.force_reraise()
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     six.reraise(self.type_, self.value, self.tb)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 139, in launch
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     return self._domain.createWithFlags(flags)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     result = proxy_call(self._autowrap, f, *args, **kwargs)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     rv = execute(f, *args, **kwargs)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     six.reraise(c, e, tb)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     rv = meth(*args, **kwargs)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 1099, in createWithFlags
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] libvirtError: internal error: process exited while connecting to monitor: outputs=1,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] 2018-03-02T05:15:27.083155Z qemu-system-x86_64: -drive file=/var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05/disk,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Could not open backing file: Could not open '/var/lib/nova/instances/_base/affe96668a4c64ef380ff1c71b4caec17039080e': Permission denied
2018-03-01 23:15:27.813 2376 ERROR nova.compute.manager [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] 
2018-03-01 23:15:27.815 2376 INFO nova.compute.manager [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Terminating instance
2018-03-01 23:15:27.822 2376 INFO nova.virt.libvirt.driver [-] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Instance destroyed successfully.
2018-03-01 23:15:27.824 2376 INFO os_vif [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] Successfully unplugged vif VIFBridge(active=False,address=fa:16:3e:b4:ed:53,bridge_name='brq03b5dd02-ac',has_traffic_filtering=True,id=042a5f68-01bb-453e-bb2a-2d798b7691d5,network=Network(03b5dd02-ac2f-49f0-b1ff-fa26059f352c),plugin='linux_bridge',port_profile=<?>,preserve_on_delete=False,vif_name='tap042a5f68-01')
2018-03-01 23:15:28.107 2376 INFO nova.virt.libvirt.driver [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Deletion of /var/lib/nova/instances/9ac0d3ec-35ad-4420-ae20-f6c0c9845f05_del complete
2018-03-01 23:15:28.501 2376 INFO nova.compute.manager [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] [instance: 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05] Took 0.68 seconds to destroy the instance on the hypervisor.
2018-03-01 23:15:29.721 2376 WARNING nova.scheduler.client.report [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] Unable to delete allocation for instance 9ac0d3ec-35ad-4420-ae20-f6c0c9845f05: (500 {"computeFault": {"message": "The server has either erred or is incapable of performing the requested operation.", "code": 500}})
2018-03-01 23:15:29.721 2376 WARNING nova.compute.manager [req-3cb9fbc5-8f4e-4244-8b14-b52ac1f0494b a6f48b630f634371ba94558f3ba576b8 5f46526ffab3410a9cf71b37fa242e11 - default default] 6 consecutive build failures

如果需要,我可以發布額外的日誌。

這是因為 debian libvirt 包中包含的 libvirt 的 AppArmor 配置文件不包括對 nova 實例目錄 (/var/lib/nova/instances) 的訪問,並且 openstack 包沒有任何 apparmor 配置文件。

通過將以下行添加到以下行來修復此錯誤/etc/apparmor.d/usr.lib.libvirt.virt-aa-helper

 /var/lib/nova/instances/_base/ r,
 /var/lib/nova/instances/_base/* r,
 /var/lib/nova/instances/** rw,

並執行:

sudo apparmor_parser -r /etc/apparmor.d/usr.lib.libvirt.virt-aa-helper

引用自:https://serverfault.com/questions/899614