Debian
禁用 VT-d 和中斷重新映射是否會破壞 MSI-X?
所以我有一個
Dell PowerEdge R610
正在執行的伺服器Debian Jessie
,根據一些文件,它的晶片組有故障:https ://www.novell.com/support/kb/doc.php?id=7014344 (Intel 5500 rev 13
)。提供的外殼測試確認了此晶片組的存在:
lspci -nn | grep -qE '8086:(340[36].*rev 13|3405.*rev (12|13|22))' && echo "Interrupt remapping is broken"
它已
VT-d
在 BIOS 中禁用並已GRUB_CMDLINE_LINUX_DEFAULT="quiet intremap=off"
添加到/etc/grub/default
.我的問題是,這是否會影響
MSI-X
NIC 卡的功能(應該在核心之間傳播中斷)?在這個特定的伺服器上,確實只有一個核心在處理中斷,但我不確定這是否是禁用 BIOS 和核心設置的結果。或者還有其他什麼可能在起作用?
(不幸的是,我無法訪問 BIOS 來更改設置並在功能開啟的情況下進行測試)。
謝謝你。
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 0: 43 0 0 0 0 0 0 0 IO-APIC-edge timer 1: 3 0 0 0 0 0 0 0 IO-APIC-edge i8042 8: 1 0 0 0 0 0 0 0 IO-APIC-edge rtc0 9: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi acpi 12: 4 0 0 0 0 0 0 0 IO-APIC-edge i8042 16: 15459 0 0 0 0 0 0 0 IO-APIC-fasteoi megasas 17: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb3 18: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb4 19: 28 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb1 20: 0 0 0 0 0 0 0 0 IO-APIC-fasteoi uhci_hcd:usb6 21: 398 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb2, uhci_hcd:usb5 23: 50805 0 0 0 0 0 0 0 IO-APIC-fasteoi ata_piix 72: 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME 73: 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME 74: 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME 75: 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME 76: 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME 95: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 96: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 97: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 98: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 99: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 100: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 101: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 102: 2 0 0 0 0 0 0 0 PCI-MSI-edge ioat-msix 103: 126464280 0 0 0 0 0 0 0 PCI-MSI-edge eth2-0 104: 92145436 0 0 0 0 0 0 0 PCI-MSI-edge eth2-1 105: 89096948 0 0 0 0 0 0 0 PCI-MSI-edge eth2-2 106: 88057555 0 0 0 0 0 0 0 PCI-MSI-edge eth2-3 107: 81812555 0 0 0 0 0 0 0 PCI-MSI-edge eth2-4 108: 89599897 0 0 0 0 0 0 0 PCI-MSI-edge eth2-5 109: 88031686 0 0 0 0 0 0 0 PCI-MSI-edge eth2-6 110: 87201395 0 0 0 0 0 0 0 PCI-MSI-edge eth2-7 112: 377125739 0 0 0 0 0 0 0 PCI-MSI-edge eth0-0 113: 72913 0 0 0 0 0 0 0 PCI-MSI-edge eth0-1 114: 1890421 0 0 0 0 0 0 0 PCI-MSI-edge eth0-2 115: 162242342 0 0 0 0 0 0 0 PCI-MSI-edge eth0-3 116: 53494 0 0 0 0 0 0 0 PCI-MSI-edge eth0-4 117: 85112 0 0 0 0 0 0 0 PCI-MSI-edge eth0-5 118: 44281 0 0 0 0 0 0 0 PCI-MSI-edge eth0-6 119: 41480 0 0 0 0 0 0 0 PCI-MSI-edge eth0-7 NMI: 39120 194 382 197 337 184 276 186 Non-maskable interrupts LOC: 217815785 1972192 4721954 2056033 2252897 1834404 1894099 1739779 Local timer interrupts SPU: 0 0 0 0 0 0 0 0 Spurious interrupts PMI: 39120 194 382 197 337 184 276 186 Performance monitoring interrupts IWI: 8 0 0 0 0 0 0 0 IRQ work interrupts RTR: 7 0 0 0 0 0 0 0 APIC ICR read retries RES: 133678 35960 21042 9616 10918 8463 8607 8143 Rescheduling interrupts CAL: 1815 3264 1626 3885 1970 3807 1972 4000 Function call interrupts TLB: 896 110 258 139 200 110 187 81 TLB shootdowns TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts MCE: 0 0 0 0 0 0 0 0 Machine check exceptions MCP: 171 171 171 171 171 171 171 171 Machine check polls HYP: 0 0 0 0 0 0 0 0 Hypervisor callback interrupts ERR: 0 MIS: 0
網卡設置:
root@server:~# lspci -v 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) Subsystem: Dell PowerEdge R610 BCM5709 Gigabit Ethernet Flags: bus master, fast devsel, latency 0, IRQ 36 Memory at d6000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [48] Power Management version 3 Capabilities: [50] Vital Product Data Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ Capabilities: [a0] MSI-X: Enable+ Count=9 Masked- Capabilities: [ac] Express Endpoint, MSI 00 Capabilities: [100] Device Serial Number 84-2b-2b-ff-fe-fe-b3-bd Capabilities: [110] Advanced Error Reporting Capabilities: [150] Power Budgeting <?> Capabilities: [160] Virtual Channel Kernel driver in use: bnx2 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) Subsystem: Dell PowerEdge R610 BCM5709 Gigabit Ethernet Flags: bus master, fast devsel, latency 0, IRQ 48 Memory at d8000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [48] Power Management version 3 Capabilities: [50] Vital Product Data Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ Capabilities: [a0] MSI-X: Enable- Count=9 Masked- Capabilities: [ac] Express Endpoint, MSI 00 Capabilities: [100] Device Serial Number 84-2b-2b-ff-fe-fe-b3-bf Capabilities: [110] Advanced Error Reporting Capabilities: [150] Power Budgeting <?> Capabilities: [160] Virtual Channel Kernel driver in use: bnx2 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) Subsystem: Dell PowerEdge R610 BCM5709 Gigabit Ethernet Flags: bus master, fast devsel, latency 0, IRQ 32 Memory at da000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [48] Power Management version 3 Capabilities: [50] Vital Product Data Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ Capabilities: [a0] MSI-X: Enable+ Count=9 Masked- Capabilities: [ac] Express Endpoint, MSI 00 Capabilities: [100] Device Serial Number 84-2b-2b-ff-fe-fe-b3-c1 Capabilities: [110] Advanced Error Reporting Capabilities: [150] Power Budgeting <?> Capabilities: [160] Virtual Channel Kernel driver in use: bnx2 02:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM5709 Gigabit Ethernet (rev 20) Subsystem: Dell PowerEdge R610 BCM5709 Gigabit Ethernet Flags: bus master, fast devsel, latency 0, IRQ 42 Memory at dc000000 (64-bit, non-prefetchable) [size=32M] Capabilities: [48] Power Management version 3 Capabilities: [50] Vital Product Data Capabilities: [58] MSI: Enable- Count=1/16 Maskable- 64bit+ Capabilities: [a0] MSI-X: Enable- Count=9 Masked- Capabilities: [ac] Express Endpoint, MSI 00 Capabilities: [100] Device Serial Number 84-2b-2b-ff-fe-fe-b3-c3 Capabilities: [110] Advanced Error Reporting Capabilities: [150] Power Budgeting <?> Capabilities: [160] Virtual Channel Kernel driver in use: bnx2 root@server:~# ethtool -k eth0 Features for eth0: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: on tx-checksum-ip-generic: off [fixed] tx-checksum-ipv6: on tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: off tx-tcp-segmentation: off tx-tcp-ecn-segmentation: off tx-tcp6-segmentation: off udp-fragmentation-offload: off [fixed] generic-segmentation-offload: off generic-receive-offload: off large-receive-offload: off [fixed] rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off [fixed] receive-hashing: on highdma: on [fixed] rx-vlan-filter: off [fixed] vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] tx-gre-segmentation: off [fixed] tx-ipip-segmentation: off [fixed] tx-sit-segmentation: off [fixed] tx-udp_tnl-segmentation: off [fixed] tx-mpls-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off [fixed] tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off [fixed] busy-poll: off [fixed] root@server:~# ethtool -i eth0 driver: bnx2 version: 2.2.5 firmware-version: 7.12.17 bc 7.10.0 NCSI 2.0.13 bus-info: 0000:01:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: yes supports-register-dump: yes supports-priv-flags: no
是的。MSI-X 需要啟用中斷重新映射才能完全工作。
來自英特爾 PDF: 面向定向 I/O 架構規範的英特爾® 虛擬化技術 2014 年 10 月
5.1 中斷重映射
中斷重映射架構使系統軟體能夠控制和審查由所有源產生的外部中斷請求,包括來自中斷控制器 (I/OxAPIC)、支持 MSI/MSI-X 的設備(包括端點、根埠和 Root-Complex 集成終端)的請求。點。