Debian

Xen VM不斷重啟,核心無法啟動init

  • February 23, 2021

我正在使用使用 Xen 管理程序和 libvirt 管理層建構的 VM。在將 Debian 版本從 8 更新到 9 後,我無法連接到 VM,因此我嘗試登錄到執行 VM 的物理機並使用 virsh 重新啟動 VM。但是,在停止並啟動 VM 後,它似乎處於重新啟動循環中。域ID不斷增加;它從 ID 8 開始,現在幾個小時後達到 400+。(這裡shell1是域名。)

$ virsh list --all
Id    Name                           State
----------------------------------------------------
412   shell1                         running

這是virsh console shell1核心日誌的結果

Escape character is ^]
0 000024 (v02 Xen   )
[    0.000000] ACPI: XSDT 0x00000000FC00A550 000054 (v01 Xen    HVM      00000000 HVML 00000000)
[    0.000000] ACPI: FACP 0x00000000FC00A280 0000F4 (v04 Xen    HVM      00000000 HVML 00000000)
[    0.000000] ACPI: DSDT 0x00000000FC001250 008FAC (v02 Xen    HVM      00000000 INTL 20160831)
[    0.000000] ACPI: FACS 0x00000000FC001210 000040
[    0.000000] ACPI: FACS 0x00000000FC001210 000040
[    0.000000] ACPI: APIC 0x00000000FC00A380 000068 (v02 Xen    HVM      00000000 HVML 00000000)
[    0.000000] ACPI: HPET 0x00000000FC00A460 000038 (v01 Xen    HVM      00000000 HVML 00000000)
[    0.000000] ACPI: WAET 0x00000000FC00A4A0 000028 (v01 Xen    HVM      00000000 HVML 00000000)
[    0.000000] ACPI: SSDT 0x00000000FC00A4D0 000031 (v02 Xen    HVM      00000000 INTL 20160831)
[    0.000000] ACPI: SSDT 0x00000000FC00A510 000031 (v02 Xen    HVM      00000000 INTL 20160831)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000018f7fffff]
[    0.000000] Initmem setup node 0 [mem 0x00000000-0x18f7fffff]
[    0.000000]   NODE_DATA [mem 0x18f7f9000-0x18f7fdfff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
[    0.000000]   Normal   [mem 0x100000000-0x18f7fffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x0009efff]
[    0.000000]   node   0: [mem 0x00100000-0xefffefff]
[    0.000000]   node   0: [mem 0x100000000-0x18f7fffff]
[    0.000000] ACPI: PM-Timer IO Port: 0xb008
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 low level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 low level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0xeffff000-0xefffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xf0000000-0xfbffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfc000000-0xffffffff]
[    0.000000] e820: [mem 0xf0000000-0xfbffffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on Xen HVM
[    0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] PERCPU: Embedded 27 pages/cpu @ffff88018f400000 s80960 r8192 d21440 u2097152
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 1549220
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-3.16.0-4-amd64 root=/dev/xvda2 ro elevator=noop console=ttyS0
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] AGP: Checking aperture...
[    0.000000] AGP: No AGP bridge found
[    0.000000] Memory: 6086104K/6282868K available (5247K kernel code, 947K rwdata, 1832K rodata, 1208K init, 840K bss, 196764K reserved)
[    0.000000] Hierarchical RCU implementation.
[    0.000000]  RCU dyntick-idle grace-period acceleration is enabled.
[    0.000000]  RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS:33024 nr_irqs:256 16
[    0.000000] xen:events: Using FIFO-based ABI
[    0.000000] xen:events: Xen HVM callback vector for event delivery is enabled
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] tsc: Detected 3598.992 MHz processor
[    0.012000] Calibrating delay loop (skipped), value calculated using timer frequency.. 7197.98 BogoMIPS (lpj=14395968)
[    0.028004] pid_max: default: 32768 minimum: 301
[    0.036012] ACPI: Core revision 20140424
[    0.051837] ACPI: All ACPI Tables successfully acquired
[    0.060540] Security Framework initialized
[    0.068009] AppArmor: AppArmor disabled by boot time parameter
[    0.080003] Yama: disabled by default; enable with sysctl kernel.yama.*
[    0.092393] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.101268] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.108550] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.120032] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.136390] Initializing cgroup subsys memory
[    0.144026] Initializing cgroup subsys devices
[    0.152026] Initializing cgroup subsys freezer
[    0.160016] Initializing cgroup subsys net_cls
[    0.168021] Initializing cgroup subsys blkio
[    0.176016] Initializing cgroup subsys perf_event
[    0.188011] Initializing cgroup subsys net_prio
[    0.196145] CPU: Physical Processor ID: 0
[    0.204006] CPU: Processor Core ID: 0
[    0.212011] mce: CPU supports 2 MCE banks
[    0.220060] Last level iTLB entries: 4KB 512, 2MB 7, 4MB 7
[    0.220060] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32, 1GB 0
[    0.220060] tlb_flushall_shift: 6
[    0.270008] Freeing SMP alternatives memory: 20K (ffffffff81a1c000 - ffffffff81a21000)
[    0.291417] ftrace: allocating 21701 entries in 85 pages
[    0.352435] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
[    0.407302] smpboot: CPU0: Intel(R) Xeon(R) CPU           X5687  @ 3.60GHz (fam: 06, model: 2c, stepping: 02)
[    0.424050] installing Xen timer for CPU 0
[    0.432177] Performance Events: unsupported p6 CPU model 44 no PMU driver, software events only.
[    0.445857] x86: Booted up 1 node, 1 CPUs
[    0.448018] smpboot: Total of 1 processors activated (7197.98 BogoMIPS)
[    0.452469] NMI watchdog: disabled (cpu0): hardware events not enabled
[    0.456253] devtmpfs: initialized
[    0.468220] pinctrl core: initialized pinctrl subsystem
[    0.472141] NET: Registered protocol family 16
[    0.476238] cpuidle: using governor ladder
[    0.480019] cpuidle: using governor menu
[    0.484089] ACPI: bus type PCI registered
[    0.488013] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.492734] PCI: Using configuration type 1 for base access
[    0.497450] ACPI: Added _OSI(Module Device)
[    0.500013] ACPI: Added _OSI(Processor Device)
[    0.504013] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.508013] ACPI: Added _OSI(Processor Aggregator Device)
[    0.522128] ACPI: Interpreter enabled
[    0.524024] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20140424/hwxface-580)
[    0.535711] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20140424/hwxface-580)
[    0.544033] ACPI: (supports S0 S3 S4 S5)
[    0.548012] ACPI: Using IOAPIC for interrupt routing
[    0.552054] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.568633] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.572021] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
[    0.576031] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    0.580343] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    0.584888] acpiphp: Slot [3] registered
[    0.588159] acpiphp: Slot [4] registered
[    0.592124] acpiphp: Slot [5] registered
[    0.596134] acpiphp: Slot [6] registered
[    0.600131] acpiphp: Slot [7] registered
[    0.604151] acpiphp: Slot [8] registered
[    0.608129] acpiphp: Slot [9] registered
[    0.612098] acpiphp: Slot [10] registered
[    0.616120] acpiphp: Slot [11] registered
[    0.620109] acpiphp: Slot [12] registered
[    0.624112] acpiphp: Slot [13] registered
[    0.628119] acpiphp: Slot [14] registered
[    0.632154] acpiphp: Slot [15] registered
[    0.636117] acpiphp: Slot [16] registered
[    0.640118] acpiphp: Slot [17] registered
[    0.644127] acpiphp: Slot [18] registered
[    0.648119] acpiphp: Slot [19] registered
[    0.652118] acpiphp: Slot [20] registered
[    0.656119] acpiphp: Slot [21] registered
[    0.660149] acpiphp: Slot [22] registered
[    0.664119] acpiphp: Slot [23] registered
[    0.668113] acpiphp: Slot [24] registered
[    0.672117] acpiphp: Slot [25] registered
[    0.676115] acpiphp: Slot [26] registered
[    0.680117] acpiphp: Slot [27] registered
[    0.684115] acpiphp: Slot [28] registered
[    0.688113] acpiphp: Slot [29] registered
[    0.692113] acpiphp: Slot [30] registered
[    0.696139] acpiphp: Slot [31] registered
[    0.700129] PCI host bridge to bus 0000:00
[    0.704018] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.708015] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
[    0.712015] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
[    0.716015] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[    0.720015] pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfbffffff]
[    0.748366] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
[    0.752022] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
[    0.756015] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
[    0.760016] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
[    0.768611] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
[    0.772162] pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
[    0.827260] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)
[    0.831799] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    0.835629] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    0.839445] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)
[    0.845239] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.848110] xen:balloon: Initialising balloon driver
[    0.856057] xen_balloon: Initialising balloon driver
[    0.860266] vgaarb: setting as boot device: PCI:0000:00:03.0
[    0.864000] vgaarb: device added: PCI:0000:00:03.0,decodes=io+mem,owns=io+mem,locks=none
[    0.864021] vgaarb: loaded
[    0.868013] vgaarb: bridge control possible 0000:00:03.0
[    0.872044] init_memory_mapping: [mem 0x190000000-0x197ffffff]
[    0.876187] PCI: Using ACPI for IRQ routing
[    0.882529] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    0.884038] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.890542] hpet0: 3 comparators, 64-bit 62.500000 MHz counter
[    0.896028] amd_nb: Cannot enumerate AMD northbridges
[    0.900036] Switched to clocksource xen
[    0.914008] pnp: PnP ACPI init
[    0.920390] ACPI: bus type PNP registered
[    0.928055] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
[    0.941437] system 00:01: [io  0x08a0-0x08a3] has been reserved
[    0.954150] system 00:01: [io  0x0cc0-0x0ccf] has been reserved
[    0.966208] system 00:01: [io  0x04d0-0x04d1] has been reserved
[    0.978430] system 00:07: [io  0xae00-0xae0f] has been reserved
[    0.989888] system 00:07: [io  0xb044-0xb047] has been reserved
[    1.002139] pnp: PnP ACPI: found 8 devices
[    1.010482] ACPI: bus type PNP unregistered
[    1.026018] NET: Registered protocol family 2
[    1.034611] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[    1.048711] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    1.062300] TCP: Hash tables configured (established 65536 bind 65536)
[    1.076282] TCP: reno registered
[    1.083473] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    1.096895] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    1.111198] NET: Registered protocol family 1
[    1.120773] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    1.132448] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    1.143536] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    1.155975] Unpacking initramfs...
[    1.653611] Freeing initrd memory: 30932K (ffff880034386000 - ffff8800361bb000)
[    1.664987] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    1.671339] software IO TLB [mem 0xebfff000-0xeffff000] (64MB) mapped at [ffff8800ebfff000-ffff8800efffefff]
[    1.680275] microcode: CPU0 sig=0x206c2, pf=0x1, revision=0x14
[    1.685279] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    1.693165] futex hash table entries: 256 (order: 2, 16384 bytes)
[    1.702078] audit: initializing netlink subsys (disabled)
[    1.710578] audit: type=2000 audit(1613960999.504:1): initialized
[    1.717050] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    1.723569] zbud: loaded
[    1.726975] VFS: Disk quotas dquot_6.5.2
[    1.732474] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    1.742456] msgmni has been set to 11947
[    1.749746] alg: No test for stdrng (krng)
[    1.757405] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    1.771209] io scheduler noop registered (default)
[    1.780259] io scheduler deadline registered
[    1.788138] io scheduler cfq registered
[    1.795348] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    1.805509] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    1.817802] GHES: HEST is not enabled!
[    1.825431] xen:grant_table: Grant tables using version 1 layout
[    1.836685] Grant table initialized
[    1.843914] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    1.897443] 00:06: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    1.912105] Linux agpgart interface v0.103
[    1.920107] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
[    1.940049] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.948666] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.958392] mousedev: PS/2 mouse device common for all mice
[    1.966923] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    1.981401] input: Xen Virtual Keyboard as /devices/virtual/input/input2
[    1.995164] input: Xen Virtual Pointer as /devices/virtual/input/input4
[    2.018214] rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
[    2.029967] rtc_cmos 00:02: alarms up to one day, 114 bytes nvram, hpet irqs
[    2.043409] ledtrig-cpu: registered to indicate activity on CPUs
[    2.055112] AMD IOMMUv2 driver by Joerg Roedel <joerg.roedel@amd.com>
[    2.067465] AMD IOMMUv2 functionality not available on this system
[    2.079793] TCP: cubic registered
[    2.086264] NET: Registered protocol family 10
[    2.094962] mip6: Mobile IPv6
[    2.100560] NET: Registered protocol family 17
[    2.108302] mpls_gso: MPLS GSO support
[    2.115426] registered taskstats version 1
[    2.124182] xenbus_probe_frontend: Device with no driver: device/vbd/51712
[    2.136918] xenbus_probe_frontend: Device with no driver: device/vif/0
[    2.150014] rtc_cmos 00:02: setting system clock to 2021-02-22 02:30:00 UTC (1613961000)
[    2.168960] Freeing unused kernel memory: 1208K (ffffffff818ee000 - ffffffff81a1c000)
[    2.184230] Write protecting the kernel read-only data: 8192k
[    2.196185] Freeing unused kernel memory: 884K (ffff880001523000 - ffff880001600000)
[    2.211466] Freeing unused kernel memory: 216K (ffff8800017ca000 - ffff880001800000)
[    2.227230] Failed to execute /init (error -8)
[    2.236746] Starting init: /sbin/init exists but couldn't execute it (error -8)
[    2.251559] Starting init: /bin/sh exists but couldn't execute it (error -8)
[    2.265834] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.
[    2.269815] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.16.0-4-amd64 #1 Debian 3.16.39-1
[    2.269815] Hardware name: Xen HVM domU, BIOS 4.8.5 01/10/2020
[    2.269815]  0000000000000000 ffffffff81514c11 ffffffff81705310 ffff8801880f3f40
[    2.269815]  ffffffff8151195e ffffffff00000008 ffff8801880f3f50 ffff8801880f3ef0
[    2.269815]  ffff8801880f3ef8 0000000000000046 000000000000093d 000000000000093d
[    2.269815] Call Trace:
[    2.269815]  [<ffffffff81514c11>] ? dump_stack+0x5d/0x78
[    2.269815]  [<ffffffff8151195e>] ? panic+0xc8/0x206
[    2.269815]  [<ffffffff81507da0>] ? rest_init+0x80/0x80
[    2.269815]  [<ffffffff81507e82>] ? kernel_init+0xe2/0xf0
[    2.269815]  [<ffffffff8151ad18>] ? ret_from_fork+0x58/0x90
[    2.269815]  [<ffffffff81507da0>] ? rest_init+0x80/0x80
[    2.269815] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)

然後ID繼續增加。

我可以做些什麼來調試這個問題?我不想使用 virsh destroy 來破壞文件系統的風險。我應該嘗試備份與 VM 儲存池關聯的 OpenAFS 卷嗎?我也無法讓 guestmount 使用,guestmount -m /dev/sda --ro -a /dev/opc-vg.shell1/shell1-disk /mnt因為它說不允許操作。

當我更新時,我的虛擬機磁碟空間不足。然後當我重新啟動時,核心無法正確載入 init,可能是由於沒有為新核心正確創建新的 initramfs。幸運的是,我仍然可以使用 暫停 VMvirsh suspend或終止 VM 從崩潰(XML 中的預設行為)重新啟動virsh destroy,然後使用 掛載 VM 的捲guestmount並複製文件。與嘗試恢復損壞的核心相比,創建一個新的虛擬機並重新同步文件要容易得多,所以這就是我要做的。

引用自:https://serverfault.com/questions/1054546