調整用作 VMWare NFS 數據儲存的捲的大小會導致 Linux VM 問題

我們必須在我們的 Netapp 上對 VMWare 使用的 NFS 數據儲存執行實時卷大小調整。調整大小後，我們所有的 Windows 虛擬機都很好。但是，我們的一些 Linux 虛擬機存在問題。

一些 Linux 虛擬機剛剛停止響應。重新啟動這些虛擬機後，我在日誌中找不到任何表明存在問題的內容。

但是，我確實在某些 VM 上找到了此類日誌消息：

May 29 14:56:02 rhel6-server-1314 kernel: INFO: task jbd2/dm-0-8:382 blocked for more than 120 seconds.
May 29 14:56:02 rhel6-server-1314 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 29 14:56:02 rhel6-server-1314 kernel: jbd2/dm-0-8   D 0000000000000000     0   382      2 0x00000000
May 29 14:56:02 rhel6-server-1314 kernel: ffff880037ce9c20 0000000000000046 ffff880037ce9be0 ffffffffa00041fc
May 29 14:56:02 rhel6-server-1314 kernel: ffff880037ce9b90 ffffffff81012b59 ffff880037ce9bd0 ffffffff8109b809
May 29 14:56:02 rhel6-server-1314 kernel: ffff880037ce1af8 ffff880037ce9fd8 000000000000f4e8 ffff880037ce1af8
May 29 14:56:02 rhel6-server-1314 kernel: Call Trace:
May 29 14:56:02 rhel6-server-1314 kernel: [<ffffffffa00041fc>] ? dm_table_unplug_all+0x5c/0x100 [dm_mod]
...             rhel6-server-1314
May 29 14:56:02 rhel6-server-1314 kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20
May 29 14:56:02 rhel6-server-1314 kernel: INFO: task master:1674 blocked for more than 120 seconds.
May 29 14:56:02 rhel6-server-1314 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 29 14:56:02 rhel6-server-1314 kernel: master        D 0000000000000000     0  1674      1 0x00000080
May 29 14:56:02 rhel6-server-1314 kernel: ffff88003d669958 0000000000000086 ffff88003d669918 ffffffffa00041fc
May 29 14:56:02 rhel6-server-1314 kernel: 0000000000000000 ffff880002216028 ffff880002215fc0 ffff88003fac2b78
May 29 14:56:02 rhel6-server-1314 kernel: ffff88003fac30f8 ffff88003d669fd8 000000000000f4e8 ffff88003fac30f8
May 29 14:56:02 rhel6-server-1314 kernel: Call Trace:
May 29 14:56:02 rhel6-server-1314 kernel: [<ffffffffa00041fc>] ? dm_table_unplug_all+0x5c/0x100 [dm_mod]
...             rhel6-server-1314
May 29 14:56:02 rhel6-server-1314 kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
May 29 14:56:02 rhel6-server-1314 kernel: INFO: task pickup:6197 blocked for more than 120 seconds.
May 29 14:56:02 rhel6-server-1314 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
May 29 14:56:02 rhel6-server-1314 kernel: pickup        D 0000000000000000     0  6197   1674 0x00000080
May 29 14:56:02 rhel6-server-1314 kernel: ffff88003da95968 0000000000000086 ffff88003da95928 ffffffffa00041fc
May 29 14:56:02 rhel6-server-1314 kernel: ffff88003da95938 ffff8800022128a0 ffff88003da95908 ffffffff81127ed0
May 29 14:56:02 rhel6-server-1314 kernel: ffff88003d90da78 ffff88003da95fd8 000000000000f4e8 ffff88003d90da78
May 29 14:56:02 rhel6-server-1314 kernel: Call Trace:
May 29 14:56:02 rhel6-server-1314 kernel: [<ffffffffa00041fc>] ? dm_table_unplug_all+0x5c/0x100 [dm_mod]
...             rhel6-server-1314
May 29 14:56:02 rhel6-server-1314 kernel: [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
May 29 14:56:02 rhel6-server-1314 kernel: mptscsih: ioc0: attempting task abort! (sc=ffff880037bfd280)
May 29 14:56:02 rhel6-server-1314 kernel: sd 2:0:0:0: [sda] CDB: Write(10): 2a 00 03 14 e8 d0 00 00 18 00
May 29 14:56:02 rhel6-server-1314 kernel: mptscsih: ioc0: WARNING - Issuing Reset from mptscsih_IssueTaskMgmt!! doorbell=0x24000000
May 29 14:56:02 rhel6-server-1314 kernel: mptscsih: ioc0: task abort: SUCCESS (rv=2002) (sc=ffff880037bfd280)
May 29 14:56:02 rhel6-server-1314 kernel: scsi target2:0:0: Beginning Domain Validation
May 29 14:56:02 rhel6-server-1314 kernel: scsi target2:0:0: Domain Validation skipping write tests
May 29 14:56:02 rhel6-server-1314 kernel: scsi target2:0:0: Ending Domain Validation
May 29 14:56:02 rhel6-server-1314 kernel: scsi target2:0:0: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127)

我的問題：

1. 有誰知道是什麼原因造成的？
2. 如果沒有，我們還應該在哪裡尋找線索？
3. 最後，有誰知道下次我們必須調整捲大小時如何緩解這種情況？

我認為，這只是 I/O 超時。

我在遠端 NFS 數據儲存上遇到了 Linux VM 的此類問題。NFS 太慢了，我們的一些 Linux 虛擬機將它們的磁碟切換到只讀模式（因此停止響應）。可能在調整大小期間，您的 NFS 數據儲存超載，這會導致問題。Linux VM 重啟後是否可以正常工作？

為避免此類問題並略微提高 Linux 客戶機的 I/O 性能，您可以嘗試將所有客戶機的 I/O 調度程序切換為“noop”或“deadline”：

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2011861

就我而言，我們大約每週在負載最多的 Linux 客戶機上遇到一次這樣的超時問題，即使使用“調度程序修復”也是如此。為了解決這個問題，我們從 NFS 切換到 iSCSI（您也可以嘗試優化您的 NFS 設置，如“rsize”、“wsize”、MTU 等，但在我的情況下仍然不夠）並嘗試減少 I/ O 盡可能對客人進行手術。

引用自：https://serverfault.com/questions/601442