Iscsi

從 XenServer 6.2 池連接到 Dell MD3200i 失敗

  • May 29, 2014

這個問題也在思傑論壇http://forums.citrix.com/thread.jspa?threadID=332289

我有一個 MD3200i,目前在我的 Xen5.6 池中執行良好,但我無法連接到新的 6.2 池以正常工作。我之前在升級 6.0 時遇到了問題(這就是為什麼舊池仍在 5.6 上的原因),但由於當時並不緊急,所以回滾而不是修復它。

此安裝是在新機器上進行的 - 我首先嘗試了 6.1(有同樣的問題),然後在安裝後的第二天發布了 6.2,所以我切換到了。

目前我還沒有從戴爾資源 DVD 中安裝任何東西——我找不到任何我應該安裝的東西,而且我讀過的所有內容都表明它不應該是必要的。

我可以從池中的兩台伺服器 ping 所有 8 個 IP 地址,iscsiadm -m discovery工作正常,我可以登錄到節點並且 iscsiadm 正確報告會話處於活動狀態。

我已將所需部分添加到 multipath.conf,但 multipath -ll 會 DM multipath kernel driver not loaded在啟動後立即報告。

以下是啟動後立即測試會話的日誌。

root@xen3 ~]# iscsiadm -m node --loginall=all
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.101,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.101,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.104,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.102,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.103,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.104,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.102,3260]
Logging in to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.103,3260]
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.101,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.101,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.104,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.102,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.103,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.104,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.130.102,3260]: successful
Login to [iface: default, target: iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91, portal: 192.168.131.103,3260]: successful                                                                                                                                               

[root@xen3 ~]# iscsiadm -m session                                                                                                                
tcp: [1] 192.168.130.101:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [2] 192.168.131.101:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [3] 192.168.131.104:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [4] 192.168.131.102:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [5] 192.168.130.103:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [6] 192.168.130.104:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [7] 192.168.130.102:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          
tcp: [8] 192.168.131.103:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91                                          

[root@xen3 ~]# service multipathd restart
ok                                                                                                                                                
Stopping multipathd daemon:                                [  OK  ]                                                                               
Starting multipathd daemon:                                [  OK  ]                                                                               

[root@xen3 ~]# multipath                                                                                                                          
Jul 04 09:58:47 | DM multipath kernel driver not loaded                                                                                           
Jul 04 09:58:47 | DM multipath kernel driver not loaded                                                                                           
[root@xen3 ~]# multipath -ll
Jul 04 09:59:03 | DM multipath kernel driver not loaded                                                                                           
Jul 04 09:59:03 | DM multipath kernel driver not loaded                                                                                           
[
root@xen3 ~]# modprobe dm_multipath                                                                                                              

[root@xen3 ~]# multipath
Jul 04 10:19:50 | 36b8ca3a0e7024800194a0bd11891cd14: ignoring map                                                                                 
create: 1Dell_Internal_Dual_SD_0123456789AB undef Dell,Internal Dual SD
size=1.9G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
 `- 7:0:0:0  sdb 8:16  undef ready  running

[root@xen3 ~]# multipath -ll
1Dell_Internal_Dual_SD_0123456789AB dm-1 Dell,Internal Dual SD
size=1.9G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=enabled
 `- 7:0:0:0  sdb 8:16  active ready  running

[root@xen3 ~]# iscsiadm -m session
tcp: [1] 192.168.130.101:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [2] 192.168.131.101:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [3] 192.168.131.104:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [4] 192.168.131.102:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [5] 192.168.130.103:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [6] 192.168.130.104:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [7] 192.168.130.102:3260,2 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
tcp: [8] 192.168.131.103:3260,1 iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91

[root@xen3 ~]# dmesg | tail -n 50
[ 1161.881010] sd 8:0:0:0: [sdf] Unhandled error code
[ 1161.881013] sd 8:0:0:0: [sdf] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1161.881017] sd 8:0:0:0: [sdf] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1161.881024] end_request: I/O error, dev sdf, sector 0
[ 1161.881031] Buffer I/O error on device sdf, logical block 0
[ 1161.881045] sd 15:0:0:0: [sdi] Unhandled error code
[ 1161.881048] sd 15:0:0:0: [sdi] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1161.881052] sd 15:0:0:0: [sdi] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1161.881058] end_request: I/O error, dev sdi, sector 0
[ 1161.881065] Buffer I/O error on device sdi, logical block 0
[ 1161.881122] sd 9:0:0:0: [sdg] Unhandled error code
[ 1161.881124] sd 9:0:0:0: [sdg] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1161.881126] sd 9:0:0:0: [sdg] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1161.881132] end_request: I/O error, dev sdg, sector 0
[ 1161.881140] Buffer I/O error on device sdg, logical block 0
[ 1168.220951]  connection6:0: ping timeout of 15 secs expired, recv timeout 10, last rx 84060, last ping 85060, now 86560
[ 1168.220957]  connection7:0: ping timeout of 15 secs expired, recv timeout 10, last rx 84060, last ping 85060, now 86560
[ 1168.220967]  connection7:0: detected conn error (1011)
[ 1168.220969]  connection4:0: ping timeout of 15 secs expired, recv timeout 10, last rx 84060, last ping 85060, now 86560
[ 1168.220973]  connection4:0: detected conn error (1011)
[ 1168.220975]  connection3:0: ping timeout of 15 secs expired, recv timeout 10, last rx 84060, last ping 85060, now 86560
[ 1168.220978]  connection3:0: detected conn error (1011)
[ 1168.220985]  connection6:0: detected conn error (1011)
[ 1168.480994] sd 14:0:0:0: [sde] Unhandled error code
[ 1168.480998] sd 14:0:0:0: [sde] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1168.481001] sd 14:0:0:0: [sde] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1168.481009] end_request: I/O error, dev sde, sector 0
[ 1168.481015] Buffer I/O error on device sde, logical block 0
[ 1168.481076] sd 11:0:0:0: [sdc] Unhandled error code
[ 1168.481078] sd 11:0:0:0: [sdc] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1168.481080] sd 11:0:0:0: [sdc] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1168.481087] end_request: I/O error, dev sdc, sector 0
[ 1168.481092] Buffer I/O error on device sdc, logical block 0
[ 1168.481144] sd 10:0:0:0: [sdd] Unhandled error code
[ 1168.481147] sd 10:0:0:0: [sdd] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1168.481150] sd 10:0:0:0: [sdd] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1168.481156] end_request: I/O error, dev sdd, sector 0
[ 1168.481163] Buffer I/O error on device sdd, logical block 0
[ 1168.481168] sd 13:0:0:0: [sdj] Unhandled error code
[ 1168.481170] sd 13:0:0:0: [sdj] Result: hostbyte=DID_TRANSPORT_DISRUPTED driverbyte=DRIVER_OK
[ 1168.481172] sd 13:0:0:0: [sdj] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
[ 1168.481178] end_request: I/O error, dev sdj, sector 0
[ 1168.481184] Buffer I/O error on device sdj, logical block 0
[ 1457.105996] device-mapper: multipath round-robin: version 1.0.0 loaded
[ 1457.106155] device-mapper: multipath: Cannot access device path 8:0: -16
[ 1457.106164] device-mapper: table: 252:1: multipath: error getting device
[ 1457.106172] device-mapper: ioctl: error adding target to table
[ 1457.171292] device-mapper: multipath: Cannot access device path 8:0: -16
[ 1457.171299] device-mapper: table: 252:1: multipath: error getting device
[ 1457.171304] device-mapper: ioctl: error adding target to table

[root@xen3 ~]# fdisk -l

Disk /dev/sda: 299.4 GB, 299439751168 bytes
255 heads, 63 sectors/track, 36404 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

  Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1           5       40131   de  Dell Utility
/dev/sda2   *           6         528     4194304   83  Linux
Partition 2 does not end on cylinder boundary.
/dev/sda3             528        1050     4194304   83  Linux
/dev/sda4            1050       36404   283986359+  8e  Linux LVM

Disk /dev/sdb: 2040 MB, 2040528896 bytes
255 heads, 63 sectors/track, 248 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

  Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         248     1992028+  83  Linux

Disk /dev/dm-1: 2040 MB, 2040528896 bytes
255 heads, 63 sectors/track, 248 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/dm-1p1               1         248     1992028+  83  Linux

[root@xen3 ~]# xe sr-probe type=lvmoiscsi device-config:target=192.168.130.101 device-config:targetIQN=iqn.1984-05.com.dell:powervault.md3200i.6782bcb0006bd850000000004ed88b91
Error code: SR_BACKEND_FAILURE_107
Error parameters: , The SCSIid parameter is missing or incorrect, <?xml version="1.0" ?>
<iscsi-target/>

注意: xml 在最後一行正確結束 - 它永遠不會返回 LUN 列表(並且這些伺服器的 SAN 上的組中有一個。

環顧其他地方的各種文章,有人建議關閉交換機上的流量控制(線索顯然是這些埠的乙太網統計數據中顯示的大量暫停數字),這無濟於事,但確實讓我看到了巨型幀支持。

ping 192.168.130.101 -s 6000 -M do(大數據包,不分段)靜默失敗,而-s 9500 -M do報告 ICMP 錯誤(如我所料)。結合日誌中的超時消息,這看起來是問題所在。

交換機設置看起來都很好,所以再次檢查 XenCenter 配置我注意到 iSCSI 的 VLAN 網路的 MTU=9000,但底層 NIC 仍設置為 1500。顯然這不僅會弄亂巨型幀(這是合理的),但不會產生 ICMP 錯誤(這對我來說似乎有點錯誤),因此一旦數據包通過 1500 並且沒有收到錯誤或回复,流量就永遠不會到達 SAN。

經驗教訓 - 確保頂級網路(我假設的 VLAN 或綁定相同)的 MTU 始終不高於它們執行的網路。

引用自:https://serverfault.com/questions/521888