Linux
如何恢復 mdadm raid6 陣列？mount：無法讀取超級塊。8/13 驅動器顯示所有驅動器存在，5/13 驅動器僅顯示 5 個驅動器存在
我今天重新啟動了我的伺服器，發現它掛了。我註釋掉了 fstab 中載入 /dev/md0 的條目，機器繼續啟動。在重新啟動之前，raid 可以訪問並正常工作，我只是因為系統更新而重新啟動。
伺服器主機板為 Supermicro X9SAE，系統有 14 個磁碟、1 個 SSD 啟動驅動器和 13 個 raid6 陣列中的磁碟。陣列中的 5/13 磁碟連接到板載 Sata。陣列中的 8/13 磁碟連接到 IT 模式 LSI 9211-8i
sudo fdisk -l 顯示所有 13 個驅動器，所有 13 個驅動器都通過智能測試
嘗試從命令行手動掛載：
sudo mount /dev/md0 /mnt/md0 mount: /mnt/md0: can't read superblock on /dev/md0. cat /proc/mdstat md0 : inactive sdh[13](S) sdc[8](S) sdm[0](S) sdb[7](S) sdj[14](S) sdd[9](S) sdk[5](S) sdg[12](S) sda[6](S) sdn[2](S) sdl[1](S) sde[10](S) sdf[11](S) 76185162872 blocks super 1.2接下來我嘗試了：
sudo mdadm --stop /dev/md0 mdadm: stopped /dev/md0 sudo mdadm --assemble /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh /dev/sdj /dev/sdk /dev/sdl /dev/sdm /dev/sdn mdadm: /dev/md0 assembled from 5 drives - not enough to start the array.我覺得奇怪的是它嘗試了 5 個驅動器，這就是板載 sdj-sdn 連接的驅動器數量。所以我試著只用卡片上的那些來組裝：
sudo mdadm --assemble /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg /dev/sdh mdadm: /dev/md0 assembled from 8 drives - not enough to start the array.通過這樣做，它試圖從 8 個驅動器組裝。所以看起來 5 個板載磁碟與卡上的 8 個驅動器不同步，但這純粹是猜測。接下來我檢查了每個驅動器。
接下來的部分會很長，它是 mdadm –examine 所有 13 個驅動器的詳細資訊。我在這裡註意到的是驅動器 sda-sdh 都顯示所有磁碟都存在，但驅動器 sdj-sdn 顯示 8 個驅動器缺少陣列狀態，還注意到事件計數，板載驅動器有事件：1913088，而那些卡上是 1913085：
sudo mdadm --examine /dev/sda /dev/sda: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 344864e7:5e19631d:0b14d41f:58264c67 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : 447ce0e - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 12 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdb /dev/sdb: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 244b3058:6919fa6f:c2499e3d:fb9db6e1 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : 5fdf7e64 - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 11 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdc /dev/sdc: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 9081b898:7782d040:acdcc700:e8621ca2 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : f4cd75b5 - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 10 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdd /dev/sdd: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 1b569018:549dae95:bbd59910:601886e3 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : 1abf13a6 - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 9 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sde /dev/sde: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 534b24a4:63d20861:502e4012:53fe2652 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : e1f47c75 - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 8 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdf /dev/sdf: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 0821860a:7b75d1cc:bb512247:3dc34a12 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : a924dd98 - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 7 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdg /dev/sdg: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 0c870013:b577639c:eba5f8d4:10a5fee4 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : e1bb7bdb - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 6 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdh /dev/sdh: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : 06713ac7:ab2a7de0:d6e0d447:52ed0231 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:38 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : 98ef9bf9 - correct Events : 1913085 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 5 Array State : AAAAAAAAAAAAA ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdj /dev/sdj: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250800 sectors, after=11440 sectors State : clean Device UUID : d6fb6b75:18fee031:84b97ab4:43ee3754 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:43 2019 Bad Block Log : 512 entries available at offset 32 sectors Checksum : 283bd3b6 - correct Events : 1913088 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 3 Array State : AAAAA........ ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdk /dev/sdk: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250792 sectors, after=11440 sectors State : clean Device UUID : aa02fdc1:15332b2a:eee48dee:9d6a91df Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:43 2019 Bad Block Log : 512 entries available at offset 72 sectors Checksum : 3283b76b - correct Events : 1913088 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 4 Array State : AAAAA........ ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdl /dev/sdl: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250792 sectors, after=11440 sectors State : clean Device UUID : 4701d341:4c70e117:8a8d37a0:19f8b474 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:43 2019 Bad Block Log : 512 entries available at offset 72 sectors Checksum : e6dd2971 - correct Events : 1913088 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 1 Array State : AAAAA........ ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdm /dev/sdm: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250792 sectors, after=11440 sectors State : clean Device UUID : ab1c4435:3c735172:3cf07404:f66f2944 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:43 2019 Bad Block Log : 512 entries available at offset 72 sectors Checksum : 68702233 - correct Events : 1913088 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 0 Array State : AAAAA........ ('A' == active, '.' == missing, 'R' == replacing) sudo mdadm --examine /dev/sdn /dev/sdn: Magic : a92b4efc Version : 1.2 Feature Map : 0x1 Array UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Name : ubuntu:0 (local to host ubuntu) Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Raid Devices : 13 Avail Dev Size : 11720794288 (5588.91 GiB 6001.05 GB) Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 11720782848 (5588.90 GiB 6001.04 GB) Data Offset : 250880 sectors Super Offset : 8 sectors Unused Space : before=250792 sectors, after=11440 sectors State : clean Device UUID : b7465e3e:502defd4:dcf61288:b0f14a84 Internal Bitmap : 8 sectors from superblock Update Time : Thu Apr 18 09:07:43 2019 Bad Block Log : 512 entries available at offset 72 sectors Checksum : 97e78eb0 - correct Events : 1913088 Layout : left-symmetric Chunk Size : 512K Device Role : Active device 2 Array State : AAAAA........ ('A' == active, '.' == missing, 'R' == replacing)這是從 4 月 17 日開始的，當時它仍在工作：
/dev/md0: Version : 1.2 Creation Time : Sat Nov 5 18:56:32 2016 Raid Level : raid6 Array Size : 64464305664 (61477.95 GiB 66011.45 GB) Used Dev Size : 5860391424 (5588.90 GiB 6001.04 GB) Raid Devices : 13 Total Devices : 13 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Tue Apr 16 19:14:14 2019 State : clean Active Devices : 13 Working Devices : 13 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 512K Consistency Policy : bitmap Name : ubuntu:0 (local to host ubuntu) UUID : 405dae60:c540c726:087bbaaf:a15c79c5 Events : 1913085 Number Major Minor RaidDevice State 0 8 192 0 active sync /dev/sdm 1 8 176 1 active sync /dev/sdl 2 8 208 2 active sync /dev/sdn 14 8 144 3 active sync /dev/sdj 5 8 160 4 active sync /dev/sdk 13 8 112 5 active sync /dev/sdh 12 8 96 6 active sync /dev/sdg 11 8 80 7 active sync /dev/sdf 10 8 64 8 active sync /dev/sde 9 8 48 9 active sync /dev/sdd 8 8 32 10 active sync /dev/sdc 7 8 16 11 active sync /dev/sdb 6 8 0 12 active sync /dev/sda讓陣列重新上線的下一步應該是什麼？
這是我所做的，Array 重新上線並且似乎正在工作。奇怪的是，最後一條消息是“不足以啟動陣列”。但它確實開始了，重新啟動後陣列處於活動狀態且乾淨，並且所有事件計數都匹配。某處可能存在少量數據損壞，但係統已經閒置了好幾天沒有出現問題，所以我懷疑它會錯過什麼。
將密切關注系統一段時間，以及事件計數。
sudo mdadm --stop /dev/md0 sudo mdadm --assemble --scan --force mdadm: forcing event count in /dev/sdh(5) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sdg(6) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sdf(7) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sde(8) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sdd(9) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sdc(10) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sdb(11) from 1913085 upto 1913088 mdadm: forcing event count in /dev/sda(12) from 1913085 upto 1913088 mdadm: clearing FAULTY flag for device 0 in /dev/md0 for /dev/sdh mdadm: clearing FAULTY flag for device 1 in /dev/md0 for /dev/sdg mdadm: clearing FAULTY flag for device 3 in /dev/md0 for /dev/sdf mdadm: clearing FAULTY flag for device 2 in /dev/md0 for /dev/sde mdadm: clearing FAULTY flag for device 4 in /dev/md0 for /dev/sdd mdadm: clearing FAULTY flag for device 5 in /dev/md0 for /dev/sdc mdadm: clearing FAULTY flag for device 6 in /dev/md0 for /dev/sdb mdadm: clearing FAULTY flag for device 7 in /dev/md0 for /dev/sda mdadm: Marking array /dev/md0 as 'clean' mdadm: /dev/md0 assembled from 13 drives - not enough to start the array.