Hard-Drive
目錄被 > 500 萬個文件淹沒後，目錄列表掛起
配置錯誤的 logrotate 在我的伺服器上的一個目錄中產生了很多文件。
ls | wc -l顯示 5,387,172 個文件，據此du -sh總計約 8 GB。dmesg顯示了許多這樣的錯誤：[16718682.749947] EXT4-fs warning (device dm-5): ext4_dx_add_entry:2209: Directory (ino: 4194315) index full, reach max htree level :2 [16718682.750028] EXT4-fs warning (device dm-5): ext4_dx_add_entry:2213: Large directory feature is not enabled on this filesystem配置現在已修復，我刪除了所有不應該存在的文件。清理後，我得到了這個：
# ls -l roundcube/logs/ total 20 -rw-r--r-- 1 www-data www-data 0 Mär 21 06:25 errors.log -rw-r--r-- 1 www-data www-data 315 Mär 21 06:25 errors.log.1 -rw-r--r-- 1 www-data www-data 0 Jan 7 06:36 errors.log.10.gz -rw-r--r-- 1 www-data www-data 0 Dez 18 06:26 errors.log.11.gz -rw-r--r-- 1 www-data www-data 0 Dez 10 06:25 errors.log.12.gz -rw-r--r-- 1 www-data www-data 321 Mär 14 06:25 errors.log.2.gz -rw-r--r-- 1 www-data www-data 272 Mär 7 06:25 errors.log.3.gz -rw-r--r-- 1 www-data www-data 354 Feb 28 06:25 errors.log.4.gz -rw-r--r-- 1 www-data www-data 20 Feb 18 16:07 errors.log.5.gz -rw-r--r-- 1 www-data www-data 0 Feb 18 18:14 errors.log.6.gz -rw-r--r-- 1 www-data www-data 0 Feb 5 13:36 errors.log.7.gz -rw-r--r-- 1 www-data www-data 0 Jan 29 09:17 errors.log.8.gz -rw-r--r-- 1 www-data www-data 0 Jan 14 06:49 errors.log.9.gz # du -sh roundcube/logs/ 632M roundcube/logs/然而，
ls通話仍然需要很長時間（我沒有看時鐘，但我認為它超過了 10 分鐘）。也可能 > 60 分鐘。而且很奇怪，du仍然報告 632MB。我啟動了一個救援系統並執行 fsck：
root@rescue /dev/mapper # fsck.ext4 vg0-mail e2fsck 1.44.5 (15-Dec-2018) vg0-mail: clean, 300078/13107200 files, 13579986/52428800 blocks root@rescue /dev/mapper # fsck.ext4 -f vg0-mail e2fsck 1.44.5 (15-Dec-2018) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information vg0-mail: 300078/13107200 files (0.5% non-contiguous), 13579986/52428800 blocks所以看起來一切都很好。我重新啟動回到正常系統，但
ls仍然很慢。只是為了確保沒有程序干擾，我重新啟動到救援系統，在那里安裝卷並嘗試列出目錄內容。它也掛了。
strace ls顯示很多行。最後幾行是：fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 read(3, "nodev\tsysfs\nnodev\ttmpfs\nnodev\tbd"..., 1024) = 309 read(3, "", 1024) = 0 close(3) = 0 access("/etc/selinux/config", F_OK) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=5547600, ...}) = 0 mmap(NULL, 5547600, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f0bd79c6000 close(3) = 0 ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, TIOCGWINSZ, {ws_row=66, ws_col=271, ws_xpixel=0, ws_ypixel=0}) = 0 stat("roundcube/logs/", {st_mode=S_IFDIR|0755, st_size=662331392, ...}) = 0 openat(AT_FDCWD, "roundcube/logs/", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3 fstat(3, {st_mode=S_IFDIR|0755, st_size=662331392, ...}) = 0 getdents64(3所以它似乎掛了
getdents64？我有點困惑，它以看似不完整的線條結尾。我讀到這可能是由硬體缺陷引起的，所以我嘗試
smartctl -t long在兩個磁碟上執行分析（raid 1）。結果如下：/dev/sda smartctl 6.6 2017-11-05 r4594 [x86_64-linux-4.19.0-14-amd64] (local build) Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Seagate Constellation ES.3 Device Model: ST2000NM0033-9ZM175 Serial Number: Z1X0QWWG LU WWN Device Id: 5 000c50 064a202c3 Firmware Version: SN07 User Capacity: 2.000.398.934.016 bytes [2,00 TB] Sector Size: 512 bytes logical/physical Rotation Rate: 7200 rpm Form Factor: 3.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2 (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Tue Mar 23 21:24:30 2021 CET SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x82) Offline data collection activity was completed without error. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 592) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 245) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x50bd) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 070 063 044 Pre-fail Always - 11974882 3 Spin_Up_Time 0x0003 096 096 000 Pre-fail Always - 0 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 25 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 7 Seek_Error_Rate 0x000f 088 060 030 Pre-fail Always - 706711710 9 Power_On_Hours 0x0032 033 033 000 Old_age Always - 59193 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 24 184 End-to-End_Error 0x0032 100 100 099 Old_age Always - 0 187 Reported_Uncorrect 0x0032 100 100 000 Old_age Always - 0 188 Command_Timeout 0x0032 100 100 000 Old_age Always - 0 189 High_Fly_Writes 0x003a 059 059 000 Old_age Always - 41 190 Airflow_Temperature_Cel 0x0022 060 050 045 Old_age Always - 40 (Min/Max 36/44) 191 G-Sense_Error_Rate 0x0032 100 100 000 Old_age Always - 0 192 Power-Off_Retract_Count 0x0032 100 100 000 Old_age Always - 18 193 Load_Cycle_Count 0x0032 099 099 000 Old_age Always - 2459 194 Temperature_Celsius 0x0022 040 050 000 Old_age Always - 40 (0 21 0 0 0) 195 Hardware_ECC_Recovered 0x001a 042 015 000 Old_age Always - 11974882 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 59192 - # 2 Short offline Interrupted (host reset) 10% 59185 - # 3 Extended offline Completed without error 00% 53208 - # 4 Extended offline Completed without error 00% 53191 - # 5 Extended offline Completed without error 00% 53178 - # 6 Short offline Completed without error 00% 53170 - # 7 Extended offline Completed without error 00% 53056 - # 8 Extended offline Completed without error 00% 53040 - # 9 Extended offline Completed without error 00% 53025 - #10 Short offline Completed without error 00% 53017 - #11 Extended offline Completed without error 00% 51660 - #12 Extended offline Completed without error 00% 51643 - #13 Extended offline Completed without error 00% 2411 - #14 Extended offline Completed without error 00% 2383 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. /dev/sdb smartctl 6.6 2017-11-05 r4594 [x86_64-linux-4.19.0-14-amd64] (local build) Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Hitachi/HGST Ultrastar 7K4000 Device Model: HGST HUS724020ALA640 Serial Number: PN2138P2GNK30J LU WWN Device Id: 5 000cca 24bc9579a Firmware Version: MF6OAA70 User Capacity: 2.000.398.934.016 bytes [2,00 TB] Sector Size: 512 bytes logical/physical Rotation Rate: 7200 rpm Form Factor: 3.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ATA8-ACS T13/1699-D revision 4 SATA Version is: SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Tue Mar 23 22:38:27 2021 CET SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x84) Offline data collection activity was suspended by an interrupting command from host. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 24) seconds. Offline data collection capabilities: (0x5b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 314) minutes. SCT capabilities: (0x003d) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000b 100 100 016 Pre-fail Always - 0 2 Throughput_Performance 0x0005 136 136 054 Pre-fail Offline - 80 3 Spin_Up_Time 0x0007 163 163 024 Pre-fail Always - 367 (Average 396) 4 Start_Stop_Count 0x0012 100 100 000 Old_age Always - 28 5 Reallocated_Sector_Ct 0x0033 100 100 005 Pre-fail Always - 0 7 Seek_Error_Rate 0x000b 100 100 067 Pre-fail Always - 0 8 Seek_Time_Performance 0x0005 145 145 020 Pre-fail Offline - 24 9 Power_On_Hours 0x0012 095 095 000 Old_age Always - 37639 10 Spin_Retry_Count 0x0013 100 100 060 Pre-fail Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 27 192 Power-Off_Retract_Count 0x0032 099 099 000 Old_age Always - 1311 193 Load_Cycle_Count 0x0012 099 099 000 Old_age Always - 1311 194 Temperature_Celsius 0x0002 150 150 000 Old_age Always - 40 (Min/Max 22/60) 196 Reallocated_Event_Count 0x0032 100 100 000 Old_age Always - 0 197 Current_Pending_Sector 0x0022 100 100 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0008 100 100 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x000a 200 200 000 Old_age Always - 0 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 37638 - # 2 Short offline Interrupted (host reset) 90% 37630 - # 3 Extended offline Completed without error 00% 31654 - # 4 Extended offline Completed without error 00% 31637 - # 5 Extended offline Completed without error 00% 30951 - # 6 Extended offline Completed without error 00% 30932 - # 7 Extended offline Completed without error 00% 30927 - # 8 Short offline Completed without error 00% 30919 - # 9 Extended offline Completed without error 00% 30882 - #10 Extended offline Completed without error 00% 30866 - #11 Extended offline Completed without error 00% 19849 - #12 Extended offline Completed without error 00% 19832 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay.我大部分都不懂，但對我來說看起來並不“壞”？！
最終我剛剛執行
mv logs logs-old; mv logs-old/* logs; rm -r logs-old了這似乎確實“解決了”我現在的問題。但我想知道這是否真的是這樣，或者是否可能還有更多。這裡可能發生了什麼導致這種目錄列表極慢的行為？現在問題可以解決了嗎？我應該找到一些損壞嗎？
這是一個已知問題：當有這麼多文件時，目錄條目（dentry）變得非常大，刪除文件不會縮小它。
更簡單的解決方案是刪除並重新創建目錄本身（即：在您的情況下，為
rm -rf roundcube/logs/ ; mkdir roundcube/logs/）請參閱此處了解更多資訊