Filesystems

VXFS (veritas) 文件系統已滿 - 但不是

  • April 9, 2015

我有一個 Solaris 10 伺服器,它報告文件系統已滿:

# df -h /opt/CSCObac/var
Filesystem             size   used  avail capacity  Mounted on
/dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL

# df -F vxfs /opt/CSCObac/var/
/opt/CSCObac/var   (/dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL): 4469796 blocks   558723 files

# df -h | fgrep '/opt/'
/dev/vx/dsk/RDU-APP-DG/RDU-APP-VOL    68G   547M    63G     1%    /opt/CSCObac
/dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL   134G   132G   2.0G    99%    /opt/CSCObac/var
/dev/vx/dsk/RDU-LOG-DG/RDU-LOG-VOL   134G   5.2G   121G     5%    /opt/CSCObac/var/rdu

除了文件系統上沒有佔用 100G+ 的文件外,df 報告它沒有用完空閒塊,也不缺少空閒 inode:

# du -sh /opt/CSCObac/var
5.1G   /opt/CSCObac/var
                  134G   132G   2.0G    99%    /opt/CSCObac/var

# df -g /opt/CSCObac/var/
/opt/CSCObac/var   (/dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL):     8192 block size          1024 frag size  
280938496 total blocks    4469536 free blocks  4190192 available             558720 total files
558691 free files     74450896 filesys id  
vxfs fstype       0x00000004 flag             255 filename length

此外,沒有打開的文件描述符:

# lsof /opt/CSCObac/var/
#

這是一個 vxfs 文件系統,有關捲和大小配置,請參見下文:

# fstyp -v /dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL
vxfs
magic a501fcf5  version 7  ctime November 15, 2013  2:15:55 PM CET
logstart 0  logend 0
bsize  1024 size  140469248 dsize  0  ninode 140469248  nau 0
defiextsize 0  ilbsize 0  immedlen 96  ndaddr 10
aufirst 0  emap 0  imap 0  iextop 0  istart 0
bstart 0  femap 0  fimap 0  fiextop 0  fistart 0  fbstart 0
nindir 2048  aulen 32768  auimlen 0  auemlen 8
auilen 0  aupad 0  aublocks 32768  maxtier 15
inopb 4  inopau 0  ndiripau 0  iaddrlen 8 bshift 10
inoshift 2  bmask fffffc00  boffmask 3ff  checksum f790321b
oltext1 32  oltext2 1282  oltsize 1  checksum2 0
free 2234899  ifree 0
efree  5 9 7 2 3 5 4 4 2 0 3 1 2 3 2 0 1 2 1 1 1 0 0 0 0 0 0 0 0 0 0 0 

# vxlist disk | grep 'VAR'
disk disk_4     RDU-VAR-DG01   RDU-VAR-DG     133.96g   608.00k imported
disk disk_8     RDU-VAR-DG02   RDU-VAR-DG     133.96g   608.00k imported

# vxlist vol RDU-VAR-VOL
Volume:  RDU-VAR-VOL
GUID:               {0de33acc-1dd2-11b2-a8f9-00212800c1e0}
Disk Group:         name=RDU-VAR-DG id=1384520670.51.rdu2a
Host:               rdu2a
Size:               133.9g
Type:               fsgen
Status:             healthy
Layout:             concat-mirror
NMirrors:           2
Read Policy:        SELECT
Flags:              open writeback
Device Path:        /dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL
Permissions:        user=root group=root mode=0600

filesystem
TY   FSTYPE     %USED VSET   MOUNT-POINT
fs   vxfs         98% -      /opt/CSCObac/var

stats
TY   OPERATION       COUNT      BLOCKS   TIME(MSEC)
stat read          1160490     1731963      17538.5
stat write         3928639   294438233       7579.4

plexes
TY   NAME             TYPE     STATUS
plex RDU-VAR-VOL-03   simple   attached

# vxlist fs | fgrep 'VAR'
fs   var       vxfs       133.96g   2046.10m     99% /dev/vx/dsk/RDU-VAR-DG/RDU-VAR-VOL   /opt/CSCObac/var

# vxassist -g RDU-VAR-DG -p maxgrow RDU-VAR-VOL
VxVM vxassist ERROR V-5-1-1178 Volume RDU-VAR-VOL cannot be extend within the given constraints

# fsadm -D /opt/CSCObac/var/ 

 Directory Fragmentation Report
            Dirs        Total      Immed    Immeds   Dirs to   Blocks to
            Searched    Blocks     Dirs     to Add   Reduce    Reduce
 total            11         3         8         0         0           0

除了 inode、塊和文件描述符之外,還有什麼可以填滿我的文件系統?

在與我們的支持供應商進行了一些冗長的故障排除後,我們找到了解決方案。在掛載 vxfs 文件系統之前啟動了一個正在執行的程序。此過程正在將數據寫入 /opt/CSCObac/var/rdu,它是 /opt/CSCObac/var/ 掛載點的嵌套目錄。終止此程序後,磁碟使用率恢復到預期的 1% 作業系統無法報告此目錄的內容,因為 rdu 掛載點掩蓋了它。在掛載文件系統之前,長時間執行的程序開始寫入此目錄掛載點中的文件,這解釋了為什麼它的大小增加而沒有看到目錄中可見文件的增加。

引用自:https://serverfault.com/questions/677583