Security

httpd 高 cpu 使用率減慢伺服器響應

  • July 1, 2013

我的客戶有一個圖像共享網站,每天有大約 100.000 名訪問者,自從今天早上我檢查程序以來,它的速度已經大大降低,我注意到 http 的 cpu 使用率很高。

最佳

top - 20:13:30 up  5:04,  4 users,  load average: 4.56, 4.69, 4.59
Tasks: 284 total,   3 running, 281 sleeping,   0 stopped,   0 zombie
Cpu(s): 12.1%us,  0.9%sy,  1.7%ni, 69.0%id, 16.4%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16037152k total, 15875096k used,   162056k free,   360468k buffers
Swap:  4194288k total,      888k used,  4193400k free, 14050008k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4151 apache    20   0  277m  84m 3784 R 50.2  0.5   0:01.98 httpd
 4115 apache    20   0  210m  16m 4480 S 18.3  0.1   0:00.60 httpd
12885 root      39  19  4296  692  308 S 13.0  0.0  11:09.53 gzip
 4177 apache    20   0  214m  20m 3700 R 12.3  0.1   0:00.37 httpd
 2219 mysql     20   0 4257m 198m 5668 S 11.0  1.3  42:49.70 mysqld
 3691 apache    20   0  206m  14m 6416 S  1.7  0.1   0:03.38 httpd

 3934 apache    20   0  211m  17m 4836 S  1.0  0.1   0:03.61 httpd
 4098 apache    20   0  209m  17m 3912 S  1.0  0.1   0:04.17 httpd
 4116 apache    20   0  211m  17m 4476 S  1.0  0.1   0:00.43 httpd
 3867 apache    20   0  217m  23m 4672 S  0.7  0.1   1:03.87 httpd
 4146 apache    20   0  209m  15m 3628 S  0.7  0.1   0:00.02 httpd
 4149 apache    20   0  209m  15m 3616 S  0.7  0.1   0:00.02 httpd
12884 root      39  19 22336 2356  944 D  0.7  0.0   0:19.21 tar
 4054 apache    20   0  206m  12m 4576 S  0.3  0.1   0:00.32 httpd

另一個頂部

top - 15:46:45 up  5:08,  4 users,  load average: 5.02, 4.81, 4.64
Tasks: 288 total,   6 running, 281 sleeping,   0 stopped,   1 zombie
Cpu(s): 18.4%us,  0.9%sy,  2.3%ni, 56.5%id, 21.8%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16037152k total, 15792196k used,   244956k free,   360924k buffers
Swap:  4194288k total,      888k used,  4193400k free, 13983368k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 4622 apache    20   0  209m  16m 3868 S 54.2  0.1   0:03.99 httpd
 4514 apache    20   0  213m  20m 3924 R 50.8  0.1   0:04.93 httpd
 4627 apache    20   0  221m  27m 4560 R 18.9  0.2   0:01.20 httpd
12885 root      39  19  4296  692  308 S 18.9  0.0  11:51.79 gzip
 2219 mysql     20   0 4257m 199m 5668 S 18.3  1.3  43:19.04 mysqld
 4512 apache    20   0  227m  33m 4736 R  5.6  0.2   0:01.93 httpd
 4520 apache    20   0  213m  19m 4640 S  1.3  0.1   0:01.48 httpd
 4590 apache    20   0  212m  19m 3932 S  1.3  0.1   0:00.06 httpd
 4573 apache    20   0  210m  16m 3556 R  1.0  0.1   0:00.03 httpd
 4562 root      20   0 15164 1388  952 R  0.7  0.0   0:00.08 top
   98 root      20   0     0    0    0 S  0.3  0.0   0:04.89 kswapd0
  100 root      39  19     0    0    0 S  0.3  0.0   0:02.85 khugepaged
 4579 apache    20   0  209m  16m 3900 S  0.3  0.1   0:00.83 httpd
 4637 apache    20   0  209m  15m 3668 S  0.3  0.1   0:00.03 httpd

ps到

[root@server ~]# ps aux | grep httpd
   root       2236  0.0  0.0 207524 10124 ?        Ss   15:09   0:03 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3087  2.7  0.1 226968 28232 ?        S    20:04   0:06 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3170  2.6  0.1 221296 22292 ?        R    20:05   0:05 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3171  9.0  0.1 225044 26768 ?        R    20:05   0:17 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3188  1.5  0.1 223644 24724 ?        S    20:05   0:03 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3197  2.3  0.1 215908 17520 ?        S    20:05   0:04 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3198  1.1  0.0 211700 13000 ?        S    20:05   0:02 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3272  2.4  0.1 219960 21540 ?        S    20:06   0:03 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3273  2.0  0.0 211600 12804 ?        S    20:06   0:03 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3279  3.7  0.1 229024 29900 ?        S    20:06   0:05 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3280  1.2  0.0      0     0 ?        Z    20:06   0:01 [httpd] <defun                                                                                            ct>
   apache     3285  2.9  0.1 218532 21604 ?        S    20:06   0:04 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3287 30.5  0.4 265084 65948 ?        R    20:06   0:43 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3297  1.9  0.1 216068 17332 ?        S    20:06   0:02 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3342  2.7  0.1 216716 17828 ?        S    20:06   0:03 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3356  1.6  0.1 217244 18296 ?        S    20:07   0:01 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3365  6.4  0.1 226044 27428 ?        S    20:07   0:06 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3396  0.0  0.1 213844 16120 ?        S    20:07   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3399  5.8  0.1 215664 16772 ?        S    20:07   0:05 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3422  0.7  0.1 214860 17380 ?        S    20:07   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3435  3.3  0.1 216220 17460 ?        S    20:07   0:02 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3463  0.1  0.0 212732 15076 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3492  0.0  0.0 207660  7552 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3493  1.4  0.1 218092 19188 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3500  1.9  0.1 224204 26100 ?        R    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3501  1.7  0.1 216916 17916 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3502  0.0  0.0 207796  7732 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3505  0.0  0.0 207660  7548 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3529  0.0  0.0 207660  7524 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3531  4.0  0.1 216180 17280 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3532  0.0  0.0 207656  7464 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3543  1.4  0.1 217088 18648 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3544  0.0  0.0 207656  7548 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3545  0.0  0.0 207656  7560 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3546  0.0  0.0 207660  7540 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3547  0.0  0.0 207660  7544 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3548  2.3  0.1 216904 17888 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3550  0.0  0.0 207660  7540 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3551  0.0  0.0 207660  7536 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3552  0.2  0.0 214104 15972 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3553  6.5  0.1 216740 17712 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3554  6.3  0.1 216156 17260 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3555  0.0  0.0 207796  7716 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3556  1.8  0.0 211588 12580 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3557  0.0  0.0 207660  7544 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3565  0.0  0.0 207660  7520 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3570  0.0  0.0 207660  7516 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   apache     3571  0.0  0.0 207660  7504 ?        S    20:08   0:00 /usr/sbin/http                                                                                            d -k start -DSSL
   root       3577  0.0  0.0 103316   860 pts/2    S+   20:08   0:00 grep httpd

httpd 錯誤日誌

[Mon Jul 01 18:53:38 2013] [error] [client 2.178.12.67] request failed: error reading the headers, referer: http://akstube.com/image/show/27023/%D9%86%DB%8C%D9%88%D8%B4%D8%A7-%D8%B6%DB%8C%D8%BA%D9%85%DB%8C-%D9%88-%D8%AE%D9%88%D8%A7%D9%87%D8%B1-%D9%88-%D9%87%D9%85%D8%B3%D8%B1%D8%B4
[Mon Jul 01 18:55:33 2013] [error] [client 91.229.215.240] request failed: error reading the headers, referer: http://akstube.com/image/show/44924
[Mon Jul 01 18:57:02 2013] [error] [client 2.178.12.67] Invalid method in request 
[Mon Jul 01 18:57:02 2013] [error] [client 2.178.12.67] File does not exist: /var/www/html/501.shtml
[Mon Jul 01 19:21:36 2013] [error] [client 127.0.0.1] client denied by server configuration: /var/www/html/server-status
[Mon Jul 01 19:21:36 2013] [error] [client 127.0.0.1] File does not exist: /var/www/html/403.shtml
[Mon Jul 01 19:23:57 2013] [error] [client 151.242.14.31] request failed: error reading the headers
[Mon Jul 01 19:37:16 2013] [error] [client 2.190.16.65] request failed: error reading the headers
[Mon Jul 01 19:56:00 2013] [error] [client 151.242.14.31] request failed: error reading the headers
Not a JPEG file: starts with 0x89 0x50

消息日誌中也有很多這樣的

Jul  1 20:15:47 server named[2426]: client 203.88.6.9#11926: query (cache) 'www.xxxmaza.com/A/IN' denied
Jul  1 20:15:47 server named[2426]: client 203.88.6.9#26255: query (cache) 'www.xxxmaza.com/A/IN' denied
Jul  1 20:15:48 server named[2426]: client 203.88.6.9#20093: query (cache) 'www.xxxmaza.com/A/IN' denied
Jul  1 20:15:48 server named[2426]: client 203.88.6.9#8672: query (cache) 'www.xxxmaza.com/A/IN' denied
Jul  1 15:45:07 server named[2426]: client 203.88.6.9#39352: query (cache) 'www.xxxmaza.com/A/IN' denied

系統資訊,它是帶有 Xeon cpu 的專用伺服器,帶有 8 個 cor 圖像正在保存在伺服器上

Processor Name  Intel(R) Xeon(R) CPU E3-1270 V2 @ 3.50GHz
Vendor ID   GenuineIntel
Processor Speed (MHz)   3492.087
Total Memory    16037152 kB
Free Memory 316004 kB
Total Swap Memory   4194288 kB
Free Swap Memory    4193400 kB
System Uptime   0 Days, 6 Hours and 8 Minutes
Apache 2.2.24   Running
DirectAdmin 1.43.0  Running
Exim 4.76   Running
MySQL 5.5.27    Running
Named 9.7.0 Running
ProFTPd 1.3.4b  Running
sshd    Running
dovecot 2.1.16  Running
Php 5.3.24  Installed

自由

[root@server ~]# free
            total       used       free     shared    buffers     cached
Mem:      16037152   15686080     351072          0     373364   14132296
-/+ buffers/cache:    1180420   14856732
Swap:      4194288        888    4193400

我的伺服器支持人員說這是一次 ddos​​ 攻擊,他們已經向我發送了這張照片,但他們之前一直錯了,所以我擔心這可能是其他問題

在此處輸入圖像描述

================================

頂部 + 1

top - 20:35:22 up  9:57,  2 users,  load average: 4.21, 4.08, 4.14
Tasks: 269 total,   2 running, 267 sleeping,   0 stopped,   0 zombie
Cpu0  : 87.3%us,  3.0%sy,  0.0%ni,  0.0%id,  9.7%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  : 12.4%us,  1.0%sy,  1.7%ni, 31.2%id, 53.7%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  : 36.1%us,  1.0%sy,  0.0%ni, 20.4%id, 42.5%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  9.3%us,  0.3%sy,  0.3%ni, 50.0%id, 40.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.3%us,  0.0%sy,  0.0%ni, 90.0%id,  9.7%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.3%us,  0.0%sy,  0.7%ni, 93.7%id,  5.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  1.0%us,  0.0%sy,  0.0%ni, 93.3%id,  5.7%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16037152k total, 15786500k used,   250652k free,   384284k buffers
Swap:  4194288k total,      932k used,  4193356k free, 14208212k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
40988 apache    20   0  216m  22m 3828 R 89.1  0.1   0:02.70 httpd
 2219 mysql     20   0 4257m 207m 5672 S 19.6  1.3  77:18.24 mysqld
23115 root      39  19  4296  692  308 D  5.0  0.0  15:32.00 gzip
40984 apache    20   0  209m  16m 4024 S  1.3  0.1   0:00.09 httpd
41006 apache    20   0  206m  12m 4476 D  1.3  0.1   0:00.80 httpd

另一個

[root@server ~]# top
top - 20:31:55 up  9:53,  2 users,  load average: 3.85, 4.04, 4.16
Tasks: 258 total,   2 running, 256 sleeping,   0 stopped,   0 zombie
Cpu0  : 42.5%us,  0.7%sy,  0.0%ni,  2.7%id, 54.2%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :  6.7%us,  1.0%sy,  3.7%ni, 56.5%id, 32.1%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :  5.6%us,  0.3%sy,  0.7%ni, 72.4%id, 20.9%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :  5.3%us,  0.0%sy,  0.3%ni, 85.4%id,  9.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  : 10.6%us,  0.7%sy,  0.0%ni, 51.7%id, 37.1%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.7%us,  0.3%sy,  0.3%ni, 98.3%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni, 94.7%id,  5.3%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni, 99.3%id,  0.7%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16037152k total, 15858928k used,   178224k free,   384208k buffers
Swap:  4194288k total,      932k used,  4193356k free, 14347484k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
40654 apache    20   0  260m  66m 4540 R 52.5  0.4   0:04.21 httpd
40512 apache    20   0  217m  23m 4692 D  9.3  0.1   0:04.43 httpd
 2219 mysql     20   0 4257m 213m 5672 S  7.0  1.4  76:59.89 mysqld
40638 apache    20   0  208m  16m 3988 S  4.7  0.1   0:00.37 httpd
23115 root      39  19  4296  692  308 S  4.3  0.0  15:22.57 gzip
23114 root      39  19 26436 7768  944 D  0.7  0.0   0:30.64 tar

我不是網站管理員,我不知道發生了什麼

也許在這裡問一個問題不是成為網站管理員的正確起點?

當您對這些事情沒有太多了解時執行自己的名稱伺服器可能不是一個好主意。

您的網路伺服器錯誤處理配置錯誤。每個實例還使用了相當多的記憶體。您沒有提供有關係統上有多少記憶體的任何資訊(我們需要查看“free”的輸出)。

您沒有提供有關正在執行的硬體的任何詳細資訊(特別是它有多少 CPU,是物理伺服器還是虛擬伺服器。也沒有提供圖像的服務方式/使用了多少頻寬。

203.88.23.* 網路與您的名稱伺服器有何關係?

為什麼在系統負載過重時以“root”身份執行備份?

實際上,我認為 CPU 使用率並沒有那麼高,儘管負載和等待時間都相對較高。

我建議您完成調整 apache 實例的基礎知識,使用 noatime 重新安裝包含您的內容和 MySQL 數據庫的磁碟。

引用自:https://serverfault.com/questions/519954