Linux

幫助!我的伺服器記憶體不足並鎖定,需要硬重啟

  • July 31, 2014

我有一個 VPS(記憶體為 256MB),今天掛了不止一次。它完全凍結了,無法通過 SSH 訪問。我必須執行提供商的“強制關機”程序才能將其恢復。然後伺服器持續了幾個小時,然後再做一次。

第一次出現關於記憶體不足並試圖殺死 mysqld 程序的消息。接下來我設法在 syslog 文件中找到了這個:

Sep 23 01:47:34: [16942.757665]  [out_of_memory+0x19e/0x1e0] out_of_memory+0x19e/0x1e0
Sep 23 01:47:34: [16942.758875] Out of memory: kill process 5983 (apache2) score 57519 or a child

我已經將 Apache 和 MySQL 配置為相當注意低 RAM,至少據我所知,我想知道為什麼會發生這種情況。我只是需要升級,還是有什麼問題?

這是我的 Apache 配置的摘錄:

#
# Timeout: The number of seconds before receives and sends time out.
#
Timeout 30

#
# KeepAlive: Whether or not to allow persistent connections (more than
# one request per connection). Set to "Off" to deactivate.
#
KeepAlive On

#
# MaxKeepAliveRequests: The maximum number of requests to allow
# during a persistent connection. Set to 0 to allow an unlimited amount.
# We recommend you leave this number high, for maximum performance.
#
MaxKeepAliveRequests 200

#
# KeepAliveTimeout: Number of seconds to wait for the next request from the
# same client on the same connection.
#
KeepAliveTimeout 3

##
## Server-Pool Size Regulation (MPM specific)
## 

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves
<IfModule mpm_prefork_module>
   StartServers          1
   MinSpareServers       1
   MaxSpareServers      3
   MaxClients          50
   MaxRequestsPerChild   1000
</IfModule>

還有我的 MySQL 配置:

[mysqld]
#
# * Basic Settings
#

#
# * IMPORTANT
#   If you make changes to these settings and your system uses apparmor, you may
#   also need to also adjust /etc/apparmor.d/usr.sbin.mysqld.
#

user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port        = 3306
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /tmp
language    = /usr/share/mysql/english
skip-external-locking
skip-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address        = 127.0.0.1
#
# * Fine Tuning
#
key_buffer      = 16K
max_allowed_packet  = 1M
thread_stack        = 64K
thread_cache_size   = 4
sort_buffer     = 64K
net_buffer_length   = 2K
#max_connections        = 100
#table_cache            = 64
#thread_concurrency     = 10
#
# * Query Cache Configuration
#
query_cache_limit       = 500k #was 1M
query_cache_size        = 8M #was 16M, lowered for RAM
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
#log        = /var/log/mysql/mysql.log
#
# Error logging goes to syslog. This is a Debian improvement :)
#
# Here you can see queries with especially long duration
#log_slow_queries   = /var/log/mysql/mysql-slow.log
#long_query_time = 2
#log-queries-not-using-indexes
#
# The following can be used as easy to replay backup logs or for replication.
# note: if you are setting up a replication slave, see README.Debian about
#       other settings you may need to change.
#server-id      = 1
#log_bin            = /var/log/mysql/mysql-bin.log
expire_logs_days    = 10
max_binlog_size         = 100M
#binlog_do_db       = include_database_name
#binlog_ignore_db   = include_database_name
#
# * BerkeleyDB
#
# Using BerkeleyDB is now discouraged as its support will cease in 5.1.12.
skip-bdb
#
# * InnoDB
#
# InnoDB is enabled by default with a 10MB datafile in /var/lib/mysql/.
# Read the manual for more InnoDB related options. There are many!
# You might want to disable InnoDB to shrink the mysqld process by circa 100MB.
skip-innodb
#
# * Security Features
#
# Read the manual, too, if you want chroot!
# chroot = /var/lib/mysql/
#
# For generating SSL certificates I recommend the OpenSSL GUI "tinyca".
#
# ssl-ca=/etc/mysql/cacert.pem
# ssl-cert=/etc/mysql/server-cert.pem
# ssl-key=/etc/mysql/server-key.pem



[mysqldump]
quick
quote-names
max_allowed_packet  = 16M

[mysql]
#no-auto-rehash # faster start of mysql but no tab completition

[isamchk]
key_buffer      = 16M

如果它有幫助,這裡ps aux是重啟後大約 20-30 分鐘的輸出:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.3   3992   920 ?        Ss   01:53   0:00 /sbin/init
root         2  0.0  0.0      0     0 ?        S<   01:53   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S<   01:53   0:00 [migration/0]
root         4  0.0  0.0      0     0 ?        S<   01:53   0:00 [ksoftirqd/0]
root         5  0.0  0.0      0     0 ?        S<   01:53   0:00 [watchdog/0]
root         6  0.0  0.0      0     0 ?        S<   01:53   0:00 [events/0]
root         7  0.0  0.0      0     0 ?        S<   01:53   0:00 [khelper]
root        18  0.0  0.0      0     0 ?        S<   01:53   0:00 [xenwatch]
root        19  0.0  0.0      0     0 ?        S<   01:53   0:00 [xenbus]
root        27  0.0  0.0      0     0 ?        S<   01:53   0:00 [migration/1]
root        28  0.0  0.0      0     0 ?        S<   01:53   0:00 [ksoftirqd/1]
root        29  0.0  0.0      0     0 ?        S<   01:53   0:00 [watchdog/1]
root        30  0.0  0.0      0     0 ?        S<   01:53   0:00 [events/1]
root        54  0.0  0.0      0     0 ?        S<   01:53   0:00 [kblockd/0]
root        55  0.0  0.0      0     0 ?        S<   01:53   0:00 [kblockd/1]
root        65  0.0  0.0      0     0 ?        S<   01:53   0:00 [kseriod]
root       104  0.0  0.0      0     0 ?        S    01:53   0:00 [pdflush]
root       105  0.0  0.0      0     0 ?        S    01:53   0:00 [pdflush]
root       106  0.0  0.0      0     0 ?        S<   01:53   0:00 [kswapd0]
root       107  0.0  0.0      0     0 ?        S<   01:53   0:00 [aio/0]
root       108  0.0  0.0      0     0 ?        S<   01:53   0:00 [aio/1]
root       124  0.0  0.0      0     0 ?        S<   01:53   0:00 [accel_watch/0]
root       125  0.0  0.0      0     0 ?        S<   01:53   0:00 [accel_watch/1]
root      2031  0.0  0.0      0     0 ?        S<   01:53   0:00 [kjournald]
root      2192  0.0  0.3  16848   944 ?        S<s  01:53   0:00 /sbin/udevd --daemon
syslog    3514  0.0  0.2  12296   728 ?        Ss   01:53   0:00 /sbin/syslogd -u syslog
root      3534  0.0  0.2   8132   592 ?        S    01:53   0:00 /bin/dd bs 1 if /proc/kmsg of /var/run/klogd/kmsg
klog      3537  0.0  0.8   5600  2292 ?        Ss   01:53   0:00 /sbin/klogd -P /var/run/klogd/kmsg
root      3556  0.0  0.4  50916  1160 ?        Ss   01:53   0:00 /usr/sbin/sshd
root      3776  0.0  0.8  36684  2152 ?        Ss   01:53   0:00 /usr/lib/postfix/master
postfix   3794  0.0  0.8  38740  2140 ?        S    01:53   0:00 pickup -l -t fifo -u -c
postfix   3795  0.0  0.8  38784  2228 ?        S    01:53   0:00 qmgr -l -t fifo -u
root      3802  0.0  0.2  12460   644 ?        Ss   01:53   0:00 /usr/sbin/dovecot
root      3815  0.0  1.0  71648  2876 ?        S    01:53   0:00 dovecot-auth
daemon    3816  0.0  0.1  16428   428 ?        Ss   01:53   0:00 /usr/sbin/atd
root      3829  0.0  0.3  18616   976 ?        Ss   01:53   0:00 /usr/sbin/cron
dovecot   3851  0.0  0.7  14148  1876 ?        S    01:53   0:00 imap-login
dovecot   3852  0.0  0.7  14148  1876 ?        S    01:53   0:00 imap-login
dovecot   3853  0.0  0.7  14148  1876 ?        S    01:53   0:00 imap-login
root      3875  0.0  0.2   3864   592 tty1     Ss+  01:53   0:00 /sbin/getty 38400 tty1
root      3913  0.0  1.1  68272  3116 ?        Ss   01:54   0:00 sshd: root@pts/0 
postfix   3917  0.0  1.1  41248  2964 ?        S    01:54   0:00 tlsmgr -l -t unix -u -c
root      3924  0.0  0.7  18908  2080 pts/0    Ss   01:54   0:00 -bash
root      3958  0.0  0.2   3864   592 ?        Ss   01:55   0:00 /sbin/getty 38400 console
root      4232  0.0  0.2   3944   608 pts/0    S    02:22   0:00 /bin/sh /usr/bin/mysqld_safe
mysql     4271  0.1  5.0 124404 13244 pts/0    Sl   02:22   0:02 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pi
root      4272  0.0  0.2   5068   768 pts/0    S    02:22   0:00 logger -p daemon.err -t mysqld_safe -i -t mysqld
root      4415  0.0  3.6 201880  9516 ?        Ss   02:29   0:00 /usr/sbin/apache2 -k start
www-data  4527  0.5 11.9 225720 31340 ?        S    02:47   0:01 /usr/sbin/apache2 -k start
www-data  4539  0.6 12.2 226492 32092 ?        S    02:50   0:00 /usr/sbin/apache2 -k start
www-data  4541  2.3 12.1 226492 31992 ?        S    02:52   0:00 /usr/sbin/apache2 -k start
root      4544  0.0  0.4  15064  1088 pts/0    R+   02:53   0:00 ps aux

有人可以幫我從這裡出去嗎?

你真的需要修剪你的服務。更好的是,您可能需要考慮使用 inetd 或 xinetd(無論哪個適用於您的安裝)。與其讓多個服務啟動並等待(並佔用寶貴的 RAM),不如在需要時啟動服務來降低 CPU 性能。這就是 inetd/xinetd 的真正意義——它們旨在提供一種低記憶體方式來允許服務按需啟動,而不是讓它們一直閒逛。當然,你的里程可能非常 - 你仍然可能會遇到記憶體資源問題 - 但如果它讓你暫時度過難關……

跟進從哪裡開始(來自評論):

我會先考慮讓 apache 移過來。將最大子數減少到只有 2 個 - 除非您為比自己更多的使用者提供服務,否則擁有 3 個正在執行的程序幾乎沒有幫助。

對於 dovecot,特別是 imap-auth 程序,您只需要一個。您也可以在 inetd 中執行它,儘管它看起來相當薄。如果您不在伺服器上使用 IMAP 或 POP,請將其關閉。最壞的情況是,您又回到了在命令行中閱讀郵件的狀態——雖然不是很漂亮但很實用。

您也可以移動 sshd,儘管我最初會在另一個埠號上執行此操作,並在您打開該埠號之前確認它已連接。

如果可能,考慮遷移到 Exim(這是 Debian/Ubuntu 設置的預設安裝)。我喜歡 Postfix,而且我不使用 Exim,但 Debian 選擇它作為預設設置是有原因的——它需要的資源很少,而且相當安全。

查看您的 /proc/sys/vm/swappiness 設置 ( echo /proc/sys/vm/swappiness) 並確定它是太高還是太低。更高的數字將有助於釋放記憶體(通過提前交換到 VM),但是當它太高時,你會像瘋了一樣交換。大多數安裝說“60”,但對於低記憶體,它可能應該更像 85。我不建議在您的設置中使用 100。

最後,想想“綠屏”。您正在談論在更舊的系統上使用已經存在很長時間的技術(因為它們也受到資源限制)。以這種方式做事有點過時了,但是如果您可以在 shell 中的客戶端程序中完成您的工作,而不是啟動服務,那麼您將進一步擴展您的計算資源。

引用自:https://serverfault.com/questions/67911