請幫助-我無法弄清楚每天關閉我的伺服器的原因

我的伺服器每天都會出現故障 - 有時一天 2 或 3 次。2-15分鐘之間

我很確定這是由於某種分佈式 DDOS 或暴力破解造成的，但使用 Cloudflare 似乎還不夠

如果有人可以查看這些並確認其導致崩潰的 apache（傳入流量），那將是一個很大的幫助

Oct  1 07:34:34 flylive kernel: httpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
Oct  1 07:34:34 flylive kernel: httpd cpuset=/ mems_allowed=0
Oct  1 07:34:34 flylive kernel: CPU: 1 PID: 6948 Comm: httpd Kdump: loaded Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
Oct  1 07:34:34 flylive kernel: Hardware name: Xen HVM domU, BIOS 4.1.5 11/28/2013
Oct  1 07:34:34 flylive kernel: Call Trace:
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f7ffa5>] dump_stack+0x19/0x1b
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f7a8c3>] dump_header+0x90/0x229
Oct  1 07:34:34 flylive kernel: [<ffffffffb9b0dadb>] ? cred_has_capability+0x6b/0x120
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c251e>] oom_kill_process+0x25e/0x3f0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9b0dbbe>] ? selinux_capable+0x2e/0x40
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c2d76>] out_of_memory+0x4b6/0x4f0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f7b3e0>] __alloc_pages_slowpath+0x5db/0x729
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c91f6>] __alloc_pages_nodemask+0x436/0x450
Oct  1 07:34:34 flylive kernel: [<ffffffffb9a18ea8>] alloc_pages_current+0x98/0x110
Oct  1 07:34:34 flylive kernel: [<ffffffffb99be427>] __page_cache_alloc+0x97/0xb0
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c0fe0>] filemap_fault+0x270/0x420
Oct  1 07:34:34 flylive kernel: [<ffffffffc0339706>] ext4_filemap_fault+0x36/0x50 [ext4]
Oct  1 07:34:34 flylive kernel: [<ffffffffb99edf6a>] __do_fault.isra.61+0x8a/0x100
Oct  1 07:34:34 flylive kernel: [<ffffffffb99ee51c>] do_read_fault.isra.63+0x4c/0x1b0
Oct  1 07:34:34 flylive kernel: [<ffffffffb99f5d80>] handle_mm_fault+0xa20/0xfb0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9e32731>] ? sock_aio_read+0x21/0x30
Oct  1 07:34:34 flylive kernel: [<ffffffffb9a4c633>] ? do_sync_read+0x93/0xe0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f8d653>] __do_page_fault+0x213/0x500
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f8d975>] do_page_fault+0x35/0x90
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f89ac9>] ? error_swapgs+0xaa/0xc0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f89778>] page_fault+0x28/0x30
Oct  1 07:34:34 flylive kernel: Mem-Info:
Oct  1 07:34:34 flylive kernel: active_anon:878702 inactive_anon:27289 isolated_anon:0#012 active_file:0 inactive_file:749 isolated_file:78#012 unevictable:0 dirty:0 writeback:0 unstable:0#012 slab_reclaimable:6602 slab_unreclaimable:10598#012 mapped:30050 shmem:78419 pagetables:6291 bounce:0#012 free:20812 free_pcp:0 free_cma:0
Oct  1 07:34:34 flylive kernel: Node 0 DMA free:15324kB min:276kB low:344kB high:412kB active_anon:296kB inactive_anon:92kB active_file:0kB inactive_file:24kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:132kB shmem:132kB slab_reclaimable:28kB slab_unreclaimable:32kB kernel_stack:0kB pagetables:24kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:39 all_unreclaimable? yes
Oct  1 07:34:34 flylive kernel: lowmem_reserve[]: 0 3580 3765 3765
Oct  1 07:34:34 flylive kernel: Node 0 DMA32 free:64680kB min:64000kB low:80000kB high:96000kB active_anon:3381472kB inactive_anon:103448kB active_file:0kB inactive_file:3104kB unevictable:0kB isolated(anon):0kB isolated(file):312kB present:3915776kB managed:3669344kB mlocked:0kB dirty:0kB writeback:0kB mapped:114384kB shmem:302416kB slab_reclaimable:20252kB slab_unreclaimable:33768kB kernel_stack:7120kB pagetables:23420kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:4822 all_unreclaimable? yes
Oct  1 07:34:34 flylive kernel: lowmem_reserve[]: 0 0 184 184
Oct  1 07:34:34 flylive kernel: Node 0 Normal free:3244kB min:3304kB low:4128kB high:4956kB active_anon:133040kB inactive_anon:5616kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:258048kB managed:189324kB mlocked:0kB dirty:0kB writeback:0kB mapped:5684kB shmem:11128kB slab_reclaimable:6128kB slab_unreclaimable:8592kB kernel_stack:2304kB pagetables:1720kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Oct  1 07:34:34 flylive kernel: lowmem_reserve[]: 0 0 0 0
Oct  1 07:34:34 flylive kernel: Node 0 DMA: 3*4kB (EM) 1*8kB (M) 3*16kB (U) 2*32kB (UE) 3*64kB (UE) 1*128kB (E) 2*256kB (UE) 2*512kB (EM) 3*1024kB (UEM) 1*2048kB (E) 2*4096kB (M) = 15300kB
Oct  1 07:34:34 flylive kernel: Node 0 DMA32: 757*4kB (UEM) 502*8kB (UEM) 400*16kB (UEM) 417*32kB (UEM) 257*64kB (UEM) 91*128kB (UEM) 17*256kB (UEM) 5*512kB (EM) 3*1024kB (M) 0*2048kB 0*4096kB = 64868kB
Oct  1 07:34:34 flylive kernel: Node 0 Normal: 223*4kB (UEM) 69*8kB (EM) 33*16kB (UEM) 24*32kB (UM) 13*64kB (UM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3572kB
Oct  1 07:34:34 flylive kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  1 07:34:34 flylive kernel: 79172 total pagecache pages
Oct  1 07:34:34 flylive kernel: 0 pages in swap cache
Oct  1 07:34:34 flylive kernel: Swap cache stats: add 0, delete 0, find 0/0
Oct  1 07:34:34 flylive kernel: Free swap  = 0kB
Oct  1 07:34:34 flylive kernel: Total swap = 0kB
Oct  1 07:34:34 flylive kernel: 1047453 pages RAM
Oct  1 07:34:34 flylive kernel: 0 pages HighMem/MovableOnly
Oct  1 07:34:34 flylive kernel: 78810 pages reserved
Oct  1 07:34:34 flylive kernel: [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Oct  1 07:34:34 flylive kernel: [  384]     0   384    13969     4769      34        0             0 systemd-journal
Oct  1 07:34:34 flylive kernel: [  386]     0   386    74209     2594      66        0             0 nova-agent
Oct  1 07:34:34 flylive kernel: [  407]     0   407    29795      369      29        0             0 lvmetad
Oct  1 07:34:34 flylive kernel: [  420]     0   420    12083      592      25        0         -1000 systemd-udevd
Oct  1 07:34:34 flylive kernel: [  534]     0   534    13883      109      26        0         -1000 auditd
Oct  1 07:34:34 flylive kernel: [  556]     0   556    13203      188      31        0             0 smartd
Oct  1 07:34:34 flylive kernel: [  557]   996   557     2145       37       9        0             0 lsmd
Oct  1 07:34:34 flylive kernel: [  558]     0   558    22642      214      47        0             0 rngd
Oct  1 07:34:34 flylive kernel: [  560]     0   560     5386       73      17        0             0 irqbalance
Oct  1 07:34:34 flylive kernel: [  561]   997   561   153256     1595      63        0             0 polkitd
Oct  1 07:34:34 flylive kernel: [  565]     0   565    57041      438      63        0             0 abrtd
Oct  1 07:34:34 flylive kernel: [  566]     0   566    56465      370      61        0             0 abrt-watch-log
Oct  1 07:34:34 flylive kernel: [  570]     0   570     6796      289      19        0             0 systemd-logind
Oct  1 07:34:34 flylive kernel: [  571]    81   571    16620      197      32        0          -900 dbus-daemon
Oct  1 07:34:34 flylive kernel: [  582]   995   582    30102      122      28        0             0 chronyd
Oct  1 07:34:34 flylive kernel: [  606]     0   606    90793     6722      95        0             0 firewalld
Oct  1 07:34:34 flylive kernel: [ 1061]     0  1061   108533     2196      45        0             0 rackspace-monit
Oct  1 07:34:34 flylive kernel: [ 1065]     0  1065   145994     3269     101        0             0 tuned
Oct  1 07:34:34 flylive kernel: [ 1296]     0  1296    23245      340      43        0             0 master
Oct  1 07:34:34 flylive kernel: [ 1303]    89  1303    23315      349      45        0             0 qmgr
Oct  1 07:34:34 flylive kernel: [ 1331]     0  1331   120817     3635     109        0             0 rsyslogd
Oct  1 07:34:34 flylive kernel: [ 1332]     0  1332    28231      256      59        0         -1000 sshd
Oct  1 07:34:34 flylive kernel: [ 1357]     0  1357     6477       52      19        0             0 atd
Oct  1 07:34:34 flylive kernel: [ 1358]     0  1358    31598      160      19        0             0 crond
Oct  1 07:34:34 flylive kernel: [ 1624]     0  1624     3022      169      11        0             0 xe-daemon
Oct  1 07:34:34 flylive kernel: [24548]    27 24548    28355       73      12        0             0 mysqld_safe
Oct  1 07:34:34 flylive kernel: [24727]    27 24727   663068   125770     322        0             0 mysqld
Oct  1 07:34:34 flylive kernel: [27028]     0 27028    13322      143      27        0             0 vsftpd
Oct  1 07:34:34 flylive kernel: [20795]     0 20795    27552       34      10        0             0 agetty
Oct  1 07:34:34 flylive kernel: [15951]     0 15951   172780    12691     260        0             0 httpd
Oct  1 07:34:34 flylive kernel: [15952]    48 15952    79019     1410     149        0             0 httpd
Oct  1 07:34:34 flylive kernel: [15957]    48 15957   888056   174687     748        0             0 httpd
Oct  1 07:34:34 flylive kernel: [16176]    48 16176   855193   177632     751        0             0 httpd
Oct  1 07:34:34 flylive kernel: [25271]    48 25271   855272   180205     835        0             0 httpd
Oct  1 07:34:34 flylive kernel: [28247]   993 28247   665696    12110     202        0             0 agent
Oct  1 07:34:34 flylive kernel: [28248]   993 28248   228276     3979      63        0             0 trace-agent
Oct  1 07:34:34 flylive kernel: [28249]   993 28249   252846     5151      81        0             0 process-agent
Oct  1 07:34:34 flylive kernel: [18206]     0 18206    31372       61      18        0             0 anacron
Oct  1 07:34:34 flylive kernel: [28219]     0 28219    28321       54      12        0             0 run-parts
Oct  1 07:34:34 flylive kernel: [28223]     0 28223    88040     3622     122        0             0 yum-cron
Oct  1 07:34:34 flylive kernel: [28224]     0 28224    28411       36      11        0             0 awk
Oct  1 07:34:34 flylive kernel: [ 6903]    48  6903   827221   120930     655        0             0 httpd
Oct  1 07:34:34 flylive kernel: [21093]    48 21093   829126    97392     555        0             0 httpd
Oct  1 07:34:34 flylive kernel: [ 3204]     0  3204    45104      229      45        0             0 crond
Oct  1 07:34:34 flylive kernel: [ 3206]     0  3206    28321       53      12        0             0 run-parts
Oct  1 07:34:34 flylive kernel: [ 3221]     0  3221   320000    12911     166        0             0 yum-cron
Oct  1 07:34:34 flylive kernel: [ 3222]     0  3222    28411       36      12        0             0 awk
Oct  1 07:34:34 flylive kernel: [ 4189]    89  4189    23271      333      46        0             0 pickup
Oct  1 07:34:34 flylive kernel: [ 6209]     0  6209     2922       54      11        0             0 xe-update-guest
Oct  1 07:34:34 flylive kernel: [ 6211]     0  6211    35820      178      26        0             0 crond
Oct  1 07:34:34 flylive kernel: [ 6217]     0  6217     3298       27      11        0             0 mkdir
Oct  1 07:34:34 flylive kernel: Out of memory: Kill process 25271 (httpd) score 186 or sacrifice child
Oct  1 07:34:34 flylive kernel: Killed process 25271 (httpd), UID 48, total-vm:3421088kB, anon-rss:628616kB, file-rss:0kB, shmem-rss:92204kB
Oct  1 07:34:34 flylive kernel: dbus-daemon invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=-900
Oct  1 07:34:34 flylive kernel: dbus-daemon cpuset=/ mems_allowed=0
Oct  1 07:34:34 flylive kernel: CPU: 3 PID: 571 Comm: dbus-daemon Kdump: loaded Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
Oct  1 07:34:34 flylive kernel: Hardware name: Xen HVM domU, BIOS 4.1.5 11/28/2013
Oct  1 07:34:34 flylive kernel: Call Trace:
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f7ffa5>] dump_stack+0x19/0x1b
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f7a8c3>] dump_header+0x90/0x229
Oct  1 07:34:34 flylive kernel: [<ffffffffb9b0dadb>] ? cred_has_capability+0x6b/0x120
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c251e>] oom_kill_process+0x25e/0x3f0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9b0dbbe>] ? selinux_capable+0x2e/0x40
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c2d76>] out_of_memory+0x4b6/0x4f0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f7b3e0>] __alloc_pages_slowpath+0x5db/0x729
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c91f6>] __alloc_pages_nodemask+0x436/0x450
Oct  1 07:34:34 flylive kernel: [<ffffffffb9a18ea8>] alloc_pages_current+0x98/0x110
Oct  1 07:34:34 flylive kernel: [<ffffffffb99be427>] __page_cache_alloc+0x97/0xb0
Oct  1 07:34:34 flylive kernel: [<ffffffffb99c0fe0>] filemap_fault+0x270/0x420
Oct  1 07:34:34 flylive kernel: [<ffffffffc0339706>] ext4_filemap_fault+0x36/0x50 [ext4]
Oct  1 07:34:34 flylive kernel: [<ffffffffb99edf6a>] __do_fault.isra.61+0x8a/0x100
Oct  1 07:34:34 flylive kernel: [<ffffffffb99ee51c>] do_read_fault.isra.63+0x4c/0x1b0
Oct  1 07:34:34 flylive kernel: [<ffffffffb99f5d80>] handle_mm_fault+0xa20/0xfb0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9a9c029>] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f8d653>] __do_page_fault+0x213/0x500
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f8d975>] do_page_fault+0x35/0x90
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f89ac9>] ? error_swapgs+0xaa/0xc0
Oct  1 07:34:34 flylive kernel: [<ffffffffb9f89778>] page_fault+0x28/0x30
Oct  1 07:34:34 flylive kernel: Mem-Info:
Oct  1 07:34:34 flylive kernel: active_anon:878702 inactive_anon:27289 isolated_anon:0#012 active_file:24 inactive_file:0 isolated_file:78#012 unevictable:0 dirty:0 writeback:0 unstable:0#012 slab_reclaimable:6602 slab_unreclaimable:10598#012 mapped:29995 shmem:78419 pagetables:6291 bounce:0#012 free:21434 free_pcp:202 free_cma:0
Oct  1 07:34:34 flylive kernel: Node 0 DMA free:15324kB min:276kB low:344kB high:412kB active_anon:296kB inactive_anon:92kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:132kB shmem:132kB slab_reclaimable:28kB slab_unreclaimable:32kB kernel_stack:0kB pagetables:24kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Oct  1 07:34:34 flylive kernel: lowmem_reserve[]: 0 3580 3765 3765
Oct  1 07:34:34 flylive kernel: Node 0 DMA32 free:67168kB min:64000kB low:80000kB high:96000kB active_anon:3381472kB inactive_anon:103448kB active_file:96kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):312kB present:3915776kB managed:3669344kB mlocked:0kB dirty:0kB writeback:0kB mapped:114164kB shmem:302416kB slab_reclaimable:20252kB slab_unreclaimable:33768kB kernel_stack:7120kB pagetables:23420kB unstable:0kB bounce:0kB free_pcp:808kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:156 all_unreclaimable? yes
Oct  1 07:34:34 flylive kernel: lowmem_reserve[]: 0 0 184 184
Oct  1 07:34:34 flylive kernel: Node 0 Normal free:3244kB min:3304kB low:4128kB high:4956kB active_anon:133040kB inactive_anon:5616kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:258048kB managed:189324kB mlocked:0kB dirty:0kB writeback:0kB mapped:5684kB shmem:11128kB slab_reclaimable:6128kB slab_unreclaimable:8592kB kernel_stack:2304kB pagetables:1720kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Oct  1 07:34:34 flylive kernel: lowmem_reserve[]: 0 0 0 0
Oct  1 07:34:34 flylive kernel: Node 0 DMA: 3*4kB (EM) 1*8kB (M) 4*16kB (UM) 2*32kB (UE) 3*64kB (UE) 1*128kB (E) 2*256kB (UE) 2*512kB (EM) 3*1024kB (UEM) 1*2048kB (E) 2*4096kB (M) = 15316kB
Oct  1 07:34:34 flylive kernel: Node 0 DMA32: 834*4kB (UEM) 536*8kB (UEM) 446*16kB (UEM) 451*32kB (UEM) 263*64kB (UEM) 91*128kB (UEM) 16*256kB (UEM) 5*512kB (EM) 3*1024kB (M) 0*2048kB 0*4096kB = 67400kB
Oct  1 07:34:34 flylive kernel: Node 0 Normal: 223*4kB (UEM) 69*8kB (EM) 33*16kB (UEM) 27*32kB (UM) 13*64kB (UM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3668kB
Oct  1 07:34:34 flylive kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  1 07:34:34 flylive kernel: 78450 total pagecache pages
Oct  1 07:34:34 flylive kernel: 0 pages in swap cache
Oct  1 07:34:34 flylive kernel: Swap cache stats: add 0, delete 0, find 0/0
Oct  1 07:34:34 flylive kernel: Free swap  = 0kB
Oct  1 07:34:34 flylive kernel: Total swap = 0kB
Oct  1 07:34:34 flylive kernel: 1047453 pages RAM
Oct  1 07:34:34 flylive kernel: 0 pages HighMem/MovableOnly
Oct  1 07:34:34 flylive kernel: 78810 pages reserved
Oct  1 07:34:34 flylive kernel: [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Oct  1 07:34:34 flylive kernel: [  384]     0   384    13969     4769      34        0             0 systemd-journal
Oct  1 07:34:34 flylive kernel: [  386]     0   386    74209     2594      66        0             0 nova-agent
Oct  1 07:34:34 flylive kernel: [  407]     0   407    29795      369      29        0             0 lvmetad
Oct  1 07:34:34 flylive kernel: [  420]     0   420    12083      5

.....

這是同一天發生的另一個例子

Oct  1 12:18:25 flylive kernel: httpd invoked oom-killer: gfp_mask=0x201da, order=0, oom_score_adj=0
Oct  1 12:18:25 flylive kernel: httpd cpuset=/ mems_allowed=0
Oct  1 12:18:25 flylive kernel: CPU: 2 PID: 16132 Comm: httpd Kdump: loaded Not tainted 3.10.0-1127.19.1.el7.x86_64 #1
Oct  1 12:18:25 flylive kernel: Hardware name: Xen HVM domU, BIOS 4.1.5 11/28/2013
Oct  1 12:18:25 flylive kernel: Call Trace:
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f7ffa5>] dump_stack+0x19/0x1b
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f7a8c3>] dump_header+0x90/0x229
Oct  1 12:18:25 flylive kernel: [<ffffffffb9b0dadb>] ? cred_has_capability+0x6b/0x120
Oct  1 12:18:25 flylive kernel: [<ffffffffb99c251e>] oom_kill_process+0x25e/0x3f0
Oct  1 12:18:25 flylive kernel: [<ffffffffb9b0dbbe>] ? selinux_capable+0x2e/0x40
Oct  1 12:18:25 flylive kernel: [<ffffffffb99c2d76>] out_of_memory+0x4b6/0x4f0
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f7b3e0>] __alloc_pages_slowpath+0x5db/0x729
Oct  1 12:18:25 flylive kernel: [<ffffffffb99c91f6>] __alloc_pages_nodemask+0x436/0x450
Oct  1 12:18:25 flylive kernel: [<ffffffffb9a18ea8>] alloc_pages_current+0x98/0x110
Oct  1 12:18:25 flylive kernel: [<ffffffffb99be427>] __page_cache_alloc+0x97/0xb0
Oct  1 12:18:25 flylive kernel: [<ffffffffb99c0fe0>] filemap_fault+0x270/0x420
Oct  1 12:18:25 flylive kernel: [<ffffffffc0339706>] ext4_filemap_fault+0x36/0x50 [ext4]
Oct  1 12:18:25 flylive kernel: [<ffffffffb99edf6a>] __do_fault.isra.61+0x8a/0x100
Oct  1 12:18:25 flylive kernel: [<ffffffffb99ee51c>] do_read_fault.isra.63+0x4c/0x1b0
Oct  1 12:18:25 flylive kernel: [<ffffffffb99f5d80>] handle_mm_fault+0xa20/0xfb0
Oct  1 12:18:25 flylive kernel: [<ffffffffb9a4e49e>] ? do_readv_writev+0x19e/0x260
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f8d653>] __do_page_fault+0x213/0x500
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f8d975>] do_page_fault+0x35/0x90
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f89ac9>] ? error_swapgs+0xaa/0xc0
Oct  1 12:18:25 flylive kernel: [<ffffffffb9f89778>] page_fault+0x28/0x30
Oct  1 12:18:25 flylive kernel: Mem-Info:
Oct  1 12:18:25 flylive kernel: active_anon:880510 inactive_anon:26048 isolated_anon:0#012 active_file:980 inactive_file:1374 isolated_file:40#012 unevictable:0 dirty:0 writeback:0 unstable:0#012 slab_reclaimable:6605 slab_unreclaimable:10583#012 mapped:30569 shmem:78464 pagetables:5854 bounce:0#012 free:21148 free_pcp:583 free_cma:0
Oct  1 12:18:25 flylive kernel: Node 0 DMA free:15336kB min:276kB low:344kB high:412kB active_anon:312kB inactive_anon:88kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:132kB shmem:132kB slab_reclaimable:28kB slab_unreclaimable:32kB kernel_stack:0kB pagetables:24kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:40 all_unreclaimable? yes
Oct  1 12:18:25 flylive kernel: lowmem_reserve[]: 0 3580 3765 3765
Oct  1 12:18:25 flylive kernel: Node 0 DMA32 free:66092kB min:64000kB low:80000kB high:96000kB active_anon:3388036kB inactive_anon:98532kB active_file:3980kB inactive_file:5876kB unevictable:0kB isolated(anon):0kB isolated(file):160kB present:3915776kB managed:3669344kB mlocked:0kB dirty:0kB writeback:0kB mapped:116456kB shmem:302596kB slab_reclaimable:20220kB slab_unreclaimable:33640kB kernel_stack:5968kB pagetables:21748kB unstable:0kB bounce:0kB free_pcp:2088kB local_pcp:168kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Oct  1 12:18:25 flylive kernel: lowmem_reserve[]: 0 0 184 184
Oct  1 12:18:25 flylive kernel: Node 0 Normal free:3164kB min:3304kB low:4128kB high:4956kB active_anon:133692kB inactive_anon:5572kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:258048kB managed:189324kB mlocked:0kB dirty:0kB writeback:0kB mapped:5688kB shmem:11128kB slab_reclaimable:6172kB slab_unreclaimable:8660kB kernel_stack:2304kB pagetables:1644kB unstable:0kB bounce:0kB free_pcp:244kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Oct  1 12:18:25 flylive kernel: lowmem_reserve[]: 0 0 0 0
Oct  1 12:18:25 flylive kernel: Node 0 DMA: 1*4kB (E) 0*8kB 5*16kB (UM) 3*32kB (UEM) 3*64kB (UE) 1*128kB (E) 2*256kB (UE) 2*512kB (EM) 3*1024kB (UEM) 1*2048kB (E) 2*4096kB (M) = 15348kB
Oct  1 12:18:25 flylive kernel: Node 0 DMA32: 1001*4kB (UEM) 609*8kB (UEM) 551*16kB (UEM) 459*32kB (UEM) 277*64kB (UEM) 92*128kB (UEM) 11*256kB (UE) 3*512kB (E) 0*1024kB 0*2048kB 0*4096kB = 66236kB
Oct  1 12:18:25 flylive kernel: Node 0 Normal: 143*4kB (UEM) 61*8kB (UEM) 35*16kB (UEM) 34*32kB (UEM) 12*64kB (UEM) 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 3476kB
Oct  1 12:18:25 flylive kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  1 12:18:25 flylive kernel: 81118 total pagecache pages
Oct  1 12:18:25 flylive kernel: 0 pages in swap cache
Oct  1 12:18:25 flylive kernel: Swap cache stats: add 0, delete 0, find 0/0
Oct  1 12:18:25 flylive kernel: Free swap  = 0kB
Oct  1 12:18:25 flylive kernel: Total swap = 0kB
Oct  1 12:18:25 flylive kernel: 1047453 pages RAM
Oct  1 12:18:25 flylive kernel: 0 pages HighMem/MovableOnly
Oct  1 12:18:25 flylive kernel: 78810 pages reserved
Oct  1 12:18:25 flylive kernel: [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Oct  1 12:18:25 flylive kernel: [  384]     0   384    13969     5091      34        0             0 systemd-journal
Oct  1 12:18:25 flylive kernel: [  386]     0   386    74209     2594      66        0             0 nova-agent
Oct  1 12:18:25 flylive kernel: [  407]     0   407    29795      369      29        0             0 lvmetad
Oct  1 12:18:25 flylive kernel: [  420]     0   420    12083      592      25        0         -1000 systemd-udevd
Oct  1 12:18:25 flylive kernel: [  534]     0   534    13883      109      26        0         -1000 auditd
Oct  1 12:18:25 flylive kernel: [  556]     0   556    13203      188      31        0             0 smartd
Oct  1 12:18:25 flylive kernel: [  557]   996   557     2145       37       9        0             0 lsmd
Oct  1 12:18:25 flylive kernel: [  558]     0   558    22642      214      47        0             0 rngd
Oct  1 12:18:25 flylive kernel: [  560]     0   560     5385       72      17        0             0 irqbalance
Oct  1 12:18:25 flylive kernel: [  561]   997   561   153256     1595      63        0             0 polkitd
Oct  1 12:18:25 flylive kernel: [  565]     0   565    57041      438      63        0             0 abrtd
Oct  1 12:18:25 flylive kernel: [  566]     0   566    56465      370      61        0             0 abrt-watch-log
Oct  1 12:18:25 flylive kernel: [  570]     0   570     6796      289      19        0             0 systemd-logind
Oct  1 12:18:25 flylive kernel: [  571]    81   571    16620      197      32        0          -900 dbus-daemon
Oct  1 12:18:25 flylive kernel: [  582]   995   582    30102      122      28        0             0 chronyd
Oct  1 12:18:25 flylive kernel: [  606]     0   606    90793     6722      95        0             0 firewalld
Oct  1 12:18:25 flylive kernel: [ 1061]     0  1061   108534     2198      45        0             0 rackspace-monit
Oct  1 12:18:25 flylive kernel: [ 1065]     0  1065   145994     3269     101        0             0 tuned
Oct  1 12:18:25 flylive kernel: [ 1296]     0  1296    23245      340      43        0             0 master
Oct  1 12:18:25 flylive kernel: [ 1303]    89  1303    23315      349      45        0             0 qmgr
Oct  1 12:18:25 flylive kernel: [ 1331]     0  1331   120817     3887     110        0             0 rsyslogd
Oct  1 12:18:25 flylive kernel: [ 1332]     0  1332    28231      256      59        0         -1000 sshd
Oct  1 12:18:25 flylive kernel: [ 1357]     0  1357     6477       52      19        0             0 atd
Oct  1 12:18:25 flylive kernel: [ 1358]     0  1358    31598      160      19        0             0 crond
Oct  1 12:18:25 flylive kernel: [ 1624]     0  1624     3022      169      11        0             0 xe-daemon
Oct  1 12:18:25 flylive kernel: [24548]    27 24548    28355       73      12        0             0 mysqld_safe
Oct  1 12:18:25 flylive kernel: [24727]    27 24727   663068   125773     322        0             0 mysqld
Oct  1 12:18:25 flylive kernel: [27028]     0 27028    13322      143      27        0             0 vsftpd
Oct  1 12:18:25 flylive kernel: [20795]     0 20795    27552       34      10        0             0 agetty
Oct  1 12:18:25 flylive kernel: [15951]     0 15951   172780    12692     260        0             0 httpd
Oct  1 12:18:25 flylive kernel: [15952]    48 15952    79019     1410     149        0             0 httpd
Oct  1 12:18:25 flylive kernel: [15957]    48 15957   888056   187683     770        0             0 httpd
Oct  1 12:18:25 flylive kernel: [16176]    48 16176   855193   194798     836        0             0 httpd
Oct  1 12:18:25 flylive kernel: [28247]   993 28247   665824    12375     202        0             0 agent
Oct  1 12:18:25 flylive kernel: [28248]   993 28248   228276     3989      63        0             0 trace-agent
Oct  1 12:18:25 flylive kernel: [28249]   993 28249   252846     5139      81        0             0 process-agent
Oct  1 12:18:25 flylive kernel: [ 6903]    48  6903   829270   156330     758        0             0 httpd
Oct  1 12:18:25 flylive kernel: [21093]    48 21093   845510   154373     726        0             0 httpd
Oct  1 12:18:25 flylive kernel: [32719]     0 32719    45104      229      45        0             0 crond
Oct  1 12:18:25 flylive kernel: [32724]     0 32724    28321       44      12        0             0 bash
Oct  1 12:18:25 flylive kernel: [32726]     0 32726    40929      802      35        0             0 python
Oct  1 12:18:25 flylive kernel: [  483]     0   483    45104      229      45        0             0 crond
Oct  1 12:18:25 flylive kernel: [  485]     0   485    28321       54      12        0             0 run-parts
Oct  1 12:18:25 flylive kernel: [  497]     0   497   314045    56125     247        0             0 yum-cron
Oct  1 12:18:25 flylive kernel: [  498]     0   498    28411       36      13        0             0 awk
Oct  1 12:18:25 flylive kernel: [ 1319]    89  1319    23271      333      45        0             0 pickup
Oct  1 12:18:25 flylive kernel: [ 2153]     0  2153     2923       60      12        0             0 xe-update-guest
Oct  1 12:18:25 flylive kernel: [ 2154]     0  2154    44078      214      42        0             0 crond
Oct  1 12:18:25 flylive kernel: Out of memory: Kill process 16176 (httpd) score 201 or sacrifice child
Oct  1 12:18:25 flylive kernel: Killed process 16176 (httpd), UID 48, total-vm:3420772kB, anon-rss:678440kB, file-rss:0kB, shmem-rss:100752kB

我只是想添加這張來自 Cloudflare 的螢幕截圖，顯示伺服器停機時間的請求激增。似乎 Cloudflare 無法阻止所有請求（我假設該網站仍然關閉了幾分鐘）

CloudFlare 截圖

正如其中一條評論中提到的，問題是伺服器記憶體不足。

這是典型的錯誤配置。

您在Internet上託管網站或其他可公開訪問的服務。應該預料到任何流量。那裡有太多“狂野”的東西：腳本惡意軟體、機器人、“探勘效應”、有針對性的攻擊。

但在任何情況下，任何流量都不應該導致記憶體不足的情況。OOM（記憶體不足）導致 Linux 搜尋它找到的最“胖”的程序以釋放記憶體。

Oct 1 07:34:34 flylive核心：記憶體不足：殺死程序25271（httpd）得分186或犧牲孩子

您可能過度分配了 Apache 工作人員，因此它使用了過多的 RAM。如果是這樣，您可能希望將它們限制為更符合您擁有的實際 RAM 的合理數字。

如果您想保證防止這樣的停機，您絕對希望將交換空間配置為故障安全，這將擴展 Linux 可用的虛擬記憶體。但請注意，交換是一件很慢的事情，因為它本質上是“慢磁碟上的 RAM”。

TLDR

• 設置足以完全防止停機的交換空間
• 將 swappinness 核心參數配置為例如 10
• 觀察free -h輸出或其他交換使用情況。如果伺服器使用大量交換，您的伺服器仍然配置錯誤，您需要進一步限制 Apache 程序之類的東西。

引用自：https://serverfault.com/questions/1036014