Linux
我怎麼知道 tcp 調整是否對我有幫助?
在 Centos7 上執行 logstash,我認為 NIC 可能已飽和。
從 logstash 伺服器,我也可以看到伺服器發送日誌的請求 q。但是我不確定根據我的設置請求 q 是否很高,或者 tcp 調整是否對我有幫助。
一些資訊:
sysctl -a | grep mem net.core.optmem_max = 20480 net.core.rmem_default = 212992 net.core.rmem_max = 212992 net.core.wmem_default = 212992 net.core.wmem_max = 212992 net.ipv4.igmp_max_memberships = 20 net.ipv4.tcp_mem = 227763 303685 455526 net.ipv4.tcp_rmem = 4096 87380 6291456 net.ipv4.tcp_wmem = 4096 16384 4194304 net.ipv4.udp_mem = 229686 306249 459372 net.ipv4.udp_rmem_min = 4096 net.ipv4.udp_wmem_min = 4096 vm.lowmem_reserve_ratio = 256 256 32 vm.memory_failure_early_kill = 0 vm.memory_failure_recovery = 1 vm.nr_hugepages_mempolicy = 0 vm.overcommit_memory = 0 netstat -na --tcp | grep :9123 tcp6 0 0 :::9123 :::* LISTEN tcp6 247834 0 192.168.123.123:9123 10.16.1.82:52289 ESTABLISHED tcp6 241242 0 192.168.123.123:9123 10.31.31.232:65293 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.1.198:53693 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.1.198:56751 ESTABLISHED tcp6 331114 0 192.168.123.123:9123 10.31.35.157:53998 ESTABLISHED tcp6 256047 0 192.168.123.123:9123 10.16.2.155:52221 ESTABLISHED tcp6 240498 0 192.168.123.123:9123 10.19.5.166:51805 ESTABLISHED tcp6 312648 0 192.168.123.123:9123 10.18.16.155:57975 ESTABLISHED tcp6 234321 0 192.168.123.123:9123 10.18.19.242:51664 ESTABLISHED tcp6 255079 0 192.168.123.123:9123 10.19.4.51:51458 ESTABLISHED tcp6 256328 0 192.168.123.123:9123 10.18.45.89:56821 ESTABLISHED tcp6 237167 0 192.168.123.123:9123 10.18.33.26:49278 ESTABLISHED tcp6 248204 0 192.168.123.123:9123 10.18.30.250:54267 ESTABLISHED tcp6 248573 0 192.168.123.123:9123 10.16.1.198:57522 ESTABLISHED tcp6 238348 0 192.168.123.123:9123 10.18.11.169:55147 ESTABLISHED tcp6 243762 0 192.168.123.123:9123 10.31.22.48:60425 ESTABLISHED tcp6 258035 0 192.168.123.123:9123 10.31.46.31:60432 ESTABLISHED tcp6 241863 0 192.168.123.123:9123 10.18.45.113:63376 ESTABLISHED tcp6 327889 0 192.168.123.123:9123 10.18.3.219:58640 ESTABLISHED tcp6 317363 0 192.168.123.123:9123 10.31.37.249:65162 ESTABLISHED tcp6 252394 0 192.168.123.123:9123 10.16.1.92:56360 ESTABLISHED tcp6 326401 0 192.168.123.123:9123 10.31.17.74:53948 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.2.12:53781 ESTABLISHED tcp6 244669 0 192.168.123.123:9123 10.18.18.100:49281 ESTABLISHED tcp6 250264 0 192.168.123.123:9123 10.18.32.116:56795 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.1.82:49304 ESTABLISHED tcp6 310864 0 192.168.123.123:9123 10.18.11.25:64230 ESTABLISHED tcp6 247973 0 192.168.123.123:9123 10.18.22.230:55209 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.17.1.8:51741 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.2.12:54507 ESTABLISHED tcp6 251552 0 192.168.123.123:9123 10.18.24.83:63499 ESTABLISHED tcp6 251481 0 192.168.123.123:9123 10.16.2.72:57268 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.1.198:53406 ESTABLISHED tcp6 312263 0 192.168.123.123:9123 10.19.12.239:52173 ESTABLISHED tcp6 238878 0 192.168.123.123:9123 10.18.5.198:57978 ESTABLISHED tcp6 322460 0 192.168.123.123:9123 10.18.5.124:53117 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.2.72:54883 ESTABLISHED tcp6 237717 0 192.168.123.123:9123 10.16.2.12:56387 ESTABLISHED tcp6 315963 0 192.168.123.123:9123 10.18.26.44:49197 ESTABLISHED tcp6 248914 0 192.168.123.123:9123 10.18.41.101:51859 ESTABLISHED tcp6 0 0 192.168.123.123:9123 10.16.2.155:49303 ESTABLISHED tcp6 316994 0 192.168.123.123:9123 10.18.44.120:49375 ESTABLISHED tcp6 236421 0 192.168.123.123:9123 10.31.43.130:51590 ESTABLISHED tcp6 240929 0 192.168.123.123:9123 10.17.1.114:63546 ESTABLISHED tcp6 306346 0 192.168.123.123:9123 10.17.2.159:61633 ESTABLISHED tcp6 239360 0 192.168.123.123:9123 10.18.39.43:54080 ESTABLISHED tcp6 245361 0 192.168.123.123:9123 10.19.13.107:52629 ESTABLISHED tcp6 244398 0 192.168.123.123:9123 10.18.11.195:53257 ESTABLISHED
您當然可以調整您的 NIC,RedHat 網路性能調整指南提供了一套相當完整的(並且是最近的,c. 2015)需要考慮的項目集。
例如,該文件建議
16384 349520 16777216
使用 TCP rmem(您是否需要調整 wmem 取決於您的設置 - 從您共享的內容來看,這似乎沒有必要)。但是,在我看來,這表明 logstash 是一個瓶頸——您的接收隊列是偵聽給定埠(logstash)的程序的緩衝區,因此增加這些緩衝區並不能真正解決根本問題。在我看來,logstash 已經(相對而言)很慢,而且更多的網路緩衝區聽起來不會讓它更快。
檢查您的 logstash 配置/解析器是否可以針對您的情況/需求進行優化(如果您使用正則表達式,這可能是一個很好的研究領域)。
您可能需要增加可用於 logstash 的資源,和/或考慮分層您的 logstash 基礎設施以減輕您的客戶端/面向源的實例的負載。
換句話說,讓前端接收數據並且幾乎不做任何實際工作(只需添加一些標籤/資訊,也許還有一些到特定處理器的路由),然後讓前端將數據通過以進一步處理到另一個實例(或者到像 Kafka 這樣的消息隊列,例如)。
值得注意的是,如果您使用 DNS 和 geoIP 外掛,則在完全解析事件/日誌/數據後,在中心位置執行這些操作通常是有意義的)。