Networking
FreeBSD 機器對流的前幾個數據包沒有響應
我在我管理的站點前面有兩台機器作為反向代理記憶體/負載平衡器執行。最近在高峰時間，我遇到了一個問題，即來自任何數據包流（ICMP、UDP、TCP 等）的前幾個數據包到達機器但沒有得到響應。
這是從某人 ping 機器的角度來看的症狀：
PING X.X.X.X (X.X.X.X): 56 data bytes Request timeout for icmp_seq 0 Request timeout for icmp_seq 1 Request timeout for icmp_seq 2 Request timeout for icmp_seq 3 Request timeout for icmp_seq 4 Request timeout for icmp_seq 5 Request timeout for icmp_seq 6 Request timeout for icmp_seq 7 Request timeout for icmp_seq 8 Request timeout for icmp_seq 9 Request timeout for icmp_seq 10 64 bytes from X.X.X.X: icmp_seq=11 ttl=62 time=47.515 ms 64 bytes from X.X.X.X: icmp_seq=12 ttl=62 time=46.108 ms 64 bytes from X.X.X.X: icmp_seq=13 ttl=62 time=48.893 ms 64 bytes from X.X.X.X: icmp_seq=14 ttl=62 time=47.466 ms 64 bytes from X.X.X.X: icmp_seq=15 ttl=62 time=49.679 ms 64 bytes from X.X.X.X: icmp_seq=16 ttl=62 time=50.011 ms 64 bytes from X.X.X.X: icmp_seq=17 ttl=62 time=49.324 ms 64 bytes from X.X.X.X: icmp_seq=18 ttl=62 time=48.989 ms 64 bytes from X.X.X.X: icmp_seq=19 ttl=62 time=51.003 ms 64 bytes from X.X.X.X: icmp_seq=20 ttl=62 time=48.612 ms這是在受影響的機器上通過 tcpdump 的 HTTP 會話視圖（XXXX -> 受影響的機器，CCCC -> 客戶端發出請求）：
21:46:27.105396 IP C.C.C.C.62425 > X.X.X.X.80: Flags [S], seq 139436485, win 65535, options [mss 1380,nop,wscale 3,nop,nop,TS val 398010008 ecr 0,sackOK,eol], length 0 21:46:28.032300 IP C.C.C.C.62425 > X.X.X.X.80: Flags [S], seq 139436485, win 65535, options [mss 1380,nop,wscale 3,nop,nop,TS val 398010017 ecr 0,sackOK,eol], length 0 21:46:28.032337 IP X.X.X.X.80 > C.C.C.C.62425: Flags [S.], seq 1108838018, ack 139436486, win 65535, options [mss 1380,nop,wscale 9,sackOK,TS val 1918451162 ecr 398010017], length 0 21:46:28.064417 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 1, win 65535, options [nop,nop,TS val 398010018 ecr 1918451162], length 0 21:46:28.064438 IP C.C.C.C.62425 > X.X.X.X.80: Flags [P.], ack 1, win 65535, options [nop,nop,TS val 398010018 ecr 1918451162], length 160 21:46:28.165372 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451296 ecr 398010018], length 0 21:46:28.165933 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451296 ecr 398010018], length 1368 21:46:28.219978 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 1369, win 65535, options [nop,nop,TS val 398010019 ecr 1918451296], length 0 21:46:28.220001 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451350 ecr 398010019], length 1368 21:46:28.220011 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451350 ecr 398010019], length 1368 21:46:28.288178 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 4105, win 65493, options [nop,nop,TS val 398010020 ecr 1918451350], length 0 21:46:28.288196 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451418 ecr 398010020], length 1368 21:46:28.288203 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451418 ecr 398010020], length 1368 21:46:28.288210 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451418 ecr 398010020], length 1368 21:46:28.288217 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451418 ecr 398010020], length 1368 21:46:28.333968 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 6841, win 65493, options [nop,nop,TS val 398010020 ecr 1918451418], length 0 21:46:28.333986 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451464 ecr 398010020], length 1368 21:46:28.333994 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451464 ecr 398010020], length 1368 21:46:28.338939 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 9577, win 65493, options [nop,nop,TS val 398010020 ecr 1918451418], length 0 21:46:28.338955 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451469 ecr 398010020], length 1368 21:46:28.338962 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 161, win 128, options [nop,nop,TS val 1918451469 ecr 398010020], length 1368 21:46:28.349943 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 12313, win 65535, options [nop,nop,TS val 398010021 ecr 1918451464], length 0 21:46:28.354190 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 15049, win 65535, options [nop,nop,TS val 398010021 ecr 1918451469], length 0 21:46:28.354206 IP X.X.X.X.80 > C.C.C.C.62425: Flags [P.], ack 161, win 128, options [nop,nop,TS val 1918451484 ecr 398010021], length 8 21:46:28.393441 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 15057, win 65535, options [nop,nop,TS val 398010021 ecr 1918451484], length 0 21:46:28.393452 IP C.C.C.C.62425 > X.X.X.X.80: Flags [F.], seq 161, ack 15057, win 65535, options [nop,nop,TS val 398010021 ecr 1918451484], length 0 21:46:28.393467 IP X.X.X.X.80 > C.C.C.C.62425: Flags [.], ack 162, win 128, options [nop,nop,TS val 1918451524 ecr 398010021], length 0 21:46:28.393481 IP X.X.X.X.80 > C.C.C.C.62425: Flags [F.], seq 15057, ack 162, win 128, options [nop,nop,TS val 1918451524 ecr 398010021], length 0 21:46:28.445126 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 15057, win 65535, options [nop,nop,TS val 398010021 ecr 1918451524], length 0 21:46:28.445138 IP C.C.C.C.62425 > X.X.X.X.80: Flags [.], ack 15058, win 65535, options [nop,nop,TS val 398010021 ecr 1918451524], length 0兩台機器位於一個 CARP 池中，其中一台處於活動狀態，一台處於備用狀態。機器的硬體和配置是相同的。該問題僅影響活動機器，並且對於流向專用機器 IP 地址和 CARPed 浮動 IP 的流量可見。將活動切換到備用，反之亦然會解決問題，所以我很確定這不是硬體問題。
他們使用 pf 作為防火牆並為他們後面的機器進行 NAT 流量。
他們正在執行 FreeBSD 8.0-RELEASE-p5。雖然他們的核心是定制的，但這只是為了添加必要的位來使用 CARP。核心配置如下：
include GENERIC ident LOADBALANCER device pf device pflog device pfsync device carp網卡是使用 em 驅動程序的 Intel 82574L。
有什麼線索嗎？
事實證明，問題出在 pf 上。
預設情況下，FreeBSD 上的 pf 將狀態表條目的數量限制為 10,000。自適應超時大部分時間都保持不變，但無法應對高峰時間。