一個看起來很奇怪的 TCP 會話腳本

我遇到了一個很奇怪的問題，我想和你分享。也許你會幫助我對正在發生的事情提出一些想法。

KVM 驅動的主機上有 3 個虛擬機。實際上有大約 50 台虛擬機，但它們都執行良好，儘管這 3 台虛擬機的行為有點不尋常。

下圖說明了這 3 個 VM 和它們之間的 2 個連結：

當一切正常時，這些會話之間的 TCP 會話（“GET / HTTP/1.0” - “HTTP 200 OK”）如下所示：

00:58:43.885118 IP 192.168.111.2.55480 > 192.168.113.2.http: Flags [S], seq 926382744, win 14600, options [mss 1460,sackOK,TS val 277997 ecr 0,nop,wscale 7], length 0
00:58:43.885380 IP 192.168.113.2.http > 192.168.111.2.55480: Flags [S.], seq 1849545379, ack 926382745, win 14480, options [mss 1460,sackOK,TS val 3702103 ecr 277997,nop,wscale 7], length 0
00:58:43.885957 IP 192.168.111.2.55480 > 192.168.113.2.http: Flags [.], ack 1, win 115, options [nop,nop,TS val 277998 ecr 3702103], length 0
00:58:43.886000 IP 192.168.111.2.55480 > 192.168.113.2.http: Flags [P.], seq 1:213, ack 1, win 115, options [nop,nop,TS val 277998 ecr 3702103], length 212
00:58:43.886061 IP 192.168.113.2.http > 192.168.111.2.55480: Flags [.], ack 213, win 122, options [nop,nop,TS val 3702104 ecr 277998], length 0
00:58:43.922286 IP 192.168.113.2.http > 192.168.111.2.55480: Flags [P.], seq 1:372, ack 213, win 122, options [nop,nop,TS val 3702140 ecr 277998], length 371
00:58:43.922335 IP 192.168.113.2.http > 192.168.111.2.55480: Flags [F.], seq 372, ack 213, win 122, options [nop,nop,TS val 3702140 ecr 277998], length 0
00:58:43.923150 IP 192.168.111.2.55480 > 192.168.113.2.http: Flags [.], ack 372, win 123, options [nop,nop,TS val 278035 ecr 3702140], length 0
00:58:43.923622 IP 192.168.111.2.55480 > 192.168.113.2.http: Flags [F.], seq 213, ack 373, win 123, options [nop,nop,TS val 278036 ecr 3702140], length 0
00:58:43.923671 IP 192.168.113.2.http > 192.168.111.2.55480: Flags [.], ack 214, win 122, options [nop,nop,TS val 3702142 ecr 278036], length 0

好的，到目前為止一切都很好。

然後我們保存 pfSense 配置，銷毀這個 VM，創建一個新的，從頭開始安裝 pfSense 並從備份文件中恢復其配置。

這是我們之後看到的：

00:46:39.218193 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [S], seq 3622924060, win 14600, options [mss 1460,sackOK,TS val 674608862 ecr 0,nop,wscale 7], length 0
00:46:39.218316 IP 192.168.113.2.http > 192.168.111.2.51674: Flags [S.], seq 152904245, ack 3622924061, win 14480, options [mss 1460,sackOK,TS va l 2977436 ecr 674608862,nop,wscale 7], length 0
00:46:39.218570 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [.], ack 1, win 115, options [nop,nop,TS val 674608862 ecr 2977436], length 0
00:46:40.417623 IP 192.168.113.2.http > 192.168.111.2.51674: Flags [S.], seq 152904245, ack 3622924061, win 14480, options [mss 1460,sackOK,TS val 2978636 ecr 674608862,nop,wscale 7], length 0
00:46:40.417947 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [.], ack 1, win 115, options [nop,nop,TS val 674610062 ecr 2977436], length 0
00:46:43.158907 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674612803 ecr 2977436], length 16
00:46:43.360103 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674613004 ecr 2977436], length 16
00:46:43.761787 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674613406 ecr 2977436], length 16
00:46:44.565890 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674614210 ecr 2977436], length 16
00:46:46.174039 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674615818 ecr 2977436], length 16
00:46:49.389921 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674619034 ecr 2977436], length 16
00:46:51.753723 IP 192.168.113.2.http > 192.168.111.2.51672: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 2989972 ecr 674560137], length 0
00:46:55.821824 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674625466 ecr 2977436], length 16
00:46:57.221625 IP 192.168.113.2.http > 192.168.111.2.51672: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 2995440 ecr 674560137], length 0
00:47:08.157575 IP 192.168.113.2.http > 192.168.111.2.51672: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 3006376 ecr 674560137], length 0
00:47:08.685886 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674638330 ecr 2977436], length 16
00:47:30.029609 IP 192.168.113.2.http > 192.168.111.2.51672: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 3028248 ecr 674560137], length 0
00:47:34.413785 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674664058 ecr 2977436], length 16
00:47:40.478757 IP 192.168.113.2.http > 192.168.111.2.51674: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 3038697 ecr 674610062], length 0
00:47:34.413785 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [P.], seq 1:17, ack 1, win 115, options [nop,nop,TS val 674664058 ecr 2977436], length 16
00:47:40.478757 IP 192.168.113.2.http > 192.168.111.2.51674: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 3038697 ecr 674610062], length 0
00:47:40.479216 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [FP.], seq 17:19, ack 2, win 115, options [nop,nop,TS val 674670123 ecr 3038697], length 2
00:47:45.946604 IP 192.168.113.2.http > 192.168.111.2.51674: Flags [F.], seq 1, ack 1, win 114, options [nop,nop,TS val 3044165 ecr 674610062], length 0
00:47:45.946979 IP 192.168.111.2.51674 > 192.168.113.2.http: Flags [.], ack 2, win 115, options [nop,nop,TS val 674675591 ecr 3044165,nop,nop,sack 1 {1:2}], length 0

看起來……我不知道，就像他們沒有聽到對方的聲音一樣。他們可以互相 ping 通，甚至可以互相互動，但看起來他們只是忽略了一些數據包。

兩個 VM 顯示相同，因此 pfSense 不會丟棄任何數據包。儘管那裡的數據包似乎出現了問題。就像他們被什麼東西弄壞了一樣。

這是我無法理解的事情，真的。如果您與我分享任何想法，那將是非常棒的。

在此先感謝大家！

重新啟動 pfsense VM 後它可以工作嗎？

根據他們的 VirtIO 文件here

使用 FreeBSD 中 VirtIO 網路驅動程序的目前狀態，有必要選中 Networking 選項卡上 System > Advanced 下的 Disable hardware checksum offload 框，並在保存設置後手動重新啟動 pfSense，即使沒有提示指示這樣做

我假設如果您只是載入配置，則與選中該框相同，但仍需要手動重新啟動。

您的行為也符合引用的錯誤。這是由於未正確計算 TCP 數據包的校驗和導致數據包在接收方看來已損壞並被丟棄，這就是像 ping 這樣的 ICMP 數據包可以正常工作的原因。

引用自：https://serverfault.com/questions/830447