Kvm-Virtualization

使用 nftables 將 HTTP(s) 流量轉發到 VM 來賓而不偽裝

  • September 28, 2020

我目前正在將安全設置為盡可能基於 KVM 的伺服器。VM 主機應該或多或少對 Internet 不可見，並且只有執行單個服務的 VM 來賓應該被公開。VM 來賓應該沒有網際網路訪問權限（–> 沒有 NAT 偽裝）。

現在我想將 VM 來賓的 HTTP(s) 埠公開192.168.122.183到 Internet。iptables幾乎不推薦使用的所有現有文件使用。所以我更喜歡使用nftables.

根據文件，這應該相當簡單，所以我的nft list ruleset外觀相應地：

table ip nat {
       chain prerouting {
               type nat hook prerouting priority 0; policy accept;
               iif "enp35s0" tcp dport { http, https } dnat to 192.168.122.183
       }
}
table inet filter {
       chain input {
               type filter hook input priority 0; policy accept;
       }

       chain forward {
               type filter hook forward priority 0; policy accept;
       }

       chain output {
               type filter hook output priority 0; policy accept;
       }
}

ip轉發也被啟動sysctl -a | grep forward：

net.ipv4.conf.all.bc_forwarding = 0
net.ipv4.conf.all.forwarding = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.default.bc_forwarding = 0
net.ipv4.conf.default.forwarding = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.enp35s0.bc_forwarding = 0
net.ipv4.conf.enp35s0.forwarding = 1
net.ipv4.conf.enp35s0.mc_forwarding = 0
net.ipv4.conf.virbr10.bc_forwarding = 0
net.ipv4.conf.virbr10.forwarding = 1
net.ipv4.conf.virbr10.mc_forwarding = 0
net.ipv4.conf.virbr10-dummy.bc_forwarding = 0
net.ipv4.conf.virbr10-dummy.forwarding = 1
net.ipv4.conf.virbr10-dummy.mc_forwarding = 0
net.ipv4.ip_forward = 1
net.ipv4.ip_forward_update_priority = 1
net.ipv4.ip_forward_use_pmtu = 0

從 VM 主機訪問 Web 伺服器可以正常工作，但從 Internet 上我沒有得到響應。我錯過了什麼？

可能與這個問題有關。

這是來自主機的 tcpdump 的摘錄。正如所見，來自 VM 主機的請求得到了正確回答，並且外部請求正在彈出，但從未得到回答。任何想法為什麼？

23:43:28.048714 IP 192.168.122.1.35584 > 192.168.122.183.http: Flags [S], seq 502099991, win 64240, options [mss 1460,sackOK,TS val 3361102460 ecr 0,nop,wscale 7], length 0
23:43:28.048746 IP 192.168.122.183.http > 192.168.122.1.35584: Flags [S.], seq 3142491522, ack 502099992, win 65160, options [mss 1460,sackOK,TS val 519662698 ecr 3361102460,nop,wscale 7], length 0
23:43:28.048801 IP 192.168.122.1.35584 > 192.168.122.183.http: Flags [.], ack 1, win 502, options [nop,nop,TS val 3361102460 ecr 519662698], length 0
23:43:28.048846 IP 192.168.122.1.35584 > 192.168.122.183.http: Flags [P.], seq 1:143, ack 1, win 502, options [nop,nop,TS val 3361102460 ecr 519662698], length 142: HTTP: GET / HTTP/1.1
23:43:28.048851 IP 192.168.122.183.http > 192.168.122.1.35584: Flags [.], ack 143, win 508, options [nop,nop,TS val 519662698 ecr 3361102460], length 0
23:43:28.048960 IP 192.168.122.183.http > 192.168.122.1.35584: Flags [P.], seq 1:325, ack 143, win 508, options [nop,nop,TS val 519662698 ecr 3361102460], length 324: HTTP: HTTP/1.1 200 OK
23:43:28.049015 IP 192.168.122.1.35584 > 192.168.122.183.http: Flags [.], ack 325, win 501, options [nop,nop,TS val 3361102460 ecr 519662698], length 0

23:43:45.263545 STP 802.1d, Config, Flags [none], bridge-id 8000.52:54:00:de:a7:a9.8002, length 35
23:43:45.537492 IP external.dip0.t-ipconnect.de.46866 > 192.168.122.183.http: Flags [S], seq 1122615229, win 64240, options [mss 1452,sackOK,TS val 2536247181 ecr 0,nop,wscale 7], length 0
23:43:47.247528 STP 802.1d, Config, Flags [none], bridge-id 8000.52:54:00:de:a7:a9.8002, length 35
23:43:47.553490 IP external.dip0.t-ipconnect.de.46866 > 192.168.122.183.http: Flags [S], seq 1122615229, win 64240, options [mss 1452,sackOK,TS val 2536249197 ecr 0,nop,wscale 7], length 0

供參考我的ip addr輸出：

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   inet 127.0.0.1/8 scope host lo
      valid_lft forever preferred_lft forever
   inet6 ::1/128 scope host 
      valid_lft forever preferred_lft forever
2: enp35s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
   link/ether a8:a1:59:0e:**:** brd ff:ff:ff:ff:ff:ff
   inet 95.217.***.***/26 brd 95.217.120.127 scope global enp35s0
      valid_lft forever preferred_lft forever
   inet6 2a01:4f9:**:****::2/64 scope global 
      valid_lft forever preferred_lft forever
   inet6 fe80::aaa1:59ff:****:*****/64 scope link 
      valid_lft forever preferred_lft forever
3: virbr10-dummy: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master virbr10 state UNKNOWN group default qlen 1000
   link/ether 52:54:00:de:a7:a9 brd ff:ff:ff:ff:ff:ff
4: virbr10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
   link/ether 52:54:00:de:a7:a9 brd ff:ff:ff:ff:ff:ff
   inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr10
      valid_lft forever preferred_lft forever
   inet6 fe80::5054:ff:fede:a7a9/64 scope link 
      valid_lft forever preferred_lft forever
5: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr10 state UNKNOWN group default qlen 1000
   link/ether fe:54:00:94:b9:ce brd ff:ff:ff:ff:ff:ff
   inet6 fe80::fc54:ff:fe94:b9ce/64 scope link 
      valid_lft forever preferred_lft forever

回答的另一個問題virt-manager與它不兼容，nftables因為它仍然使用iptables. 但是 iptable 規則是空的並且iptable_nat沒有載入 iptables 模組（尤其是）

# lsmod | grep ip
nft_chain_route_ipv4    16384  0
nft_chain_nat_ipv4     16384  1
ipt_MASQUERADE         16384  0
nf_nat_ipv4            16384  2 ipt_MASQUERADE,nft_chain_nat_ipv4
nf_nat                 36864  2 nft_nat,nf_nat_ipv4
ipt_REJECT             16384  0
nf_reject_ipv4         16384  1 ipt_REJECT
nf_conntrack          172032  5 xt_conntrack,nf_nat,nft_nat,ipt_MASQUERADE,nf_nat_ipv4
nf_defrag_ipv6         20480  1 nf_conntrack
nf_defrag_ipv4         16384  1 nf_conntrack
nf_tables             143360  13 nft_chain_route_ipv4,nft_compat,nft_nat,nft_chain_nat_ipv4,nft_counter,nf_tables_set
ip_tables              28672  0
x_tables               45056  7 xt_conntrack,nft_compat,xt_tcpudp,ipt_MASQUERADE,xt_CHECKSUM,ipt_REJECT,ip_tables

必須滿足以下條件才能使 VM 能夠接收來自外部的連接：

  1. VM 有一個預設路由指向192.168.122.1.
  2. 防火牆已配置 DNAT 規則。

在您的情況下，第一個可能失去。這會導致 VM 網路堆棧丟棄 TCP 協議發送的 SYN-ACK 數據包。發生這種情況是因為客戶端 VM 不知道將響應數據包轉發到哪裡。

您要麼需要明確的源 NAT 或偽裝，您的虛擬機可能會回答，但您不會重寫傳出數據包的源 IP，因此外部客戶端接收的數據包沒有意義。

引用自：https://serverfault.com/questions/1035544

相關問答

Ubuntu

在 libvirt / KVM 中將埠轉發給來賓

  • November 17, 2021
檢視問答
Smtp

NFTables：是否可以在不偽裝的情況下轉發流量？

  • September 29, 2020
檢視問答
Linux

保留傳出數據包的源埠

  • October 13, 2014
檢視問答
Networking

KVM nat 命令行

  • January 15, 2022
檢視問答
Nginx

iptables 埠轉發，同時保留客戶端 IP

  • November 18, 2021
檢視問答
Linux

為什麼 MASQUERADE SNAT 可以阻止 localhost 連接？

  • July 17, 2021
檢視問答