Keepalived

Keepalived 在 Tinc VPN 網格中，選舉後無法 ping VIP

  • March 25, 2020

描述

配置

我有 3 個節點，使用 Tinc VPN 連接在一起，我想在其中安裝 HAproxy 並擁有一個 VIP，以便 HAproxy 本身處於高可用性模式。

以下是節點詳細資訊：

  • 節點 1在介面vpn上的 IP 地址為10.0.0.222/32
  • 節點 2在介面vpn上的 IP 地址為10.0.0.13/32
  • 節點 3在介面vpn上的 IP 地址為10.0.0.103/32

為此，我keepalived在每台機器上都安裝了。

我還啟用了以下 sysctl：

net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

節點 1 具有以下**/etc/keepalived/keepalived.conf**文件：

global_defs {
 enable_script_security
 router_id node-1
}

vrrp_script haproxy-check {
   script "/usr/bin/killall -0 haproxy"
   interval 2
   weight 2
}

vrrp_instance haproxy-vip {
   state MASTER
   priority 150
   interface vpn
   virtual_router_id 1
   advert_int 1

   virtual_ipaddress {
       10.0.0.1/32
   }

   track_script {
       haproxy-check
   }
}

節點 2 和 3 具有以下**/etc/keepalived/keepalived.conf**文件：

global_defs {
 enable_script_security
 router_id node-2 # Node 3 has "node-3" here.
}

vrrp_script haproxy-check {
   script "/usr/bin/killall -0 haproxy"
   interval 2
   weight 2
}

vrrp_instance haproxy-vip {
   state BACKUP
   priority 100
   interface vpn
   virtual_router_id 1
   advert_int 1

   virtual_ipaddress {
       10.0.0.1/32
   }

   track_script {
       haproxy-check
   }
}

當所有節點都在執行keepalived時，節點 1 是主節點，並且 VIP10.0.0.1配置良好，其他 2 個節點 ping 它。

節點 1 日誌

啟動時的日誌keepalived：

Dec  5 14:07:53 node-1 systemd[1]: Starting Keepalive Daemon (LVS and VRRP)...
Dec  5 14:07:53 node-1 Keepalived[5870]: Starting Keepalived v1.3.2 (12/03,2016)
Dec  5 14:07:53 node-1 systemd[1]: Started Keepalive Daemon (LVS and VRRP).
Dec  5 14:07:53 node-1 Keepalived[5870]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Dec  5 14:07:53 node-1 Keepalived[5870]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:07:53 node-1 Keepalived[5871]: Starting Healthcheck child process, pid=5872
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Initializing ipvs
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Registering Kernel netlink reflector
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Registering Kernel netlink command channel
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:07:53 node-1 Keepalived[5871]: Starting VRRP child process, pid=5873
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Registering Kernel netlink reflector
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Registering Kernel netlink command channel
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Registering gratuitous ARP shared channel
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:07:53 node-1 Keepalived_healthcheckers[5872]: Using LinkWatch kernel netlink reflector...
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: Using LinkWatch kernel netlink reflector...
Dec  5 14:07:53 node-1 Keepalived_vrrp[5873]: VRRP_Script(haproxy-check) succeeded
Dec  5 14:07:54 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) Transition to MASTER STATE
Dec  5 14:07:54 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) Changing effective priority from 150 to 152
Dec  5 14:07:55 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) Entering MASTER STATE
Dec  5 14:07:57 node-1 ntpd[946]: Listen normally on 45 vpn 10.0.0.1:123

節點 1 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.222/24 scope global vpn
  valid_lft forever preferred_lft forever
inet 10.0.0.1/24 scope global secondary vpn
  valid_lft forever preferred_lft forever

節點 2 和 3 日誌

Dec  5 14:14:32 node-2 systemd[1]: Starting Keepalive Daemon (LVS and VRRP)...
Dec  5 14:14:32 node-2 Keepalived[13745]: Starting Keepalived v1.3.2 (12/03,2016)
Dec  5 14:14:32 node-2 Keepalived[13745]: WARNING - default user 'keepalived_script' for script execution does not exist - please create.
Dec  5 14:14:32 node-2 Keepalived[13745]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:14:32 node-2 Keepalived[13746]: Starting Healthcheck child process, pid=13747
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Initializing ipvs
Dec  5 14:14:32 node-2 systemd[1]: Started Keepalive Daemon (LVS and VRRP).
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Registering Kernel netlink reflector
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Registering Kernel netlink command channel
Dec  5 14:14:32 node-2 Keepalived[13746]: Starting VRRP child process, pid=13748
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Registering Kernel netlink reflector
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Registering Kernel netlink command channel
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Registering gratuitous ARP shared channel
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Opening file '/etc/keepalived/keepalived.conf'.
Dec  5 14:14:32 node-2 Keepalived_healthcheckers[13747]: Using LinkWatch kernel netlink reflector...
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: Using LinkWatch kernel netlink reflector...
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Entering BACKUP STATE
Dec  5 14:14:32 node-2 Keepalived_vrrp[13748]: VRRP_Script(haproxy-check) succeeded
Dec  5 14:14:33 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Changing effective priority from 100 to 102

節點 2 和 3 ip addr：

節點 2

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.13/24 scope global vpn
  valid_lft forever preferred_lft forever

節點 3

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.103/24 scope global vpn
  valid_lft forever preferred_lft forever

問題

但是，當我停keepalived在節點 1 上時，節點 3 被選為主節點，並註冊了 VIP，只有節點 3 ping 10.0.0.1。

節點 1 日誌

停止時：

Dec  5 14:15:26 node-1 systemd[1]: Stopping Keepalive Daemon (LVS and VRRP)...
Dec  5 14:15:26 node-1 Keepalived[5871]: Stopping
Dec  5 14:15:26 node-1 Keepalived_healthcheckers[5872]: Stopped
Dec  5 14:15:26 node-1 Keepalived_vrrp[5873]: VRRP_Instance(haproxy-vip) sent 0 priority
Dec  5 14:15:27 node-1 Keepalived_vrrp[5873]: Stopped
Dec  5 14:15:27 node-1 Keepalived[5871]: Stopped Keepalived v1.3.2 (12/03,2016)
Dec  5 14:15:27 node-1 systemd[1]: Stopped Keepalive Daemon (LVS and VRRP).
Dec  5 14:15:28 node-1 ntpd[946]: Deleting interface #45 vpn, 10.0.0.1#123, interface stats: received=0, sent=0, dropped=0, active_time=451 secs

節點 1 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.222/24 scope global vpn
  valid_lft forever preferred_lft forever

節點 2 日誌

Dec  5 14:15:27 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Transition to MASTER STATE
Dec  5 14:15:27 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Received advert with higher priority 102, ours 102
Dec  5 14:15:27 node-2 Keepalived_vrrp[13748]: VRRP_Instance(haproxy-vip) Entering BACKUP STATE

節點 2 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.13/24 scope global vpn
  valid_lft forever preferred_lft forever

節點 3 日誌

Dec  5 14:15:27 node-3 Keepalived_vrrp[31252]: VRRP_Instance(haproxy-vip) Transition to MASTER STATE
Dec  5 14:15:27 node-3 Keepalived_vrrp[31252]: VRRP_Instance(haproxy-vip) Received advert with lower priority 102, ours 102, forcing new election
Dec  5 14:15:28 node-3 Keepalived_vrrp[31252]: VRRP_Instance(haproxy-vip) Entering MASTER STATE
Dec  5 14:15:29 node-3 ntpd[27734]: Listen normally on 36 vpn 10.0.0.1:123

節點 3 ip addr：

vpn: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 500
link/none
inet 10.0.0.103/24 scope global vpn
  valid_lft forever preferred_lft forever
inet 10.0.0.1/24 scope global secondary vpn
  valid_lft forever preferred_lft forever

更多細節

跟踪路由

我用來traceroute嘗試獲取有關該問題的更多資訊。

當所有節點都在執行keepalived並且 ping VIP 無處不在時，traceroute顯示所有節點：

$ traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
1  10.0.0.1 (10.0.0.1)  0.094 ms  0.030 ms  0.019 ms

當keepalived在節點 1 上停止，並且節點 3 當選時，節點 1 無法確定 VIP 在哪裡：

$ traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
1  * * *
2  * * *
...
29  * * *
30  * * *

節點 2 期望節點 1 擁有 VIP：

$ traceroute 10.0.0.1
traceroute to 10.0.0.1 (10.0.0.1), 30 hops max, 60 byte packets
1  10.0.0.222 (10.0.0.222)  0.791 ms  0.962 ms  1.080 ms
2  * * *
3  * * *
...

並且節點 3 有 VIP，所以它可以工作。

我有設備類型

我閱讀了一些郵件存檔，建議DeviceType = tap在 Tinc 配置中使用 以便傳輸 ARP 包（據我了解），但它沒有幫助。

實際上，隨著選舉的發生，我不確定 Tinc 是根本原因。

嘗試不使用 Tinc

我更改了keepalived配置，使其使用公共網際網路介面，使用單播。

我在每個節點上的每個 keepalived 配置中添加了以下塊（這裡是 for node-1）：

   unicast_src_ip XXX.XXX.XXX.XXX # node's public IP address
   unicast_peer {
       XXX.XXX.XXX.XXX # other node's public IP address
       XXX.XXX.XXX.XXX # other node's public IP address
   }

但是行為與上面描述的完全一樣，所以 Tinc 不應該是相關的。

要求

誰能幫我找出問題所在並解決這個問題，以便在進行新的選舉時，節點可以在新位置找到 VIP？

Mode = switch我剛剛通過添加到我的 tinc.conf解決了類似的問題。

我面臨的問題與您描述的類似；keepalived 將按預期在我的 3 個節點之間轉換我設置的虛擬 ip（指向一個簡單的 nginx 伺服器）。但是，唯一能夠訪問該服務的節點是當選的 MASTER。這是因為路由表是從主機配置文件而不是從 ARP 數據靜態建構的。

我確實覺得很奇怪，你沒有 tinc 的嘗試失敗了。當我更改配置以在具有路由器的本地網路上執行時，keepalived 和 haproxy 按預期執行，並且 vip 在路由器的 ARP 表中可見。您確定您為本地測試更改了 haproxy 和 keepalived 配置嗎？

祝你好運！

參考：

  1. https://www.tinc-vpn.org/documentation/Main-configuration-variables.html
  2. https://www.tinc-vpn.org/pipermail/tinc/2010-February/002191.html

引用自：https://serverfault.com/questions/994427

相關問答

Backup

查看 Keepalived 的目前狀態

  • April 8, 2022
檢視問答
Ssl

HAProxy 作為 TCP 負載平衡器（SSL 直通）不起作用？

  • April 3, 2022
檢視問答
Linux

當 Firewalld 執行時，Keepalived 腦裂

  • March 28, 2022
檢視問答
Kubernetes

我可以在所有 Keepalived 節點上擁有相同的配置嗎？

  • January 5, 2022
檢視問答
Centos

KeepAlived 在不同的子網上

  • December 29, 2021
檢視問答
Linux

執行 keepalived 的兩台伺服器都成為主伺服器並具有相同的虛擬 IP

  • December 3, 2021
檢視問答