創建一千個macvlan，ping本地ip失敗

我想macvlan用不同的 ip 創建 1000 個。並使用ip rule多個介面將不同的公共 IP 路由到網際網路。

首先創建1000個macvlan：

ip link add link eth0 address %02x:%02x:%02x:%02x:%02x:%02x eth0_%d type macvlan

%02x 是 MAC 地址，%d 是 0-999。

然後使用 ifconfig 設置每個 macvlan 不同的公共 ip。最後，使用 ip 規則：

ip route add default via ${router} dev ${interface} src ${ip} table ${interfaceidx}
ip rule add from ${ip} table ${interfaceidx}

每個 macvlan 都會添加一個規則和一個表。

使用其他伺服器 ping 任何 macvlan ip 都可以。

root@ubuntu:/tmp# ping 222.217.107.102
PING 222.217.107.102 (222.217.107.102) 56(84) bytes of data.
64 bytes from 222.217.107.102: icmp_seq=1 ttl=56 time=57.5 ms
64 bytes from 222.217.107.102: icmp_seq=2 ttl=56 time=58.0 ms
64 bytes from 222.217.107.102: icmp_seq=3 ttl=56 time=60.1 ms
64 bytes from 222.217.107.102: icmp_seq=4 ttl=56 time=57.5 ms

但是主機中的 ping macvlan 會丟棄數據包：

[root@localhost ~]# ping 222.217.107.102
PING 222.217.107.102 (222.217.107.102) 56(84) bytes of data.
64 bytes from 222.217.107.102: icmp_seq=1 ttl=64 time=0.124 ms
ping: sendmsg: Invalid argument
64 bytes from 222.217.107.102: icmp_seq=3 ttl=64 time=0.049 ms
ping: sendmsg: Invalid argument

ping 127.0.0.1 也會丟棄數據包：

[root@localhost ~]# ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
ping: sendmsg: Invalid argument
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.050 ms
ping: sendmsg: Invalid argument
ping: sendmsg: Invalid argument
64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.061 ms
ping: sendmsg: Invalid argument
ping: sendmsg: Invalid argument
ping: sendmsg: Invalid argument


[root@localhost ~]# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 
[root@localhost ~]# uname -a
Linux localhost.localdomain 3.10.0-957.10.1.el7.x86_64 #1 SMP Mon Mar 18 15:06:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

我認為我的路由表沒問題。其他 pc ping 沒問題。現代 Linux 核心支持 4294967295 個表 id，實現為 32 位長度的 rtnetlink 屬性 RTA_TABLE。https://bird.network.cz/pipermail/bird-users/2013-November/008706.html

327654: from 113.15.163.120 lookup 1429 
327655: from 113.15.163.121 lookup 1511 
327656: from 113.15.163.122 lookup 1522 
327657: from 113.15.163.123 lookup 1186 
327658: from 113.15.163.125 lookup 1513 
327659: from 113.15.163.124 lookup 1190 
327660: from all lookup main 
327670: from all lookup default

linux預設的arp表是1000。

net.ipv4.neigh.default.gc_thresh1 = 8192
net.ipv4.neigh.default.gc_thresh2 = 32768
net.ipv4.neigh.default.gc_thresh3 = 65536

1000 macvlan會導致arp失去，所以ping會掉。將這些行添加到sysctl.conf. sysctl -p.

引用自：https://serverfault.com/questions/961596