Firewall

kubectl:與伺服器 XXX.XXX.XXXXXX 的連接被拒絕

  • February 27, 2017

我正在嘗試連接到 Google Cloud Engine 中的 kubernetes master(集群)。

kubectl嘗試訪問 kubernetes master 時,我總是得到的錯誤是:

與伺服器 XXX.XXX.XXX.XXX 的連接被拒絕 - 您是否指定了正確的主機或埠?

例如:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server XXX.XXX.XXX.XXX was refused - did you specify the right host or port?

到目前為止,我檢查客戶端使用的是與伺服器相同的版本(版本 1.5.2)。但出於某種奇怪的原因,它拒絕連接。

$ gcloud beta container get-server-config
Fetching server config for europe-west1-c
defaultClusterVersion: 1.5.2
defaultImageType: GCI
validImageTypes:
- CONTAINER_VM
- GCI
validMasterVersions:
- 1.5.2
- 1.4.8
validNodeVersions:
- 1.5.2
- 1.5.1
- 1.4.8
- 1.4.7
- 1.4.6
- 1.3.10
- 1.2.7

在 kubernetes 主集群(伺服器版本)中,我收到以下錯誤:

# kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?

我按照以下步驟創建 Kubernetes 集群主伺服器:

export APP_NAME=brand-project
export GOOGLE_CONTAINER_NAME=b.gcr.io/brand/project
gcloud container clusters create $APP_NAME --zone europe-west1-c --machine-type g1-small --num-nodes 1

我得到並完美地設置了憑據:

gcloud config set container/cluster $APP_NAME
gcloud container clusters get-credentials $APP_NAME
gcloud auth application-default login

描述很好:

gcloud container clusters describe $APP_NAME

Google配置也是:

gcloud config list

上下文似乎也合法:

kubectl config get-contexts

甚至我可以通過 ssh 連接到 kubernetes 主集群,但只有 SSH,沒有 HTTP 或 HTTPS,或者例如執行正確的 kubectl。

我也在Kubernetes 文件中閱讀過:

Google Container Engine 使用 SSH 隧道來保護 Master -> Cluster 通信路徑。在此配置中,apiserver 向集群中的每個節點發起 SSH 隧道(連接到偵聽埠 22 的 ssh 伺服器),並通過該隧道傳遞所有發往 kubelet、節點、pod 或服務的流量。此隧道確保流量不會暴露在執行集群的私有 GCE 網路之外。

所以我不知道如何在 Kubernetes Cluster master 中打開 8000 埠以允許連接(並且在 Google Cloud Engine 中打開防火牆中的所有埠似乎也不起作用)。

我沒有想法,我主要搜尋所有與穀歌相關的條目。所以我不知道如何解決與伺服器的連接或者我在這個過程中做錯了什麼。非常感謝任何幫助!

編輯:

檢查“ Container Registry Deprecation Notices ”後,容器位置已更新為 eu.gcr.io 而不是 b.gcr.io,根據:

2017 年 2 月 28 日,b.gcr.io 和 bucket.gcr.io 等“自帶儲存桶”系統資料庫的使用被視為已棄用。在該日期之後,Container Registry 將不再提供您在這些儲存桶中擁有的任何容器映像。

但問題仍然存在。

解決我自己的答案。看來真正的問題是通過 DNS 訪問和連接到 accounts.google.com。在我檢查我有 ping 之後:

$ ping accounts.google.com
PING accounts.google.com (216.58.201.141) 56(84) bytes of data.
64 bytes from mad06s25-in-f13.1e100.net (216.58.201.141): icmp_seq=1 ttl=56 time=21.9 ms
64 bytes from mad06s25-in-f13.1e100.net (216.58.201.141): icmp_seq=2 ttl=56 time=19.0 ms
64 bytes from mad06s25-in-f13.1e100.net (216.58.201.141): icmp_seq=3 ttl=56 time=20.4 ms
^C
--- accounts.google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 19.070/20.468/21.914/1.173 ms

並在命令期間跟踪所有打開的文件:

$ strace -eopenat kubectl version
openat(AT_FDCWD, "/proc/sys/net/core/somaxconn", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/stat", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/sys/kernel/hostname", O_RDONLY|O_CLOEXEC) = 3
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
openat(AT_FDCWD, "/home/shakaran/.kube/config", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/home/shakaran/.config/gcloud/application_default_credentials.json", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/nsswitch.conf", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/resolv.conf", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/sys/kernel/hostname", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/etc/hosts", O_RDONLY|O_CLOEXEC) = 3
The connection to the server 104.155.120.114 was refused - did you specify the right host or port?
+++ exited with 1 +++

我試圖找出打開的連接:

$ systemd-resolve --status | cat
Global
        DNS Servers: 127.0.1.1
                     8.8.8.8
                     8.8.4.4
         DNSSEC NTA: 10.in-addr.arpa
                     16.172.in-addr.arpa
                     168.192.in-addr.arpa
                     17.172.in-addr.arpa
                     18.172.in-addr.arpa
                     19.172.in-addr.arpa
                     20.172.in-addr.arpa
                     21.172.in-addr.arpa
                     22.172.in-addr.arpa
                     23.172.in-addr.arpa
                     24.172.in-addr.arpa
                     25.172.in-addr.arpa
                     26.172.in-addr.arpa
                     27.172.in-addr.arpa
                     28.172.in-addr.arpa
                     29.172.in-addr.arpa
                     30.172.in-addr.arpa
                     31.172.in-addr.arpa
                     corp
                     d.f.ip6.arpa
                     home
                     internal
                     intranet
                     lan
                     local
                     private
                     test

Link 10 (vboxnet3)
     Current Scopes: LLMNR/IPv4 LLMNR/IPv6
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

Link 9 (vboxnet2)
     Current Scopes: none
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

Link 8 (vboxnet1)
     Current Scopes: none
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

Link 7 (vboxnet0)
     Current Scopes: none
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

Link 6 (docker0)
     Current Scopes: none
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

Link 5 (tun0)
     Current Scopes: none
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

Link 3 (wlan0)
     Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: no
        DNS Servers: 8.8.8.8
                     8.8.4.4

Link 2 (eth0)
     Current Scopes: none
      LLMNR setting: yes
MulticastDNS setting: no
     DNSSEC setting: allow-downgrade
   DNSSEC supported: yes

在我執行禁用界面後,我發現我啟用了 tun0 的 openvpn(阻止與 accounts.google.com 的連接):

sudo ifconfig tun0 down

我得到完美:

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:52:34Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
So sorry for all the noise. But probably it is a good idea add this in FAQ's or so for warning the users about VPNs

所以問題主要是拒絕連接。使用 -v=4 調試kubernetes 項目中的問題 #41975 可能很有用,例如:

$ kubectl version -v=4
Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"08e099554f3c31f6e6f07b448ab3ed78d0520507", GitTreeState:"clean", BuildDate:"2017-01-12T04:57:25Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"}
I0224 11:32:36.914299   30751 helpers.go:221] Connection error: Get https://XXX.XXX.XXX.XXX/api: Post https://accounts.google.com/o/oauth2/token: dial tcp: lookup accounts.google.com on 127.0.1.1:53: read udp 127.0.0.1:46403->127.0.1.1:53: read: connection refused
F0224 11:32:36.914378   30751 helpers.go:116] The connection to the server XXX.XXX.XXX.XXX was refused - did you specify the right host or port?

引用自:https://serverfault.com/questions/834479