Nginx

如何在 nginx 入口後面正確配置對 kubernees 儀表板的訪問

  • June 8, 2020

我正在嘗試配置 nginx 入口以訪問多個服務,如下所示:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: ingress-monit
spec:
 rules:
 - host: grafana.localhost
   http:
     paths:
     - path: /
       backend:
         serviceName: prometheus-grafana
         servicePort: 80
 - host: kubernetes-dashboard.localhost
   http:
     paths:
     - path: /
       backend:
         serviceName: kubernetes-dashboard
         servicePort: 80

我可以毫無問題地訪問 grafana 服務,我的問題是 kubernetes-dashboard。我已經將 kubernetes-dashboard 配置為允許使用此配置的 HTTP 流量

kind: Service
apiVersion: v1
metadata:
 labels:
   k8s-app: kubernetes-dashboard
 name: kubernetes-dashboard
 namespace: monit
spec:
 ports:
   - port: 80
     targetPort: 9090
 selector:
   k8s-app: kubernetes-dashboard

---

kind: Deployment
apiVersion: apps/v1
metadata:
 labels:
   k8s-app: kubernetes-dashboard
 name: kubernetes-dashboard
 namespace: monit
spec:
 replicas: 1
 revisionHistoryLimit: 10
 selector:
   matchLabels:
     k8s-app: kubernetes-dashboard
 template:
   metadata:
     labels:
       k8s-app: kubernetes-dashboard
   spec:
     containers:
       - name: kubernetes-dashboard
         image: kubernetesui/dashboard:v2.0.0-beta8
         imagePullPolicy: Always
         ports:
           - containerPort: 9090
             protocol: TCP
         args:
           - --namespace=monit
           - --insecure-bind-address=0.0.0.0
           - --insecure-port=9090
           - --enable-insecure-login
           # Uncomment the following line to manually specify Kubernetes API server Host
           # If not specified, Dashboard will attempt to auto discover the API server and connect
           # to it. Uncomment only if the default does not work.
           # - --apiserver-host=http://my-address:port
         volumeMounts:
           - name: kubernetes-dashboard-certs
             mountPath: /certs
             # Create on-disk volume to store exec logs
           - mountPath: /tmp
             name: tmp-volume
         livenessProbe:
           httpGet:
             scheme: HTTP
             path: /
             port: 9090
           initialDelaySeconds: 30
           timeoutSeconds: 30
         securityContext:
           allowPrivilegeEscalation: false
           readOnlyRootFilesystem: true
           runAsUser: 1001
           runAsGroup: 2001
     volumes:
       - name: kubernetes-dashboard-certs
         secret:
           secretName: kubernetes-dashboard-certs
       - name: tmp-volume
         emptyDir: {}
     serviceAccountName: kubernetes-dashboard
     nodeSelector:
       "beta.kubernetes.io/os": linux
     # Comment the following tolerations if Dashboard must not be deployed on master
     tolerations:
       - key: node-role.kubernetes.io/master
         effect: NoSchedule

我還有一個有效的令牌,當我使用 ClusterIP 時,我可以使用它來訪問 kubernetes 儀表板。但是,當我通過 ngress 訪問它時,即使使用有效令牌,我也無法瀏覽登錄頁面(參見螢幕截圖)。

在此處輸入圖像描述

我查看了 Nginx 日誌中的問題/錯誤,但一切似乎都很好

$ kubectl logs -n monit ingress-nginx-controller-bbdc786b4-6nl9h  -f
192.168.65.3 - - [03/Jun/2020:02:03:13 +0000] "GET /api/v1/csrftoken/login HTTP/1.1" 200 85 "http://kubernetes-dashboard.localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" 479 0.001 [monit-kubernetes-dashboard-80] [] 10.1.0.123:9090 85 0.001 200 59fc952888dfadf0223740c31e562ef8
192.168.65.3 - - [03/Jun/2020:02:03:13 +0000] "POST /api/v1/login HTTP/1.1" 200 1508 "http://kubernetes-dashboard.localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" 1545 0.005 [monit-kubernetes-dashboard-80] [] 10.1.0.123:9090 1508 0.005 200 241388246b11031765557475bea603ff
192.168.65.3 - - [03/Jun/2020:02:03:13 +0000] "GET /api/v1/plugin/config HTTP/1.1" 200 185 "http://kubernetes-dashboard.localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" 477 0.003 [monit-kubernetes-dashboard-80] [] 10.1.0.123:9090 185 0.003 200 45371469793ce4f35c45dec70530bea0
192.168.65.3 - - [03/Jun/2020:02:03:13 +0000] "GET /api/v1/login/status HTTP/1.1" 200 108 "http://kubernetes-dashboard.localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" 476 0.001 [monit-kubernetes-dashboard-80] [] 10.1.0.123:9090 108 0.001 200 49171f5e9316a2d6da883d1c4f0b50df
192.168.65.3 - - [03/Jun/2020:02:03:13 +0000] "GET /api/v1/login/status HTTP/1.1" 200 108 "http://kubernetes-dashboard.localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" 476 0.001 [monit-kubernetes-dashboard-80] [] 10.1.0.123:9090 108 0.001 200 c69b9d166f1527f00e7cd175696ec8c7
192.168.65.3 - - [03/Jun/2020:02:03:13 +0000] "GET /api/v1/login/status HTTP/1.1" 200 108 "http://kubernetes-dashboard.localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36" 476 0.001 [monit-kubernetes-dashboard-80] [] 10.1.0.123:9090 108 0.001 200 1f9c27ca407bca57dcc0c26bca65be58

我的入口配置中缺少什麼?

**更新:**我嘗試使用此配置為儀表板設置 https 入口

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: https-ingress-monit
 annotations:
   nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
 rules:
 - host: kubernetes-dashboard.localhost
   http:
     paths:
     - path: /
       backend:
         serviceName: kubernetes-dashboard
         servicePort: 443

但這似乎不起作用,沒有配置端點

$ kubectl describe ingress https-ingress-monit -n monit
Name:             https-ingress-monit
Namespace:        monit
Address:          localhost
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
 Host                            Path  Backends
 ----                            ----  --------
 kubernetes-dashboard.localhost  
                                 /   kubernetes-dashboard:443 (<error: endpoints "kubernetes-dashboard" not found>)
Annotations:                      nginx.ingress.kubernetes.io/backend-protocol: HTTPS
Events:
 Type    Reason  Age   From                      Message
 ----    ------  ----  ----                      -------
 Normal  CREATE  87s   nginx-ingress-controller  Ingress monit/https-ingress-monit
 Normal  UPDATE  74s   nginx-ingress-controller  Ingress monit/https-ingress-monit

現在,當我嘗試訪問http://kubernetes-dashboard.localhost/我看到503 Service Temporarily Unavailable

TL;博士

您無法通過Sign In缺少HTTPS.

正如我在評論中所說:

登錄不可用

如果您的登錄視圖顯示以下錯誤,這意味著您正在嘗試通過 HTTP 登錄,並且出於安全原因它已被禁用。

僅當用於訪問儀表板的 URL 以以下開頭時,才能登錄:

  • http://localhost/...
  • http://127.0.0.1/...
  • https://<domain_name>/...

Github.com:Kubernetes:儀表板:登錄不可用

您只能通過以下方式登錄Kubernetes Dashboard 而不使用 HTTPS

  • http://localhost/...
  • http://127.0.0.1/...

您需要HTTPS登錄到您Kubernetes Dashboard的:

  • https://IP.ADDRESS
  • https://DOMAIN.NAME

endpoints "kubernetes-dashboard" not found

但這似乎不起作用,沒有配置端點

這意味著Ingress資源找不到Endpoint將流量發送到的資源。這發生在您的案例中,因為:

  • Ingressdefault命名空間中
  • Service命名kubernetes-dashboardmonit命名空間中

為了使其工作,您可以(其中一種方法)Ingress專門在命名空間中創建另一個資源monit

您可以呼叫以下命令來獲取有關您的資源的更多資訊:

  • $ kubectl get services -n monit
  • $ kubectl get endpoints -n monit

Kubernetes 中的資源嚴格連接到namespaces. 您可以在此處了解更多資訊:Kubernetes.io:概念:使用對象:命名空間


您有多種部署方式Kubernetes Dashboard。這取決於您使用的解決方案(minikubebare metal kubeadm clustereksgke等)。

部署Kubernetes Dashboard的一般步驟Nginx-ingress

  • 部署Nginx-ingress
  • 下載並修改Dashboard定義
  • Dashboard配置訪問權限Ingress
  • 測試一下

部署Nginx-ingress

請遵循有關部署的官方文件Nginx-ingressKubernetes.github.io:Ingress-nginx:部署

下載並修改Dashboard定義

Kubernetes 的安裝DashboardKubernetes.io:Web ui 儀表板:部署

上面的連結可用於部署Dashboard,但需要進行少量調整。

假設如下:

  • kubernetes-dashboard命名空間中的每個資源
  • 參數Dashboard
  - args:
    - --namespace=kubernetes-dashboard
    - --enable-insecure-login
    - --insecure-bind-address=0.0.0.0
  • Dashboard監聽埠9090
  • Services以及與Dashboardset to port相關的健康檢查9090/TCP/HTTP

辯論技巧!

enable-skip-login false 啟用後,將顯示登錄頁面上的跳過按鈕。

Github.com:Kubernetes:儀表板:參數

您的儀表板定義需要Service在集群外部公開。Service您可以為下面的類似範例創建自己的定義,也可以編輯YAML上面安裝中包含的定義。

下面的例子:

kind: Service
apiVersion: v1
metadata:
 name: dashboard-service
 namespace: kubernetes-dashboard
 labels:
   k8s-app: kubernetes-dashboard
spec:
 selector:
   k8s-app: kubernetes-dashboard
 ports:
   - port: 80
     targetPort: 9090
     nodePort: 30001
     name: dashboard-port
 type: NodePort

請具體看一下:

 ports:
   - port: 80
     targetPort: 9090
     nodePort: 30001
     name: dashboard-port

流量將根據自身 參數的要求發送到Dashboard埠上的 pod 。9090``Dashboard

Dashboard配置訪問權限Ingress

假設您Ingress的部署正確,您可以使用以下範例來公開Dashboard

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: dashboard-ingress
 namespace: kubernetes-dashboard
 annotations:
   kubernetes.io/ingress.class: "nginx"
spec:
 tls:
 - secretName: tls-secret # NON-EXISTENT
 rules:
 - host:
   http:
     paths:
     - path: /
       backend:
         serviceName: dashboard-service
         servicePort: dashboard-port 

請具體看零件:

  • - secretName: tls-secret # NON-EXISTENT- 它將配置控制器使用假證書並允許HTTPS連接
  • namespace: kubernetes-dashboard``Dashboard- 命名空間與其他資源完全相同
  • serviceName: dashboard-service- 相關服務的名稱Dashboard
  • servicePort: dashboard-port- 相關服務的埠名稱Dashboard

測試一下

完成此步驟後,您應該能夠在 Web 瀏覽器中輸入 IP 地址或域名並打開Dashboard面板。

請確保您連接到Dashboardhttps://

如果您將您的配置Dashboard為需要身份驗證,則應提供身份驗證令牌。您可以通過呼叫以下命令找到您的令牌:

  • $ kubectl describe secret NAME_OF_THE_SECRET -n NAMESPACE

引用自:https://serverfault.com/questions/1019919