Kubernetes

使用 kubernetes ingress 暴露簡單的 pod

  • May 19, 2021

嗨，我正在學習 kubernetes，但在公開服務時遇到了麻煩。我想將流量從 HAProxy 路由到我的集群。我正在使用自己的裸機伺服器。

編輯：我還創建了一個入口控制器。

現在，當我描述我的入口時，我可以看到工作機器的 IP 地址，但我仍然知道 Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>) 並且不知道如何訪問我的 pod…

範例配置：

部署.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
 name: apache
 labels:
   app: apache-test
spec:
 replicas: 1
 selector:
   matchLabels:
     app: apache-test
 template:
   metadata:
     labels:
       app: apache-test
   spec:
     containers:
     - name: apache
       image: httpd
       ports:
       - containerPort: 80

服務.yaml

apiVersion: v1
kind: Service
metadata:
 name: apache-test-service
spec:
 selector:
   app: apache-test
 ports:
 - protocol: TCP
   port: 80
   targetPort: 80
   name: http

入口.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: apache-test-ingress
spec:
 rules:
 - host: apache-test.com
   http:
     paths:
     - pathType: Prefix
       path: "/"
       backend:
         service:
           name: apache-test-service
           port:
             number: 80

怎麼了？

描述入口：

Name:             apache-test-ingress
Namespace:        default
Address:          192.168.6.72
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
 Host         Path  Backends
 ----         ----  --------
 apache-test  
              /   apache-test-service:80 (10.44.0.1:80)
Annotations:   <none>
Events:        <none>

描述服務：

Name:              apache-test-service
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=apache-test
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.104.63.167
IPs:               10.104.63.167
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.44.0.1:80
Session Affinity:  None
Events:            <none>

描述控制器：

Name:         ingress-nginx-controller-55bc4f5576-vpsgb                                                                                                                                                    
Namespace:    ingress-nginx
Priority:     0
Node:         kubernetes-node02/192.168.6.72
Start Time:   Sun, 16 May 2021 16:47:26 +0200
Labels:       app.kubernetes.io/component=controller
             app.kubernetes.io/instance=ingress-nginx
             app.kubernetes.io/name=ingress-nginx 
             pod-template-hash=55bc4f5576
Annotations:  <none>
Status:       Running
IP:           10.36.0.1
IPs:
 IP:           10.36.0.1
Controlled By:  ReplicaSet/ingress-nginx-controller-55bc4f5576
Containers:
 controller:
   Container ID:  docker://7daf566a039aba0d06f856b0adcc03659423ec2462c33d9a79f820b58dfcbf98
   Image:         k8s.gcr.io/ingress-nginx/controller:v0.46.0@sha256:52f0058bed0a17ab0fb35628ba97e8d52b5d32299fbc03cc0f6c7b9ff036b61a
   Image ID:      docker-pullable://k8s.gcr.io/ingress-nginx/controller@sha256:52f0058bed0a17ab0fb35628ba97e8d52b5d32299fbc03cc0f6c7b9ff036b61a
   Ports:         80/TCP, 443/TCP, 8443/TCP
   Host Ports:    0/TCP, 0/TCP, 0/TCP
   Args:
     /nginx-ingress-controller
     --election-id=ingress-controller-leader
     --ingress-class=nginx
     --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
     --validating-webhook=:8443
     --validating-webhook-certificate=/usr/local/certificates/cert
     --validating-webhook-key=/usr/local/certificates/key
   State:          Running
     Started:      Sun, 16 May 2021 16:47:28 +0200
   Ready:          True
Restart Count:  0
   Requests:
     cpu:      100m
     memory:   90Mi
   Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
   Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
   Environment:
     POD_NAME:       ingress-nginx-controller-55bc4f5576-vpsgb (v1:metadata.name)
     POD_NAMESPACE:  ingress-nginx (v1:metadata.namespace)
     LD_PRELOAD:     /usr/local/lib/libmimalloc.so
   Mounts:
     /usr/local/certificates/ from webhook-cert (ro)
     /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ftnfs (ro)
Conditions:
 Type              Status
 Initialized       True 
 Ready             True 
 ContainersReady   True 
 PodScheduled      True 
Volumes:
 webhook-cert:
   Type:        Secret (a volume populated by a Secret)
   SecretName:  ingress-nginx-admission
   Optional:    false
 kube-api-access-ftnfs:
   Type:                    Projected (a volume that contains injected data from multiple sources)
   TokenExpirationSeconds:  3607
   ConfigMapName:           kube-root-ca.crt
   ConfigMapOptional:       <nil>
   DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                            node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

來自 POD 的日誌不顯示任何內容…來自入口控制器的日誌：

I0516 14:47:28.871207       8 flags.go:208] "Watching for Ingress" class="nginx"
W0516 14:47:28.871287       8 flags.go:213] Ingresses with an empty class will also be processed by this Ingress controller
W0516 14:47:28.872068       8 client_config.go:614] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0516 14:47:28.872594       8 main.go:241] "Creating API client" host="https://10.96.0.1:443"
I0516 14:47:28.887394       8 main.go:285] "Running in Kubernetes cluster" major="1" minor="21" git="v1.21.0" state="clean" commit="cb303e613a121a29364f75cc67d3d580833a7479" platform="linux/amd64"
I0516 14:47:29.768986       8 main.go:105] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0516 14:47:29.772688       8 main.go:115] "Enabling new Ingress features available since Kubernetes v1.18"
W0516 14:47:29.775841       8 main.go:127] No IngressClass resource with name nginx found. Only annotation will be used.
I0516 14:47:29.793896       8 ssl.go:532] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0516 14:47:29.829161       8 nginx.go:254] "Starting NGINX Ingress controller"
I0516 14:47:29.848934       8 event.go:282] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"0cf6bc98-71b3-4387-a535-7d3dcb956fc8", APIVersion:"v1", ResourceVersion:"401441", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0516 14:47:30.936661       8 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"apache-test-ingress", UID:"6e3c5757-28cf-4a68-be98-827fd69ee86f", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"400092", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
I0516 14:47:31.030103       8 nginx.go:296] "Starting NGINX process"
I0516 14:47:31.030266       8 leaderelection.go:243] attempting to acquire leader lease ingress-nginx/ingress-controller-leader-nginx...
I0516 14:47:31.030658       8 nginx.go:316] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0516 14:47:31.031274       8 controller.go:146] "Configuration changes detected, backend reload required"
I0516 14:47:31.040799       8 leaderelection.go:253] successfully acquired lease ingress-nginx/ingress-controller-leader-nginx
I0516 14:47:31.041189       8 status.go:84] "New leader elected" identity="ingress-nginx-controller-55bc4f5576-vpsgb"
I0516 14:47:31.054203       8 status.go:204] "POD is not ready" pod="ingress-nginx/ingress-nginx-controller-55bc4f5576-vpsgb" node="kubernetes-node02"
I0516 14:47:31.129614       8 controller.go:163] "Backend successfully reloaded"
I0516 14:47:31.129922       8 controller.go:174] "Initial sync, sleeping for 1 second"
I0516 14:47:31.130053       8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-55bc4f5576-vpsgb", UID:"16d9fca9-8ac9-4fc1-be40-056540857035", APIVersion:"v1", ResourceVersion:"401513", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I0516 14:48:31.054140       8 status.go:284] "updating Ingress status" namespace="default" ingress="apache-test-ingress" currentValue=[] newValue=[{IP:192.168.6.72 Hostname: Ports:[]}]
I0516 14:48:31.067947       8 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"apache-test-ingress", UID:"6e3c5757-28cf-4a68-be98-827fd69ee86f", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"401625", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync

描述 POD

Name:         apache-67487b7c8b-8jbgb
Namespace:    default
Priority:     0
Node:         kubernetes-node01/192.168.6.71
Start Time:   Sun, 16 May 2021 15:13:07 +0200
Labels:       app=apache-test
             pod-template-hash=67487b7c8b
Annotations:  <none>
Status:       Running
IP:           10.44.0.1
IPs:
 IP:           10.44.0.1
Controlled By:  ReplicaSet/apache-67487b7c8b
Containers:
 apache:
   Container ID:   docker://70e4e3c4e01dffa11aa3c945f297e2cf3bc8af249c8d900c8aa30381ce7f56e6
   Image:          httpd
   Image ID:       docker-pullable://httpd@sha256:e4c2b93c04762468a6cce6d507d94def02ef4dc285278d0d926e09827f4857db
   Port:           80/TCP
   Host Port:      0/TCP
   State:          Running
     Started:      Sun, 16 May 2021 15:13:10 +0200
   Ready:          True
   Restart Count:  0
   Environment:    <none>
   Mounts:
     /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-c8dfx (ro)
Conditions:
 Type              Status
 Initialized       True 
 Ready             True 
 ContainersReady   True 
 PodScheduled      True 
Volumes:
 kube-api-access-c8dfx:
   Type:                    Projected (a volume that contains injected data from multiple sources)
   TokenExpirationSeconds:  3607
   ConfigMapName:           kube-root-ca.crt
   ConfigMapOptional:       <nil>
   DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                            node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>

編輯：我使用了一個入口控制器：

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.46.0/deploy/static/provider/baremetal/deploy.yaml

TL;博士

您應該使用與您在供應期間創建的類型相關聯的埠，而不是使用80/的埠。443``ServiceNodePortIngress controller

  • $ kubectl get services -n
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.233.39.209   <none>        80:30983/TCP,443:32636/TCP   4h30m
ingress-nginx-controller-admission   ClusterIP   10.233.54.211   <none>        443/TCP                      4h30m

在此範例中，您應該使用您的 IP 地址Nodes和相應埠之一（嘗試從外部連接到它時）：

  • curl http://IP_ADDRESS:30983或者curl -v -k https://IP_ADDRESS:32636

解釋

專注於YAML您使用的清單部分：

# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service
metadata:
 annotations:
 labels:
   <-- REDACTED --> 
 name: ingress-nginx-controller
 namespace: ingress-nginx
spec:
 type: NodePort # <-- IMPORTANT 
 ports:
   - name: http
     port: 80
     protocol: TCP
     targetPort: http
   - name: https
     port: 443
     protocol: TCP
     targetPort: https
 selector:
   app.kubernetes.io/name: ingress-nginx
   app.kubernetes.io/instance: ingress-nginx
   app.kubernetes.io/component: controller

當您應用整個清單時，您創建了一個Servicetype NodePort。

引用官方文件：

  • NodePort: 在每個節點的 IP 上的靜態埠 (the NodePort) 上公開服務。ClusterIPService 路由到的ServiceNodePort是自動創建的。您可以NodePort通過請求從集群外部聯繫服務<NodeIP>:<NodePort>。

– Kubernetes.io：文件：概念：服務網路：服務：發布服務服務類型

類型節點埠

如果將該type欄位設置為NodePort，則 Kubernetes 控制平面會從標誌指定的範圍內分配一個埠--service-node-port-range（預設值：30000-32767）。每個節點都將該埠（每個節點上的相同埠號）代理到您的服務中。.spec.ports[*].nodePort您的服務在其欄位中報告分配的埠。

– Kubernetes.io：文件：概念：服務網路：服務：Nodeport

這NodePort是您的Ingress控制器的入口點。您需要向其埠發送請求以聯繫您的Ingress控制器（然後Ingress控制器會將流量相應地路由到Ingress資源）。

您可以通過呼叫（前面提到）來檢查應該將流量發送到哪個埠：

  • $ kubectl get services -n
NAME                                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.233.39.209   <none>        80:30983/TCP,443:32636/TCP   4h30m
ingress-nginx-controller-admission   ClusterIP   10.233.54.211   <none>        443/TCP                      4h30m

在此範例中，它應如下所示：

  • curl http://IP_ADDRESS:30983為了HTTP
  • curl -v -k https://IP_ADDRESS:32636為了HTTPS

如果您想Nginx Ingress在埠上公開您的控制器80/443您可以選擇使用支持Service的類型LoadBalancer：

Metallb將允許您創建可用於分配Service類型的 IP 地址池LoadBalancer。

旁注！

請記住，使用Ingress您指定的資源，您應該發送一個帶有 a 的請求，Host: apache-test.com否則您將得到一個404. 出於測試目的，您可以設置：

  • - host: 代替：
  • - host: apache-test.com

解決問題的以下部分：

現在，當我描述我的入口時，我可以看到工作機器的 IP 地址，但我仍然知道Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)並且不知道如何訪問我的 pod…

這不會限制您連接到您的能力，Services因為這是一個在不匹配資源中的任何規則時將流量發送到的Ingress資源。

其他資源：

引用自：https://serverfault.com/questions/1063732

相關問答

Kubernetes

在生產環境中更改 Kubernetes 中的容器埠

  • May 30, 2022
檢視問答
Nginx

Kubernetes Nginx 入口控制器指標

  • January 12, 2022
檢視問答
Kubernetes

如何更改 Kubernetes Ingress 中的預設證書鏈

  • October 1, 2021
檢視問答
Kubernetes

大使服務保持“待定”

  • May 24, 2021
檢視問答
Domain-Name-System

單 IP 訪問 Kubernetes 集群入口

  • February 28, 2021
檢視問答
Google-Cloud-Platform

ingress-nginx、GCE 和靜態 IP

  • February 13, 2021
檢視問答