Kubernetes
Kibana 不信任 Kubernetes Helm 設置中的 Let’s Encrypt CA
我有一個配置了 calico 作為網路系統的 k8s 集群。我正在執行啟用了安全性的 helm elastic/elasticsearch 和 elastic/kibana 的略微定製版本。為了安全起見，我使用 Let’s Encrypt 證書。啟動 kibana 時，與 elasticsearch 實例的連接失敗並出現此錯誤
{"type":"log","@timestamp":"2021-06-01T13:09:55+00:00","tags":["debug","elasticsearch","query","data"],"pid":952,"message":"[ConnectionError]: unable to get issuer certificate"}我可以通過在 kibana 中禁用證書驗證來解決此問題。任何人都可以看到，為什麼它會失敗？
密鑰庫創建
cat cert1.pem > store.pem cat privkey1.pem >> store.pem cat chain1.pem >> store.pem cat fullchain1.pem >> store.pem openssl pkcs12 -export -in store.pem -out keystore.pkcs12values_elastic.yaml
replicas: 1 minimumMasterNodes: 1 esConfig: elasticsearch.yml: | xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12 xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12 xpack.security.http.ssl.enabled: true xpack.security.http.ssl.truststore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12 xpack.security.http.ssl.keystore.path: /usr/share/elasticsearch/config/certs-gen/keystore.pkcs12 xpack.security.enabled: true extraEnvs: - name: ELASTIC_PASSWORD valueFrom: secretKeyRef: name: elastic-credentials key: password - name: ELASTIC_USERNAME valueFrom: secretKeyRef: name: elastic-credentials key: username secretMounts: - name: elastic-certificates secretName: elastic-certificates path: /usr/share/elasticsearch/config/certs-gen/ protocol: https service: labels: {} labelsHeadless: {} type: NodePort nodePort: 30001 annotations: {} httpPortName: http transportPortName: transport loadBalancerIP: "" loadBalancerSourceRanges: [] externalTrafficPolicy: "" clusterHealthCheckParams: "wait_for_status=yellow&timeout=1s"values_kibana.yaml
elasticsearchHosts: "redacted its a TLD with appropriate port" extraEnvs: - name: "NODE_OPTIONS" value: "--max-old-space-size=1800" - name: 'ELASTICSEARCH_USERNAME' valueFrom: secretKeyRef: name: elastic-credentials key: username - name: 'ELASTICSEARCH_PASSWORD' valueFrom: secretKeyRef: name: elastic-credentials key: password - name: 'KIBANA_ENCRYPTION_KEY' valueFrom: secretKeyRef: name: kibana key: encryptionkey - name: 'LOGGING_VERBOSE' value: "true" secretMounts: - name: elastic-certificates secretName: elastic-certificates path: /usr/share/kibana/config/certs-gen/ kibanaConfig: kibana.yml: | server.ssl: enabled: true key: /usr/share/kibana/config/certs-gen/privkey1.pem certificate: /usr/share/kibana/config/certs-gen/fullchain1.pem elasticsearch.ssl: certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem verificationMode: certificate xpack.reporting.encryptionKey: ${KIBANA_ENCRYPTION_KEY} xpack.security.encryptionKey: ${KIBANA_ENCRYPTION_KEY} xpack.encryptedSavedObjects.encryptionKey: ${KIBANA_ENCRYPTION_KEY} protocol: https service: type: NodePort loadBalancerIP: "" port: 5601 nodePort: 30002 labels: {} annotations: {} # cloud.google.com/load-balancer-type: "Internal" # service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0 # service.beta.kubernetes.io/azure-load-balancer-internal: "true" # service.beta.kubernetes.io/openstack-internal-load-balancer: "true" # service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true" loadBalancerSourceRanges: [] # 0.0.0.0/0 httpPortName: HTTPkubectl 獲取 pv,pvc,nodes,pods,svc
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE persistentvolume/elk-data 30Gi RWO Retain Bound default/elasticsearch-master-elasticsearch-master-0 40m NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE persistentvolumeclaim/elasticsearch-master-elasticsearch-master-0 Bound elk-data 30Gi RWO 32m NAME STATUS ROLES AGE VERSION node/kubeloadbalan Ready control-plane,master 28h v1.21.1 NAME READY STATUS RESTARTS AGE pod/elasticsearch-master-0 1/1 Running 0 13m pod/kibana-kibana-7fdbd7c66d-bg5xb 0/1 Running 0 7m1s pod/nginx-deployment-868c6bb874-tsbg4 1/1 Running 0 40m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/elasticsearch-master NodePort 10.97.47.66 <none> 9200:30001/TCP,9300:32185/TCP 13m service/elasticsearch-master-headless ClusterIP None <none> 9200/TCP,9300/TCP 13m service/kibana-kibana NodePort 10.101.78.90 <none> 5601:30002/TCP 7m1s service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28h service/nginx-service NodePort 10.100.231.27 <none> 80:30000/TCP 40m從容器內部：
kubectl exec pod/kibana-kibana-7fdbd7c66d-bg5xb -it bash curl -k -u redacted:redacted https://redacted:30001 { "name" : "elasticsearch-master-0", "cluster_name" : "elasticsearch", "cluster_uuid" : "iXEuqB7iQ9abptIZ_Gp1yg", "version" : { "number" : "7.13.0", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "5ca8591c6fcdb1260ce95b08a8e023559635c6f3", "build_date" : "2021-05-19T22:22:26.081971330Z", "build_snapshot" : false, "lucene_version" : "8.8.2", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } curl -u redacted:redacted https://redacted:30001 curl: (60) SSL certificate problem: unable to get local issuer certificate More details here: https://curl.haxx.se/docs/sslcerts.html curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.這表明，即使 curl 也無法從容器內部驗證合法性。這可能表明問題也出在 elasticsearch 上。由於詳細程度增加，完整的 Kibana 日誌完全是一團糟。我可以根據要求發布。
我為此沉迷了好幾天。我筋疲力盡，不知道從哪裡繼續。
通過在評論中討論問題的可能根本原因（缺少CA），我自己找到了解決方案。該問題是由添加錯誤的 CA 引起的
elasticsearch.ssl: certificateAuthorities: /usr/share/kibana/config/certs-gen/fullchain1.pem verificationMode: certificate刪除這些條目可以解決問題。