Dok Docs
Github Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage
  1. 欢迎阅读DOK的官方文档
  2. /
  3. Docs
  4. /
  5. kubeadm
Edit page

kubeadm

概述

本文主要介绍 DOK 是如何设置 kubeadm 的配置文件，并且通过 kubeadm 来创建集群的，另外需要注意的是，可以在执行机定义 DOK_KUBEADM_CONFIG 这个环境变量，指向自定义的配置文件，因为 /root/dok-release/bin/k8s/dok.yaml 是默认值，因此如果没有这个环境变量，DOK 就会根据自定义的地址来读取环境变量。

kubeadm init --upload-certs --config "${DOK_KUBEADM_CONFIG-/root/dok-release/bin/k8s/dok.yaml}"

证书

DOK 使用的 kubeadm 并不是社区开源的 kubeadm，主要的修改点是在证书方面，社区的 kubeadm 证书是1年的，为了拓展相关的应用服务，以最小化的原则修改到 100年，减少了运维在切换更换证书的时候可能引发的问题，为了保证 DOK 创建的集群使用的是修改过的 kubeadm，会在安装的过程中进行强制的检查，以防用户重新制作安装包的时候 kubeadm 使用了开源的版本。

默认证书情况 

# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Aug 01, 2023 07:13 UTC   364d                                    no
apiserver                  Aug 01, 2023 07:13 UTC   364d            ca                      no
apiserver-etcd-client      Aug 01, 2023 07:13 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Aug 01, 2023 07:13 UTC   364d            ca                      no
controller-manager.conf    Aug 01, 2023 07:13 UTC   364d                                    no
etcd-healthcheck-client    Aug 01, 2023 07:13 UTC   364d            etcd-ca                 no
etcd-peer                  Aug 01, 2023 07:13 UTC   364d            etcd-ca                 no
etcd-server                Aug 01, 2023 07:13 UTC   364d            etcd-ca                 no
front-proxy-client         Aug 01, 2023 07:13 UTC   364d            front-proxy-ca          no
scheduler.conf             Aug 01, 2023 07:13 UTC   364d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Jul 29, 2032 07:12 UTC   9y              no
etcd-ca                 Jul 29, 2032 07:12 UTC   9y              no
front-proxy-ca          Jul 29, 2032 07:12 UTC   9y              no

修改点 

diff -r kubernetes/cmd/kubeadm/app/constants/constants.go kubernetes-1.21.7/cmd/kubeadm/app/constants/constants.go
49,50c49
< 	// CertificateValidity = time.Hour * 24 * 365
< 	CertificateValidity = time.Hour * 24 * 365 * 100
---
> 	CertificateValidity = time.Hour * 24 * 365
diff -r kubernetes/staging/src/k8s.io/client-go/util/cert/cert.go kubernetes-1.21.7/staging/src/k8s.io/client-go/util/cert/cert.go
66,67c66
< 		// NotAfter:              now.Add(duration365d * 10).UTC(),
< 		NotAfter:              now.Add(duration365d * 100).UTC(),
---
> 		NotAfter:              now.Add(duration365d * 10).UTC(),
diff -r kubernetes/vendor/k8s.io/client-go/util/cert/cert.go kubernetes-1.21.7/vendor/k8s.io/client-go/util/cert/cert.go
66,67c66
< 		// NotAfter:              now.Add(duration365d * 10).UTC(),
< 		NotAfter:              now.Add(duration365d * 100).UTC(),
---
> 		NotAfter:              now.Add(duration365d * 10).UTC(),

修改完证书相关的代码，在 Kubernetes 仓库执行这个命令就会正常进行编译和打包 build/run.sh make kubeadm KUBE_BUILD_PLATFORMS=linux/amd64，重新编译之后，最好再检查一下版本，确认一下 kubeadm 编译的时间。

# _output/dockerized/bin/linux/amd64/kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.7", GitCommit:"1f86634ff08f37e54e8bfcd86bc90b61c98f84d4", GitTreeState:"archive", BuildDate:"2022-07-04T05:37:38Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}

最后创建出集群之后，检查一下证书期限，默认情况下，DOK 安装包内的 kubeadm 就是支持100年证书的。

# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Jul 11, 2122 01:19 UTC   99y                                     no
apiserver                  Jul 11, 2122 01:18 UTC   99y             ca                      no
apiserver-etcd-client      Jul 11, 2122 01:19 UTC   99y             etcd-ca                 no
apiserver-kubelet-client   Jul 11, 2122 01:18 UTC   99y             ca                      no
controller-manager.conf    Jul 11, 2122 01:19 UTC   99y                                     no
etcd-healthcheck-client    Jul 11, 2122 01:19 UTC   99y             etcd-ca                 no
etcd-peer                  Jul 11, 2122 01:19 UTC   99y             etcd-ca                 no
etcd-server                Jul 11, 2122 01:19 UTC   99y             etcd-ca                 no
front-proxy-client         Jul 11, 2122 01:18 UTC   99y             front-proxy-ca          no
scheduler.conf             Jul 11, 2122 01:19 UTC   99y                                     no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Jul 11, 2122 01:18 UTC   99y             no
etcd-ca                 Jul 11, 2122 01:18 UTC   99y             no
front-proxy-ca          Jul 11, 2122 01:18 UTC   99y             no

配置文件

DOK 通过 kubeadm 来创建进去，实际上是执行了这样的命令 kubeadm init --upload-certs --config /root/dok-release/bin/k8s/dok.yaml 因此对于 kubeadm 的配置问题，可以查看文件 /root/dok-release/bin/k8s/dok.yaml ，如果对 Kubernetes 组件的配置有其他要求，可以在下载完 DOK 的安装包之后，编辑上述文件，然后重新打包压缩，并且创建安装压缩包的 md5 文件即可。

kube-apiserver

EphemeralContainers=true 是为了能够使用 kubectl debug。

apiServer:
  timeoutForControlPlane: 4m0s
  extraArgs:
    feature-gates: TTLAfterFinished=true,EphemeralContainers=true
  extraVolumes:
    - name: localtime
      hostPath: /etc/localtime
      mountPath: /etc/localtime
      readOnly: true
      pathType: File
  certSANs:
    - "127.0.0.1"
    - "localhost"

kube-control-manager 

controllerManager:
  extraArgs:
    bind-address: 0.0.0.0
    feature-gates: TTLAfterFinished=true,EphemeralContainers=true
    experimental-cluster-signing-duration: 876000h
  extraVolumes:
    - hostPath: /etc/localtime
      mountPath: /etc/localtime
      name: localtime
      readOnly: true
      pathType: File

kube-scheduler 

scheduler:
  extraArgs:
    bind-address: 0.0.0.0
    feature-gates: TTLAfterFinished=true,EphemeralContainers=true
  extraVolumes:
    - hostPath: /etc/localtime
      mountPath: /etc/localtime
      name: localtime
      readOnly: true
      pathType: File

kube-proxy

主要是 ipvs 的模式。

apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"
metricsBindAddress: "0.0.0.0:10249"

etcd

metrics 将会通过 HTTP 的方式暴露。

etcd:
  local:
    extraArgs:
      listen-metrics-urls: "http://0.0.0.0:2381"