kubeadm – How to “upgrade” (update) your configuration
As you may know, you can easily configure a Kubernetes master or high availability (HA) cluster using kubeadm. All the instruction are on the kubernetes.io website. However, there’s no details about how to play with the configuration or how to apply a modification to the configuration. After a bit of searching on google I was not able to find how to update my current cluster configuration “the proper way” using kubeadm and it’s configuration.
After you run the kubeadm tool in order to install your cluster (or server) generate a kubeadm-config within the ConfigMap on the kube-system namespace. If you edit that ConfigMap, the changes won’t be propagated on the master nodes. If you want to apply the changes, you will have to apply the configuration yourself master by master.
In this article, I will explain how to apply such changes.
Get your configuration in kubeadm format
The command is simple and easy to find all over the internet. Basically you want to view the current configuration and save it into a file. In linux, you simply type the following command:
kubeadm config view > kubeadm-config.yaml
Now that you have the configuration applied during the kubeadm init (or join) phase you can now edit and do your changes. I would absolutely not recommend to change your CIDR here. In my case, the kube-apiserver does not run using the first network interface. This bring some issues since kubeadm take for granted that you will be using your first network interface. For example, in vagrant, it means that it can access the kubernetes network using the enp0s8 which is the second NIC. The first NIC is used by vagrant itself during the installation and to connect to it using the NAT on the enp0s3. To ensure that the reconfiguration will be working, I absolutely need to inject the advertise-address in the configuration.
Edit your configuration
Bellow, you will see that I’ve added some legacy API (deprecated) used by let’s say Gitlab-ce and the advertise-address. Beware that the advertise-address should be different on each master you reconfigure.
apiServer:
extraArgs:
advertise-address: 192.168.5.102
authorization-mode: Node,RBAC
runtime-config: apps/v1beta1=true,apps/v1beta2=true,extensions/v1beta1/daemonsets=true,extensions/v1beta1/deployments=true,extensions/v1beta1/replicasets=true,extensions/v1beta1/networkpolicies=true,extensions/v1beta1/podsecuritypolicies=true
timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: kube-lb:6443
controllerManager: {}
dns:
type: CoreDNS
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.17.2
networking:
dnsDomain: cluster.local
serviceSubnet: 10.96.0.0/12
scheduler: {}
Validate your changes
To verify if your configuration match what you want to do, you can do a diff with the current server.
kubeadm upgrade diff --config kubeadm-config.yaml
Apply your changes
Once you’re happy with your adjustments, you can then apply them. In the following command I’ve enabled the verbose mode to give all the output of what it is doing. You can skip it if you wish. I personally like to see what’s happening.
kubeadm upgrade apply --config kubeadm-config.yaml --ignore-preflight-errors all --force --v=5
In case you are not so sure about what you’re going to do, you might want to do a –dry-run before applying it.
Once applied, the ConfigMap will be updated. The configuration there only represent the last “kubeadm” configuration you’ve applied. So, each time you re-apply a new version of that configuration will be shown.
Upgrade other master node
The upgrade does not update all your “master” on the cluster, it only upgrade the current one where you are. You will need to apply the config on each master by proceeding the last part of this article (change IP of advertise address in my case and then run the upgrade apply).
That’s it for today 😉