Rebuild home k8s with containerd

dc

• 2 min read

Somethings I need to describe before:

  • Why 18.04? The most stable OS for Rock64, if you choose Raspberry PI nodes, just upgrade to official recommended Ubuntu 20.04 or later.
  • Why containerd? I choose CRI-O before, but there is no ARM64 build for Ubuntu 18.04, so I give up, I don't want to build it myself, from source, and I don't want to use docker, so I choose containerd.
  • Why Flannel network? I don't know, maybe I used it before, I forgot. I think it is simple for my home k8s scenarios.

Control panel

RaspberryPI 4B with 2GB memory, 32G SD card

OS: Ubuntu 18.04.5(official)

Turn off SWAP

sudo swapoff -a

Modules

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# Setup required sysctl params, these persist across reboots.
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.ipv4.ip_forward                 = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

Install containerd

sudo apt-get remove docker docker-engine docker.io containerd runc
sudo apt-get update
sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update
sudo apt-get install containerd.io

Configure

sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo systemctl restart containerd

Change to use systemd as cgroup driver

modify /etc/containerd/config.toml
search SystemdCgroup = true

sudo systemctl restart containerd

Work node

Rock64 with 4GB memory, 32G SD card(eMMC)

OS: Ubuntu 18.04(bionic-minimal-rock64-0.9.14-1159-arm64)

same as control panel

Install k8s

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Initialize control plane

add cgroup_enable=memory cgroup_memory=1 to the end of /boot/firmware/nobtcmd.txt

why nobtcmd.txt, because it's the value of cmdline in /boot/firmware/config.txt
sudo kubeadm init

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install network add-on

kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

Join cluster

sudo kubeadm join xxx