Search Articles   Keywords



Deploy a K3s Cluster using Ansible

Deploy a K3s Cluster using Ansible


  1. 1. Description
  2. 2. Requirements
    1. 2.1. Before you start
  3. 3. Install Ansible on the Laptop[1]
    1. 3.1. Deploy k3s using Ansible[2]
  4. 4. Install Kubectl on Laptop
    1. 4.1. Setup Kubectl to administer the cluster[5]
  5. 5. Resetting the cluster
  6. 6. Sources

Description

This article will document how to deploy K3s to a cluster of Raspberry Pis.

Requirements

I am using the following:

  • One (1) Laptop (Running Ubuntu 21.10)
  • Three (3) Raspberry Pis running Raspberry Pi OS, 64-bit (Debian 10 buster)
    • Two (2) raspberry pi model 4s, with 8gb RAM
    • One (1) Raspberry Pi 400 with 4gb RAM
    • Each Pi is using a Samsung T7 SSD connected via USB 3.0 for storage

Before you start

Make sure that each Raspberry Pi is running and accessible on the network. You will need a list of each Pi’s IP address.

Install Ansible on the Laptop[1]

To configure the PPA on your machine and install Ansible run these commands:[1:1]

1
2
3
4
$ sudo apt update
$ sudo apt install software-properties-common
$ sudo add-apt-repository --yes --update ppa:ansible/ansible
$ sudo apt install ansible

Deploy k3s using Ansible[2]

https://k3s.io/ maintains an Ansible playbook that can set everything up for us.[2:1]

It is available here: https://github.com/k3s-io/k3s-ansible [3]

Download and unzip the repository to the following location, /home/<user>/Downloads/k3s-ansible-master.

Open a command line and navigate to the k3s-ansible-master project directory:

1
$ cd /Downloads/k3s-ansible-master

Edit the hosts.ini file,

1
$ nano inventory/sample/hosts.ini

It needs to contain IP addresses for each Raspberry Pi node in our cluster:

Example:

1
2
3
4
5
6
7
8
9
10
[master]
10.32.25.224

[node]
10.32.25.111
10.32.25.159

[k3s_cluster:children]
master
node

Now edit the group_vars/all.yml file to supply the Ansible username.

1
$ nano inventory/sample/group_vars/all.yml

Example:

1
2
3
4
5
6
7
---
k3s_version: v1.17.5+k3s1
ansible_user: <username>
systemd_dir: /etc/systemd/system
master_ip: "{{ hostvars[groups['master'][0]]['ansible_host'] | default(groups['master'][0]) }}"
extra_server_args: ""
extra_agent_args: ""

Remove the config files from the sample directory.

1
$ mv inventory/sample/* inventory/

Run the following command to start the Ansible playbook:

1
$ ansible-playbook site.yml -i inventory/hosts.ini

Note: I actually couldn’t get this to work using SSH passwordless authentication. I resorted to using the following command:

1
$ ansible-playbook site.yml -K -i inventory/hosts.ini 

The -K option prompts for the password before the playbook kicks off. Using this method it finished without error.

Install Kubectl on Laptop

Download the latest release with the command:[4]

1
$ curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

Install kubectl:[4:1]

1
$ sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Setup Kubectl to administer the cluster[5]

1
$ scp <user>@master_ip:~/.kube/config ~/.kube/pi-cluster-config

Note: you may need to create the ~/.kube directory if it doesnt already exist.

Set an environment variable to tell kubectl to use our pi-cluster-config file,

1
$ export KUBECONFIG=~/.kube/pi-cluster-config

Now you can run the following command to see if your pi cluster is up and running:

1
$ kubectl get nodes

It should return something like this:

1
2
3
4
NAME        STATUS   ROLES    AGE   VERSION
silverfox Ready master 95m v1.17.5+k3s1
whitefoot Ready <none> 95m v1.17.5+k3s1
cleareyes Ready <none> 95m v1.17.5+k3s1

You can also ssh into the master node and administer the cluster.

Resetting the cluster

It’s best practice to rebuild a cluster frequently.[6]

Regardless of the reason, here’s how to quickly wipe the cluster clean (without re-flashing all the Raspberry Pis from scratch):

In the k3s-ansible repository directory (which you used to set up the cluster), run:

1
ansible-playbook -i inventory/hosts.ini reset.yml

This command will likely have a few failures relating to files that can’t be cleaned up until after a reboot.

Reboot the Raspberry Pis (in the same directory):

1
ansible -i inventory/hosts.ini all -m reboot -b

Run the reset playbook a second time, to clean up the stragglers:

1
ansible-playbook -i inventory/hosts.ini reset.yml

Re-install K3s on the cluster:

1
ansible-playbook -i inventory/hosts.ini site.yml

[5:1]

Sources


  1. Ansible Documentation: Installing Ansibl on Ubuntu ↩︎ ↩︎

  2. K3S: Lightweight Kubernetes ↩︎ ↩︎

  3. GitHub.com/k3s-io/k3s-ansible: Build a Kubernetes cluster using k3s via Ansible ↩︎

  4. Kubernetes.io: Install Kubectl - Linux ↩︎ ↩︎

  5. Raspberry Pi Cluster Ep 3 - Installing Kubernetes (K3s) on the Turing Pi ↩︎ ↩︎

  6. geerlingguy turing-pi-cluster ↩︎