Skip to main content

Installing a Rancher managed cluster on OpenStack

This guide will validate running an RKE1 (Rancher) cluster on an OpenStack environment. We'll be following the official rancher documentation: Setting up a High-availability RKE Kubernetes Cluster.

RKE1 is the first iteration of Rancher's Kubernetes deployment system. RKE2 is available and also works within OpenStack. We'll be creating a guide on RKE2 in the near future. To learn the differences between RKE1 and RKE2, please see RKE1 vs RKE2.

Setting up an RKE1 cluster on OpenStack is rather simple. First, we need create the nodes for our cluster within OpenStack. Then we create an RKE configuration file that points to each of our nodes. Finally, we use RKE to install Kubernetes on our nodes.

We'll be performing these steps on a Standard - OpenMetal Cloud. You can follow along by deploying a trial, then accessing one of your hardware nodes via SSH.

Prerequisites​

This guide requires access to the OpenStack CLI. Complete the following steps to install the OpenStack CLI.

Create Nodes​

We'll need to create 3 nodes for our cluster. Before we can create our nodes, we need to create a network, subnet, and router. We'll also need to create a security group so traffic can reach our cluster.

Create a Project​

Creating a project will separate the resources and make it easy to cleanup later.

openstack project create --domain default --description "RKE1 Cluster" rke1
openstack role add --project rke1 --user admin admin

Update the OpenStack CLI​

Update the following environment variables.

Note: Replace <project_id> with the project id from the previous step.

export OS_PROJECT_ID=<project_id>
export OS_PROJECT_NAME=rke1

Network​

We'll need to create some networking resources for our nodes.

Create a Network​

openstack network create \
  --project rke1 \
  rke1

Create a Subnet​

openstack subnet create \
  rke1-subnet \
  --project rke1 \
  --network rke1 \
  --subnet-range 172.31.0.0/28

Create a Router​

openstack router create \
  rke1-router \
  --project rke1

Add a Subnet to the Router​

openstack router add subnet \
  rke1-router \
  rke1-subnet

Set the Router's External Gateway​

openstack router set --external-gateway \
  $(openstack network list --external -f value -c ID) \
  rke1-router

Create a Security Group​

Create a security group that allows traffic on common ports required by Kubernetes deployment systems.

Note: This is not a definitive list. In production deployments you'll want to lock down your security groups to only allow traffic to the nodes and ports that need it.

openstack security group create rke1 --project rke1
openstack security group rule create --protocol icmp --dst-port 1:65535 rke1
openstack security group rule create --protocol tcp --dst-port 22:22 rke1
openstack security group rule create --protocol tcp --dst-port 53:53 rke1
openstack security group rule create --protocol tcp --dst-port 179:179 rke1
openstack security group rule create --protocol tcp --dst-port 6443:6443 rke1
openstack security group rule create --protocol tcp --dst-port 2380:2380 rke1
openstack security group rule create --protocol tcp --dst-port 7080:7080 rke1
openstack security group rule create --protocol tcp --dst-port 8472:8472 rke1
openstack security group rule create --protocol tcp --dst-port 8080:8080 rke1
openstack security group rule create --protocol tcp --dst-port 9100:9100 rke1
openstack security group rule create --protocol tcp --dst-port 10250:10250 rke1
openstack security group rule create --protocol udp --dst-port 8472:8472 rke1
openstack security group rule create --protocol tcp --dst-port 30000:32767 rke1

Create a Key Pair​

This key will be used by the deployment environment to access the nodes.

Generate a key pair​

ssh-keygen -t ed25519 -N '' -f /root/.ssh/id_rke1

Upload the public key to OpenStack​

openstack keypair create --public-key /root/.ssh/id_rke1.pub rke1-key

Create Instances​

Prepare Docker installation script​

Each of your nodes will need docker installed on them. This script will install any dependencies after the VM is created.

cat <<EOF > ./install_docker.sh
#!/bin/bash

curl https://releases.rancher.com/install-docker/20.10.sh | sh
sudo usermod -aG docker ubuntu
EOF

Create Servers​

These nodes will serve as your 3 Kubernetes cluster nodes. We'll use the --user-data flag to pass the script we created above to the nodes. This will install docker on the nodes. We'll also use the --max flag to create 3 nodes at once.

openstack server create --flavor m1.medium \
  --image="Ubuntu 20.04 (focal-amd64)" \
  --network rke1 \
  --key-name rke1-key \
  --security-group rke1 \
  --user-data ./install_docker.sh \
  --max 3 \
  rke1

Deployment node​

We'll also deploy a node that we'll use to deploy our cluster. This node will be used to run the RKE installer. It's not required, but it's a good idea to keep the deployment environment separate from the cluster. This will allow you to easily destroy the deployment environment without affecting your cluster.

openstack server create --flavor m1.medium \
  --image="Ubuntu 20.04 (focal-amd64)" \
  --network rke1 \
  --key-name rke1-key \
  --security-group rke1 \
  rke1-launcher2

Add a floating IP to the deployment node​

At this point, you should have 4 nodes running. Each node is currently only accessible from the private network. We'll need to add a floating IP to the deployment node so we can access it.

openstack floating ip create \
  --description "RKE1 Cluster - Deployment Node" \
  --project rke1 \
  External
openstack server add floating ip rke1-launcher2 <floating-ip>

Note: You'll need to replace <floating-ip> with the floating IP you created for this and the following commands.

Copy the SSH key to deployment node​

We'll need to copy the SSH key we created earlier to the deployment node. This will allow RKE to deploy Kubernetes to the 3 cluster nodes.

scp -i ~/.ssh/id_rke1  ~/.ssh/id_rke1 ubuntu@<floating-ip>:/home/ubuntu/.ssh/id_rsa

SSH to deployment node​

ssh -i ~/.ssh/id_rke1 ubuntu@<floating-ip>

Deploy Your Cluster​

Install RKE​

curl -OL https://github.com/rancher/rke/releases/download/v1.3.14/rke_linux-amd64
chmod +x rke_linux-amd64 && sudo mv rke_linux-amd64 /usr/local/bin/rke

Install Kubectl​

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Prepare your configuration file​

Create a configuration file for your RKE cluster. Substitute the address with the IP addresses of the nodes you created. Save this file as rancher-cluster.yml

Full documentation for the configuration file can be found here: Configuration Options

Example Configuration file: Example Configuration

You can view the IP addresses of your nodes by running:

openstack server list --project rke1

Configuration File​

nodes:
  - address: <node_1_IP_address>
    user: ubuntu
    role:
      - controlplane
      - worker
      - etcd
  - address: <node_2_IP_address>
    user: ubuntu
    role:
      - controlplane
      - worker
      - etcd
  - address: <node_3_IP_address>
    user: ubuntu
    role:
      - controlplane
      - worker
      - etcd

services:
  etcd:
    snapshot: true
    creation: 6h
    retention: 24h

ingress:
  provider: nginx
  options:
    use-forwarded-headers: "true"

Run RKE​

This process takes about 2 minutes. After it completes, you should have a working Kubernetes cluster.

rke up --config rancher-cluster.yml

Output:

...
INFO[0171] [ingress] Setting up nginx ingress controller
INFO[0171] [ingress] removing admission batch jobs if they exist
INFO[0171] [addons] Saving ConfigMap for addon rke-ingress-controller to Kubernetes
INFO[0171] [addons] Successfully saved ConfigMap for addon rke-ingress-controller to Kubernetes
INFO[0171] [addons] Executing deploy job rke-ingress-controller
INFO[0181] [ingress] removing default backend service and deployment if they exist
INFO[0181] [ingress] ingress controller nginx deployed successfully
INFO[0181] [addons] Setting up user addons
INFO[0181] [addons] no user addons defined
INFO[0181] Finished building Kubernetes cluster successfully

Verify Installation​

Set config as default​

mkdir ~/.kube && cp kube_config_rancher-cluster.yml ~/.kube/config

Fetch resources​

Get pods in all namespaces.

kubectl get pods -A

Output:

ubuntu@rke1-launcher:~/rancher$ kubectl get pods -A
NAMESPACE       NAME                                       READY   STATUS      RESTARTS   AGE
ingress-nginx   ingress-nginx-admission-create-9s6t2       0/1     Completed   0          3m55s
ingress-nginx   ingress-nginx-admission-patch-2brpz        0/1     Completed   0          3m55s
ingress-nginx   nginx-ingress-controller-j286h             1/1     Running     0          3m55s
ingress-nginx   nginx-ingress-controller-nm5m7             1/1     Running     0          3m55s
ingress-nginx   nginx-ingress-controller-x65pp             1/1     Running     0          3m55s
kube-system     calico-kube-controllers-74df54cbb7-49xm7   1/1     Running     0          4m25s
kube-system     canal-jkzvb                                2/2     Running     0          4m26s
kube-system     canal-mz67r                                2/2     Running
...

Credentials​

Save a copy of the following files in a secure location:

  • rancher-cluster.yml: The RKE cluster configuration file.
  • kube_config_cluster.yml: The Kubeconfig file for the cluster, this file contains credentials for full access to the cluster.
  • rancher-cluster.rkestate: The Kubernetes Cluster State file, this file contains credentials for full access to the cluster.

What's Next?​

You've now deployed a Kubernetes cluster using RKE1. You can now deploy workloads to the cluster. However, likely want to deploy an OpenStack load balancer and persistent volumes as well.

To do this, you'll need to setup OpenStack Cloud Provider resources. We're in the process of creating guides on how to do this.

You can find more information on the OpenStack Cloud Provider resources here: OpenStack Cloud Provider.