With many moving to Proxmox in their home lab or even production environments, you may be interested in your options for running Kubernetes. How does this work with Proxmox? Well, there are several ways you can configure your deployment of Kubernetes running in Proxmox. Let’s take a look at Proxmox Kubernetes configurations you need to know and see the architecture of each.
Proxmox is a great platform for K8s
Since so many are looking at Proxmox as their hypervisor of choice, those who want to experiment or run Kubernetes also want to take a look at the best methods to run Kubernetes in Proxmox. There are a number of ways that you can do this.
You have a lot of options with Proxmox since it has the built-in LXC container functionality in the platform and you can also take the traditional virtual machine route as well, along with running Kubernetes using K3D if you want in containers. So, lots of methods. I encourage you to try out each of the various configurations and see which
Proxmox Kubernetes configurations
We will consider the following Proxmox Kubernetes configurations that provide great options for home lab or production environments:
- Traditional Kubernetes virtual machine nodes
- Talos Linux Kubernetes
- Kubernetes in LXC containers
- K3D
- Run on Proxmox bare metal
1. Traditional Kubernetes virtual machine nodes
This is the most basic type of deployment where you simply spin up 3 traditional Linux virtual machines and run your choice of Kubernetes distro inside the virtual machines. My choice lately for doing this is running Ubuntu Server virtual machines with Microk8s installed.
Microk8s is a production-ready Kubernetes distribution that is easy to install and administer due to the fact that all the commands are baked into the snap package and you can easily enable or disable components in your Microk8s cluster.
Microk8s also works very well with Microceph and integrates your Kubernetes storage management with Microceph using the rook-ceph components for Microk8s.
Pros
- Very easy and traditional way to configure K8s
- Backups of your Kubernetes nodes are easy since they are traditional VMs
Cons
- Takes more resources
- Management of virtual machine lifecycle, updates, etc
2. Talos Linux Kubernetes
There is another Kubernetes option that is massively gaining popularity and that is Talos Linux Kubernetes. Talos Linux is a very streamlined distribution that is very security focused. You can’t SSH into your Talos Linux hosts and everything is API-driven.
Talos Linux Kubernetes has a bootstrapping process to get up and running where you deploy the VMs and then bootstrap the cluster and bring it online. If you want to know more details on how to set this up in Proxmox, read my detailed blog post here on how to install Proxmox Kubernetes with Talos Linux.
Pros
- Very secure
- API-driven
- No SSH access
Cons
- Steeper learning curve
- May have challenges compared to other Kubernetes distributions
3. Kubernetes in LXC containers
There is another great option, especially for home labs to run Kubernetes in Proxmox, especially for home lab environments. Proxmox natively supports LXC containers. We can spin these up easily in Proxmox in the web GUI or from the command line. LXC containers bridge the gap between a full-blown virtual machine and a Docker container. It fits somewhere in between.
It is a more operating system-like container than Docker which is more application-centric. These are still much more lightweight than a full VM and you can run Kubernetes inside of your LXC containers. This provides a very lightweight and streamlined environment where you can run Kubernetes.
Pros
- Very lightweight
- Easy for lab environments
Cons
- May not be the best for production environments
- Can have limitations
4. K3D
Another option to run Proxmox Kubernetes clusters is using the K3D project. What is K3D? K3D is a utility that serves as a lightweight tool that lets you run your K3s Kubernetes environment inside Docker containers.
You can spin up either a single node cluster or multi-node cluster using the K3s distribution inside Docker. This is a great option for home labs that provides a very lightweight way to run your Kubernetes clusters, especially for labbing and learning scenarios. There are also a lot of community scripts out there that use K3D to automate spinning up a Kubernetes cluster.
It is free and open-source and is not an official Rancher solution. You can learn more about K3D from the official site here: k3d.
Pros
- Easy to spin up
- Great for lab environments
- Lightweight
Cons
- Probably not the best for production
- Steeper learning curve
5. Run on Proxmox bare metal
This is an option I think if you want to run your Kubernetes cluster on bare metal and have a dual purpose setup where you also want to run VMs and containers. Since Proxmox is based on Debian, you can simply treat your Proxmox host like a Kubernetes bare metal node and install your Kubernetes distro directly on the hosts.
I think this is the least desirable of the 5 configurations though as using virtualization technologies as the underpinning for your Kubernetes cluster opens up so many additional possibilities, IMHO. However, just know this is a thing and an option if you want to try it out.
Pros
- You have the performance and performance of bare metal
- You have the platform host ready to go by default
- Ok for lab environments
Cons
- Not for production
- Mixes your physical host environment with your virtualization host
Wrapping up
Hopefully this blog post gives you a few ideas of Proxmox Kubernetes configurations you can try in your home lab or even production environment to find the best solution that fits your needs. Each method has its own pros and cons, so keep this in mind when choosing. Above all, make full use of a lab environment before going into production. This helps to flesh out any issues or limitations with a specific configuration before these are found in production.
0 Comments