Goal: reproduce a cloud-style platform (load balancer, ingress, managed-database feel, monitoring) on servers you own, for an application that cannot or should not run in a public cloud.
Versions in use on this date. MicroK8s 1.32, which carries Kubernetes 1.32, was announced on 16 December 2024 and is the newest stable track on 7 April 2025. The nodes run Ubuntu 24.04 LTS. MicroK8s runs containers with containerd, so Docker Engine 28 (released February 2025) is needed only on the build or CI machine, not on the cluster nodes.
Target architecture
flowchart TB
U[Users and office network] --> FW[Firewall and internal DNS]
FW --> VIP[MetalLB virtual IP]
VIP --> ING[Ingress controller<br/>TLS termination]
subgraph CL[MicroK8s cluster of three nodes]
ING --> APP[apps namespace<br/>services and workers]
APP --> DATA[data namespace<br/>MongoDB MySQL PostgreSQL Redis]
APP --> CE[cloud-emulation namespace<br/>LocalStack]
DATA --> ST[Storage layer<br/>hostpath or replicated]
end
ZBX[Zabbix server<br/>separate VM] -.->|agent 2 and API checks| CL
DATA -->|dumps and snapshots| BK[Backup target<br/>NAS or object storage]
| Layer | Choice | Reason |
|---|---|---|
| Nodes | Three servers, each 8 vCPU, 32 GB RAM, 500 GB NVMe (example sizing) | Three voters give a highly available control plane that survives one node failure |
| Datastore | Built-in dqlite, clustered across the three nodes | MicroK8s uses dqlite instead of etcd, and forms an HA control plane when three or more nodes join |
| Networking | Calico (the MicroK8s default CNI) | Supports network policies |
| Load balancer | MetalLB addon with a reserved address range | On-premises clusters have no cloud load balancer, so LoadBalancer services need MetalLB |
| Ingress | MicroK8s ingress addon (NGINX-based) |
Terminates TLS and routes by host name |
| Storage | hostpath-storage to start; a replicated option for important data |
Hostpath volumes live on one node and are not highly available |
Prepare each node
Static IPs and DNS names first, then, on every Ubuntu 24.04 node:
sudo apt update && sudo apt full-upgrade -y
sudo swapoff -a && sudo sed -i '/ swap / s/^/#/' /etc/fstab
sudo timedatectl set-ntp true
sudo snap install microk8s --classic --channel=1.32/stable
sudo usermod -aG microk8s $USER && newgrp microk8s
microk8s status --wait-readyOpen the cluster ports between nodes only (the MicroK8s documentation lists them: API server 16443, kubelet 10250, cluster agent 25000, dqlite 19001, and others), and publish only 80 and 443 on the ingress addresses.
Form the cluster
On the first node, generate a join token; on each of the others, run the join command it prints:
# node 1
microk8s add-node
# node 2 and node 3 (use a fresh token for each)
microk8s join 10.20.0.11:25000/<token>
# check
microk8s kubectl get nodes
microk8s status # high-availability: yes once three nodes are inThen enable the addons:
microk8s enable dns hostpath-storage metrics-server
microk8s enable metallb:10.20.0.200-10.20.0.220
microk8s enable ingress
microk8s enable cert-managerStorage decision
hostpath-storage is fast and simple, but a pod using it is tied to one node, and a node failure means that data is unavailable until the node returns. For databases that need to survive a node loss, use one of:
| Option | Use when |
|---|---|
| Database-level replication (primary plus replicas on different nodes, each on hostpath) | The database software handles replication; the simplest dependable option |
| Longhorn | You want replicated block volumes managed by Kubernetes |
| Rook-Ceph (a MicroK8s addon exists) | You have at least three nodes with spare raw disks and operational capacity for Ceph |
Write down what is not highly available, so nobody discovers it during an outage.
Namespaces, quotas and network policy
Separate apps, data, cloud-emulation and monitoring, each with limits, and deny traffic by default:
apiVersion: v1
kind: Namespace
metadata:
name: data
labels: { pod-security.kubernetes.io/enforce: baseline }
---
apiVersion: v1
kind: ResourceQuota
metadata: { name: data-quota, namespace: data }
spec:
hard: { requests.cpu: "8", requests.memory: 24Gi, persistentvolumeclaims: "20" }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: default-deny-ingress, namespace: data }
spec:
podSelector: {}
policyTypes: [Ingress]
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata: { name: allow-from-apps, namespace: data }
spec:
podSelector: {}
policyTypes: [Ingress]
ingress:
- from:
- namespaceSelector:
matchLabels: { kubernetes.io/metadata.name: apps }Decisions to write down first
- IP plan: node addresses, the MetalLB range, DNS names, and pod and service CIDRs that do not clash with the office network.
- Disk layout: fast SSD for databases, separate volume for logs.
- Backups: target, schedule, retention and a tested restore. MicroK8s can back up its datastore with
microk8s dbctl backup, but application data needs its own dumps. - Ownership: who renews certificates, patches nodes (
snap refreshchannel policy) and answers alerts. - Upgrades: rolling, one node at a time, with the cluster checked between nodes.