Tutorials & Guides

K3s + Helm + Kyverno Quickstart

Install K3s

curl -sfL https://get.k3s.io | sh -

sudo systemctl restart k3s
sudo systemctl status k3s

mkdir -p ~/.kube
sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config
sudo chown $USER:$USER ~/.kube/config
export KUBECONFIG=~/.kube/config
echo 'export KUBECONFIG=~/.kube/config' >> ~/.bashrc

Install Helm

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

Install Kyverno

helm repo add kyverno https://kyverno.github.io/kyverno/
helm repo update
helm install kyverno kyverno/kyverno -n kyverno --create-namespace --wait

Install Policy Reporter

helm repo add policy-reporter https://kyverno.github.io/policy-reporter
helm repo update

helm install policy-reporter policy-reporter/policy-reporter \
  --create-namespace -n policy-reporter \
  --set ui.enabled=true \
  --set kyverno-plugin.enabled=true

Access the UI

kubectl port-forward service/policy-reporter-ui 8082:8080 -n policy-reporter

ssh -L 8082:localhost:8082 sama@[this-vm-ip]

http://localhost:8082/

(Optional) Install the Pod Security Standard policy library

helm install kyverno-policies kyverno/kyverno-policies -n kyverno
kubectl get cpol

Install workloads

1. A basic compliant workload

Create file nginx-compliant.yaml with the following:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-compliant
  namespace: demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-compliant
  template:
    metadata:
      labels:
        app: nginx-compliant
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          resources:
            limits:
              cpu: \"200m\"
              memory: \"256Mi\"
            requests:
              cpu: \"100m\"
              memory: \"128Mi\"

Apply it:
kubectl create namespace demo
kubectl apply -f nginx-compliant.yaml

2. A non-compliant workload (no resource limits)

Create file redis-noncompliant.yaml with the following:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-noncompliant
  namespace: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis-noncompliant
  template:
    metadata:
      labels:
        app: redis-noncompliant
    spec:
      containers:
        - name: redis
          image: redis:7

Apply it:
kubectl apply -f redis-noncompliant.yaml

Check Kyverno policies

kubectl get cpol

Example output:
NAME  ADMISSION  BACKGROUND  READY  AGE  MESSAGE
disallow-capabilities  true  true  True  15m  Ready
...
restrict-sysctls  true  true  True  15m  Ready

More workloads

kubectl create deployment busybox --image=busybox --namespace=demo -- sleep 3600
kubectl run standalone-pod --image=alpine --namespace=demo -- sleep 3600

Every container image reference has this shape:
[registry]/[namespace]/[image]:[tag]

If you omit parts, defaults kick in silently:
No registry -> docker.io (Docker Hub)
No namespace -> library (official images)
No tag -> :latest

Equivalent explicit forms:
kubectl create deployment busybox --image=docker.io/library/busybox:latest --namespace=demo -- sleep 3600
kubectl run standalone-pod --image=docker.io/library/alpine:latest --namespace=demo -- sleep 3600

Check pod image path:
kubectl get pod standalone-pod -n demo -o jsonpath='{.status.containerStatuses[0].image}'

Check Kyverno registry endpoint:
curl -I https://reg.kyverno.io/v2/

Keycloak deployment (dev)

kubectl create namespace keycloak

Create file keycloak.yaml with the following (Deployment + Service):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keycloak
  namespace: keycloak
spec:
  replicas: 1
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
    spec:
      containers:
        - name: keycloak
          image: quay.io/keycloak/keycloak:26.0
          args: ["start-dev"]
          env:
          - name: KEYCLOAK_ADMIN
            value: "admin"
          - name: KEYCLOAK_ADMIN_PASSWORD
            value: "admin"
          ports:
          - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: keycloak
  namespace: keycloak
spec:
  type: NodePort
  selector:
    app: keycloak
  ports:
    - port: 8080
      targetPort: 8080

Apply and verify:
kubectl apply -f keycloak.yaml
kubectl -n keycloak get pods
kubectl -n keycloak get svc keycloak

Get this VM's IP:
ip a | grep "inet " | grep -v 127.0.0.1

Port-forward to access admin console:
kubectl port-forward -n keycloak service/keycloak 8080:8080

Alternative mapping:
kubectl port-forward -n keycloak service/keycloak 8080:8085

Or SSH tunnel:
ssh -L 8085:localhost:8085 sama@<vm-ip>

Troubleshooting pods crashing

Check node resource pressure
kubectl describe node sama | grep -A 5 "Conditions:"
kubectl top node

Disk space
df -h /

sudo du -sh /var/lib/rancher/* 2>/dev/null | sort -rh
sudo du -sh /var/log/* 2>/dev/null | sort -rh | head -10
sudo journalctl --disk-usage

sudo du -sh /var/lib/rancher/k3s/* 2>/dev/null | sort -rh
sudo du -sh /var/lib/rancher/k3s/agent/* 2>/dev/null | sort -rh

Logs
kubectl -n keycloak logs deployment/keycloak

Configure Keycloak

ACCESS KEYCLOAK GUI:
http://localhost:8085/

CONFIGURE KEYCLOAK

1. Create the realm
Top-left realm dropdown (currently shows "master")
Create realm
Name: policy-reporter
Create

2. Create the client

Make sure the policy-reporter realm is selected (check the dropdown), then:

Clients -> Create client

General settings:
Client type: OpenID Connect
Client ID: policy-reporter
Next

Capability config:
Client authentication: ON
Leave Authorization off, Standard flow checked
Next

Login settings:
Valid redirect URIs: http://localhost:8082/callback
Web origins: http://localhost:8082
Save

3. Get the client secret
Credentials tab on the client -> copy the secret

1. Create the user
In the Keycloak admin console (make sure you're in the policy-reporter realm):

Left sidebar -> Users
Add user
Fill in:
Username: e.g. testuser
Email: optional, e.g. testuser@example.com
First name / Last name: optional
Email verified: toggle ON (avoids Keycloak prompting for email verification on first login)
Create

2. Set the user's password
You'll land on the user's detail page after creation — go to the Credentials tab
Set password
Enter a password
Temporary: toggle this OFF
Save, confirm

4. Get the NodePort and VM IP
kubectl -n keycloak get svc keycloak
ip a | grep "inet " | grep -v 127.0.0.1

1. Create the Kubernetes secret
Once you have the client secret from Keycloak's Credentials tab, run (replace <client-secret>, <vm-ip>, and <node-port>):

kubectl create secret generic policy-reporter-oidc-secret \
  -n policy-reporter \
  --from-literal=clientId='policy-reporter' \
  --from-literal=clientSecret='<client-secret>' \
  --from-literal=discoveryUrl='http://<vm-ip>:<node-port>/realms/policy-reporter/.well-known/openid-configuration'

If you already created this secret on a previous attempt, delete it first:
kubectl -n policy-reporter delete secret policy-reporter-oidc-secret --ignore-not-found

2. Verify the discovery URL works before touching Policy Reporter
Test it directly from your laptop's browser or via curl on the VM:

curl -s http://<vm-ip>:<node-port>/realms/policy-reporter/.well-known/openid-configuration | head -c 300

Should return JSON starting with
{"issuer":"http://<vm-ip>:<node-port>/realms/policy-reporter", ...}

3. Create/update the Helm values file
cat <<EOF > ~/policy-reporter-values.yaml
ui:
  enabled: true
  openIDConnect:
    enabled: true
    callbackUrl: "http://localhost:8082/callback"
    secretRef: "policy-reporter-oidc-secret"
    scopes:
      - openid
      - profile
      - email
EOF

4. Apply via Helm upgrade
helm upgrade policy-reporter policy-reporter/policy-reporter \
  -n policy-reporter \
  -f ~/policy-reporter-values.yaml \
  --reuse-values

5. Watch the rollout
kubectl -n policy-reporter rollout status deployment/policy-reporter-ui
kubectl -n policy-reporter logs deployment/policy-reporter-ui --tail=50

6. Test the full login flow
On the VM: kubectl port-forward service/policy-reporter-ui 8082:8080 -n policy-reporter
On your laptop: ssh -L 8082:localhost:8082 sama@<vm-ip>
Browse to http://localhost:8082/

Kyverno post-install notes & scaling

- Admission controller replicas < 2 are not HA.
- PolicyExceptions are disabled by default; enable via --enablePolicyException=true if needed.

Scale admission controller to 3 replicas:
kubectl scale deployment kyverno-admission-controller -n kyverno --replicas=3

K3s troubleshooting quick checks

View recent K3s logs:
sudo journalctl -u k3s -n 100 --no-pager

Check API/server ports (10250, 10248):
sudo ss -tlnp | grep -E '10250|10248'

Terminate a stuck process by PID:
sudo kill -9 (PID)

Remove Kyverno (cleanup)

helm uninstall kyverno -n kyverno
kubectl delete namespace kyverno

Sample app: 2048 game (with Traefik IngressRoute)

Create 2048.yaml with the following:

apiVersion: v1
kind: Namespace
metadata:
  name: game-2048
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: game-2048
  namespace: game-2048
  labels:
    app: game-2048
spec:
  replicas: 2
  selector:
    matchLabels:
      app: game-2048
  template:
    metadata:
      labels:
        app: game-2048
    spec:
      containers:
        - name: game-2048
          image: alexwhen/docker-2048
          ports:
          - containerPort: 80
          resources:
            requests:
              cpu: "50m"
              memory: "64Mi"
            limits:
              cpu: "200m"
              memory: "128Mi"
---
apiVersion: v1
kind: Service
metadata:
  name: game-2048
  namespace: game-2048
spec:
  selector:
    app: game-2048
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP
---
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
  name: game-2048
  namespace: game-2048
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`2048.local`)
      kind: Rule
      services:
      - name: game-2048
        port: 80

Apply and watch:
kubectl apply -f 2048.yaml
kubectl get pods -n game-2048 -w

Optional: pre-pull and roll out alternate image:
sudo k3s crictl pull marcincuber/2048-game:latest
kubectl set image deployment/game-2048 game-2048=marcincuber/2048-game:latest -n game-2048

Update the YAML to pin the new image:
sed -i 's|alexwhen/docker-2048|marcincuber/2048-game:latest|' 2048.yaml
grep image 2048.yaml

Expand disk on Ubuntu (LVM)

Fix GPT and rescan:
sudo sgdisk -e /dev/sda
sudo partprobe /dev/sda

Grow partition (sda3):
sudo apt install cloud-guest-utils -y || true
sudo growpart /dev/sda 3

Resize LVM and filesystem:
sudo pvresize /dev/sda3
sudo lvextend -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv
sudo resize2fs /dev/mapper/ubuntu--vg-ubuntu--lv

Verify:
df -h /