Untrusted code belongs in a VM, not a namespace

A container feels like a box. It isn't. It's a normal process on the host, wrapped in namespaces and cgroups so it can't see most of the system - but it's still running on the host's kernel, sharing it with every other container and with the node itself. Namespaces are an isolation convenience. They are not a security boundary against code that is actively trying to get out.

For most workloads that's a fine trade. For code you genuinely don't trust - say, an AI agent executing arbitrary tasks you didn't write - it isn't. One kernel bug, one container escape, and "isolated workload" becomes "process on your node with your node's privileges." The boundary you were relying on was never really there.

Give it its own kernel

Kata Containers change the shape of the boundary. Instead of running the pod as a process on the shared kernel, Kata boots each pod inside its own lightweight virtual machine - a microVM with its own guest kernel, behind the hardware hypervisor (KVM). Now an escape doesn't land you on the host; it lands you inside a throwaway VM, with a real CPU-enforced boundary (VT-x) between you and everything else.

The lovely part is that nothing about the operator experience changes. It's still a pod. You still kubectl apply it, it still gets scheduled, mounts volumes, shows up in logs - you just set one field (runtimeClassName: kata) and the container runtime quietly boots a VM instead of a namespace. The ergonomics of Kubernetes, the isolation of a hypervisor.

How you know it's real

It's easy to claim hardware isolation; it's worth proving. The cleanest proof is the kernel itself: exec into the pod and check the kernel version. If it's a different kernel from the host, it is - definitionally - a different kernel, which means a real VM with its own guest, not a namespace dressed up as one. (On my setup the guest reports one kernel and the Talos host another; that mismatch is the whole proof.) It needs hardware virtualisation enabled (VT-x / KVM) - without it, Kata can't boot the guest and fails honestly rather than silently downgrading.

The cost is real, so design for it

A VM per pod isn't free. Each one carries a fixed memory overhead for the guest kernel and a cold-start measured in seconds, not milliseconds. That changes how you plan:

• It's a throughput tier, not a low-latency one. Seconds-to-start means batch-shaped work, not request-per-second serving. Fine for "run this task"; wrong for "answer this instantly."
• Capacity is designed, not discovered. Because the untrusted tier is, by definition, the part most likely to misbehave, it gets a hard resource quota - a ceiling on memory and concurrent pods - so a runaway (or hostile) sandbox can never starve the control plane. The scheduler also has to know about the per-VM overhead, or it will quietly over-pack the node and OOM under load. You bound the blast radius with policy, not hope.

The honest caveat

Kata, plus a default-deny network around it, is a superb isolation floor: it contains an escape and it blocks the pod from talking to things it shouldn't. But notice what it does not do. If the untrusted code is handed a legitimate tool - a credential, an API it's allowed to call - hardware isolation won't stop it from misusing that tool. The VM contains a break-out; it does nothing about exfiltration through a sanctioned door. That's a different problem, and it needs a different answer - a broker that holds the keys the agent never sees.

Running on a single-node Talos cluster as the untrusted-execution tier of an AI-agent platform: Kata microVMs + a default-deny floor + a designed capacity ceiling, verified guest-kernel-up.