The Cloud Native Computing Foundation published guidance this week making a specific, important argument for anyone deploying LLM workloads on Kubernetes: the infrastructure primitives K8s provides (routing, network isolation, pod-level RBAC, secrets management) are necessary but not sufficient for LLM security. The underlying observation is that an LLM is not a stateless compute workload. It is a system that accepts untrusted input and decides what to do with it, including whether to invoke tools, emit sensitive data, or follow instructions that appear inside its prompt. That threat model is different from "containerize this microservice and isolate it," and it needs controls that live at the application and prompt layer, not the cluster layer.
CNCF's specific list of what Kubernetes cannot do is useful. Kubernetes cannot determine whether a prompt should be allowed through to the model, whether a response contains sensitive data that should be redacted before it reaches a user, or whether the model should have access to a particular tool for a particular request. The infrastructure layer handles routing and pod isolation. The application layer has to handle authentication, authorization, input validation, output filtering, tool-access decisions, and audit logging of what the model actually did. The OWASP Top 10 for LLM Applications is the best current checklist of the failure modes that application-layer controls need to address: prompt injection, sensitive information disclosure, supply chain attacks, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption. If your LLM deployment does not have explicit controls for each of these items, the fact that it runs on a secure Kubernetes cluster does not make it secure.
This fits the larger 2026 pattern running through pieces covered here all week: the governance layer for agent and LLM workloads is becoming a first-class category separate from the model layer and separate from the infrastructure layer. AWS Agent Registry ships the enterprise-catalog-and-audit version of this; CNCF is publishing the open-standards version for anyone running Kubernetes-based LLM infrastructure. The Kubernetes AI Conformance program is expanding later in 2026 to include Sovereign AI standards, which ties directly to the UK Sovereign AI Fund and the broader sovereignty pattern across the industry. For builders who want to deploy LLMs in regulated or self-hosted environments, the open-source K8s-plus-AI-conformance path is a real option that lets you sidestep hyperscaler lock-in while getting real security structure. For hyperscaler-committed enterprises, AWS Agent Registry with Bedrock AgentCore is the managed version. The two stacks will compete for the same enterprise procurement dollars.
For teams running LLM workloads on Kubernetes, three moves follow. First, audit your current deployment against the OWASP Top 10 for LLM Applications; if you do not have named controls for each of the ten items, start with the ones that cover input validation, output filtering, and tool-access authorization, because those close the largest attack surface. Second, separate infrastructure security posture from application-layer security posture in your internal documentation; a compliance audit that says "our K8s cluster is CIS-benchmarked" is not the same as "our LLM deployment is production-safe," and the two belong to different teams with different skill sets. Third, watch the CNCF AI Conformance program through the rest of 2026; the Sovereign AI standards coming later this year will probably become the de-facto reference architecture for regulated-industry LLM deployments, and being early on that spec is worth more than catching up after it ships. For teams on managed hyperscaler stacks, the same principles apply with different product names: Bedrock, Vertex AI, and Azure AI Foundry all have equivalent application-layer control surfaces that need to be configured, not assumed.