Artificial intelligence is moving from prototype to production faster than traditional security paradigms can adapt. For CISOs and platform engineering teams, the challenge is clear: you need to protect proprietary model weights, defend against novel application-layer threats like prompt injection, and enforce strict regulatory compliance—all without slowing down your AI developers.
To meet all of these security goals, you need more than just a place to run containers; you need a platform that compounds layers of security out-of-the-box.
Today, we’re sharing our blueprint for Best practices for AI workload security on Google Kubernetes Engine (GKE). This blueprint consolidates controls across multiple Google Cloud services and GKE features to help you to build a secure-by-default GKE platform that handles the realities of AI at scale.
The AI workload security blueprint for GKE identifies three critical layers of the AI stack. Here’s how Google Cloud and GKE approach security at each of these layers.
Infrastructure Layer: Hardware-Attested Execution
You can’t have a secure AI workload on an insecure cluster. The infrastructure layer is where GKE provides a security baseline that most enterprises spend years building independently.
-
Confidential Accelerators: Heavy inference workloads handle your most sensitive data. Confidential GKE Nodes extend hardware-level memory encryption and attestation capabilities to high-performance accelerators, including Confidential GPUs (e.g., NVIDIA H100) and TPUs. This protects your intellectual property from hypervisor-level compromise and infrastructure operator scraping, providing hardware-attested confidentiality.
-
Zero-Trust Networking & Identity: GKE enforces least-privilege by default. Workload Identity Federation for GKE ensures inference pods can securely fetch model weights from Cloud Storage without long-lived keys, while VPC Service Controls create a strong perimeter around regulated workloads to prevent data exfiltration.
Model Security: Provenance and Behavioral Integrity
If you are deploying your own models—whether fine-tuned or open-source—you own the safety and integrity of the weights.
GKE integrates deeply with Google Cloud’s supply chain tools to ensure what you train is exactly what you serve. Traditional SBOMs do not capture AI artifacts. GKE uses k8s-aibom (AI Bill of Materials for Kubernetes) to generate comprehensive inventories of your models, datasets, and frameworks and give you enhanced supply chain visibility.
Application Security: Defending the Inference Path
The application layer is where you have content access and where novel AI-specific threats (like prompt injection and data leakage) emerge. Google Cloud provides purpose-built services that sit directly in your GKE inference path.
-
Content-Layer Defense: Model Armor sits between your application and the inference endpoint. It inspects every prompt and response for prompt injection, sensitive data exposure (PII), and harmful content generation.
-
Session Management: The GKE Inference Gateway provides session-level observability and quota enforcement. It allows you to enforce per-user rate limits and detect abuse patterns, such as session manipulation or inference cost abuse.
-
Agentic Isolation: When your AI acts as an agent—executing generated code or interacting with unverified third-party tools—it must be contained. GKE Sandbox (gVisor) provides a secure isolation boundary that prevents container escapes and protects the underlying node from unpredictable agent behavior.
A Phased Approach to Security
Security on GKE compounds. We recommend a phased approach to securing your AI deployments:
-
Phase 1 — Deploy (Your Baseline): Implement the foundational configurations. Enable Workload Identity, deploy Model Armor in front of inference endpoints, and run sensitive workloads on Confidential GKE Nodes.
-
Phase 2 — Operate (Your Hardening): Turn your prototype into a production system. Enforce signed-image policies with Binary Authorization, tune Model Armor profiles, and aggregate audit logs for cross-layer SIEM correlation.
-
Phase 3 — Govern (Enterprise Scale): Automate compliance. Establish organization-level guardrails with Organization Policy Service, enforce admission-time policies via Kubernetes webhooks, and automate incident response for high-confidence detections.
Our AI workload security blueprint provides you with recommended controls and security measures for each of these phases. Additionally, the blueprint includes foundational guidance for observing your environment over time.
Next Steps
The race to deploy AI should not be a race to the bottom for security. By building on GKE and integrating with Google Cloud, platform teams inherit the infrastructure security baseline that Google has been refining for over a decade, paired with purpose-built AI defenses.
To dive deeper into the specific threat models, architectural patterns, and the complete maturity self-assessment, read the full Best practices for AI workload security on GKE.
