Kubernetes 1.34/1.35 Certificate Revolution: From Manual Hell to Zero-Trust Heaven

Recently upgraded to 1.35 and discovered that certificate management changes are nothing short of revolutionary—especially for self-managed K8s users, where operational overhead has been cut in half.

In the past, certificate issues were the “silent killer” of security incidents: expired certificates causing outages, token leaks, and manual rotation consuming 30% of ops time. Versions 1.34/1.35 introduce native automated mTLS, making zero trust no longer exclusive to Istio. Today, let’s dive into these new features and compare them in a self-managed K8s vs. cloud K8s hands-on scenario.

Kubernetes 1.34: Pod Certificates (Alpha → Beta)

In a nutshell: Pods automatically request “identity cards” like people do, with hour-long short-lived certificates and mTLS without sidecars.

Core Mechanism

graph LR
    A[Pod starts] --> B[kubelet generates CSR]
    B --> C[API Server signs]
    C --> D[Auto-mounts to /run/workload-spiffe-credentials]
graph LR
    A[Pod starts] --> B[kubelet generates CSR]
    B --> C[API Server signs]
    C --> D[Auto-mounts to /run/workload-spiffe-credentials]
graph LR
    A[Pod starts] --> B[kubelet generates CSR]
    B --> C[API Server signs]
    C --> D[Auto-mounts to /run/workload-spiffe-credentials]

Hands-on:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
apiVersion: apps/v1
kind: Deployment
meta
  name: qwen-72b-secure
  labels:
    app: qwen-72b
    security: zero-trust 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: qwen-72b
  template:
    meta
      labels:
        app: qwen-72b
    spec:
      serviceAccountName: llm-inference-sa
      nodeSelector:
        node.kubernetes.io/instance-type: "g5.12xlarge"
      
      containers:
      - name: vllm-server
        image: vllm/vllm-openai:v0.9.1
        
        # === Application directly consumes certificates ===
        # vLLM natively supports HTTPS, pointing directly to the auto-mounted path
        command: ["python3", "-m", "vllm.entrypoints.openai.api_server"]
        args:
          - "--model=/data/models/Qwen-72B-Int4"
          - "--tensor-parallel-size=4"
          - "--gpu-memory-utilization=0.92"
          # 🔐 Enable mTLS/HTTPS 
          # Use certificate files auto-generated by Pod Certificates
          - "--ssl-certfile=/run/workload-spiffe-credentials/tls.crt"
          - "--ssl-keyfile=/run/workload-spiffe-credentials/tls.key"
        
        ports:
        - containerPort: 8000
          name: https # Marked as HTTPS port
        
        resources:
          limits:
            nvidia.com/gpu: "4"
        
        volumeMounts:
        # === 1.35 standard mount path ===
        # Mount to SPIFFE standard location, no sidecar injection needed
        - mountPath: /run/workload-spiffe-credentials
          name: pod-identity-cert
          readOnly: true
        - mountPath: /data/models
          name: model-storage

      volumes:
      - name: model-storage
        persistentVolumeClaim:
          claimName: qwen-models-pvc
      
      # === 1.35 PodCertificate volume declaration ===
      - name: pod-identity-cert
        projected:
          sources:
          - podCertificate:
              # Key: Specify Signer (EKS/Cloud environments usually have dedicated Signers)
              # For self-managed K8s, use internal-ca or kubernetes.io/kube-apiserver-client
              signerName: "eks.amazonaws.com/pod-ca" 
              
              # Key: Custom certificate validity period (1.35 Alpha/Beta feature)
              # Force 1-hour rotation
              expirationSeconds: 3600 

1.34 Exclusive Features:

  • Kubelet server certificate auto-rotation: --rotate-certificates enabled by default, node certificates never expire.
  • Deprecation of weak TLS ciphers: Prevents POODLE attacks, enforces modern cipher suites.
  • ImagePullSecrets OIDC integration: ECR pulls with zero static tokens.

Kubernetes 1.35: Security Upgraded (Beta Stable + New Alpha)

In a nutshell: Stricter certificate validation, anti-spoofing + automated renewal become standard.

New Game-Changers

  • KubeletCertCNValidation (Alpha): API Server enforces certificate CN = node hostname, not just IP.
    • Scenario: ARP spoofing attack killer, essential for EKS multi-tenant environments.
  • PodCertificates spec.userConfig: Custom SAN, KeyUsage for more flexible enterprise CA integration.
  • kubeadm upgrade integrated renew: Automatically backs up and renews control plane certificates during upgrades.

1.35 CN Validation Verification Command

1
kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="KubeletCNValid")].status}{"\n"}{end}'

⚠️ Common Pitfalls and Limitations

Before deploying in production, be aware of:

  1. Fixed certificate lifecycle: Default TTL is 1 hour. Long-lived connections need to handle reconnection themselves.
  2. Private key never leaves the node: The private key generated by kubelet exists only in memory/temp disk; Pods cannot export it.
  3. Signer limitations: Currently mainly supports built-in cluster Signers; integrating with external PKI still requires cert-manager bridging.

Self-Managed K8s vs. Cloud K8s: Pain Points → Solution Hands-On Comparison

1. Self-Managed K8s (kubeadm/kops): From Hell to Automation

Before upgrade: 100-node cluster, 5% certificate expiration outages, 1-2 days of manual renewal per month.

After upgrade:

1
2
3
4
# Enable Feature Gates
kubeadm init --feature-gates=PodCertificates=true,KubeletCertCNValidation=true
# Upgrade with auto-renewal
kubeadm upgrade plan v1.35 --certificate-renewal=true

Self-Managed ROI Matrix:

Pain PointOld SolutionNew Solution (1.35)ROI
Manual renewalkubeadm certs monthlykubelet auto-rotateMTTR 15min → 0min
No mTLSIstio sidecarPodCertificates nativeCPU savings 10%
Token leaksNon-expiring SA tokensHourly TTL certificatesSecurity incidents down 80%

Self-Managed Migration Trap: Self-built CAs need to explicitly support the pod profile in ca-config.json.

2. Cloud K8s (EKS/AKS/GKE): Out-of-the-Box Accelerator

Before EKS: Control plane managed, but node certs still manual; Fargate has no kubelet issues.

EKS 1.35 Upgrade Hands-on (Corrected Steps):

  1. Console Operation: EKS Console → Cluster → Update → Version 1.35.
  2. Enable Features (EKS requires explicit enablement):
    1
    2
    
    # Must update API Server config to enable Alpha/Beta features
    kubectl patch cm kube-apiserver -n kube-system -p '{"data":{"featureGates":"PodCertificates=true,KubeletCertCNValidation=true"}}'
  3. Node Group Rotation:
    1
    
    aws eks update-nodegroup-version --cluster-name my-cluster --nodegroup-name gpu-nodes

Cloud Provider Support Comparison:

ProviderSetup DifficultyUnique Advantage
EKS⭐⭐Bedrock mTLS + KMS native integration
AKS⭐⭐⭐AAD seamless integration, enterprise zero-trust support
GKEWorkload Identity enhanced

Detailed Cost Comparison Table (Updated)

SolutionSoftware LicenseCPU OverheadStorage CostOps TimeTotal Cost/Year (100 nodes)
cert-manager$0 (Open Source)2-4 vCPU3GB+~120h$12,000
Istio mTLSLicense/Ent10-15%5GB+~60h$25,000+
K8s 1.35 Native$0<0.5%<100MB~10h$2,500

Migration Full Guide: Zero-Disruption Roadmap

Pre-Migration Checklist

  • CA Compatibility: Confirm Cluster CA supports issuing Client Auth certificates.
  • Node Hostname: Run openssl x509 -in /etc/kubernetes/pki/kubelet.crt -text | grep CN to ensure CN matches Hostname.
  • API Version: Scan all manifests, remove old certificates.k8s.io/v1beta1 references.

General Steps

  1. Week 1: kube-score + certificate expiration scanning.
  2. Week 2: Canary 10% node upgrade, enable Metrics monitoring.
  3. Week 3: Full Feature Gates, kubectl test Pod signing.
  4. Week 4: Disable old ServiceAccount Token mounts, switch to full mTLS.
  1. Phase 1: EKS 1.34 → Enable PodCertificates + ImagePullSecrets OIDC.
  2. Phase 2: 1.35 → Enable Kubelet CN validation + DRA GPU scheduling.
  3. Phase 3: Bedrock Agent full mTLS, configure Transit Gateway security groups.

Verification Commands:

1
2
kubectl get pods -l app=llm -o yaml | grep podCertificate
kubectl exec -it qwen-inference -- openssl x509 -in /run/workload-spiffe-credentials/tls.crt -text

Conclusion: 2026 K8s Security New Baseline

The 1.34/1.35 certificate features transform Kubernetes from a “container orchestrator” into a true AI-native infrastructure. For my EKS+Bedrock stack, PodCertificates+mTLS directly doubles the security factor of the RAG system.

Strongly Recommended: Test environment to 1.35 immediately, production blue-green follow-up.

References

  • https://kubernetes.io/blog/2025/08/27/kubernetes-v1-34-release/
  • https://kubernetes.io/blog/2025/12/17/kubernetes-v1-35-release/
  • https://github.com/kubernetes/enhancements/issues/4317
  • https://aws.github.io/aws-eks-best-practices/security/docs/

Updated on 2026-01-18 with latest 1.35 GA details.


Want updates? Subscribe via RSS


Related Content