Happy Dussehra! May good always win. Happy Diwali! Wishing you a bright year ahead. Merry Christmas! Warm wishes from Vizag. Happy New Year! Here’s to a great year ahead.

Archive · Kubernetes

Kubernetes 1.22: Removed APIs to Fix Before Upgrading

Kubernetes 1.22 removes several long-deprecated v1beta1 APIs including Ingress and CustomResourceDefinition. How to find what you use, convert manifests and Helm charts, and plan the upgrade.

3 min read

  • Kubernetes 1.22
  • kubectl-convert
  • pluto
  • Helm 3

Kubernetes 1.22 was released on 4 August 2021 and is the first release in a long time to remove many beta APIs at once. Manifests, Helm charts and operators that still use them will fail to apply on a 1.22 cluster, so the work to do is on your side, before the control plane is upgraded.

Timing. Managed services lag the upstream release: as of today the major cloud providers’ managed offerings are at 1.21 or earlier, so you have time, but charts and controllers take weeks to update. Use the time to inventory and fix.

Upgrade readiness flow

flowchart TB
  A[Inventory API usage] --> B[Scan manifests and charts<br/>pluto or kubent]
  A --> C[Check live cluster<br/>deprecated API metrics]
  B --> D[Update to stable versions]
  C --> D
  D --> E[Upgrade controllers and operators<br/>ingress cert-manager prometheus]
  E --> F[Test on a 1.22 kind cluster]
  F --> G[Upgrade staging control plane]
  G --> H[Upgrade production]

The main removals in 1.22

Removed API (version) Use instead
extensions/v1beta1 and networking.k8s.io/v1beta1 Ingress, IngressClass networking.k8s.io/v1
apiextensions.k8s.io/v1beta1 CustomResourceDefinition apiextensions.k8s.io/v1
admissionregistration.k8s.io/v1beta1 Validating/MutatingWebhookConfiguration admissionregistration.k8s.io/v1
certificates.k8s.io/v1beta1 CertificateSigningRequest certificates.k8s.io/v1
apiregistration.k8s.io/v1beta1 APIService apiregistration.k8s.io/v1
rbac.authorization.k8s.io/v1beta1 Role, ClusterRole, bindings rbac.authorization.k8s.io/v1
scheduling.k8s.io/v1beta1 PriorityClass scheduling.k8s.io/v1
storage.k8s.io/v1beta1 CSIDriver, CSINode, StorageClass, VolumeAttachment storage.k8s.io/v1
coordination.k8s.io/v1beta1 Lease; authentication and authorization v1beta1 reviews the v1 versions

Always confirm against the official Deprecated API Migration Guide for your exact versions.

Find what you use

In files and charts (before they reach a cluster):

# Fairwinds pluto: scans YAML directories and Helm releases
pluto detect-files -d k8s/ --target-versions k8s=v1.22.0
pluto detect-helm --target-versions k8s=v1.22.0
helm template ./chart | pluto detect - --target-versions k8s=v1.22.0
grep -rE "apiVersion: (extensions/v1beta1|networking.k8s.io/v1beta1|apiextensions.k8s.io/v1beta1|rbac.authorization.k8s.io/v1beta1)" k8s/

In a running cluster:

kubectl get --raw /metrics | grep apiserver_requested_deprecated_apis       # which deprecated APIs have clients called
kubectl get ingress -A -o jsonpath='{range .items[*]}{.apiVersion}{"  "}{.metadata.namespace}/{.metadata.name}{"\n"}{end}'

The API server serves each object in several versions, so kubectl get ingress already shows the new version even if your file still uses the old one. The risky usages are in files, Helm charts and operators that submit old versions.

Convert manifests

Ingress changed in more than the version string:

# old (extensions/v1beta1 or networking.k8s.io/v1beta1)
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations: { kubernetes.io/ingress.class: nginx }
spec:
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /
            backend: { serviceName: web, servicePort: 80 }
# new (networking.k8s.io/v1)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
spec:
  ingressClassName: nginx
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend: { service: { name: web, port: { number: 80 } } }

Differences: pathType is required; the backend becomes a nested service object; the class moves from an annotation to spec.ingressClassName. For other kinds, the kubectl-convert plugin (downloadable alongside kubectl) rewrites manifests: kubectl convert -f old.yaml --output-version networking.k8s.io/v1. Review the output; conversion cannot guess pathType choices.

CRDs moving to v1 require schema (structural) with served/storage per version, and preserveUnknownFields: false, so third-party CRDs are best fixed by upgrading the operator or chart that installs them.

Helm and controllers

  • Upgrade charts whose templates use removed versions (ingress-nginx, cert-manager, Prometheus operator, Argo CD, Istio and others have fixed releases; check each project’s compatibility table).
  • For releases already deployed with old API versions, install the helm mapkubeapis plugin so Helm’s stored release manifests are updated: helm mapkubeapis my-release -n my-namespace.
  • Keep controllers one step ahead of the cluster: upgrade ingress and cert-manager on 1.21 first, verify, then upgrade the control plane.

Plan

  1. Inventory with the commands above; list owners for each finding.
  2. Fix Git, charts and CI templates; add pluto to CI so removed APIs cannot come back.
  3. Rehearse on a disposable cluster (kind with a 1.22 node image) and apply every manifest with kubectl apply --dry-run=server.
  4. Upgrade staging, run smoke tests, then production, one minor version at a time (1.21 to 1.22), control plane first, then nodes.

Also remember that Docker-as-runtime (dockershim) removal is still planned for a later release; check your node runtimes while you are at it.