Description

This rule checks that surge upgrades are enabled on DOKS node pools. With surge upgrades, DigitalOcean provisions additional temporary (duplicate) nodes during an upgrade, drains and reschedules workloads onto them, and only then removes the old nodes, minimizing disruption. It inspects the cluster's surge_upgrade setting.

Without surge upgrades, nodes are replaced more conservatively and workloads are more likely to experience capacity shortfalls or eviction pressure during routine version upgrades. Surge upgrades require enough Droplet quota to stand up the extra nodes (the limit must accommodate up to ten additional nodes).

  • Clusters with surge_upgrade disabled.

  • GPU node pools on contract-only Droplets where surge is off by default and may need a capacity plan.

  • Clusters near their Droplet quota where surge cannot allocate the extra nodes.

Rationale

Upgrading a node without surge capacity means old nodes must be drained before replacements exist, which can leave pods Pending and cause an eviction storm on the remaining nodes. Surge upgrades invert that order by creating upgraded nodes first, so workloads move onto healthy capacity before anything is torn down, keeping services available through the upgrade.

This is the mechanism that makes routine, frequent Kubernetes patching safe to run. Combined with PodDisruptionBudgets, surge upgrades let the platform respect application availability guarantees while it rolls nodes.

  • Old nodes are drained onto new nodes, not into thin air.

  • Reduces the blast radius of every version upgrade.

Impact

Pros:

  • Near-zero-downtime node replacement during upgrades.

  • Fewer Pending pods and eviction storms during maintenance.

  • Makes frequent patching operationally routine rather than risky.

Cons:

  • Temporary extra nodes incur brief additional cost during the upgrade.

  • Requires Droplet quota headroom (limit must allow up to ~10 extra nodes).

  • Benefits are only fully realized if workloads also define PodDisruptionBudgets.

Default Value

Surge upgrades are enabled by default for new DOKS clusters, but are disabled by default for GPU node pools that use contract-only Droplets.

Pre-Requisite

  • DigitalOcean Control Panel access to the cluster's Settings, or an API token / authenticated doctl.

  • Droplet quota headroom of at least the current node count plus up to ten nodes.

  • PodDisruptionBudgets defined on critical workloads so surge capacity is used effectively.

Remediation

Test Plan

Using the DigitalOcean Control Panel:

  1. Sign in to cloud.digitalocean.com and open Kubernetes.

  2. Select the cluster and open the Settings tab.

  3. Locate the surge upgrade setting and confirm whether it is enabled.

  4. Pass: surge upgrades are enabled for the cluster's node pools.

  5. Fail: surge upgrades are disabled (excluding GPU contract-only pools evaluated against your capacity plan).

Using doctl or the API:

Read the surge_upgrade flag from the cluster object. (doctl's --format has no SurgeUpgrade column, so read the full JSON with -o json.)

doctl kubernetes cluster get <cluster-id> -o json | jq '.[0].surge_upgrade'
curl -s -H "Authorization: Bearer $DIGITALOCEAN_TOKEN" "https://api.digitalocean.com/v2/kubernetes/clusters/$CLUSTER_ID" | jq '.kubernetes_cluster.surge_upgrade'

Pass if surge_upgrade = true. Fail if surge_upgrade = false.

Implementation Steps

Using the DigitalOcean Control Panel:

  1. Open the cluster's Settings tab in the Control Panel.

  2. Enable the surge upgrade option for the cluster.

  3. Save the change.

  4. Confirm your Droplet limit can accommodate the additional upgrade nodes.

Using doctl or the API:

Enable surge upgrades with doctl or by updating the cluster.

doctl kubernetes cluster update <cluster-id> --surge-upgrade=true
curl -s -X PUT -H "Authorization: Bearer $DIGITALOCEAN_TOKEN" -H 'Content-Type: application/json' -d '{"surge_upgrade":true}' "https://api.digitalocean.com/v2/kubernetes/clusters/$CLUSTER_ID"

Backout Plan:

  1. Open the cluster Settings (or run the update command again).

  2. Set surge upgrades to disabled: doctl kubernetes cluster update <cluster-id> --surge-upgrade=false.

  3. Save. Note that disabling surge increases upgrade disruption risk.

References:

  1. DOKS - Upgrade a cluster (surge upgrades)

  2. DOKS - Features

  3. doctl kubernetes cluster reference