- remediation
- CVE
- base images
CVE patching in Kubernetes: fix versions, base images, and what "fixed" means
A CVE in a Kubernetes pod is fixed by a new image, not apt-get. How distro backports change “fixed in”, what “no fix” and “won’t fix” mean, and the order to patch in.
StackRadar Team
10 min read
On a server you patch a CVE with apt-get upgrade. In Kubernetes that does not stick: anything you change inside a running container is gone at the next restart. The fix for a CVE in a pod is always a new image. If you build the image, that means rebuilding it on a newer base or bumping a dependency. If someone else builds it, you bump the chart or the image tag. Then you roll it out and confirm that the digest the kubelet runs actually changed.
The hard parts come after that: the "fixed in" version depends on who packaged the software, "no fix available" covers four different situations, and the order you work in decides whether the list gets shorter. This post covers each one, and ends with a playbook.
The unit of fix is the image, deployed by something
A finding lives in an image: a specific digest that contains a specific package version. That image runs because something deployed it, such as a Helm release pinning a chart version, an Argo CD application pointing at a Git path, or a Deployment someone applied by hand. So every fix has two halves, and scanners tend to forget the second:
- Produce an image without the vulnerable package version. Rebuild your own on a newer base. For someone else's, find the tag that has the fix.
- Change the thing that deploys it (a chart bump, a values change, a Git commit) so the new digest is what the kubelet runs.
| Where the vulnerable package is | Half one: the new image | Half two: the rollout |
|---|---|---|
| A dependency your build added (npm, PyPI, Go, Maven) | Bump the dependency, rebuild | New tag in values or Git |
| The OS layer of an image you build | Rebuild on a newer base image | New tag in values or Git |
| A third-party image | Wait for the maintainer's fixed release | Chart upgrade or image-tag override |
Many of the images in a typical cluster are third-party: ingress controllers, cert-manager, monitoring agents, CSI drivers, the service mesh. For those, a list of CVEs is not actionable, but a chart version is. That is why StackRadar attributes every finding to the Helm release or Argo CD application that deployed it. The guidance on a release page reads "upgrade cert-manager from v1.15.3 to v1.16.2", not "libcurl in 14 images".
What "fixed in" means: backports change the version numbers
An advisory says a vulnerability is fixed in some version. Which version depends on the ecosystem, and comparing across ecosystems is the most common way to get the wrong answer.
| Ecosystem | "Fixed in" looks like | Meaning |
|---|---|---|
| Upstream / language (npm, PyPI, Go) | 4.19.2 → 4.20.0 | A new upstream release. OSV records exact ranges. |
| Debian / Ubuntu | 3.0.11-1~deb12u2 → 3.0.15-1~deb12u1 | Backported into the stable branch; upstream may be at 3.3 |
| Red Hat / UBI | 1.1.1k-7.el8_6 → 1.1.1k-12.el8_9 | Same upstream version, fix in the release suffix |
| Alpine | 3.3.2-r0 → 3.3.2-r1 | Package revision bump, tracked in the aports secfixes block |
| Go module in a binary | v0.24.0 → v0.25.0 | Whoever built the binary has to rebuild it |
Here is what goes wrong. A scanner that compares your Debian openssl 3.0.15-1~deb12u1 against upstream's "fixed in 3.3.2" reports a vulnerability that Debian fixed months ago:
Good matchers check distro packages against the distro's own security data. That is why the package URL carries the distro (pkg:deb/debian/openssl@3.0.15-1~deb12u1?distro=debian-12), and why version-range evaluation has to understand Debian's epoch-and-tilde ordering, RPM's release field and Alpine's -rN. When two scanners disagree about a distro package, this is nearly always the reason.
"No fix available" and "won't fix"
Many findings on a Debian- or Ubuntu-based image have no fixed version. Four different situations hide behind that, and each needs a different response:
- Not yet. The advisory is new and the distro has not shipped a backport (Debian shows it as unfixed, Ubuntu as
neededorneeds-triage). Serious issues are often fixed within days, while low-priority ones can wait much longer. Track it, and mitigate in the meantime if it is being exploited. - Deliberately not. The distro has decided the issue does not justify an update. In Debian it is
<no-dsa>, typically with the note "Minor issue". That means no advisory, though a fix may still arrive in a point release, and the sub-states<postponed>and<ignored>tell you which way it is leaning. Ubuntu marks such issuesdeferredorignored, often with prioritynegligible. Red Hat uses "Will not fix": the product is affected, but no fix is planned. The finding is real, and upgrading will not fix it. - End of life. The release line gets no security fixes at all, so every future CVE in it will show as "no fix". Debian 11 is the current example: its LTS ended on 31 August 2026. Debian tags packages it has stopped supporting
<end-of-life>, and Red Hat's "Out of support scope" means the product's lifecycle phase no longer covers issues of that severity. The same happens to charts: ingress-nginx was retired in March 2026 and will not get another release. The only remedy is to migrate, which is a bigger piece of work than a tag bump. StackRadar labels these findings as an end-of-life base rather than as waiting for a fix, and it never marks a release line EOL when it cannot tell. - Not affected. The distro says the vulnerable code is not in its build (Debian
<not-affected>, Ubuntunot-affected, Red Hat "Not affected"). No action is needed. If your scanner still reports it, you are looking at the upstream-versus-distro mismatch from the previous section.
For the deliberately-not class you have three honest options. You can move to a base image that ships the fix (Alpine, distroless or a Wolfi-based image often does), you can mitigate it (for example, by turning off the vulnerable feature), or you can record an accepted risk with an owner, a reason and an expiry so that someone revisits it. Suppressing a finding without an expiry is how a cluster ends up with three-year-old "temporary" exceptions.
Base images do most of the work
On an application image built on a full distro base, most findings are usually in the OS layer, and none of them is fixed by changing your code. They are fixed by rebuilding on a newer base, and three practices make that cheap:
- Pin the base by digest and automate the bump. Write
FROM debian:12-slim@sha256:…and let Renovate or Dependabot open a pull request when the tag moves. Each fix then arrives as a reviewed one-line change, and builds stay reproducible. - Rebuild images nobody is touching. A digest bump only helps an image that gets rebuilt, and an image built six months ago carries six months of unpatched OS packages. Rebuild and redeploy every image when its base bump merges, or at least weekly, even when its code has not changed.
- Use a smaller base. Fewer packages means fewer findings and fewer "won't fix" rows. Distroless, Alpine, Wolfi-based and UBI-micro images ship a fraction of the packages of a full Debian or Ubuntu base. You give up debuggability, and with Alpine you also switch to musl.
# Pinned by digest; Renovate bumps the digest when a new 12-slim is published.
FROM debian:12-slim@sha256:<digest-of-the-current-12-slim>
# No apt-get upgrade here: the fix comes as a base-image bump, not a build-day surprise.
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
&& rm -rf /var/lib/apt/lists/*apt-get upgrade in the Dockerfile does fix CVEs, and many teams use it. The cost is that the same Dockerfile produces a different image on Tuesday than on Monday, so the SBOM you attested on Monday describes an image nobody runs. Pinning by digest gets you the same fix as a reviewed change.Before merging a bump, check that it clears what you expect. The free stackradar scan CLI needs no account. It scans a candidate image locally with your Docker credentials and folds the base-image findings into one line. For images you build, StackRadar also checks your base every night against its current tag and the newest tag of the same variant, and tells you which rebuild removes more findings.
stackradar scan --image registry.example.com/checkout:1.42.1 --min-severity highA patching playbook: the order to work in
The list is long, and the order you work in matters more than how fast you go. This sequence works:
- Start from what is running, not what is in the registry. Scan the digests the kubelet reports. An image nobody runs is a cleanup task, not an incident.
- Fix anything in CISA KEV this week, whatever image it is in. These are observed exploitation, and the list is short. If there is no fix yet, mitigate now and track the backport.
- Then fix anything with EPSS ≥ 0.1 that has a fix, highest CVSS first. That list is still short.
- Upgrade third-party charts, starting with the biggest win. Rank chart upgrades by how many findings each one closes. A single chart that deploys several images can close dozens with one version bump. Read the changelog, because chart majors do break values.
- Bump the base images of your own images, oldest base first. This is where most of the long tail goes, and with the digest-pin setup above it is mostly merging pull requests.
- Deal with the no-fix residue deliberately. Migrate end-of-life bases, mitigate what you can, and record the rest as accepted risk with an owner and an expiry.
- Confirm every fix by digest, as the next section shows. Nothing is fixed until the vulnerable digest stops running.
In StackRadar, each finding's contribution to the Radar Score is likelihood × consequence. KEV membership sets the likelihood to certainty and EPSS weights the rest, so steps 2 and 3 come out at the top of the list. Helm release pages cover step 4, showing the findings a release carries, the images it deploys and what an upgrade would clear. Image pages cover step 5 with the rebuild advice above. StackRadar deliberately does not claim that a bump is safe. It tells you what the change fixes, and reading the changelog is still your job.
Confirming it is actually fixed
The Deployment says :1.42.1, but the pod might still be running :1.42.0. The rollout may have failed or been paused, or a node may have reused a cached copy of a tag that was pushed again (imagePullPolicy: IfNotPresent). Wait for the rollout, then check the digest each container is actually running:
kubectl rollout status deployment/checkout -n checkout
kubectl get pods -n checkout -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .status.containerStatuses[*]}{.imageID}{" "}{end}{"\n"}{end}'A scanner keyed on running digests stops reporting the finding when the vulnerable digest stops running, and not before. A ticket marked "done" is not proof. The fix is real when the kubelet reports the new digest.
Frequently asked questions
How do you patch a CVE in a Kubernetes pod?
You replace the image rather than patch the pod. A package upgrade run inside a running container is lost the next time the container restarts, so the fix has to be a new image that contains the fixed package. For images you build, rebuild on a newer base image or bump the dependency. For third-party workloads, upgrade the Helm chart or override the image tag once the maintainer has published a fixed release. Then roll it out and check that the running image digest changed.
What does "fixed version" mean for a CVE?
It is the first version of that package, as packaged by that ecosystem, that contains the fix. It is often not a new upstream release: Debian, Ubuntu, Red Hat and Alpine backport fixes into the version they already ship and bump a suffix. The fixed OpenSSL on Debian 12 might be 3.0.15-1~deb12u1 while upstream is on 3.3, so comparing your package against the upstream version number gives the wrong answer.
What does "no-dsa" mean in the Debian security tracker?
That the Debian security team judged the issue not to warrant an immediate Debian Security Advisory, usually with a note such as "Minor issue". It is not a promise that the issue will never be fixed: no-dsa issues can still be fixed in a point release or alongside a later, more serious fix. The sub-states <ignored> and <postponed> say which way it is leaning.
What does "no fix available" mean?
That the ecosystem has not published a version containing the fix. Either it has not got to it yet, it has decided not to (Debian no-dsa, Ubuntu deferred, Red Hat "Will not fix"), or the release line is end-of-life and will never get one. The first case is worth waiting for. For the other two the remedies are a different base image, a mitigation, or an accepted-risk record with an owner and an expiry.
Should I run apt-get upgrade in my Dockerfile to fix CVEs?
It fixes CVEs, but it makes builds non-reproducible: the image you get depends on the day you build it. Pinning the base image by digest and letting Renovate or Dependabot open the bump gets you the same fixes as a reviewed one-line change, and the SBOM of a build still describes the image you shipped.
How do I know a CVE is actually fixed in production?
Check the running digest, not the manifest. The kubelet reports the digest each container is running in status.containerStatuses[].imageID. If it has not changed, the fix has not landed, whatever the Deployment says. A scanner that keys on running digests, such as the StackRadar scanner or Trivy Operator, stops reporting the finding only once the vulnerable digest is no longer running.
More from the blog
All postsMost of your container vulnerabilities are not yours to fix
A container scan mixes three kinds of finding: your dependencies, the base image, and images someone else builds. How to separate them without hiding an exploited CVE.
7 min read
StackRadar is open source: a self-hosted Kubernetes vulnerability dashboard under AGPL-3.0
The server, the dashboard and the Helm chart are on GitHub under AGPL-3.0, the scanner and CLI under Apache-2.0. One Helm command installs the lot in your cluster, with no telemetry and nothing leaving it. What is open, why these licences, and where the cloud fits.
8 min read
A new CVE should not need a new scan: how we keep 900,000 advisories current
Most scanners match an image once, when it is scanned. We mirror OSV.dev every hour, EPSS and KEV every day, and re-match every stored SBOM when the mirror changes. How, and what it does not solve.
10 min read
Zero findings is not a clean image: two matching bugs and how we found them
We compared our scanner with Trivy and OSV-Scanner on one Helm chart. It invented 140 findings on one image and missed every OS finding on three distros.
10 min read