A new CVE should not need a new scan: how we keep 900,000 advisories current

A vulnerability scan is a snapshot. It says what was known about an image's packages at the moment the scanner ran. The image does not change after that until somebody deploys a new one, which for a running service might be next week or next year. What is known about its packages changes every hour.

That gap is where most of the real risk in a cluster sits. The CVE that matters is rarely the one that was already public when the image was built. It is the one published on a Tuesday in October against a library inside an image that has been running unchanged since January, and the question is how long it takes for that advisory to turn into a finding on that image. For a scanner that matches at scan time, the answer is "the next time someone scans it", and for an image nobody is rebuilding, that can be never.

The short answer: we keep a full local copy of OSV.dev, update it every hour, join FIRST.org's EPSS scores and CISA's KEV catalogue onto it every day, and re-match every stored SBOM whenever the copy changes. A new advisory against a running image shows up as a line in the feed within the day, marked as a change in the world rather than a change in the cluster. This post is how that works, what it costs, and the three things it still does not fix.

Two clocks

A finding is a join between two things: what an image contains, and what is known about what it contains. The two change on different clocks.

The image's clock ticks when you deploy. Between deploys the package list is fixed, down to the byte, and there is nothing new to learn by scanning it again. This is why StackRadar keeps one SBOM per image digest per cluster and does not re-scan: a second syft run on the same digest produces the same list.

The advisory clock never stops. OSV.dev's change feed lists about 220 modified records in a typical hour, which is new advisories, widened or narrowed version ranges, added aliases, and withdrawals. EPSS republishes every CVE's exploitation probability daily. CISA adds to the KEV catalogue several times a week. None of that needs anything to happen in your cluster.

So the work has to be split along the same line. Build the package list once, when the image appears. Keep the advisory data current on its own schedule. And re-run the join whenever the second side moves, because that is the only time the answer can change.

A mirror, not an API

OSV.dev has a query API, and the obvious design is to call it for each package at match time. We did not, for three reasons.

• The volume. One SBOM can hold a thousand components. A fleet of a few hundred images re-matched every hour is a few hundred thousand lookups an hour against somebody else's service, and the batch endpoint's limits are not written with that in mind.
• The failure mode. A matcher that depends on a third party being reachable does not fail at a convenient time. It fails during the hour everyone is looking, and it fails as a timeout rather than a wrong answer, so the scan stalls.
• The keys. Matching is mostly about spelling, as the last post showed. With the records in our own database we compute a canonical lookup key for every affected package once, at ingest, and the match is an indexed equality rather than a normalisation at query time.

The copy is a little over 900,000 advisories, exploded into about four million rows of "this ecosystem, this package, this version range", which is the table the matcher reads. The price is that the copy has to be kept current, and that is the part worth describing, because it is where a scanner quietly becomes stale.

Keeping 900,000 advisories current

The first import

A fresh mirror starts from OSV's all.zip, about 1.35 GB compressed. The import does not write into the live table. It builds a new generation of the affected-package rows beside the one currently serving the matcher, and when the whole archive has been read it moves one pointer and deletes the old generation in a single transaction. Every read of the mirror is scoped to the live generation, so the matcher never sees a half-built copy.

If the import is judged bad at the end, because too many entries failed to parse or the archive produced no records, the new generation is dropped and the old one stays live. A failed rebuild is a no-op, not an outage. This is also why the rebuild has its own schedule: a check runs twice a day and exits immediately unless the mirror has never completed, so a restored database recovers on its own within twelve hours, and the hourly job refuses to start a multi-hour import it would be killed in the middle of.

Every hour after that

OSV publishes a CSV of every record's last modification time. It is 45.8 MB and roughly 896,000 lines, sorted newest first. The hourly run streams it and stops reading as soon as it has passed the time of the last change it applied, less a ten-minute overlap for clock skew. In a normal hour that is a few hundred lines out of 896,000. Each changed record is fetched, hashed, and written only if the hash differs from what is stored, so a record that OSV touched without changing costs one GET and no write.

INFO: Processing update candidates linesParsed=271 stoppedEarly=true
      orderingViolated=false candidateCount=218 deduplicatedCandidates=53
INFO: Incremental sync complete recordsProcessed=218 inserted=5 updated=201
      unchanged=12 affectedPackagesExtracted=1204 parseFailures=0 lagSeconds=42

Three things in that log are the ones we watch.

• The watermark only moves forward over things that landed. A record that could not be fetched caps the watermark below its timestamp, so the next run tries it again. Nothing below the watermark is ever revisited, which means a swallowed write error would be a permanent hole reported as a healthy sync. So a write failure is fatal to the run rather than logged.
• Burst days fall back to archives. A bulk republish, or a window missed while something was down, can mean tens of thousands of changed records at once. Above 5,000 candidates the run pulls each affected ecosystem's archive once instead of issuing one request per record.
• Lag is measured every run. The difference between now and the newest change applied is the number that says whether the mirror is current. It is usually under a minute. A warning fires at six hours, and an alert on the same metric is what would tell us before a customer did.

The record count matters less than it looks. What makes a mirror trustworthy is not that it has 900,000 advisories but that the one published forty minutes ago is in it, and that when it is not, something is already complaining.

The daily feeds: EPSS and KEV

Two more sources join the mirror, and they are what turn a list of findings into an order. FIRST.org publishes an EPSS score for about 360,000 CVEs, refreshed every day. CISA publishes the Known Exploited Vulnerabilities catalogue, about 1,600 entries, as a single JSON file. We fetch EPSS at 06:15 UTC and KEV at 07:15 UTC and write both onto the advisory rows.

The risk score of a finding is a stored column computed by the database from those rows: CVSS for impact, EPSS for likelihood, a KEV listing overriding likelihood to one. It is a generated column, which means no process ever writes it. Postgres recomputes it whenever the row changes. So the daily EPSS refresh is the recomputation of every finding's score, and a score that disagrees with the data it was computed from is not a state the database can be in. The Radar Score page has the formula.

Both syncs have one guard worth knowing about, because it is the opposite of what a sync usually does. Each refuses a file that is too short. For EPSS the floor is 150,000 rows: a gzip stream cut off mid-transfer decompresses cleanly up to the cut, and every CVE missing from the truncated file would have its score blanked, which reorders the whole priority list. For KEV the floor is 500 entries, and the reason is sharper. The KEV sync is a full replacement rather than an update, because CISA does occasionally remove entries, and a listing that outlived its withdrawal would pin a finding to the top of every list forever. A full replacement from a truncated file would be a mass delisting. So both syncs roll back rather than apply a short file.

Re-matching everything that is stored

An up-to-date mirror on its own changes nothing. Matching only runs inside a scan job, and for a long time the only things that started a scan job were an SBOM upload and a person. An image was matched once, when it first appeared, and an advisory published against it afterwards never surfaced. The data was current and the findings were not.

Now every hourly run that inserted or updated at least one record ends by queuing one scan job for every image that has an SBOM. The matcher is local database work, and an image is one job however many namespaces it runs in, so the fleet re-matches in minutes. Three details keep it from becoming its own problem.

• A run that changed nothing queues nothing. Re-matching against an identical mirror cannot change a verdict, and doing it anyway is pure load.
• Jobs do not stack. An image with a scan already pending or running is skipped, so a slow worker sees a bounded queue no matter how many hourly runs pass while it catches up.
• Fresh uploads go first. The hourly re-match runs at the lowest priority. An SBOM that just arrived from a cluster has never been matched at all, and its first findings matter more than the fleet's refresh.

Each scan brackets itself. It reads the image's findings before its first write and after its last, diffs the two sets on the cross-image identity of a finding, so a GHSA that becomes its CVE is one finding on both sides, and writes one event per image per scan carrying what was added and what was removed. Never one event per finding. The feed shows it as one line: this image, these workloads, three findings added, one removed, and a cause.

The cause is the part that was missing from every scan report we had used before. A finding that appeared because the mirror changed is labelled as such: nothing was deployed, the world changed. A finding that appeared because a new SBOM was uploaded is labelled the other way: the findings came with the image. The reader's next action is different in the two cases, and a report that only says "seven new findings" makes them look up which one it was.

What this does not solve

Three things, in the order they are most likely to bite.

• The mirror is only as complete as OSV.dev. OSV publishes nothing for Amazon Linux, Oracle Linux, CentOS, Fedora or Photon OS, so an hourly sync of a feed that does not cover your distribution is an hourly sync of nothing. The previous post covers which distributions are affected and how to test for it.
• The SBOM side does not refresh itself. The package list is built once per digest with the syft version current at the time. When a newer syft learns to see a package the old one missed, images already in the mirror keep the old list until their SBOM is replaced. A rebuilt image is a new digest and gets a new SBOM; an image that keeps running is not.
• "Within the day" is the honest number, not "within the hour". OSV itself ingests from NVD, GitHub and the distribution trackers on its own schedule, then our run picks up the change within the hour, then the queue drains at the worker's pace. Each step is short. Together they are hours, and a finding tied to a daily EPSS or KEV change moves on the daily clock.

The re-match itself is also deliberately blunt: every image after any change, even an advisory for a package none of them contain. The incremental run knows exactly which packages it touched, so narrowing the re-match to images that contain one of them is a change to one query. We have not made it because the queue has not asked for it, and a blunt sweep that runs is better than a precise one that has a bug in its filter.

How other tools answer the same question

The question to ask of any scanner is not how many advisories its database holds but when a finding for an image that is already running first appears. The answers differ by design, not by quality, and each is the right one for the job the tool was built for.

The row that surprises people is the first one. A pipeline scanner is the most common answer to "we scan our images", and it is also the one with no path at all from a new advisory to a running image. It scans what is being built, and the images with the oldest packages are the ones that are not being built.

Measure it on your own cluster

Whatever you run, the number worth knowing is the delay between an advisory being published and your tool reporting it on an image that did not change. It takes a few minutes to measure.

1. Pick an image that has been running for a while

kubectl get pods -A -o jsonpath='{range .items[*]}{.status.startTime}{"\t"}{range .status.containerStatuses[*]}{.imageID}{"\n"}{end}{end}' \
  | sort | head

The digest in imageID is what is actually running, whatever the tag says. Note the start date.

2. Find a package in it with a recent advisory

List the image's packages with syft, pick a few common ones (an OpenSSL, a curl, a popular library in the application's language), and ask OSV what has been published against each since the pod started:

curl -s https://api.osv.dev/v1/query \
  -d '{"package":{"name":"openssl","ecosystem":"Debian:12"}}' \
  | jq -r --arg since 2026-09-01 \
      '.vulns[] | select(.published >= $since) | "\(.published[:10])  \(.id)"'

The ecosystem is spelled OSV's way, so Debian:12, Alpine:v3.21, npm, PyPI or Go. If nothing comes back for the packages you picked, choose an older pod or a busier package.

3. Check whether your tool shows it, and when it did

An advisory that applies to the installed version and is published after the pod started should be on that image in your tool now, without a redeploy. If it is, the time between the advisory's publication and its first appearance in your tool is the number. If it is not, the number is whatever the next rebuild date is, and that is the answer to write down.

A StackRadar account answers the same question from the feed: filter it to the image and look for the line marked as a mirror change. The free CLI scans what your cluster runs with OSV-Scanner embedded, so it is always current and always a snapshot, like every other one-shot scanner. The stored SBOM and the hourly re-match are what the hosted product adds, and this post is the whole of what they do. For how the matching itself works, see the matching reference.