# The vulnerability database

The Qualor server keeps its own copy of the Trivy vulnerability database and checks the
dependencies of your tracked branches and versions against it. It downloads the copy itself, so no
scan has to. Instance admins manage it in **Settings → Vulnerability database**.

| | |
|---|---|
| Datasets | three: the Trivy database; the CISA KEV catalog and the FIRST EPSS scores (Business and Enterprise, `vuln.priority`) |
| Source | one setting for the three datasets: the official sources, your mirror, or files you upload. Under **Your mirror**, KEV and EPSS each take an optional URL of their own |
| Download | about 131 MiB, only when the database changed |
| Disk per replica | 1.68 GB (1.56 GiB) extracted, plus room for one update and for Trivy's work files: mount 4 GiB (Helm `vulnerabilityDatabase.cacheSizeLimit`, 6Gi by default) |
| Memory during a re-check | 1 GiB is enough: a server limited to 1 GiB re-checked 2 000 projects (8 051 branches and versions, 5 414 distinct packages) in about 8 minutes, at a peak of 929 MiB. A re-check runs one Trivy at a time, on at most 1 000 packages. The database file (1.56 GiB) is read through the page cache, which counts toward a container's memory limit until the kernel reclaims it |
| Check interval | 1, 3, 6, 12 or 24 hours (6 by default) |
| Switch | **Vulnerability monitoring**, on by default |

Where the results show: [Dependency vulnerabilities](https://qualor.dev/docs/dependency-vulnerabilities.md), the project's
**Vulnerabilities** tab.

Scanners can take their database from this copy: see [Scanners](#scanners).

## Scanners

By default, a scanner takes the database from this server instead of the one in its image, so every
scan checks against the data the server checks against. The scanner asks the server on every
analysis, downloads a database only when it needs a newer one, and reuses the copy it kept. Under
**Scanners**, an instance admin sets:

| Setting | Values |
|---|---|
| **Scanners take the database from this server** | on (default). Off: scanners use the database in their image |
| Policy | **At most once a day** (default): a scanner downloads a newer database when its copy is a day old. **At every analysis**: whenever the server has a newer one |

Any authenticated caller, scanner tokens included, can download the database. The
[public demo account](https://qualor.dev/docs/users-projects-tokens.md#a-public-read-only-demo) is offered none.

The server serves at most `QUALOR_VULNDB_MAX_DOWNLOADS` downloads at once (4 by default, 1 to 64).
It answers more callers with `503 DOWNLOADS_BUSY` and `retry-after: 5`. A scanner retries the
download once when `retry-after` is 10 seconds or less. When that fails too, it uses its cached copy
or the image and warns. The server cuts a download that stalls for 60 seconds.

While **Vulnerability monitoring** is off, or before the first database is ready, the server offers
scanners nothing and they use their image.

How the scanner chooses, caches and checks the copy, and its warnings, are in
[Dependencies (Trivy)](https://qualor.dev/docs/languages-and-analyzers.md#the-vulnerability-database).

## Where the data comes from

| Source | What the server calls |
|---|---|
| **Official Trivy database** | `ghcr.io/aquasecurity/trivy-db:2`, and `pkg-containers.githubusercontent.com`, the host that registry sends the file from |
| **Your mirror** | the registry you name, and the token service and file host it points to |
| **Upload a file (no internet)** | nothing |

With the `vuln.priority` feature (Business and Enterprise), the server also downloads two small
files over HTTPS:

| Dataset | URL | Host |
|---|---|---|
| CISA KEV | `https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json` | `www.cisa.gov` |
| FIRST EPSS | `https://epss.empiricalsecurity.com/epss_scores-current.csv.gz` | `epss.empiricalsecurity.com` (the URL redirects within this host) |

The KEV file is about 1.5 MB, the EPSS file about 300 000 rows. KEV and EPSS are checked in the
same run as the Trivy database. The server sends conditional requests, so an unchanged file is not
downloaded again. A file is read gzipped or plain and is at most 32 MiB.

The server asks the source for the newest database at every check and downloads it only when its
digest differs from the current one. It checks the digest of the download and that the archive
holds a valid Trivy database (`metadata.json` with `Version` 2, and `trivy.db`) before it uses it.
A failed or cut download changes nothing: the previous database stays.

The server keeps the two newest databases in its PostgreSQL, about 131 MiB each. It unpacks the
current one into the cache directory (`QUALOR_VULNDB_DIR`) for Trivy. After a restart or a new pod
it unpacks it again from PostgreSQL and downloads nothing.

The requests carry the header `User-Agent: Qualor/<version>` and nothing about your installation.
See [Telemetry](https://qualor.dev/docs/telemetry.md#vulnerability-database).

The server does not use `HTTPS_PROXY` or `HTTP_PROXY`, for these requests as for every call it
makes. Where a proxy is the only way out, the check fails with `UNREACHABLE`: use a mirror inside
your network, or upload the file.

### A mirror

Select **Your mirror** and fill in the **Repository**, for example
`registry.corp.example.com/mirrors/trivy-db:2`. The repository must hold the Trivy database as
Aqua publishes it, for example copied with `oras copy`. Add a **User** and a
**Password or token** if it needs them.

- The registry must serve HTTPS. A registry over plain HTTP is not supported: put TLS in front of
  it, or upload the file.
- A private certificate authority works like it does for GitLab: give the server the CA in
  `NODE_EXTRA_CA_CERTS` ([Server settings](https://qualor.dev/docs/install-server.md#server-settings)).
- A registry on a private address needs its host in `QUALOR_VULNDB_INTERNAL_HOSTS`,
  comma-separated, each with an optional port (`registry.corp:5000`). This lifts the address check
  only. It does not allow HTTP.
- The server sends the credentials only to the registry and to the token service it names, never
  to the host a file is sent from.
- If you change the registry's host and enter no new password, the stored password is dropped.

With `vuln.priority`, two more fields under **Your mirror** are optional: **KEV URL** and
**EPSS URL**. Each is an HTTPS URL without credentials or a fragment, up to 2048 characters.
Empty means the official source. A private host needs its name in `QUALOR_VULNDB_INTERNAL_HOSTS`,
as for the registry. Saving a changed URL starts a check at once.

### Offline

For a server without internet, get the file on a machine that has it and upload it under
**Upload a file (no internet)**. The upload is up to `QUALOR_VULNDB_UPLOAD_MAX_BYTES` (1 GiB by
default). Upload the layer as the registry serves it, `db.tar.gz`. The server refuses an archive
with pax or long-name headers, which `tar` often writes when you pack the files again.

A reverse proxy in front of Qualor needs the same body size limit for the upload's path, or it
refuses the file: see [Reverse proxy and TLS](https://qualor.dev/docs/install-server.md#reverse-proxy-and-tls). The Helm
chart sets it for its ingress.

With `oras`:

```sh
oras pull ghcr.io/aquasecurity/trivy-db:2
```

It writes `db.tar.gz` into the current directory. Without it, use `curl`:

```sh
TOKEN=$(curl -s "https://ghcr.io/token?scope=repository:aquasecurity/trivy-db:pull" \
  | sed 's/.*"token":"\([^"]*\)".*/\1/')
DIGEST=$(curl -s -H "Authorization: Bearer $TOKEN" \
  -H "Accept: application/vnd.oci.image.manifest.v1+json" \
  https://ghcr.io/v2/aquasecurity/trivy-db/manifests/2 \
  | sed 's/.*"layers":\[{[^}]*"digest":"\(sha256:[0-9a-f]*\)".*/\1/')
curl -L -H "Authorization: Bearer $TOKEN" -o db.tar.gz \
  "https://ghcr.io/v2/aquasecurity/trivy-db/blobs/$DIGEST"
```

With `vuln.priority`, the server takes the two other files too, each up to 32 MiB, gzipped or
plain. Upload them with `PUT /api/v0/system/vulnerability-db/file/kev` and `…/file/epss` as
`application/octet-stream`.

```sh
curl -O https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json
curl -L -O https://epss.empiricalsecurity.com/epss_scores-current.csv.gz
```

`-L` follows the redirect the EPSS URL answers with.

An uploaded database does not update itself. Upload a newer file at least every 3 days: after
that it counts as stale.

## States

The state shows on the page, under **State**.

| State | Meaning | What to do |
|---|---|---|
| Up to date | The data is at most 3 days old and the last check worked | nothing |
| Updating… | A download is running | wait |
| Stale: the data is N days old | The data is more than 3 days old, so the checks cannot find new vulnerabilities | fix the error shown under the state, or upload a newer file. Qualor sends one email to the instance admins for each stale database, in every edition ([Email](https://qualor.dev/docs/email.md)) |
| Failed: message | The last check or the last re-check did not work. The page shows the message of each. The previous data, if there is any, stays in use | see the table below |
| No data yet | Nothing has been downloaded or uploaded | wait for the first check, or select **Update now**. With **Upload a file (no internet)**, upload one |
| Off | **Vulnerability monitoring** is off | switch it on |

KEV and EPSS each have their own state, shown beside the Trivy database's, with the same values.
Each of their rows shows its last check. A KEV catalog is stale when the source last confirmed it
more than 3 days ago. An EPSS file is stale when the earlier of its score date and its storage time
is more than 3 days old. A file without a score date counts from its storage time. An uploaded file
counts from its dates in the same way.

**Update now** checks at once. After a failed re-check, it also re-checks every project. A re-check
message stays until a re-check of every project succeeds. The status line under the state shows
the last and next check, and how long the last full re-check took.

A failed state shows a code and a message.

| Code | Cause | Fix |
|---|---|---|
| `AUTH` | the registry or its token service refused the user and password, or the stored password cannot be read | enter the user and password again under **Your mirror** |
| `NOT_FOUND` | the repository or tag does not exist | check the **Repository** |
| `NO_LAYER` | the repository holds no Trivy database layer | point it at a copy of `ghcr.io/aquasecurity/trivy-db:2` |
| `BLOCKED` | the host is on a private address, or the registry sent a URL that is not HTTPS | list the host in `QUALOR_VULNDB_INTERNAL_HOSTS`, or serve the mirror over HTTPS |
| `UNREACHABLE` | no connection, a timeout, or a TLS error | open the firewall for the hosts above; for a private CA set `NODE_EXTRA_CA_CERTS`; or use a mirror or an upload |
| `BAD_ANSWER` | the registry answered something Qualor cannot use | check that the repository is an OCI registry with the Trivy database; look at the server log |
| `TOO_LARGE` | the file is over its limit. Trivy database: 2 GiB for a download, `QUALOR_VULNDB_UPLOAD_MAX_BYTES` for an upload. KEV and EPSS: 32 MiB as sent; unpacked, 8 MiB and 100 000 entries for KEV, 64 MiB and 1 000 000 lines for EPSS. The KEV and EPSS limits are fixed: `QUALOR_VULNDB_UPLOAD_MAX_BYTES` does not raise them | check the source; for the Trivy database, raise the upload limit |
| `DIGEST_MISMATCH` | the download does not match the digest the registry announced | check again; look at the mirror |
| `INVALID_DATA` | the KEV or EPSS file does not parse, downloaded or uploaded | from a mirror: check the **KEV URL** or **EPSS URL**, or upload the file. For an upload, send the file as the source serves it: `known_exploited_vulnerabilities.json` or `epss_scores-current.csv.gz` |
| `INVALID_DATABASE` | the file is not a Trivy database (no `metadata.json` with `Version` 2, no `trivy.db`, or an archive with pax or long-name headers) | upload the `db.tar.gz` as the registry serves it |
| `STREAM_FAILED` | the data stopped before its end | check again, or check the connection |
| `STORE_FAILED`, `UPDATE_FAILED` | the server could not store the database | look at the server log |
| `DISK` | the server cannot write to `QUALOR_VULNDB_DIR` | mount a writable volume there, with the space the table at the top gives |
| `RECHECK_FAILED` | Trivy could not run, or failed, during a re-check | look at the server log, then select **Update now** |

## The re-check

A re-check matches the dependencies of the tracked targets of a project with the current database.
The targets are:

- the project's main branch,
- the branches that match the branch patterns in the project's
  [vulnerability monitoring](https://qualor.dev/docs/releases.md) settings,
- the versions the project tracks.

It runs:

- after an analysis of a tracked target,
- after a version is stored,
- after the project's tracking or monitoring settings change,
- after a new database becomes current,
- when **Vulnerability monitoring** is turned on,
- on **Update now**, after a failed re-check.

A target that is not tracked keeps its last results and is not checked again. It is checked again
when it is tracked again.

The `.trivyignore` of a project applies as it does in a scan. An entry with `exp:` stops
applying at 00:00 UTC on that date. Packages without a package URL, for example from an ecosystem
Trivy does not know, are counted as not checked.

Re-checks and database updates share one worker, which runs one job at a time: an update that waits
runs first. A re-check runs Trivy 0.74.0 offline against the local copy. Trivy's work files, up to
32 MiB of output per run, go in `QUALOR_VULNDB_DIR` too, not in `/tmp`. Memory: see the table at
the top.

## The switch

**Vulnerability monitoring** is on by default. Turning it off stops the downloads and the
re-checks. The server then makes no request to the registry and offers scanners no database, so
they use the one in their image. Turning it on starts a check and a re-check of every project.

## Backups and replicas

- Each new database adds about 131 MiB to the table `vulndb_chunks` in PostgreSQL, so the
  write-ahead log and the backups grow by that much. The server keeps the two newest.
- To leave the files out of a backup, dump with
  `pg_dump --exclude-table-data=vulndb_chunks --exclude-table-data=vulndb_datasets`. A server
  restored from such a dump downloads the database again at its next check. With the upload
  source, upload the file again.
- Each EPSS update rewrites about 300 000 rows, once a day. One update writes about 90 MB of
  write-ahead log, measured on PostgreSQL 18, and the table holds about 50 MB. The KEV catalog is
  small.
- A small installation writes less with a longer check interval (12 or 24 hours): each check
  downloads at most one database.
- Each replica needs a `QUALOR_VULNDB_DIR` of its own. Never share one volume (ReadWriteMany)
  between replicas. The Helm chart gives each pod its own.

## Sources and licences

The Trivy database combines public advisory sources. The Trivy project lists them under
[Data Sources](https://trivy.dev/latest/docs/scanner/vulnerability/#data-sources) in its own
documentation. For the language packages, they are:

| Ecosystem | Upstream sources |
|---|---|
| PHP | PHP Security Advisories Database, GitHub Advisory Database |
| Python | GitHub Advisory Database |
| Ruby | Ruby Advisory Database, GitHub Advisory Database |
| Node.js | Ecosystem Security Working Group, GitHub Advisory Database |
| Java | GitHub Advisory Database |
| Go | GitHub Advisory Database, Go Vulnerability Database |
| Rust | Open Source Vulnerabilities (crates.io) |
| .NET | GitHub Advisory Database |
| C/C++ | GitLab Advisories Community (delayed by one month) |
| Dart, Elixir, Swift | GitHub Advisory Database |

The same page lists the sources of operating system packages (such as Debian, Ubuntu, Alpine and
Red Hat) and says that the National Vulnerability Database serves as a fallback for severity.

The two priority datasets:

| Dataset | Publisher | Terms |
|---|---|---|
| CISA KEV | CISA | [CC0 1.0](https://www.cisa.gov/sites/default/files/licenses/kev/license.txt) |
| FIRST EPSS | FIRST | free of charge, with no registration; attribution is requested |

EPSS is maintained by the EPSS Special Interest Group at FIRST. Scores are generated by Empirical
Security and published freely to the community.

Each source keeps its own terms. Trivy's page names them and does not state them. Read the terms of
the sources you rely on at the links on that page.