chainguard-json-processor

Status: Live Source: packages.cgr.dev/chainguard/security.json (Alpine secdb format) Type: json (single bulk feed, SHA-256 freshness) Source slug: chainguard — as the adpOrgId / vendor on CVEAffected, not as the owning CVEMetadata.source Schedule: Runs daily at 08:30 UTC (cron(30 8 * * ? *)).

Overview

Chainguard publishes hardened, minimal container images and tracks which of their apk packages have fixed a given CVE. This processor turns that into per-image answerability: for a CVE that a scanner reports against a Chainguard image, we can say which apk package is implicated and which package version fixed it, and we can attribute the finding to the chainguard-images registry so a consumer knows the advisory applies to cgr.dev/chainguard/* rather than to upstream Alpine.

Crucially it is an ADP (Authorized Data Publisher) enricher, not an advisory source: it never claims authorship of a CVE. Each vulnerability id in the feed is resolved to the existing CVEMetadata row that already owns it (cve.org → nist-nvd → github → osv → any other), and the affected/version rows are hung off that row with containerType="adp", adpOrgId="chainguard". Only when no processor has ever seen the id is a minimal placeholder created under the id’s natural authority.

Without it, Chainguard image scans have no fixed-version data and no registry attribution, so every CVE present in the base image reads as unfixed.

The feed shape is identical to Wolfi’s, so both binaries are thin shims over the shared internal/apksecdb package. See APK Secdb Processors for the feed format, the "0"-is-wildcard rule, and the shared storage model.

Records produced

ConditionRecords
Every (package, vulnID) with a resolvable base rowCVEAffected upsert at (cveId, baseSource, containerType="adp", affectedHash) — adpOrgId/vendor = chainguard, product/packageName = apk name, collectionURL = https://images.chainguard.dev, one wildcard CPE in cpes
Every observed fix versionCVEAffectedVersion — secfixes["0"] → Version="*"; secfixes["X.Y.Z-rN"] → Version="0", LessThan="X.Y.Z-rN"; VersionType="apk"
Vuln id unknown to every sourceMinimal placeholder CVEMetadata under the id’s natural authority (CVE-*→cve.org, GHSA-*→github, else sourceident.FromPrefix), written by db.EnsureMinimalCVEMetadata — state='PUBLISHED', dataVersion='5.1', datePublished=0, no title, no raw payload. It is a foreign-key anchor for the affected rows, not a record; the natural authority’s own processor is expected to fill it in. Self-sourced placeholders are dropped — if the natural source resolves to chainguard itself, no row is written, because no upstream authority would ever enrich it.
Every affected row writtenContainerOriginAdvisory via db.UpsertContainerOriginAdvisory — registrySlug="chainguard-images", imageReference="cgr.dev/chainguard/wolfi-base". Non-fatal: a failure is logged and counted but does not roll back the affected rows.
Every affected row writtenDependency / Registry / PackageVersion / PackageVersionCVE via db.EnrichAffectedWithDependency (PURL pkg:apk/chainguard/<pkg>). Manages its own savepoint and swallows its own errors.
Clean full run onlyBulkDataDumpTracker row chainguard_secdb keyed on the feed body SHA-256
Feed bodyS3 archive / quarantine (see § S3 Persistence)

No CVEDescription, CVEMetric, CVEProblemType or CVEAlias rows are written — the ADP contribution is affected-product data only.

Production scale as of this audit: ~277.7k CVEAffected rows with adpOrgId='chainguard' and ~16.0k ContainerOriginAdvisory rows for the chainguard-images registry.

Performance shape

Base-source resolution for every unique vuln id in the feed is done before the write loop, in chunks of 1000 via = ANY($1::text[]) on the read pool. Chunking matters: at 5000 elements the planner mis-estimates selectivity against the ~4.8M-row CVEMetadata and flips to a seq scan, blowing the read pool’s 60 s statement_timeout (SQLSTATE 57014). This pre-resolve is what took per-package latency from ~10 s to a few hundred ms.

Each package is then written in its own short transaction with a SAVEPOINT per vuln, so no run holds row locks across packages and a deadlock with a sibling processor costs at most one vuln.

Failure modes

SymptomCause
feed unchanged, no workTracker SHA-256 matches the fetched body — task.no_work, exit 0
apksecdb: feed not foundUpstream 404. Expected topology for some (branch, repo) pairs on the Alpine sibling; for Chainguard’s single feed it means the endpoint moved
soft deadline reached, stopping earlyOnly when EXPECTED_DURATION_MINUTES is set (ECS). The deadline is correctly left at its zero value for local backfills, so just go-chainguard-json-backfill runs to completion
Per-package errors, exit 0Partial success is deliberate — the ECS task is not retried. The tracker is not advanced, so the next scheduled run retries the whole feed
exit 1Only when errored > 0 and affectedUpserted == 0, i.e. nothing at all succeeded

When any per-package error was recorded the run reports task.errored (not task.completed) so the per-item details reach Slack, even though the exit code may still be 0.

Local execution

just go-chainguard-json-backfill          # local DB
just go-chainguard-json-backfill prod     # .env.production

S3 Persistence

  • Archive path: chainguard/files/{sha256}/{filename} ✓
  • Quarantine path: failed-feeds/chainguard-json-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
  • Failure reasons emitted: parse-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

flowchart LR SRC[Source feed] --> PROC[chainguard-json-processor] PROC -->|success| ARCHIVE[("S3: chainguard/files/{sha256}/{filename}")] PROC -->|failure| Q[("S3: failed-feeds/chainguard-json-processor/{date}/{reason}/{filename}")] PROC --> DB[(PostgreSQL)]

See the S3 Persistence Contract for the full reason taxonomy.