alpine-json-processor

Status: Live Source: secdb.alpinelinux.org Type: json (one JSON document per {branch}/{repo}, no pagination) Source slug: alpine — used as the CVEAffected vendor and adpOrgId, not as a CVEMetadata.source Schedule: Runs daily at 05:00 UTC (cron(0 5 * * ? *)), 512 CPU / 1024 MB, expected_duration_minutes = 60.

Overview

Alpine is the base image for a very large share of the container estate, and Alpine’s own secdb feed is the only authority on which apk package version carries the fix for a given CVE. Before this processor existed, alpine package data reached the VDB only indirectly (through the OSV mirror and anchore-adp-git-processor), and Docker Hub registry attribution existed only as a one-shot migration (20260525000003_backfill_alpine_chainguard_wolfi_registries). This processor is the ongoing canonical writer for both.

It does not mint its own advisory namespace. Alpine secdb has no Alpine-prefixed identifier — the (package, fixed-version, CVE) triple is the advisory — so every row it writes is attached to the CVE id itself, under whichever source is already authoritative for that CVE.

Branch discovery scrapes the secdb directory listing (apksecdb.DiscoverAlpineBranches) unless --branches overrides it, then crosses it with main and community (--repos). 46 feeds are tracked in production as of 2026-08-06 (23 branches × 2 repos); each (branch, repo) pair is fetched, SHA-256’d and tracked independently, so a run where only v3.20/community.json changed re-processes only that feed.

{branch}/community.json did not exist before v3.3, so a 404 is expected topology and is counted as “unchanged”, never alarmed on (internal/apksecdb/runner.go:29-33).

Source attribution

For each vuln id in a secfixes slot, bulkLookupBaseSources resolves the source that already owns the CVE, preferring cve.org → nist-nvd → github → osv (internal/apksecdb/mapper.go:117-128). All rows are then written against that (cveId, source) pair. When no source knows the id yet, naturalSourceFor maps the prefix (CVE-* → cve.org, GHSA-* → github) and a minimal placeholder CVEMetadata row is created under that source so the authoritative processor fills it in later. Ids whose natural source would be alpine itself are dropped rather than left as permanent orphans (mapper.go:244-255).

Records produced

ConditionRecords
Every (package, fixed-version, vuln id) tripleCVEAffected — containerType="adp", adpOrgId="alpine", vendor alpine, collectionURL="https://pkgs.alpinelinux.org", cpes JSON, affectedHash = MD5 of vendor|##|product|##|collectionURL|##|packageName|##|adp:alpine
SameCVEAffectedVersion — the secfixes key is the fix version, modelled as version="0" … lessThan=<fix>, versionType="apk"; the literal "0" key becomes a pinned wildcard row
Vuln id unknown to every sourceCVEMetadata placeholder via db.EnsureMinimalCVEMetadata under the id’s natural source (cve.org / github / prefix-derived) with datePublished = 0 for the authority to overwrite
Every affected rowContainerOriginAdvisory (registryId="docker-hub-alpine", imageReference="docker.io/library/alpine") via db.UpsertContainerOriginAdvisory
Every affected rowDependency, DependencyRegistry, PackageVersion, PackageVersionCVE via db.EnrichAffectedWithDependency with PURL pkg:apk/alpine/<pkg>[@<ver>]
Feed changed and ≥1 row written with no errorsBulkDataDumpTracker (source="alpine_secdb_{branch}_{repo}", sha256 = feed body hash)

No CVEMetadata row is ever written under source='alpine', and no CVEAlias edges are emitted — the feed carries no alias information.

Scheduling and failure modes

  • Feeds are processed sequentially; the per-package write phase inside apksecdb.Run uses one short transaction per package with a SAVEPOINT per vuln, so nothing holds row locks across packages (internal/apksecdb/mapper.go:263-330).
  • A per-feed failure is recorded via notifier.RecordError and the loop continues to the next feed (cmd/alpine-json-processor/main.go:133-136).
  • Every feed unchanged → NoWork. Any recorded error → Errored. The process exits non-zero only when nothing at all succeeded (agg.Errored > 0 && agg.AffectedUpserted == 0), so partial success does not make EventBridge re-fire (main.go:174-192).
  • The soft deadline is applied only when EXPECTED_DURATION_MINUTES is set (ECS), and is honoured both in the outer feed loop and inside apksecdb.Run’s package loop, so a local backfill runs to completion.

S3 filenames

The {filename} slot is always the literal security.json — one object per feed body per successful parse (internal/apksecdb/runner.go:85), and the same name on the quarantine path (runner.go:79). Because the key is content-addressed by the feed SHA-256, distinct (branch, repo) feeds never collide.

Flags

FlagDefaultDescription
--branches(discovered)Comma-separated branch override, e.g. v3.20,edge — skips the directory-listing scrape
--reposmain,communityComma-separated repositories to fetch per branch

Local run: just go-alpine-json-backfill prod. Expect 5–15 minutes on a first pass and seconds thereafter (every feed unchanged → NoWork).

Shared code

internal/apksecdb is the single mapper for the Alpine secdb wire format and is shared with chainguard-json-processor and wolfi-json-processor; the only differences are the feed URL, source slug, collection URL and registry slug passed in apksecdb.Config.

S3 Persistence

  • Archive path: alpine/files/{sha256}/{filename} ✓
  • Quarantine path: failed-feeds/alpine-json-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
  • Failure reasons emitted: parse-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

flowchart LR SRC[Source feed] --> PROC[alpine-json-processor] PROC -->|success| ARCHIVE[("S3: alpine/files/{sha256}/{filename}")] PROC -->|failure| Q[("S3: failed-feeds/alpine-json-processor/{date}/{reason}/{filename}")] PROC --> DB[(PostgreSQL)]

See the S3 Persistence Contract for the full reason taxonomy.