ZDI Advisory Processor Design

Overview

Zero Day Initiative (ZDI/Trend Micro) publishes per-year RSS advisory feeds at https://www.zerodayinitiative.com/rss/published/{YYYY}/ covering 2005 to current year. Each advisory maps to one or more CVEs. This processor fetches these feeds, parses advisories, maps them to CVE data, and stores them directly into the existing CVEMetadata / reference / metric tables — matching the pattern used by osv-file-processor and mitre-cve-processor.

ECS schedule: hourly at :50 (current year only, small incremental feed). Backfill: --start-year flag processes 2005 → current year locally.

ZDI Feed Structure

Every <item> has exactly 5 elements:

Description extraction patterns:

• CVSS: The ZDI has assigned a CVSS rating of (\d+(?:\.\d+)?).
• CVEs: CVE-\d{4}-\d{4,} (all occurrences)
• Attack vector: remote attackers / local attackers / network-adjacent attackers
• Auth required: Authentication is not required → false; default true
• User interaction: User interaction is required → true

Decision: Skip advisories with no CVE IDs (cannot key CVEMetadata).

Package Layout

internal/zdi/
  types.go   — Feed/Item/Advisory structs + Source = "zdi"
  parser.go  — XML parse + regex extraction functions
  mapper.go  — Maps Advisory → osv.CVESourceData

cmd/zdi-processor/
  main.go    — Main binary (ECS + backfill flags)
  s3.go      — S3 uploader (same pattern as other processors)

Flags

Processing Flow

1. Parse flags, validate year range (2005 ≤ startYear ≤ endYear ≤ currentYear)
2. Connect DB (60-minute timeout)
3. Build S3 uploader if S3_BUCKET_NAME set
4. For each year in range:
   • Fetch RSS feed with 3-attempt retry
   • SHA256 early-exit if feed unchanged and !force
   • Parse all items (skip items with no CVE IDs)
   • Load resume set (cveID → sourceFileHash)
   • Process in batches of 100 via db.WithTx + savepoints
   • db.UpsertTracker with per-year tracker source zdi_YYYY
5. Exit 1 if any errors

DB Impact

No new tables. Uses existing tables:

• CVEMetadata — source='zdi', sourceAdvisoryRef='ZDI-YY-NNN'
• CVEAlias — secondary CVE IDs + ZDI advisory ID as alias
• CVEDescription — full advisory description text
• CVEReference — advisory URL with type='advisory'
• CVEMetric — CVSS score (no full vector string from feed)
• CVEAffected — vendor/product from title parsing
• BulkDataDumpTracker — per-year SHA256 dedup (source='zdi_YYYY')
• Artifact — one row per advisory whose raw payload is archived to S3, recording the bucket, key, file size, content type and checksum (db.InsertArtifact, bomFormat='zdi', type='OTHER')
• Link — joins the advisory URL to that Artifact as PLAIN_JSON (db.InsertLinkWithArtifact); the resulting link id is attached to the CVEMetadata row as fileLinkId

Artifact and Link are shared tables that other processors also write. They were missing from this list until the efficacy audit, which is what failed the code-mirrors-design gate — the writes were correct, but discoverable only by reading storeAdvisory.

Decisions

• No CVE IDs: Skipped (logged at info level). ZDI CAN ID is not a stable public identifier.
• CVSS version: cvssV3_1 for 2016+, cvssV2_0 for earlier (approximation; no vector string in feed).
• One tracker per year: Enables per-year change detection.
• Vendor/product from title: Best-effort heuristic (first word = vendor, second = product).

S3 Persistence

• Archive path: zdi/files/{sha256}/{filename} ✓
• Quarantine path: failed-feeds/zdi-processor/{YYYY-MM-DD}/{reason}/{filename} ⚠ not yet wired
• Failure reasons emitted: parse-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

See the S3 Persistence Contract for the full reason taxonomy.