zoom-fetch-processor — Design

1. Overview

• Purpose: scrape Zoom Security Bulletins from https://www.zoom.com/en/trust/security-bulletin/ (listing) + per-bulletin /ZSB-YYNNN/ detail pages, emit CVEMetadata rows under source="zoom" plus a single CRIT envelope per CVE keyed to the Zoom service inferred from the bulletin title.
• Source URL: https://www.zoom.com/en/trust/security-bulletin/
• Owner: Zoom Security (no public CNA assignment; Zoom internal).
• Schedule: Runs weekly on Wednesdays at 06:00 UTC (cron(0 6 ? * WED *)). 1h offset from Phase 1.17 Elastic at 09:00.
• ECS resources: cpu_units=256, memory_mb=512, expected_duration_minutes=15.
• Reads: Zoom listing + per-bulletin detail pages (HTTP), BulkDataDumpTracker.zoom, LoadProcessedHashes for resume.
• Writes: CVEMetadata, CVEDescription, CVEMetadataReferences, CVEAlias (ZSB-* alias), BulkDataDumpTracker, S3 archive + quarantine, CritRecord (single envelope/CVE — enabled in ECS). GCVE issuance + alias only for items that lack a CVE-* alias. No CVEMetric, CVEProblemType, CVEAffected or PackageVersion rows; internal/aienrich is not wired.
• Freshness gate: compares now - BulkDataDumpTracker.lastProcessedAt for source zoom against a hardcoded 86400 s (frequencySecs), not against the tracker row’s own frequency column.

2. Source contract (Phase 0 recon)

3. Architecture

Listing → URLs → per-detail fetch (polite delay) → parse → store → CRIT.

4. Source → DB field mapping

⚠ datePublished is the scrape time, not the publication date. zoom.MapToSourceData sets DatePublished = fetchedAt.Unix() (internal/zoom/map.go:32-35), where fetchedAt is time.Now() from the processing loop. Every source='zoom' row therefore claims to have been published on the day we scraped it, and the value moves on every re-store. No ZSB publication date is parsed from either the listing or the detail page. The ZSB-YYNNN alias-shell rows (26 of the 55 source='zoom' rows) carry datePublished = 0 and no title — those are FK placeholders minted by db.InsertAliases, not records, and must be excluded from field-coverage measurements.

CVEMetric and CVEProblemType are never written: Zoom’s CVSS severity and affected-product list are JS-hydrated and out of scope for the static scrape (see §11). internal/aienrich is not wired, so no CWE is inferred either.

5. Identifier policy

• CVEMetadata.cveId: prefer first CVE-YYYY-NNNN in detail body. When absent, mint ZSB-YYYY-NNNN via db.LoadMaxGcveSequence("ZSB", year).
• CVEAlias: ZSB-YYNNN always written as alias.
• ⚠ The mint path increments seqByYear[currentYear] (main.go:207-210) but UpsertGcveIssuance derives its year from datePublished and reads seqByYear[year] (main.go:331-338). Because datePublished is the scrape time these agree today, but they are two different sources of truth for the same sequence and will diverge if a real publication date is ever wired in. The same mismatch exists on the failure path (main.go:232-235), which decrements a possibly different year’s counter. No minted ZSB-YYYY-NNNN id exists in production yet, so nothing is currently mis-sequenced.
• ⚠ There is no lookup of an already-issued ZSB-YYYY-NNNN for a known ZSB, so a --force run on a CVE-less bulletin would mint a second id for it (the same class of defect that produced duplicate TWILIO-… rows).

6. CRIT / VEX

• Single envelope per CVE keyed to title-inferred service.
• Title keyword → service mapping (priority order):
  • “workplace” / “client” / “desktop” → workplace/application (customer)
  • “marketplace” / “sdk” / “app sdk” → marketplace/application (customer)
  • “phone” → phone/line (provider)
  • “webinar” → webinar/event (provider)
  • “rooms” → rooms/device (provider; client also affected)
  • “contact center” → contact_center/tenant (provider)
  • default → meetings/session (provider)
• VEX status: "fixed" (ZSB only ships when fixed).
• temporal.service_available_date: 2013-01-25 (Zoom launch).
• temporal.provider_fix_date: no publication date is parsed, so this falls back to the run date.
• --emit-crit defaults to false in the binary but the ECS command passes --emit-crit=true (terraform/go-schedules.tf:4476), so the CRIT path is live in production for this producer.

7. S3 layout

• Archive on success: zoom/files/{sha256}/{ZSB-YYNNN}.html
• Quarantine on failure: failed-feeds/zoom-fetch-processor/{date}/{reason}/{filename}

8. Error handling & Slack

• Per-record failures non-fatal.
• Listing fetch failure fatal.
• Slack: Started, Errored, NoWork, Completed.

9. Performance

• Listing fetch (~270KB) ~1s + per-detail fetch (~250KB) ~1s with 1s polite delay between detail fetches → 150 detail fetches ~5 min worst case.
• Resume via per-bulletin sha256 in LoadProcessedHashes.

10. Backfill

• Same binary; justfile recipe unsets EXPECTED_DURATION_MINUTES.
• Full historical: ~150 ZSBs covered by single listing-walk.

11. Out of scope

• JS-rendered CVSS Severity, Affected Products list. A future enhancement could swap to JSON-LD or chromium-based scrape if the structured fields become valuable.

S3 Persistence

• Archive path: zoom/files/{sha256}/{filename} ✓
• Quarantine path: failed-feeds/zoom-fetch-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
• Failure reasons emitted: store-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

See the S3 Persistence Contract for the full reason taxonomy.