mongodb-rss-processor — Design

1. Overview

  • Purpose: Ingest MongoDB security bulletins from the public RSS feed into CVEMetadata and related tables.
  • Source URL: https://www.mongodb.com/alerts/rss
  • Owner: MongoDB Security Team
  • Licence: Public RSS feed, no auth required
  • Schedule cadence: cron(0 6 ? * 1 *) — weekly Monday 06:00 UTC. MongoDB releases security patches in coordinated batches every 1–2 months.
  • ECS resources: cpu_units=256, memory_mb=512, expected_duration_minutes=15
  • Reads: External RSS feed, BulkDataDumpTracker (feed SHA256), LoadProcessedHashes (resume set)
  • Writes: CVEMetadata, CVEAlias, CVEDescription, CVEReference, CVEMetric, CVEAffected, CVEAffectedVersion, BulkDataDumpTracker, S3 archive

2. Source contract (from Phase 0 recon)

  • Cadence: Batched — coordinated security releases every 1–2 months, with 3–15 CVEs per batch.
  • Volume: ~96 security CVEs total (2019–2026); ~3–15 new per quarter.
  • Identifier shape: Native CVE-prefixed IDs (e.g. CVE-2024-1351). JIRA ticket IDs (SERVER-50605) as secondary refs.
  • Backfill feasibility: Full archive available in single RSS feed (no pagination). No backfill binary needed.
  • Anti-bot / auth / rate-limit: None — public RSS, no rate limiting observed.
  • Pagination/freshness: No pagination. Feed contains full archive. Freshness tracked via feed body SHA256 + per-item hash resume.

3. Architecture diagram

graph LR Source[RSS Feed] --> HTTP[httpclient.NewHTTP1] --> Parse[mongodb.ParseFeed] Parse --> Item[mongodb.ParseItem] --> Map[mongodb.MapToSourceData] Map --> Tx[(pgx Tx)] --> Pipeline[processor.StoreCVESourceData] Pipeline --> CVEMetadata Pipeline --> CVEAlias[(db.InsertAliases)] Pipeline --> CVEMetric & CVEDescription & CVEReference & CVEAffected Tx -.commit.-> S3[s3client.ArchiveRecord]

4. Source → DB field mapping

Source FieldTarget Table / ColumnNotes
item.titleCVEMetadata.titleCVE ID stripped for cleanliness
item.description (HTML stripped)CVEDescription.valueContainerType=“cna”, Lang=“en”
item.pubDateCVEMetadata.datePublishedRFC 2822 → Unix seconds
item.linkCVEReference.urlType=“advisory”, ReferenceSource=“mongodb”
item.guid—Used for item hash (resume key)
CVSS score textCVEMetric.baseScoreMetricType=“cvssV3_1”, ContainerType=“cna”
Affected product textCVEAffected.productBest-effort parsing from description
Affected version textCVEAffectedVersionBest-effort version range extraction
JIRA ticket IDCVEAlias.aliasCveIde.g. SERVER-50605

5. Identifier policy

  • CVEMetadata.cveId: The native CVE ID from the RSS title (e.g. CVE-2024-1351). Always uppercase.
  • CVEAlias: The JIRA ticket ID (e.g. SERVER-50605, RUST-591, COMPASS-7260) is stored as an alias. No bundle suppression needed — each RSS item maps to exactly one CVE.
  • No minted IDs: MongoDB bulletins natively emit CVE-prefixed IDs, so no minted-prefix or GCVE- issuance is required.

6. CRIT / VEX

Decision: CRIT staging is not included in the initial implementation. MongoDB Atlas is a cloud service, but the RSS feed does not provide structured (Provider, Service, ResourceType) triples with sufficient fidelity for reliable CRIT mapping. The affected product text is free-form prose requiring NLP-level extraction. CRIT staging can be added later if:

  1. MongoDB provides a structured API with explicit service/resource_type mappings, or
  2. We implement reliable heuristic classification of description text into Atlas vs Server vs Driver categories.

If added later, the dictionary would live at internal/critutil/dictionaries/extended/mongodb.json with:

  • Provider: mongodb
  • Services: atlas (cluster, serverless_instance), server (replica_set, sharded_cluster, standalone)

7. S3 / source-file archive layout

  • Bucket: $S3_BUCKET_NAME
  • Archive prefix: mongodb/files/{sha256}/{cveId}.json
  • Quarantine prefix: failed-feeds/mongodb-rss-processor/{YYYY-MM-DD}/{reason}/{cveId}.json
  • Payload schema: Normalised JSON of the mongodb.Advisory struct

8. Error handling & Slack

  • Per-record vs batch: Per-record failure semantics. One item failing does not abort the batch.
  • Retry profile: 3 attempts, exponential backoff. Transient connection errors (unexpected EOF, connection reset) get longer backoff.
  • Slack events:
    • Started — at processor startup
    • Completed — with stats {stored: N, errors: M}
    • Errored — if any per-record errors accumulated
    • NoWork — not used (feed is always processed; resume set handles skipping)

9. Performance

  • Concurrency: Single-threaded. RSS feed is small (~96 items) and batched releases are infrequent.
  • Per-request timeout: 90 seconds for HTTP fetch.
  • Rate-limit pacing: 200ms delay between items (polite to source).
  • Soft deadline: 10 minutes before EXPECTED_DURATION_MINUTES.
  • Resume strategy: db.LoadProcessedHashes at startup (key=cveId, value=itemHash). Tracker bump uses feed SHA256.

10. Backfill

No backfill binary is warranted. The full historical archive is available in the single RSS feed. Running the processor with --force reprocesses all items. The justfile recipe go-mongodb-rss-backfill supports this.

S3 Persistence

Not used. This processor does not currently archive payloads or quarantine failures to S3. Per the S3 Persistence Contract this is non-compliant — see the compliance matrix for the implementation roadmap.

⚠ Not in the compliance matrix — status needs verification.

Expected paths when implemented:

  • Archive: mongodb/files/{sha256}/{filename}
  • Quarantine: failed-feeds/mongodb-rss-processor/{YYYY-MM-DD}/{reason}/{filename}
  • Likely reasons: (none documented)