Design: ABB CSAF Processor

Overview

Fetches ABB PSIRT security advisories published in CSAF 2.0 (Common Security Advisory Framework) JSON format from the public TLP:WHITE feed index and stores the parsed vulnerability data into the VDB PostgreSQL database.

Source identifier: abb Data type: csaf ECS task name: go-abb-csaf-processor Schedule: Runs weekly on Fridays at 04:00 UTC (cron(0 4 ? * FRI *)).

Data Source

Feed Index Structure

{
  "feed": {
    "id": "abb-security-advisories-csaf-feed-tlp-white",
    "entry": [
      {
        "id": "3adr011536",
        "title": "Cyber Security Advisory - AC500 V3 Stack buffer overflow...",
        "published": "2026-03-12T12:00:00Z",
        "updated": "2026-03-12T12:00:00Z",
        "link": [
          { "rel": "self", "href": "https://psirt.abb.com/csaf/2026/3adr011536.json" },
          { "rel": "hash", "href": "https://psirt.abb.com/csaf/2026/3adr011536.json.sha256" }
        ],
        "content": { "src": "https://psirt.abb.com/csaf/2026/3adr011536.json", "type": "application/json" }
      }
    ]
  }
}

CSAF Advisory Structure

Each advisory JSON follows the CSAF 2.0 standard:

• document.tracking.id — Advisory ID (e.g. 3ADR011536)
• document.tracking.initial_release_date / current_release_date — Publication dates
• document.title — Advisory title
• document.references[] — Document-level references (PDF + CSAF self-links + external)
• document.notes[] — Summary, support info, FAQ, recommendations
• vulnerabilities[] — One or more CVE entries, each with:
  • cve — CVE identifier
  • scores[] — CVSS v3 scoring (v3.1 with temporal/environmental metrics)
  • notes[] — Descriptions
  • cwe — Singular CWE identifier (ABB uses singular cwe field, not array cwes)
  • product_status.known_affected[] — Affected product IDs
  • references[] — Per-vulnerability references (NVD links)
  • remediations[] — Fix/workaround/mitigation information
• product_tree — Hierarchical product catalog (vendor→family→product→version with relationships)

Data Mapping

CVEMetadata (one row per CVE per advisory)

CVEDescription

• Source: vulnerabilities[].notes where category is description, summary, or general
• Falls back to document.notes if vulnerability has no matching notes
• containerType = "cna", lang = "en"

CVEMetadataReferences

All references are deduplicated by URL across document + vulnerability + remediation levels:

ReferenceSource = "abb" on all rows.

CVEMetric

One row per unique CVSS vector string:

CVEProblemType

Note: ABB uses singular cwe object per vulnerability. The mapper also supports the standard cwes array if present.

CVEAffected

One row per unique product name in product_status.known_affected:

• vendor = "ABB"
• product = resolved from product_tree by product ID
• affectedHash = MD5(vendor|##|product|##||##|)
• containerType = "cna"

Artifact + Link

• Artifact: type = "OTHER", bomFormat = "abb", R2 bucket/key set
• S3 key: abb/advisories/{sha256}/{cveId}.json
• Link: contentType = "PLAIN_JSON", URL = advisory self-link
• CVEMetadata.fileLinkId points to the Link record

Incremental Strategy

1. Feed-level: SHA256 of the full feed index JSON stored in BulkDataDumpTracker with key abb_csaf_feed. Run is skipped entirely if unchanged (unless --force).
2. Advisory-level: SHA256 of each advisory JSON stored in CVEMetadata.sourceFileHash. Advisory is skipped if hash matches existing record (unless --force).

This means ECS runs are O(1) network + O(changed) DB writes on most days.

DEFERRED advisories

A feed entry whose advisory URL returns a permanent HTTP error (403 / 404 / 410) is not retried. Instead storeDeferred (main.go:402) writes a placeholder CVEMetadata row keyed on the feed entry id with state='DEFERRED', dataVersion='5.0', sourceAdvisoryRef set and nothing else. Because osv.CVESourceData.DatePublished is nil on that path, the row lands with datePublished = 0.

Note that UpsertCVEMetadata overwrites title and rawDataJSON with EXCLUDED values on conflict, so an advisory that was previously fetched successfully and later starts returning 403 has its title and raw CSAF replaced with NULL. ABB currently has no DEFERRED rows in production, but the code path is identical to the Siemens one, where it has caused exactly that.

Flags

Processing Pipeline

main()
 ├─ Fetch feed index (5-attempt retry, 30 s per attempt)
 ├─ Parse feed → []FeedEntry
 ├─ Check feed SHA256 vs BulkDataDumpTracker → skip if unchanged (NoWork)
 ├─ db.LoadProcessedHashes(source="abb") → resume map (cveId → sha256)
 ├─ Dispatcher goroutine → workCh
 ├─ N=--workers fetch workers, each:
 │   ├─ stop dispatching once (softDeadline − 10 min) has passed
 │   ├─ wait for the shared 200 ms ticker
 │   ├─ Fetch advisory JSON (5 attempts; 403/404/410 → DEFERRED, no retry)
 │   ├─ abb.ParseAdvisory()  (failure → S3 quarantine `parse-error`)
 │   ├─ abb.MapAdvisory() → []*osv.CVESourceData (one per CVE in the advisory)
 │   └─ Skip when resumeSet[items[0].CveID] == advisory SHA256
 ├─ Single consumer goroutine, per item:
 │   └─ storeItem: one transaction per item, up to 3 attempts
 │       ├─ S3 upload → InsertArtifact → InsertLinkWithArtifact
 │       ├─ processor.StoreCVESourceData() [shared pipeline]
 │       └─ UpdateCVEMetadataFileLinkID() [best-effort]
 └─ db.UpsertTracker(abb_csaf_feed, newSHA256, processedCount)

There is no 50-item batch and no per-item savepoint: each mapped CVE gets its own short transaction with a 2-minute context and a 3-attempt retry loop that sanitises invalid UTF-8 in rawDataJSON between attempts.

Soft deadline

softDeadline is now + (EXPECTED_DURATION_MINUTES − 10) when the env var is set, else now + 30 min unconditionally (main.go:59-63). Workers then stop dispatching at softDeadline − 10 min (main.go:176), so the effective work window is EXPECTED_DURATION_MINUTES − 20. The unconditional 30-minute fallback also applies to just go-abb-csaf-backfill, which is why a local full backfill stops dispatching after 20 minutes.

Files

Local Development

# Incremental run (default — processes only new/changed advisories)
just go-abb-csaf-backfill

# Full backfill of all advisories
just go-abb-csaf-backfill FORCE="true"

# Limit to first 5 advisories (quick test)
just go-abb-csaf-backfill LIMIT="5"

# Run in prod environment
just go-abb-csaf-backfill TARGET="prod" LIMIT="5"

# Run in local ARM64 container (mirrors ECS)
just abb-csaf-processor

Deployment

The processor is built as part of Containerfile.go-processors (target: abb-csaf-processor) and pushed to ECR as go-abb-csaf-processor-latest. The ECS task definition and EventBridge schedule are managed in terraform/go-schedules.tf under module "abb_csaf_processor".

Known Limitations

• ABB advisories use a singular cwe object per vulnerability rather than the standard CSAF cwes array. The mapper handles both formats.
• Some advisories may contain a single advisory ID with multiple CVEs. Each CVE is stored as a separate CVEMetadata row sharing the same raw advisory JSON and S3 artifact.
• Product names resolved from product_tree may include relationship-derived names (e.g. “ABB AC500 V3 Firmware 3.9.0 installed on ABB AC500 V3 PM5xxx”). The affectedHash deduplicates by vendor+product pair.

S3 Persistence

• Archive path: abb/files/{sha256}/{filename} ✓
• Quarantine path: failed-feeds/abb-csaf-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
• Failure reasons emitted: parse-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

See the S3 Persistence Contract for the full reason taxonomy.