SUSE CSAF Processor

Data Source

Publisher: SUSE Product Security Team
Format: CSAF 2.0 JSON advisories + CSAF VEX documents
CSAF Index: https://ftp.suse.com/pub/projects/security/csaf/index.txt
CSAF Base URL: https://ftp.suse.com/pub/projects/security/csaf/
VEX Index: https://ftp.suse.com/pub/projects/security/csaf-vex/index.txt
VEX Base URL: https://ftp.suse.com/pub/projects/security/csaf-vex/
Volume: ~29 000 CSAF advisories, ~53 000 VEX documents (the index grows continuously; ingestion is still catching up)
Update frequency: Daily; existing advisories may be revised
Language: English (en)

Index Format

CSAF Index

Plain text file with one filename per line:

opensuse-ru-2026_20010-1.json
opensuse-su-2024_0331-1.json
suse-su-2024_1234-1.json

Advisory prefixes:

opensuse-su — openSUSE security updates
opensuse-ru — openSUSE recommended updates
suse-su — SLES/SLED security updates

VEX Index

Plain text file with one filename per line (lowercase CVE-based):

cve-1999-0002.json
cve-2024-38528.json

CSAF Advisory Structure

Standard CSAF 2.0 with category: "csaf_security_advisory":

Publisher: SUSE Product Security Team, category “vendor”
Language: English
Product status: Uses recommended array (not known_affected in many cases); processor handles both known_affected, last_affected, and recommended
CVSS scores: May or may not be present depending on advisory
CWE identifiers: May or may not be present
Product tree: Hierarchical vendor/product/arch branches with relationships
Aggregate severity: English text (moderate, important, critical, etc.)
Remediations: vendor_fix category with YaST/zypper installation instructions

VEX Format

CSAF 2.0 with category: "csaf_vex":

Tracking ID: CVE identifier (e.g., “CVE-2024-38528”)
Product tree: Extensive product branches with CPE and PURL identifiers
Revision history: Can have 40+ revisions for actively tracked CVEs
References: Links to related SUSE advisories and CVE pages
Vulnerabilities: May or may not include a vulnerabilities array

VEX Storage

VEX files are stored as S3 artifacts only (no OpenVex model records):

Artifact type: VEX, bomFormat: suse
S3 key: suse/vex/{sha256}/{cveId}.json
Link record created with VEX source URL
VEX URL added as reference on CVEMetadata (type=“advisory”)

Data Mapping

CSAF Field	CVEMetadata Field
`vulnerabilities[].cve`	`cveId`
`"suse"`	`source`
`"5.0"`	`dataVersion`
`"PUBLISHED"`	`state`
`tracking.initial_release_date`	`datePublished` (Unix seconds)
`tracking.current_release_date`	`dateUpdated` (Unix seconds)
First CVSS vector (if any)	`vectorString`
`document.title`	`title`
Advisory URL	`sourceAdvisoryRef`
Vendor from product tree	`affectedVendor` (default: “SUSE”)
First affected product name	`affectedProduct`
SHA256 of advisory JSON	`sourceFileHash`
Full CSAF document	`rawDataJSON`

Relations

CVEDescription: Vulnerability notes (category: description/summary/general), fallback to document notes; lang: "en"
CVEMetadataReferences: Advisory URL + document refs + vulnerability refs + remediation URLs + VEX URL (if correlated)
CVEMetric: CVSS v2/v3/v4 from vulnerability scores (if present)
CVEProblemType: CWE identifiers (if present)
CVEAffected: Products from product_status.known_affected + last_affected + recommended, vendor extracted from product tree branches

S3 Artifacts

Artifact	Type	S3 Key Pattern
CSAF advisory	`OTHER`	`suse/advisories/{sha256}/{cveId}.json`
VEX document	`VEX`	`suse/vex/{sha256}/{cveId}.json`

Incremental Processing

Fetch index.txt, compute SHA256
Compare against BulkDataDumpTracker key suse_csaf_index
If unchanged and --force not set, exit early (reports task.nowork)
Fetch VEX index and cache in memory as map[string]bool (a VEX-index fetch failure disables correlation for the run but does not fail it)
Load per-CVE sourceFileHash resume set from DB
URL pre-filter — db.LoadProcessedAdvisoryRefs(source='suse') returns every sourceAdvisoryRef already stored, and any index filename whose advisory URL is already present is dropped before it is ever fetched (main.go:151-165). See the caveat below.
Per surviving advisory (fetched concurrently by --workers goroutines behind a shared 500 ms ticker ≈ 2 req/s): compute SHA256, skip the store if the hash matches the record for the advisory’s first CVE
Per CVE: look the lowercased {cveId}.json up in the cached VEX index; fetch and archive the VEX document when present, and append its URL as an advisory reference on every CVE in the advisory
Store each mapped CVE in its own transaction (2-minute context, up to 3 attempts) — there is no batch commit
Update tracker on completion

Revisions are not re-ingested. Step 6 filters on advisory URL, and SUSE revises an advisory in place at the same URL (suse-su-2024_1234-1.json). Once a URL is stored, no scheduled run ever fetches it again, so the per-advisory sourceFileHash check in step 7 can never fire for it. Picking up revisions requires --force, which bypasses the pre-filter.

Soft deadline

softDeadline = now + (EXPECTED_DURATION_MINUTES − 10), defaulting to now + 120 min when the env var is unset; workers stop dispatching at softDeadline − 10 min. The ECS budget is 60 minutes (terraform/go-schedules.tf, scripts/task-manager.toml), so the effective work window is 40 minutes — well short of the ~29 k advisories the CSAF index lists, which is why the source is still catching up (8 305 suse rows in production as of 2026-08). The unconditional 120-minute fallback also caps just go-suse-csaf-backfill at 110 minutes.

DEFERRED advisories

An index entry whose advisory URL returns 403 / 404 / 410 is stored as a placeholder row keyed on the advisory id: state='DEFERRED', sourceAdvisoryRef set, no title, no raw JSON, datePublished = 0. Production currently holds zero DEFERRED suse rows.

Flags

Flag	Default	Description
`--force`	`false`	Reprocess all advisories regardless of hash, and bypass the URL pre-filter
`--limit`	`0`	Max advisories to fetch (0 = all)
`--workers`	`3`	Concurrent fetch workers sharing one 500 ms ticker (≈2 req/s)

Files

Path	Purpose
`cmd/suse-csaf-processor/main.go`	Entry point, fetch/parse/batch loop with VEX correlation
`cmd/suse-csaf-processor/s3.go`	S3 artifact upload helper
`internal/suse/types.go`	CSAF 2.0 type definitions + SUSE constants
`internal/suse/parser.go`	Index, VEX index, and advisory parsing
`internal/suse/mapper.go`	CSAF -> CVESourceData conversion

Local Development

# Direct Go run (fastest for testing)
just go-suse-csaf-backfill

# With production DB
just go-suse-csaf-backfill TARGET=prod

# Force reprocess all
just go-suse-csaf-backfill FORCE=true

# Limit to 10 advisories
just go-suse-csaf-backfill LIMIT=10

# Container build (mirrors ECS)
just suse-csaf-processor

Deployment

ECS schedule: Daily at 03:30 UTC (cron(30 3 * * ? *))
Resources: 256 CPU / 512 MB memory
expected_duration_minutes: 60
ECR tag: go-suse-csaf-processor-latest
Log group: /ecs/vdb-scheduler/go-suse-csaf-processor

S3 Persistence

Archive path: suse/files/{sha256}/{filename} ✓
Quarantine path: failed-feeds/suse-csaf-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
Failure reasons emitted: parse-error, schema-violation

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

flowchart LR SRC[Source feed] --> PROC[suse-csaf-processor] PROC -->|success| ARCHIVE[("S3: suse/files/{sha256}/{filename}")] PROC -->|failure| Q[("S3: failed-feeds/suse-csaf-processor/{date}/{reason}/{filename}")] PROC --> DB[(PostgreSQL)]

See the S3 Persistence Contract for the full reason taxonomy.