CERT-BE Fetch Processor — Design

Overview

Fetches security advisories from the Belgian Centre for Cybersecurity (CERT-BE / CCB). The RSS feed provides item URLs and titles, but the description is hardcoded (“CCB Advisories”). Each advisory page is fetched individually to extract CVE IDs, CVSS vectors, CWE IDs, affected products, and descriptions.

Records are stored under original CVE IDs (source=cert-be). Items without CVE IDs are skipped.

Feed

Property	Value
URL	`https://ccb.belgium.be/advisories.xml`
Auth	None — fully public
Format	RSS 2.0 with Atom namespace
Language	English
Items	~10 advisories
Note	`<description>` is hardcoded “CCB Advisories” — must fetch linked pages

Page Structure

Advisory pages use Drupal HTML with two key content regions:

Date div: <div class="field--node-field-advisories-date">Published : DD/MM/YYYY</div>
Content div: <div class="field--node-field-advisories-content"> containing:
- Blockquote header: Last update, Affected software, Type (CWE), CVE/CVSS
- h2 sections: Sources, Risks, Description, Recommended Actions, References

Parsing

Blockquote Metadata Header

Field	Format
Last update	`<li><strong>Last update</strong>: DD/MM/YYYY</li>`
Affected software	`<li><strong>Affected software:</strong><br/>→ Product 1<br/>→ Product 2</li>`
Type	`<li><strong>Type</strong>:<br/>→ CWE-35: Path Traversal</li>`
CVE/CVSS	`<li><strong>CVE/CVSS</strong><br/>→ CVE-2026-22557: CVSS 10 (CVSS:3.1/AV:N/...)</li>`

h2 Sections

Heading	Data Extracted
Sources	Vendor advisory URLs with labels
Risks	Risk assessment text
Description	Vulnerability description text
Recommended Actions	Patch/monitor guidance text
References	NVD/CVE.org URLs

Storage

No new tables or columns. All tables already exist.

Table	Rows inserted
`CVEMetadata`	One per CVE ID; `source="cert-be"`
`CVEDescription`	One per CVE; `lang="en"`
`CVEMetadataReferences`	Advisory URL + vendor/technical refs
`CVEMetric`	Per-CVE CVSS v3.1/v4.0 vectors with scores
`CVEProblemType`	CWE IDs from “Type” field
`CVEAffected`	Products from “Affected software” field
`CVEAlias`	Cross-references between CVEs in same advisory — only for two-CVE advisories

Per-CVE vectors are written to CVEMetric.vectorString but not to the denormalised CVEMetadata.vectorString column — the mapper never sets CVESourceData.VectorString. Consumers reading the metadata column see NULL for every cert-be row, which is the cheapest way to consume the one thing this source uniquely provides.
Multi-CVE advisories emit CVE↔CVE aliases from the mapper, but db.InsertAliases suppresses them: when a CVE-prefixed primary id carries more than one CVE-prefixed alias the list is treated as bundling and the CVE entries are dropped (internal/db/cvealias.go:65). The rows are still cross-linked to their peers in other sources by the same-cveId backfill.

Incremental Strategy

On startup, load all sourceAdvisoryRef values from CVEMetadata where source='cert-be' into a map[string]bool. Per advisory: if URL is in the set and --all/--force is false, skip. Rate limit: 200ms between page fetches.

Flags

Flag	Default	Description
`--all`	false	Reprocess all advisories
`--limit`	0	Maximum advisories to process (0 = unlimited)
`--force`	false	Force reprocessing

ECS Schedule

Currently DISABLED (schedule_enabled = false in terraform/go-schedules.tf). ccb.belgium.be blocks AWS egress at the IP level — a Fargate task in the scheduler’s own subnet gets HTTP 403 on every path (including the site root), while the identical client from a residential IP gets 200. It is not a User-Agent/header problem, so no code change fixes it; the task’s last runs errored and ingested nothing. The cron below is retained for when the feed is reachable again from AWS (or the fetch is routed through non-datacenter egress):

Nominal schedule: once daily (cron(0 6 * * ? *)), expected_duration_minutes = 60.

The feed itself is healthy from a non-datacenter IP (verified 2026-08-06: HTTP 200, 10 items), so a local backfill still works — 297 rows are stored and lastFetchedAt stops at 2026-06-30, ten days before the schedule was turned off.

Error Alerting

Feed fetch or parse failure is fatal (notifier.Errored + os.Exit(1)). Per-advisory page-fetch failures and per-CVE transaction failures both call notifier.RecordError, but the run always ends in notifier.Completed, which does not flush RecordError messages (internal/notify/notify.go:213). Because the 403 wall produces a page-fetch failure per item rather than a feed-level error, a completely blocked run reported task.completed with stored: 0 and every 403 buried in the logs.

Key Files

File	Purpose
`cmd/cert-be-fetch-processor/main.go`	Main processor with page fetching
`internal/certbe/types.go`	RSS feed, advisory, CVE entry, CWE entry structs
`internal/certbe/parser.go`	Feed parsing, HTML page scraping, section extraction
`internal/certbe/mapper.go`	Advisory → CVESourceData mapping
`schemas/cert_be_fetch_advisory.schema.json`	JSON Schema Draft 7

S3 Persistence

Archive path: cert-be/files/{sha256}/{filename} ✓
Quarantine path: failed-feeds/cert-be-fetch-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
Failure reasons emitted: parse-error, store-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

flowchart LR SRC[Source feed] --> PROC[cert-be-fetch-processor] PROC -->|success| ARCHIVE[("S3: cert-be/files/{sha256}/{filename}")] PROC -->|failure| Q[("S3: failed-feeds/cert-be-fetch-processor/{date}/{reason}/{filename}")] PROC --> DB[(PostgreSQL)]

See the S3 Persistence Contract for the full reason taxonomy.