CERT-AU (AUSCERT) RSS Processor — Design

Overview

Fetches security bulletins from the AUSCERT (Australian Computer Emergency Response Team) RSS 2.0 feed and creates CVEMetadata rows (source=cert-au) under original upstream CVE IDs.

AUSCERT is a non-profit CERT that redistributes advisories from upstream vendors (Red Hat, Ubuntu, SUSE, Debian, Cisco, Microsoft, etc.) with standardised summaries including CVSS and EPSS data. Each bulletin always references one or more upstream CVE IDs. Items without CVE IDs are skipped.

Feed

Payload Format

The <content:encoded> field is not HTML content. It contains an HTML-escaped <pre> block whose inner text is AUSCERT’s proprietary fixed-width bulletin format:

===========================================================================
             AUSCERT External Security Bulletin Redistribution

                               ESB-2026.3146
                        389-ds-base security update
                               31 March 2026

===========================================================================

        AUSCERT Security Bulletin Summary
        ---------------------------------

Product:           389-ds-base
Publisher:         Red Hat
Operating System:  Red Hat
Resolution:        Patch/Upgrade
CVE Names:         CVE-2025-14905

Original Bulletin:
   https://access.redhat.com/errata/RHSA-2026:6220

Comment: CVSS (Max):  7.2 CVE-2025-14905 (CVSS:3.1/AV:N/AC:L/PR:H/UI:N/S:U/C:H/I:H/A:H)
         CVSS Source: Red Hat
         Calculator:  https://www.first.org/cvss/calculator/3.1#CVSS:3.1/...

         EPSS (Max):  0.5% (64th) CVE-2025-14905 2026-03-30

Multi-CVE bulletins have a multi-line CVE Names: block:

CVE Names:         CVE-2025-14905 CVE-2025-14906
                   CVE-2025-14907

Continuation lines start with whitespace and contain additional CVE IDs until a blank line or the next key-value field.

Parsing

Step 1 — RSS XML unmarshalling

Standard RSS 2.0 XML. The <content:encoded> field is bound via the http://purl.org/rss/1.0/modules/content/ namespace.

Step 2 — Payload extraction

1. HTML-unescape the content:encoded value (handles <, &, etc.)
2. A second unescape pass handles double-encoded entities (rare)
3. Strip the <pre> / </pre> wrapper
4. Iterate lines with a state machine

Step 3 — State machine line parsing

Step 4 — Comment block extraction

parseCommentLine() applies three regexes to each line:

Field summary

ID Strategy

One CVEMetadata row is created per AUSCERT bulletin, using the ESB advisory ID (e.g. ESB-2026.3146) as the primary cveId. All CVE IDs referenced in the bulletin (CVE-YYYY-NNNN) are stored as CVEAlias entries pointing back to the ESB ID.

Two skip conditions apply, in this order:

1. Bulletins whose parsed CVEIDs list is empty are skipped before mapping (cmd/cert-au-rss-processor/main.go:131) — nothing is stored for them.
2. Bulletins with no ESB ID return nil from MapAdvisory (internal/certau/mapper.go:16) and are counted as noCVEs.

Because the primary id is ESB-… rather than CVE-…, the multi-CVE bundle suppression in db.InsertAliases does not apply: every CVE listed in the bulletin gets an alias edge. Edges are only written for CVE ids that already have a CVEMetadata row from some other source — a CVE that no other source has seen yet is skipped rather than materialised as a placeholder.

Storage

No new tables or columns. All tables already exist.

Alias fan-out. MapAdvisory passes the bulletin’s whole CVE list through as Aliases (internal/certau/mapper.go:34). db.InsertAliases then writes one edge per (alias CVE, source that carries it) pair, so a bulletin listing N CVEs produces roughly N × (number of sources holding those CVEs) rows. AUSCERT redistribution bulletins routinely list hundreds of CVEs — the largest currently stored lists 1,210 — and the multi-CVE bundle guard in db.InsertAliases does not apply here because it only fires when the primary id is itself CVE-prefixed. cert-au accounts for ~373k CVEAlias rows as a result.

The parsed CVSS vector is written to CVEMetric.vectorString but not to the denormalised CVEMetadata.vectorString column — the mapper never sets CVESourceData.VectorString. Consumers that read the metadata column (rather than joining CVEMetric) see NULL for every cert-au row.

Reference type classification

The OriginalBulletinURL is classified by domain:

Incremental Strategy

On startup, load all sourceAdvisoryRef values from CVEMetadata where source='cert-au' into a map[string]bool. Per bulletin: if the URL is in the set and --all/--force is false, skip. After successful processing, add the URL to the in-memory set.

Resume is URL-based (not hash-based) because AUSCERT bulletin URLs are stable and permanent. The --force flag re-stores all bulletins regardless of whether they’ve been seen before.

Flags

HTTP Client

Standard httpclient.New(30s) with browser headers. The AUSCERT portal is publicly accessible without IP restrictions; no special HTTP/1.1 forcing is required. Feed is fetched before database connection. Up to 3 retries with attempt * 2s backoff.

ECS Schedule

Runs once daily (cron(0 6 * * ? *)). The rolling-window feed returns ~100 items; deduplication ensures only new bulletins are stored on each run.

Key Files

S3 Persistence

• Archive path: cert-au/files/{sha256}/{filename} ✓
• Quarantine path: failed-feeds/cert-au-rss-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
• Failure reasons emitted: parse-error, store-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

See the S3 Persistence Contract for the full reason taxonomy.