ACSC Alert RSS Processor — Design

Overview

Fetches security alerts from the Australian Cyber Security Centre (ACSC) RSS 2.0 feed (https://www.cyber.gov.au/rss/alerts) and creates first-class CVEMetadata rows (source=acsc) with minted ACSC-YYYY-N identifiers.

CVE IDs embedded in alert titles and descriptions are extracted and stored as GcveAlias and CVEAlias records linking them to the minted identifier.

Feed

We consume the smallest of three ACSC channels. Verified live 2026-08-06: /rss/alerts serves 2 items, /rss/advisories serves 3 (e.g. “Russian state-supported cyber actors … Zimbra Collaboration Suite”, “ClickFix distributing Vidar Stealer via WordPress targeting Australian infrastructure”), and /rss/news serves 14. Only /rss/alerts is in feedURL (main.go:37), so the advisories channel — which is squarely vulnerability/threat content — is not ingested at all. Because both feeds are rolling windows and the poll is daily, anything that enters and leaves a window between two runs is lost permanently.

Parsing

ID Generation

Each advisory is assigned a ACSC-{year}-{seq} identifier:

1. On startup, query MAX(sequenceNumber) from GcveIssuance where gcveId LIKE 'GCVE-110-ACSC-{year}-%'
2. For each new advisory, increment the sequence counter for the advisory’s publication year
3. Store ACSC-YYYY-N as CVEMetadata.cveId and GCVE-110-ACSC-YYYY-N as GcveIssuance.gcveId
4. Link extracted CVE IDs and advisory slug as GcveAlias records
5. Store CVE IDs in CVEAlias table linking them to the ACSC identifier

Two hazards follow from minting an identifier per run rather than deriving one from the advisory:

• --all duplicates advisories rather than refreshing them. The dedup check is if !*all && processedURLs[advisory.URL] (main.go:148), so with --all the run falls through to seqByYear[year]++ (main.go:159) and allocates a new ACSC-YYYY-N for an advisory that already has one. Observed: the 2026-08-06 18:30 AEST prod run minted ACSC-2026-0008 and ACSC-2026-0009 for the same two sourceAdvisoryRef URLs already held by ACSC-2026-0007 and ACSC-2026-0006. There is no unique constraint on (source, sourceAdvisoryRef) to catch it. The twins also disagree on datePublished by exactly 36,000s — the AEST offset — so one of each pair has the wrong publication time.
• Only two years of sequence state are loaded. main.go:114 iterates y := currentYear - 1; y <= currentYear, but the year is taken from the advisory’s own pubDate (main.go:155). An item dated 2024 gets seqByYear[2024] == 0 and is therefore minted as ACSC-2024-0001, colliding with whatever already holds that id — and UpsertCVEMetadata will overwrite it rather than fail.

Storage

No new tables or columns. All tables already exist.

Incremental Strategy

On startup, load all sourceAdvisoryRef values from CVEMetadata where source='acsc' into a map[string]bool. Per advisory: if the URL is in the set and --all is false, skip.

Flags

HTTP Client

cyber.gov.au sits behind Lagoon + Signal Sciences (SigSci/Fastly) WAF. SigSci’s IP reputation service flags AWS datacenter IP ranges as “cloud/hosting” and silently hangs HTTP responses (TCP connects, TLS completes, request is sent — but response headers never arrive). Browser header spoofing (Sec-Fetch-*, Accept-Language, etc.) addresses TLS fingerprinting checks but does not bypass IP reputation scoring.

This is no longer the case. The schedule is ENABLED — verified live with aws scheduler get-schedule --name go-acsc-rss-processor (State ENABLED, cron(30 15 * * ? *)) — and the Fargate runs fetch the feed successfully: the 2026-08-05 15:30 UTC run in /ecs/vdb-scheduler/acsc-rss-processor logs "feed fetched","items":2 followed by Outcome=success in 0.13s. Whatever SigSci was doing to us, the forced HTTP/1.1 client plus the full browser header set below appears to have resolved it. Keep the hardening; treat the paragraph above as history.

The local recipe (just go-acsc-rss-backfill) remains useful for reprocessing — but see the --all hazard under ID Generation before using ALL=true against production.

The HTTP client uses HTTP/1.1 (Lagoon sends HTTP/2 INTERNAL_ERROR stream resets to Go’s HTTP/2 implementation), 90s timeout, 30s dial, ResponseHeaderTimeout: 60s, full browser headers, gzip decompression, and up to 5 retries with attempt * 5s backoff. The feed is fetched before opening the DB pool because pgx background activity was observed to cause additional hangs.

ECS Schedule

ENABLED — daily at 15:30 UTC, cron(30 15 * * ? *) (terraform/go-schedules.tf:2904), 256 CPU / 512 MB, expected_duration_minutes = 30. Typical run is ~0.13s and stores nothing, because a 2-item feed is almost always unchanged.

The soft deadline is defaultTimeout = 30 * time.Minute unconditionally (main.go:42, main.go:62), so unsetting EXPECTED_DURATION_MINUTES in the backfill recipe does not remove it. Harmless at this feed size, but it is the same hardcoded-margin pattern internal/rundeadline.Soft exists to replace.

Key Files

S3 Persistence

Compliant. The generator printed “Not used” here because scripts/docs/s3-status.yaml recorded status: none for this slug; that entry was wrong (cf. ORCH-09) and has been corrected.

• Archive: acsc/files/{sha256}/{ACSC-YYYY-N}.json — uploader.ArchiveRecord(ctx, source, payload) at main.go:181, success path only, payload built by s3client.MarshalRecord at main.go:168 before the transaction opens so no S3 PUT holds a DB connection.
• Quarantine: failed-feeds/acsc-rss-processor/{YYYY-MM-DD}/{reason}/…
• Failure reasons emitted: parse-error for the whole feed body (main.go:84, before the DB pool is opened) and store-error per advisory (main.go:176). fetch-error is not emitted — a fetch that fails all 5 attempts exits 1 at main.go:79 with no payload to store, which is contract-consistent.

Skipped entirely when S3_BUCKET_NAME is unset — s3client.NewFromEnv returns a nil uploader and both methods are nil-receiver no-ops.