Design: SAP Patch Day Fetch Processor

Overview

Scrapes the public SAP Security Patch Day archive pages on support.sap.com. Per-Note bodies (linked at me.sap.com/notes/...) are auth-walled (S-user login required), but each row in the support archive table carries enough information to write a useful row: note ID, primary CVE, additional CVE list, title, product, version list, severity label, CVSS score.

Source identifier: sap Data type: fetch (HTML scraping; no public RSS / JSON / CSAF) ECS task name: go-sap-patch-day-fetch-processor ECS schedule: Every Tuesday 16:00 UTC (cron(0 16 ? * TUE *)) — picks up Patch Tuesday after publication Pattern: A — inline per-record CRIT staging Phase: 1.9

Two operating modes

archiveStartYear = 2021 — earliest year SAP exposes via the public archive URL pattern.

current 404s gracefully when the month’s page hasn’t been published yet (e.g. fetching may-2026.html on May 6, before that month’s Patch Tuesday). The processor logs “page not found, skipping” and continues.

Data Source

URLs

Per-row HTML shape

<tr>
  <td width="10%"><p><a href="https://me.sap.com/notes/3719353">3719353</a></p></td>
  <td width="70%">
    <p>[<a href="https://www.cve.org/CVERecord?id=CVE-2026-27681">CVE-2026-27681</a>] <b>SQL Injection vulnerability in SAP Business Planning and Consolidation and SAP Business Warehouse</b></p>
    <p><u>Product</u> - SAP Business Planning and Consolidation and SAP Business Warehouse<br />
    Version(s) - HANABPC 810, BPC4HANA 300, SAP_BW 750, 752, 753, 754, 755, 756, 757, 758, 816</p>
  </td>
  <td width="10%"><p>Critical</p></td>
  <td width="7%"><p><a href="https://www.first.org/cvss/calculator/3.0#CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H">9.9</a></p></td>
</tr>

The </u> is followed by a non-breaking space ( ) before the dash separator — the parser normalises NBSP to regular space at fetch time.

Multi-CVE rows

<p><u>Additional CVE</u> - <a href="https://www.cve.org/CVERecord?id=CVE-2026-0496">CVE-2026-0496</a>, <a href="https://www.cve.org/CVERecord?id=CVE-2026-0495">CVE-2026-0495</a></p>

Extracted into note.AdditionalCVEs.

Malicious-package rows (CVE-less)

<p><b>Malicious open-source packages in SAP Cloud Application Programming Model & MTA Build Tool</b></p>
<p><u>Product</u> - SAP Cloud Application Programming Model & MTA Build Tool<br />
Package versions: cap-js/sqlite - v2.2.2, ...</p>

PrimaryCVE is empty; Versions captures the package list. Stored under cveId = SAP-{noteID} with no CVE alias.

Architecture / Data Flow

Source → Database Mappings

CVEMetadata

CVEAlias

Primary CVE + every Additional CVE entry → db.InsertAliases(ctx, tx, "SAP-{noteID}", "sap", aliases, logger). Empty alias list still triggers the same-cveId backfill per AGENTS.md alias-write contract.

InsertAliases canonicalises the edge direction — CVE-* IDs become primaryCveId, SAP-{noteID} becomes aliasCveId.

CVEMetadataReferences

Always inserted:

1. https://me.sap.com/notes/{noteID} (advisory; SAP-Note canonical link, with title)
2. support.sap.com/.../{month/bulletin}-{year}.html (advisory; the page URL the row was scraped from)
3. https://www.cve.org/CVERecord?id={primaryCVE} (advisory) when primary CVE present
4. https://www.cve.org/CVERecord?id={cve} (advisory) — one per Additional CVE

CVEDescription

Single row, joining title + product + versions + severity/CVSS:

SQL Injection vulnerability in SAP Business Planning...
Product: SAP Business Planning and Consolidation and SAP Business Warehouse
Version(s): HANABPC 810, BPC4HANA 300, SAP_BW 750, 752, 753, ...
Severity: Critical (CVSS 9.9)

Columns and tables this producer does NOT write

The parser extracts more than the writer persists. These are gaps, not design:

S3 archive

sap/files/{sha256}/SAP-{noteID}.json

Carries the full ParsedNote struct as JSON for downstream re-processing.

Service Mapping

service_map.go:Resolve walks the row’s Product text and returns the first matching (service, resource_type) tuple.

Synonym table (initial)

Extended-dict layer (internal/critutil/dictionaries/extended/sap.json): 16 entries — netweaver, abap_platform, s4hana (cloud-tenant + on-premise), business_one, commerce_cloud, successfactors, concur, ariba, fieldglass, customer_data_cloud, signavio, solution_manager, business_objects, hana_database, gateway. Templates use template_format ∈ {sap_btp_url, sap_odata_url, sap_sf_url} — all from the spec’s allowed enum.

The spec dict’s sap.json (55 entries / 12 services) covers SObject-record-level resources (S/4HANA company codes, SuccessFactors employees, BTP subaccounts) — disjoint from this product-cloud layer.

When the product text matches no synonym, the producer logs "crit skipped: unmapped product" with the product string. CVEMetadata + references + description still land. Operators can grow the synonym table by inspecting these warnings.

Business Rules

URL planning (fetch.go:PlanURLs)

NBSP normalisation (parser.go:ParsePage)

Row extraction (parser.go:ParsePage)

Hash + resume (main.go:noteHash)

CRIT mapper (crit_mapper.go:mapSAPNoteToCRIT)

fix_propagation inference (crit_mapper.go:inferFixPropagation)

Without auth-walled body content, this is heuristic — driven by service shape + severity rather than text patterns.

Confidence ladder (crit_mapper.go:confidenceFor)

Auth-walled body means we can’t text-match remediation language. Confidence ladder reflects this:

Coupling rules (shared with Phase 1.x producers)

• R28 automatic ⇒ provider_only.
• R29 existing_deployments_remain_vulnerable = !(provider_only && automatic).

Drain (main.go)

• R30 Drain skipped on CRIT_DISABLE_INPROCESS_DRAIN.
• R31 4 workers.

Quarantine (main.go:storeNote)

• R32 Failed transaction → quarantine the payload | if !storeNote(...) { uploader.QuarantineRecord(ctx, "store-error", payload); continue }.
• R33 Archive only on success | if storeNote ok { uploader.ArchiveRecord(...) } — failed stores never appear in sap/files/.

Verification Queries

-- Note count
SELECT count(*) FROM "CVEMetadata" WHERE source='sap';
-- expected (after backfill): ≥400 (covers 2021..present, ~80-300 notes/year)

-- SAP-* primary keys
SELECT count(*) FROM "CVEMetadata"
 WHERE source='sap' AND "cveId" LIKE 'SAP-%';

-- CVE alias edges
SELECT count(*) FROM "CVEAlias" WHERE "discoveredFrom"='sap';
-- expected: high — every CVE'd note produces 1+ edges per source-pair

-- Reference coverage (every row has ≥1)
SELECT count(*) FROM "CVEMetadataReferences"
 WHERE source='sap'
   AND "cveId" IN (SELECT "cveId" FROM "CVEMetadata" WHERE source='sap')
 GROUP BY "cveId" HAVING count(*) = 0;
-- expected: 0 rows

-- Pending should be empty
SELECT "processingStatus", count(*) FROM "S3QueueObject"
 WHERE source='sap-patch-day-fetch-processor' GROUP BY 1;
-- expected: only `inserted`

-- Service distribution — historical SAP Patch Day pattern
SELECT service, count(*) FROM "CritRecord"
 WHERE provider='sap' GROUP BY 1 ORDER BY 2 DESC;
-- expected weighting: netweaver dominant, then s4hana, business_objects,
-- commerce_cloud, business_one, solution_manager, hana_database

Risk Surface

S3 Persistence

• Archive path: sap/files/{sha256}/{filename} ✓
• Quarantine path: failed-feeds/sap-patch-day-fetch-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
• Failure reasons emitted: store-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

See the S3 Persistence Contract for the full reason taxonomy.