atlassian-json-processor

Status: Live Source: api.atlassian.com/vuln-transparency/v1/products Type: json (single unauthenticated document, no pagination) Source slug: atlassian Schedule: Runs daily at 06:00 UTC (cron(0 6 * * ? *)) with --emit-crit=true, 256 CPU / 512 MB, expected_duration_minutes = 15.

Overview

Atlassian is a CNA, so most of these CVEs also exist under cve.org — what the Transparency API adds is Atlassian’s own per-product, per-version view: which Jira / Confluence / Bitbucket / Bamboo builds a CVE actually lands in, plus the atl_tracking_url into the public Jira ticket. That is the data a Data Center / Server operator needs to decide whether their pinned build is affected, and it is not in the CVE record.

The endpoint is not the documented one. /vuln-transparency/v1/cves still answers 200 — permanently, with content-length: 0. A client reading it sees a successful run that ingests nothing, which is exactly how this feed silently went empty. The processor reads /v1/products instead, and an empty or cve_metadata-less payload is raised as an error rather than an empty success (internal/atlassian/products.go:16-20, 57-67).

The /v1/products shape is two cross-referenced maps, not a list: cve_metadata keyed by CVE id, and products → versions → [{CVE: state}]. ParseProductsPayload inverts the second into CVE → [products] and emits the same Resource struct the mapper already consumed, sorted by CVE id so repeated runs hash and archive identically (products.go:69-106).

The payload has no CVSS vector field, but the description states the vector verbatim (“with a CVSS Vector of CVSS:3.1/…”), so it is recovered with a regex (products.go:46-48).

Records produced

Condition	Records
Every CVE in `cve_metadata`	`CVEMetadata` (`source="atlassian"`, native `CVE-YYYY-NNNN` id, `title` = `cve_summary` truncated to 500, `affectedVendor="Atlassian"`, `datePublished` from `cve_publish_date`, `sourceFileHash` = SHA-256 of the canonical resource JSON)
Always	`CVEAlias` via `db.InsertAliases` — empty list, so only the same-`cveId` cross-source backfill fires
Always	`CVEDescription` (`cve_description`, falling back to `cve_summary`)
`atl_tracking_url` present	`CVEMetadataReferences` (type `issue`)
`advisory_url` present	`CVEMetadataReferences` (type `advisory`) — absent from the `/v1/products` payload, so in practice not emitted
Vector recovered or `cve_severity > 0`	`CVEMetric` (metric type derived from the vector prefix, `baseScore` = `cve_severity`, `baseSeverity` banded per FIRST CVSS v3.x)
One per affected product	`CVEAffected` (vendor `Atlassian`, product = product key, no `collectionURL`)
`--emit-crit` and a dictionary match	CRIT candidate envelope staged to `crit-candidates/pending/`, then drained in-process into `CritRecord` (+ VEX sibling) by `critpublisher.DrainKeys`
End of run	`BulkDataDumpTracker` (`source="atlassian"`, `sha256` = listing hash over the document)

SuppressDerivedMetrics is set, so the shared pipeline does not derive a Vulnetix CVSS v4 vector from the description — the API already supplies a score (internal/atlassian/map.go:37-40).

CRIT staging

The feed is Atlassian-scoped by identity, so the matcher is constructed with ImplicitProviders: ["atlassian"] — a bare product token such as jira or bamboo counts without an in-text “Atlassian” (cmd/atlassian-json-processor/main.go:140). Records that produce no envelope are offered to CRIT inference instead (main.go:221-223). Staged keys are drained in-process at end of run unless CRIT_DISABLE_INPROCESS_DRAIN is set.

Resume

loadProcessedHashes reads every non-null sourceFileHash for source='atlassian' into a set and skips any resource whose canonical-JSON SHA-256 is already present (main.go:265-285). --force bypasses it. The tracker hash is informational — it does not short-circuit the run.

Failure modes

Empty upstream — atlassian.ErrEmptyResponse exits 1 with an Errored notification rather than reporting success (main.go:95-99).
Denormalised columns stay empty — the mapper sets AffectedVendor but never AffectedProduct or CVESourceData.VectorString, so CVEMetadata.affectedProduct and CVEMetadata.vectorString are NULL for every row even though CVEAffected and CVEMetric rows exist.
No AI enrichment — no aienrich.Enricher is constructed.
The usable processing window is 3 minutes, not 15 — the soft deadline is EXPECTED_DURATION_MINUTES - 10 (main.go:59-61) and the per-record check subtracts a further 2 minutes (main.go:166), so with expected_duration_minutes = 15 the loop stops 12 minutes into the run. That is enough for ~336 single-record transactions today, but it is not the budget the task definition implies, and the margin should come from rundeadline.Soft rather than two stacked constants.
Tracker is written unconditionally (main.go:244), including after a soft-deadline break. Harmless here only because the tracker hash is never read back — resume is per-record via sourceFileHash.

S3 persistence

Archive: atlassian/files/{sha256}/{CVE-ID}.json — the mapped resource’s raw JSON, after the store transaction commits (main.go:196).
Quarantine: failed-feeds/atlassian-json-processor/{YYYY-MM-DD}/store-error/{CVE-ID}.json (main.go:189).

Flags

Flag	Default	Description
`--force`	false	Reprocess every record even when its hash is unchanged
`--limit`	0	Cap the number of records stored (0 = unlimited)
`--emit-crit`	false (`true` in ECS)	Stage CRIT candidate envelopes

Local run: just go-atlassian-json-backfill prod.

S3 Persistence

Archive path: atlassian/files/{sha256}/{filename} ✓
Quarantine path: failed-feeds/atlassian-json-processor/{YYYY-MM-DD}/{reason}/{filename} ✓
Failure reasons emitted: store-error

Uses s3client.Uploader from internal/s3client/uploader.go. Skipped when S3_BUCKET_NAME is unset (local dev).

flowchart LR SRC[Source feed] --> PROC[atlassian-json-processor] PROC -->|success| ARCHIVE[("S3: atlassian/files/{sha256}/{filename}")] PROC -->|failure| Q[("S3: failed-feeds/atlassian-json-processor/{date}/{reason}/{filename}")] PROC --> DB[(PostgreSQL)]

See the S3 Persistence Contract for the full reason taxonomy.