malware-actor-backfill

Status: Local-only tool — no ECS task definition, no EventBridge schedule, no task-manager.toml entry Source: the database itself — CVEMetadata where isMaliciousPackage = true and source <> 'osm', joined to CVEAffected for the package identity, plus GitHub/registry lookups via internal/actorintel Type: backfill (cmd/malware-actor-backfill) Source slug: none — it writes no CVEMetadata Run with: just go-malware-actor-backfill prod "" 250 (target, sources, request delay in ms)

Overview

Every ecosystem malware processor runs an inline malwareactor.PostPass after its batch, but that pass is bounded by the run’s soft deadline and only ever looks at its own source. Three gaps follow: a source whose processor predates the post-pass never got attributed at all, a source with a large historical backlog can never catch up in a 25-minute window, and OSV-derived malware (oss-malicious-packages, 228k records) has no owning registry processor to run a post-pass at all.

This tool is the comprehensive net for those gaps. It selects purely on isMaliciousPackage = true AND source <> 'osm' — deliberately regardless of which processor stored the record — pages through everything without a MalwareAttribution row, resolves actor identities, and persists them. The OSM-specific tables (OsmThreat*) are never touched; only the generic attribution tables are written, which is what the /malware and /threat-actors API surfaces and the malware-campaign pages read.

Attribution is what turns a list of bad packages into a story: 350 AUR packages published by one persona is one campaign, not 350 incidents. Without it the malware corpus can answer “is this package bad” but not “who did this, and what else did they publish”.

What it writes

Table	Rows
`ThreatActor`	one per resolved identity (platform, identifier, actor type, display name, email, profile/avatar URL, company, location, raw JSON)
`ThreatActorKey`	one per identity key (email, handle, commit signature, …)
`MalwareThreatActor`	one edge per (cveId, cveSource, actor) with `attributionForm`, `attributionBasis`, `confidence`, and the commit hash / repo URL when known
`MalwareAttribution`	exactly one row per malware CVE, carrying the terminal status and the claimed/victim repo metadata. This row is also the idempotency key — a record that has one is excluded from the next page

Persistence is one short write transaction per record (internal/malwareactor/enrich.go:170); the network lookups happen outside it.

MalwareAttribution.status is the outcome, not a success flag. Production distribution: no-commit-hash 83,482, attributed 34,127, no-repo 2,992, unresolved 73 — i.e. most records are resolvable to something but not to a signed commit.

False-positive gates

Attribution is where a legitimate maintainer can be libelled as a malware author, so three independent guards apply:

Guard	Where	Effect
Known-legit maintainer allowlist	`internal/actorintel/resolve.go:116` (`legitmaintainer.IsLegit` on identifier and email)	the identity is dropped before an edge exists
Hijack/victim gate	`internal/malwareactor/enrich.go:57` — `BuildSubject` derives `Tags` with `actorintel.HijackTagsFromText(title)`	a compromised-maintainer advisory attributes the attacker, not the hijacked maintainer whose name is on the package
Customer curation	`internal/curationgate` — `ActorAllowed` (≥3 orgs flagged it) skips the actor entirely; `ActorDowngraded` (≥1 org) forces `confidence = "low"` so it cannot stand alone	applied per edge inside the transaction

The Tags derivation matters specifically: without it the hijack gate can never fire on this path, and a compromised-maintainer advisory would brand the real registry maintainer as the threat actor.

Resume and pacing

There is no tracker. The loader (db.LoadPendingMalwareSubjects) is an anti-join against MalwareAttribution, so every persisted record drops out of the next page and a fixed page size of 200 walks the whole backlog (internal/malwareactor/enrich.go:102-134). The tool can be interrupted and re-run at any point.

--sources restricts to a comma-separated subset (default: every non-OSM malware source).
--req-delay paces per-record GitHub lookups in milliseconds. Use it: a GitHub PAT allows 5,000 req/hr, and without a PAT the limit is 60 req/hr and identities degrade to bare handles.
--github-pat overrides GITHUB_PAT / GITHUB_TOKEN.

Loading the pending page is a heavy read (DISTINCT ON over all malicious CVEMetadata × CVEAffected plus the anti-join), so the justfile recipe sets READ_STATEMENT_TIMEOUT=30min to beat the read pool’s 60-second default — the same pattern as summary-processor. Transient load failures are retried five times with exponential backoff (internal/malwareactor/enrich.go:139-156).

No deadline — by design

The recipe explicitly unset EXPECTED_DURATION_MINUTES and Backfill carries no soft deadline, per the AGENTS.md backfill rule. The sibling EnrichPendingForSource (the inline post-pass used by the scheduled processors) does take a deadline; this entry point deliberately does not.

Remaining backlog

As of 2026-08-06 the net is far from closed — pending = malicious records with no MalwareAttribution row:

Source	Malicious	Attributed	Pending
`oss-malicious-packages`	228,649	81,925	146,724
`go`	4,214	1,372	2,842
`gitlab`	2,139	78	2,061
`vulnetix`	542	203	339
`cargo`	1,470	1,465	5

Every registry processor with an inline post-pass (aur, homebrew, packagist, pypi, hex, conan, cran, …) is fully attributed; the backlog is concentrated in the sources that have no owning processor to run one.

S3 Persistence

Not applicable. This tool consumes no feed — its input is the database and live GitHub/registry lookups, and its output is attribution rows. There is no payload to archive and no record to quarantine.