maven-json-processor

Status: Live, but producing nothing — the legacy search.maven.org solr GAV index it walks stopped receiving new documents around 2026-06-10 (its newest document timestamp equals the stored watermark), so every run reaches the watermark on the first page and stops. There are zero maven advisories in production as a result. Fixing this means moving to the Central Portal API or the repo1 incremental index — see Failure modes. Source: Maven Central search + repo1.maven.org Type: json (solrsearch GAV feed + per-artifact POM + JAR fetch) Source slug: maven Schedule: Runs hourly (at minute 25) (cron(25 * * * ? *)).

Overview

Continuously watches Maven Central for newly published artifacts and flags malicious publishes. Each run pages the solrsearch GAV core (search.maven.org/solrsearch/select?q=*:*&core=gav&sort=timestamp+desc) from the head; because the feed is sorted by publish timestamp descending, walking stops as soon as a document’s timestamp is at or below the stored watermark. The cursor watermark is the maximum publish timestamp seen, which Maven returns as epoch milliseconds, stored as a string in the tracker. Each document is a group:artifact:version (GAV) coordinate; within a run only the newest version of each group:artifact is processed.

For every candidate artifact it fetches the standalone POM (repo1.maven.org/maven2/<group-with-slashes>/<artifact>/<version>/<artifact>-<version>.pom) and gathers:

• Artifact metadata — group, artifact, version, packaging, homepage (POM <url>), repository (POM <scm><url>/<url> when github.com), and license (POM <licenses>).
• POM build manifest — the POM declares build plugins (exec-maven-plugin, gmavenplus-plugin, groovy-maven-plugin, maven-antrun-plugin, frontend-maven-plugin) that can run arbitrary code at build time, the classic Maven supply-chain execution vector. The POM XML feeds the install-script detectors.
• JAR embedded scripts — for artifacts whose POM declares a script-running build plugin (the high-risk subset), the published .jar (a ZIP) is downloaded (bounded) and its MANIFEST.MF, any *.xml/*.properties, and embedded *.sh/*.py/*.js scripts concatenated for deep static detection. .class filenames are recorded for typosquat/recon signal (bytecode itself is not scannable).
• Upstream GitHub repo (only when the POM scm/url is a github.com project) — repository, contributors, and license, persisted GitHub-first in its own committed transaction.

Records produced

Gates before minting

Three independent gates sit between a detection and an advisory:

• LLM false-positive gate (internal/aimalgate, main.go:320-333) — when PIX_INFERENCE_ENABLED + an AI-Gateway token are configured, the evidence set and the scanned source are put to the model before minting. Only a confident legitimate verdict drops the detection; any gateway or parse error fails open and keeps the verdict, so the gate can only remove false positives, never suppress a real detection on infrastructure failure. Cleared artifacts are counted as aiCleared in the run summary.
• Human curation gate (curationgate.PackageCleared, main.go:356) — a coordinate/version a reviewer has already cleared is never re-minted, even when the detectors fire again. The PackageVersion row is still written.
• Legitimate-maintainer guard (legitmaintainer.SkipActor, main.go:425) — a well-known upstream identity is never branded a threat actor, which is what stops an ownership-reputation signal cascading onto every artifact a legitimate maintainer publishes.

Detection

Reuses the shared malscan-engine detect engine. The POM build manifest is scanned by the install-script detectors; the POM plus any JAR-embedded scripts and MANIFEST by the general / shell / source-url detectors. Maven/Java-specific vectors of interest include POM build plugins that run scripts at build (exec-maven-plugin, gmavenplus-plugin), Runtime.getRuntime().exec / ProcessBuilder command execution, base64-decode chained into defineClass / Class.forName, and remote URLClassLoader loading. Findings carry one of three classes: evidence (a factual malicious behaviour — mints the advisory on its own), trigger (a weak corroborating signal such as a high-entropy embedded payload or a supply-chain ownership/identity change — never mints alone), and context (reputation/risk, recorded as PackageVersion.metadata only). detect.CombinedVerdict mints on any evidence, a known-bad package owner, a high-entropy payload combined with an ownership/identity change, or two independent identity-change families changing together. The full finding-class model and the per-ecosystem capability config (22 capabilities) live in Malware Detection.

Known-bad IOC matching — the package’s declared source is also matched against the public per-ecosystem known-bad STIX feed (domains, IPs, URLs), loaded once per run and matched in memory. A hit is evidence (CWE-506) folded into the finding set before the combination gate, and is recorded as a MalwareIoc row whose references carry the file/line and STIX provenance.

Resume

A publish-timestamp watermark (epoch milliseconds, as a string) is stored in BulkDataDumpTracker under source maven-recent (the sha256 column holds the last processed timestamp). The first run (no tracker) or a --force run sweeps the head pages (bounded by --max-pages); subsequent runs stop as soon as the descending feed reaches the saved watermark.

Failure modes

• Frozen upstream index (current) — search.maven.org/solrsearch is the legacy Central search index and has not indexed a new artifact since 2026-06-10. Because the watermark already equals that newest timestamp, every run legitimately scans nothing and exits 0, so the failure is invisible in alerting: there is no error, just no work. Detect it by comparing BulkDataDumpTracker.sha256 for maven-recent against wall-clock time.
• Silent sort override — solrsearch rewrites the requested sort=timestamp desc to score desc,timestamp desc,g asc,a asc,v desc. The descending-walk assumption only holds because q=*:* scores every document equally; changing the query term would break the watermark logic without any error.
• Truncated run advances the watermark — a run stopped by --limit or the soft deadline still persists the newest timestamp it saw, so skipped coordinates are not revisited.
• Newest version wins — only the newest version of each group:artifact in the window is scanned; an older malicious release published in the same window is skipped.

S3 Persistence

• Archive path: maven/files/maven-{group}-{artifact}-{version}/{group:artifact}/sources.txt ✓
• Quarantine path: failed-feeds/maven-json-processor/{YYYY-MM-DD}/{reason}/{group:artifact}/sources.txt ✓
• Failure reasons emitted: store-error

The unit of work is an artifact coordinate, not a file: the concatenated pom.xml plus embedded-script text that was actually scanned is archived after a successful store (main.go:414) and quarantined when the store transaction fails (main.go:408). The archive key slot holds the maven-{group}-{artifact}-{version} coordinate rather than a content hash, so re-scanning the same GAV overwrites in place. Skipped when S3_BUCKET_NAME is unset (local dev).