Insights Org Graph
Insights Org Graph
Processor: insights-org-graph-processor
Schedule: hourly (cron(0 * * * ? *))
Reads: InsightsRun, InsightsCrossRepoEdge, InsightsContributor
Writes: InsightsOrgEdge, and an ORG-scoped InsightsRun with its InsightsMetric / InsightsEvidence rows
Why this exists
vulnetix analyze scans one repository, and never reads a second one. That is deliberate:
it means the CLI can run in any repository’s CI with only that repository’s checkout and no
special access to anything else.
The consequence is that no scan can see the organisation. Each one publishes what its repository provides and what it consumes, as normalised join keys:
| Join kind | Provided when | Consumed when |
|---|---|---|
PACKAGE | the repo declares a module (go.mod, package.json) | the repo depends on it |
HTTP_ROUTE | a file stands up a router and declares the route | — |
TOPIC | a publish call names it | a subscribe call names it |
CONTAINER_IMAGE | — | a FROM or compose image: names it |
WORKFLOW | the repo defines a reusable workflow | a workflow uses: it |
An edge exists wherever one repository’s consumes meets another’s provides on the same key. Somebody has to do that matching, and it has to be somebody who can see all of it. That is this job.
What it produces
1. InsightsOrgEdge — the materialised org graph
Rebuilt wholesale per organisation on each run, from the latest scan of each repository. (That “latest” matters: history is retained in full, so without it a repository scanned twenty times would contribute its keys twenty times and every edge would be duplicated twentyfold.)
Three rules govern the matching:
- A key consumed by us and provided by nobody in the organisation is an external dependency,
not a dangling edge. Every repository consumes
cobra; none publishes it. Skipped, not reported as a problem. - A repository consuming what it also provides is not an org edge. That relationship is internal and already lives in that repository’s own graph.
- An edge’s confidence is the minimum of the two sides — an edge is only as good as the weaker of the two claims behind it, and averaging would let a confident provider launder a shaky consumer.
Each edge records the two runs it was formed from, so it can always be traced back to the scans that produced it. An edge with no provenance is an assertion nobody can check.
2. The metrics a single repository cannot compute
Written as ordinary InsightsMetric rows against an ORG-scoped InsightsRun, so they reuse
the evidence model and the read API that already exist. An org metric is a metric, and it is held
to the same standard: a value of 23 has 23 evidence rows.
| Metric | What it is |
|---|---|
org.repos.orphaned | Repositories where none of the recent committers is still committing anywhere else in the organisation. Not a quiet repository — a finished library is quiet and fine — but one whose knowledge has left the building: if it broke tomorrow, nobody still here has ever worked on it. From inside the repository this is invisible; everybody there looks perfectly present. |
org.bus_factor.commits | The smallest number of contributors whose cumulative share of commits, ranked descending, first exceeds 50% across the whole organisation. Strictly scarier than any single repository’s: a repository can have a healthy bus factor of four while three of those four are the same three people holding up every other repository too. |
org.packages.high_fan_out | Internally-published packages consumed by three or more repositories — the blast radius of a release. |
Bots are excluded from every one of them. A repository kept ticking over by Dependabot alone is still orphaned, and counting the bot as a committer would be the single easiest way to make it look alive.
Idempotency
The whole job is idempotent per organisation, inside one transaction:
- the organisation’s edges are deleted and rewritten — an edge is a fact about the current state of the organisation, not a history, and the runs that produced it are retained anyway
- the previous ORG run is replaced — unlike a repository scan, this is not an observation of a moment, it is a derivation from observations that are themselves kept. Retaining every derivation would grow without bound and tell you nothing a re-derivation could not.
A concurrency lock (insights_org_graph) prevents overlapping runs.
Caveats, which travel with the run
An absent edge may mean an unscanned repository rather than an absent dependency. A repository
that has never been analyzed contributes nothing to the graph, so the org graph is only ever as
complete as its coverage. This is recorded in the run’s diagnosticsJson and surfaced in the
console, rather than left for a reader to work out on their own.
S3 Persistence
Not used. This processor does not currently archive payloads or quarantine failures to S3. Per the S3 Persistence Contract this is non-compliant — see the compliance matrix for the implementation roadmap.
⚠ Not in the compliance matrix — status needs verification.
Expected paths when implemented:
- Archive:
insights-org-graph/files/{sha256}/{filename} - Quarantine:
failed-feeds/insights-org-graph/{YYYY-MM-DD}/{reason}/{filename} - Likely reasons: (none documented)