How this data is verified

“Verified” is the whole pitch, so it has to be checkable. This page is the exact process every row goes through.

The pipeline

Discovery starts from public server indexes and community seed lists; seed lists are URL pointers only, and every field is re-derived from primary evidence.
Deduplication: URL canonicalisation plus fork/mirror clustering, so the same server never counts twice.
Collection uses documented public machine-readable sources only. No HTML scraping, no login-walled sources, no private personal data.
Liveness verification at snapshot time wherever the evidence allowed one; rows with nothing to ping or inspect ship as an explicit null rather than a verdict.
Computed fields (maintenance status, commercial-use mapping, auth shape, install method) derived deterministically from the collected metadata, by rules published in full.
LLM-assisted categorisation and extraction, validated against allow-lists and hand-checked on a random sample; a field is re-run rather than shipped if the sampled error rate exceeds 10%.
Categorisation refuses to guess: a row with no tools, no topics and no package coordinates gets category = other rather than a plausible label. That rule is why 20.7% of the corpus lands on other, and 48% of the public registry exposes only a name.
Cross-review: every published document is reviewed by a different model family than the one that drafted it, and the review is logged.

No HTML scraping: documented machine-readable sources only, robots.txt respected.
No personal data beyond the public org/handle already attached to a repository.
No login-walled or paywalled sources.
No invented values: a field we could not verify ships as an explicit null.
If a source's machine-readable policy blocks the path we would use, we drop that source rather than work around it. The machine-readable signal wins over our reading of the acceptable-use policy.

Public server indexes: canonical server identity and package coordinates.
Public repository metadata: SPDX license, activity signals and maintenance evidence.
Public package metadata: package existence, versions and release dates.
Community awesome-lists: discovery seeds only, credited in the SOURCES.md that ships in the pack; every field re-derived from the primary evidence above.

The pipeline is scripted and re-runnable. Each snapshot is dated. No refresh cadence is promised here; later snapshots, if they are offered, are offered separately.