paracelsus

Running Paracelsus against your own catalog

Companion to README.md, which covers the demo catalog. This file covers pointing Paracelsus at a DataHub instance of your own — what works out of the box, and the four things that will need your attention.


What is already portable

Nothing in the scoring path is tied to our demo catalog. Paracelsus talks to DataHub only through the GraphQL API and the MCP server, and it discovers datasets by enumerating whatever the catalog contains.

git clone https://github.com/fiya-chris-and-AI/paracelsus.git
cd paracelsus
make preflight && make venv

Pointing it at your instance is then a two-line change in .env:

DATAHUB_GMS_URL=https://your-datahub.example.com/api/gms
DATAHUB_GMS_TOKEN=<a personal access token from your instance>

Then make risk. Steps 4 and 5 of the Quickstart (make up, which brings up a local DataHub and loads the demo datapack) are not needed — you already have a catalog.

We have not run this against a second catalog. The claim above is about architecture, not about a run we performed. Four things will need attention, and we would rather name them than let you discover them:

1. The bands need re-anchoring; the ranking does not

action_threshold in engine/weights.yaml is anchored to a written reference case from our catalog — a specific, documented defect that we decided should sit exactly at the line between “act” and “don’t”. Everything else follows from it: Critical is 2× the threshold, Monitor is ⅓×, and the safety margin printed next to every score is action_threshold / risk.

That anchor is ours, not yours. On your catalog the relative ordering of findings is meaningful immediately — that is the part the method produces. The labels are not, until you pick your own reference case: one asset, one defect, where your team agrees this is exactly worth acting on. Set action_threshold to that asset’s score and the bands mean something again.

We deliberately did not make this automatic. A threshold that calibrates itself is a threshold nobody has to defend, and the entire point of this tool is that every number can be defended in a sentence out loud.

2. Skip the usage-stats import

engine/import_usage_stats.py exists to work around a DataHub 1.7.0 behaviour: the usageFeatures aspect is write-only — it can be ingested but not read back through any of the seven read paths we tried. We filed this as datahub#19014. Our workaround re-emits the datapack’s own query counts as datasetUsageStatistics, which is readable.

If your catalog already carries datasetUsageStatistics — most production instances with usage ingestion configured do — skip that step entirely. Exposure will read your real query volumes.

If your catalog has no usage data at all, Paracelsus still runs, and this is where its central idea does the work: missing usage evidence is not treated as “unused, therefore safe”. It raises the uncertainty factor. You will see higher scores and wider safety margins, which is the correct answer to “we cannot see who touches this”.

3. The first run only builds a baseline

The detector compares the live schema against state/schema_baseline.json, which it maintains itself. On your catalog, the first make risk establishes that baseline and will legitimately report nothing — there is no “before” to compare against yet. Real findings appear from the second run onward, as your schemas actually drift.

Do not carry over seed/schema_baseline_seeded.json. That fixture exists so the demo can show findings in three minutes instead of three weeks; it describes our catalog’s assets and means nothing on yours.

4. Two hard-coded numbers to change

Scale

Dataset enumeration paginates through GraphQL search a page at a time, which is fine for catalogs in the low hundreds. A catalog with thousands of datasets needs scroll/cursor pagination in engine/datahub_client.py::search_dataset_urns. We knew this and chose not to build it — see the comment at that function.