The invariant suite is committed to your repository. The signed bundles are written to your S3 bucket or Snowflake stage. The telemetry schema is published. If EvalQA disappeared tomorrow, you would keep every test, every finding and every statistic, and the notary records would still verify against the published verification key.
1. What lives where
| Artefact | Where it lives | Format | Depends on EvalQA to read? |
|---|---|---|---|
| Invariant suite (synthesised + authored) | Your repo, alongside dbt | YAML + SQL; SQLGlot-dialect SQL | No — runnable with any SQL client |
| Challenge queries and counterfactual probes | Your repo | SQL + JSON parameters | No |
| Signed audit bundles | Your S3 / Snowflake stage | HTML + JSON (evalqa.schema.json v9.3) | No — signatures verify with the published public key |
| Adjudication records, rater competency signatures | Inside the bundle | JSON | No |
| Notary records | EvalQA | Hash, date, protocol, EAL, status | The endpoint, or the public key + your manifest |
| qabit console ratings (where used) | EvalQA hosting | JSONL / CSV export on demand | No, after export |
2. The export and recreate drill
Once per engagement year, on request, we run the drill with you: export everything, delete our copy of the Mode 2 diagnostics, and demonstrate that your team can re-execute the suite, re-verify a bundle against the public key, and reproduce the Tier-3 statistics from the JSON with standard libraries. If any step needs us, that is a defect in the guarantee and we fix it.
3. Ingesting what you already have
Existing dbt tests, Great Expectations suites, custom golden-query sets and Cortex verified queries are ingested as customer golden truth at intake (validated, counted, and subject to the 30% independent floor). Nothing has to be rewritten into a proprietary format to be used.
4. Supported export formats
- Suites: dbt-compatible YAML tests where the invariant maps to one; otherwise SQL assertions with a JSON manifest.
- Findings: JSON per
evalqa.schema.json; CSV for the scorecard table; SARIF for security-class findings where a customer’s tooling wants it. - Statistics: the raw counts behind every interval, so the interval can be recomputed.
5. The written exit covenant
- No artefact required to reproduce an assurance case is held only by EvalQA.
- Exports are provided within 10 business days of a request, at no charge, in the formats above.
- The verification public key and the telemetry schema are published and versioned; a superseded key remains published for verification of records signed under it.
- On termination, Mode 2 diagnostics are deleted by cryptographic erasure and confirmed in writing; notary records remain, because they contain no customer data and a notary that forgets is not a notary.
- If EvalQA is acquired or ceases operation, this covenant binds the successor for records already issued, and the public keys are escrowed with the customer on request.