Sample assurance report — the three-tier glass-box readout

Bundle header

Agent: cortex_finance — Snowflake Cortex Agents, semantic model finance_v14.yaml
Consequence tier: Tier 1 — financial reporting (board revenue pack)
Evidence mode / EAL: Mode 1 local-only / EAL-2 customer-attested
Surface denominator: 41 models, 612 columns; frozen 2026-09-02, signed by owner and lead methodologist
Suite commitment: sha256:9f3a…c41e · nonce n_7f21…e0
Manifest: sha256:2b7c…91ad · dual-signed · notary

Tier 1 — Executive Board Scorecard

One page. No SQL. Written for the person who signs the residual-risk acceptance.

Business risk score (0–100)

$1.9M

Financial exposure at risk (ARR restatement path)

57%

ICFY — critical/high findings beyond native evaluation (95% CI 34–77%)

88%

RWAC — pre-frozen surface under active invariants

Recommendation: UNSAFE FOR UNATTENDED PRODUCTION

Two confirmed Sev-1 defects on the revenue path. Both are deterministic in the semantic layer and reproducible on every trial; neither is a model-sampling artefact. Human-in-the-loop operation is defensible; unattended publication to the board pack is not, until INV-017 and INV-042 are remediated and re-tested.

Decision owner: CDAO. This recommendation supports the customer’s internal go/no-go gate; it does not grant or withhold clearance. Evidence Assurance Level EAL-2.

Finding	Severity	Domain	Trials	Exposure	Decision changed?
INV-017 · fact_subscriptions × dim_invoices fan-out inflates ARR	Sev-1	Relational	20/20	$1.4M	Yes — publication paused
INV-042 · `is_current` applied to Q1-2025 point-in-time	Sev-1	Temporal	20/20	$0.5M	Yes — semantic model fix
INV-063 · non-sargable predicate on `event_ts` clustering key	Sev-2	Performance	20/20	$5.1k/mo	Yes — prompt guidance
INV-088 · “active customer” has three definitions in the manifest	—	Ambiguity	—	—	`INDETERMINATE` → owner SLA

Tier 2 — Engineering Triage Dashboard

For the analytics engineer who has to fix it. Exact SQL, failed invariant, affected lineage, suggested correction.

INV-042 — effective-dated join violated

Question: “What was customer ARR in Q1 2025 by tier?”
Invariant: For any effective-dated dimension joined to a fact filtered by a historical period, the join must bound the validity window by the fact event date.
Lineage: fact_subscriptions → dim_customers (SCD-2, valid_from/valid_to) → rpt_board_arr
Counterfactual: Re-running with the corrected join changes Q1-2025 ARR from $27.1M to $25.2M (−7.4%); 312 customers change tier.
Suggested fix: Add a point_in_time verified-query example to finance_v14.yaml; add a dbt test asserting no is_current predicate in historical marts.

-- generated (fails INV-042)
JOIN dim_customers d ON s.customer_id = d.customer_id
WHERE s.subscription_date BETWEEN '2025-01-01' AND '2025-03-31'
  AND d.is_current = TRUE

-- assurance formulation (passes)
JOIN dim_customers d ON s.customer_id = d.customer_id
 AND s.subscription_date >= d.valid_from
 AND s.subscription_date <  COALESCE(d.valid_to, '9999-12-31')

Tier 3 — Scientific Audit Appendix

For risk, internal audit and the audit committee. Every number with its interval, every rater with their competency signature, every method with its citation.

Quantity	Value	95% interval	Method
Invariants executed	84	—	61 synthesised, 23 authored (Type B: 15, Type C: 8)
PASS / DEFECT_CONFIRMED / INDETERMINATE / EXCLUDED	71 / 3 / 2 / 8	—	Four-state taxonomy
ICFY	57%	34–77%	Wilson, `statsmodels.stats.proportion`, n = 14 critical/high
Finding Overturn Rate	0%	0–41%	Wilson, n = 5 adjudicated
Sev-1 sequential bound (INV-017, INV-042)	p_fail > 0.95	—	SPRT stopped on failure at trial 1; 20 confirmatory trials run
Rater agreement (QC overlap, 18 items)	α = 0.81	AC₁ = 0.86, P_o = 0.94	Krippendorff, Gwet
Reviewer Brier / BSS (T3-0142)	0.09 / 0.41	trailing 90 days	Adjudicated truth
Authoring hours	31.5	—	Against the 50-hour Sprint-1 baseline; target <5 by Sprint 10

The appendix also carries: the frozen denominator with both signatures; the challenge-suite commitment and nonce; the runner provenance (build_version, cosign_verified: true); the evidence-mode declaration; the twelve-element assurance case; and the standards mapping for NIST AI 100-1, NIST AI 600-1, ISO/IEC 42001 and IEEE 1012-2024.

What is deliberately absent

A pass/fail badge. There is a recommendation with a stated EAL and a decision owner. There is no seal.
A single overall percentage. “92% accurate” hides a Sev-1. The scorecard leads with exposure.
Any raw row. Counterfactual results are aggregates computed in the perimeter; the bundle contains no table contents.

Get one of these for your agent → How each number is computed