1. Getting in
- Apply on the guild page. We reply within a week.
- The battery. Fifty items, four hours, proctored, unpaid. Drawn from the eight validated failure families with matched clean controls. Passing standard: at most one Sev-1 miss, ≥85% Sev-2/3 recall, ≤5% false positives, Brier ≤ 0.15 with positive skill.
- Calibration. Paid at your rate from the first hour: a dense overlapping set of ≥50 challenge queries rated alongside other reviewers, which is what makes the latent-class model identifiable and your competency vector meaningful.
- The roster. Identity verification, confidentiality agreement, tax and payment setup. Then work, as it exists.
What we promise about timing
A reply to every application within a week; a battery result within five business days of sitting it; payment weekly for approved hours. If we miss one of these, tell us and we will say why.
2. Pay
| Role | Rate | Gate |
|---|---|---|
| Qualified reviewer (Tier 3) | $65 per hour, loaded | ≥98% Sev-1 recall on the 50-item battery; ≤5% false-positive rate on matched controls |
| Senior assurance engineer | $120 per hour | Authors Type B/C fixtures; dbt and Snowflake execution-plan fluency |
| Lead adjudicator (Tier 4) | $150 per hour | Resolves Sev-1 disputes; Brier score ≤0.15 and BSS > 0 over the trailing 90 days |
Calibration hours are paid at your rate. The only unpaid step is the 50-item qualification battery itself. Rates are published per role and change with 30 days notice, never retroactively. Competency is a vector (gold accuracy, sample size, recency, dialect, domain, overturn rate, Brier), not a years-of-experience proxy. Reviewers never pay us. Not for the qualification battery, not for calibration, not to stay on the roster. We make money when you find real defects, not when you sit an exam.
3. How items reach you
Deterministic AST checks and the Tier-2 semantic judge clear most items. You see Tier-3 items — edge cases, invariant exceptions, high-consequence vectors — and, if you are a lead adjudicator, Tier-4 disputes. Items arrive with the invariant, the generated SQL or its pseudonymised AST, the execution plan, and the counterfactual result. You rate the outcome state, name the defect family, and give a probability that the anomaly is a true Sev-1/2 defect. The probability is scored: it is your Brier score.
4. Quality
- Hidden gold. Five percent of your items are canaries with known answers. Your rolling error rate is Bayesian, so one miss in forty is treated differently from one in four hundred.
- Emergency suspension. Two consecutive canary misses pause your queue and start a conversation, not a silent removal.
- Overlap. 15–25% of items are also rated by someone else, to measure agreement and catch drift. Paid like any other item.
- Your vector. Gold accuracy, sample size, recency (90-day half-life), dialect, domain, overturn rate, Brier and skill score. You can see it at any time.
5. Confidentiality and evidence modes
In Mode 1 you never see a customer’s raw query or data; you adjudicate on sanitised evidence. In Mode 2 you see pseudonymised AST snippets with a 14-day TTL. In Mode 3 you work inside the customer’s VDI with clipboard, downloads and capture disabled; that session is regulated processing and your processing location at the time is recorded. Never copy, photograph, transcribe or discuss customer material anywhere outside the platform. Your confidentiality agreement is indefinite.
6. If something goes wrong
- You disagree with an adjudication. Say so in the record. Overturns are tracked as a rate, and a lead adjudicator with a high overturn rate is the one under review, not you.
- You think an invariant is wrong. Flag it. If you are right, it triggers the recall protocol and you are credited in the RCA.
- A payment is late. Email [email protected]; we answer within two business days.
- You are suspended. You get the canary items you missed, the reasoning, and a re-calibration path. Suspension is not termination.
7. Difficult content
Warehouse SQL rarely contains distressing material, but prompts can. You may decline any item without penalty. Tell us and we re-route it.
8. What we ask of you
- Rate what is in front of you, not what you assume the customer meant. If the definition is ambiguous, the answer is
INDETERMINATE, and that is a good answer. - Give honest probabilities. A reviewer who is always 99% sure has a bad Brier score.
- Keep your processing location current.
- Do not identify or contact a customer outside the platform.
9. What this is not
- Not a certification. Passing the battery is an observed qualifying score on a specific challenge set. We do not issue a credential and ask you not to describe it as one.
- Not employment. Reviewers are independent contractors with a formal multi-client business; you control how and when you work within the SLAs you accept.
- Not a promise of volume. Today there is little work. The roster grows with contracted ARR and backlog, not ahead of it.