Calibrate a judge for a Claude Code Skill revision
Calibrate a source-grounded second-model judge for each criterion, then compare one owned text-output Claude Code Skill revision with its baseline on real development and sealed held-out cases.
The big change
What changes: A source-grounded second model can assess a criterion that literal string checks cannot, with its own calibration receipts and an explicit unknown result when evidence is thin.
How it works: The same frozen input, truth, tests, model controls and one-attempt rule bind both Skill revisions. Every calibration and grading call records its input and output hashes before a score is computed.
Why it matters: A passing text match cannot establish source fidelity. The owner can inspect the judged answer, disagreements and regressions before choosing accept, hold or revert; the decision does not activate the revision.







