AI evaluation protocol
Defines a versioned dataset, acceptance threshold and failure handling before a model is released.
Synthetic evaluation-method sample. Version 1.0, published November 7, 2026. Authored by Newton & Carter for demonstration; no client, participant or measured outcome is implied.
Newton & CarterDefine what ready means.
01
Scope
02
Build
03
Review
Inspect the artifact
The complete, machine-readable source is below. Use it to discuss the structure and acceptance of an engagement.
{
"version": "1.0",
"dataset": "Fictional CivicAnswer corpus and 50-question benchmark",
"decision": "Can the system select an approved source or refuse an unsupported question?",
"measures": [
"Expected source/refusal agreement",
"Source-text fidelity",
"Unsupported-question refusal"
],
"acceptance": "Expected source/refusal agreement is assessed against the published cases. Review every failure; do not infer general model quality.",
"humanReview": [
"Confirm the expected cases",
"Inspect false refusals and unsafe answers",
"Approve deployment boundary"
],
"limits": "A deterministic extractive demonstration is not model inference, a client result or evidence of general AI performance."
}