← All scenariosAI quality / LEARNING SCENARIO

Does the source support the answer?

Review an assistant's draft response before showing it to a customer.

01 / CONTEXT

The only source: “The Team plan of the fictional service Contour includes 3 workspaces and up to 50 members. An administrator can manually export a report as CSV. Automatic reports and PDF export are not available in the Team plan.” Customer question: “Can Team provide automatic PDF reports and unlimited workspaces?” Assistant's draft: “Yes, Team supports automatic PDF reports and an unlimited number of workspaces.” Assess only whether the draft is supported by the provided source.

02 / QUESTIONS

verdictchoice

How does the draft relate to the source?

supportedAll claims in the draft are supported
contradictedAt least one claim directly contradicts the source
insufficientThe source is insufficient to verify the claims, with no direct contradiction

groundingscore

How many of the draft's two main claims (automatic PDF reports; unlimited workspaces) are supported by the source?

0Neither
1One of the two
2Both

contains_contradictionnoul

The draft contains a claim that directly contradicts the provided source.

03 / REAL API RECORDING

What the model returned.

Recorded on October 17, 2026, using shvydko-latest in free preview, with English inputs. This is a response to a fictional learning example, not a guarantee of quality on your data. The full dataset includes answers that disagree with the authored expectations; inspect the checks in Workflow Lab. The current model may be updated.

Question	Author's expectation	Actual response	Match
`verdict`	"contradicted"	contradicted	Yes
`grounding`	{"min":0,"max":0.5}	0.0011	Yes
`contains_contradiction`	true	0.0067	No

{
  "verdict": {
    "type": "choice",
    "choice": "contradicted",
    "probabilities": {
      "supported": 0.0005,
      "contradicted": 0.9991,
      "insufficient": 0.0004
    },
    "confidence": 0.9987
  },
  "grounding": {
    "type": "score",
    "score": 0.0011,
    "probabilities": {
      "0": 0.9991,
      "1": 0.0007,
      "2": 0.0002
    },
    "confidence": 0.9984,
    "legend": {
      "0": "Neither",
      "1": "One of the two",
      "2": "Both"
    }
  },
  "contains_contradiction": {
    "type": "noul",
    "noul": 0.0067
  }
}

Compare two situations Full recording and expectations

04 / YOUR FIRST RUN

Edit. Run. Check.

Run this request in Playground or evaluate a set of examples in your account. Compare the new response with the recording above, the rules in the context, and your own edge cases.

{
  "state": "The only source: “The Team plan of the fictional service Contour includes 3 workspaces and up to 50 members. An administrator can manually export a report as CSV. Automatic reports and PDF export are not available in the Team plan.” Customer question: “Can Team provide automatic PDF reports and unlimited workspaces?” Assistant's draft: “Yes, Team supports automatic PDF reports and an unlimited number of workspaces.” Assess only whether the draft is supported by the provided source.",
  "questions": {
    "verdict": {
      "type": "choice",
      "instructions": "How does the draft relate to the source?",
      "criteria": {
        "supported": "All claims in the draft are supported",
        "contradicted": "At least one claim directly contradicts the source",
        "insufficient": "The source is insufficient to verify the claims, with no direct contradiction"
      }
    },
    "grounding": {
      "type": "score",
      "instructions": "How many of the draft's two main claims (automatic PDF reports; unlimited workspaces) are supported by the source?",
      "criteria": [
        "Neither",
        "One of the two",
        "Both"
      ]
    },
    "contains_contradiction": {
      "type": "noul",
      "instructions": "The draft contains a claim that directly contradicts the provided source."
    }
  }
}

Put the decision into code Evaluate quality on your own dataset