{"id":"support-routing-original","state":"An online store customer reports: “For the last 15 minutes, no customer has been able to pay for an order by card. The page shows payment unavailable. Payment on delivery works. We checked on three different devices. No money is being charged to the cards.” Routing policy: website failures go to engineering; questions about charges and invoices go to finance; delivery times go to logistics. A payment-method outage affecting many customers requires urgent human review.","questions":{"team":{"type":"choice","instructions":"Which team should handle this request? Choose based on the main cause of the problem.","criteria":{"engineering":"Engineering: website or integration failure","finance":"Finance: invoices and refunds of charged payments","logistics":"Logistics: shipping and delivery"}},"impact":{"type":"score","instructions":"Assess how the described issue affects placing orders.","criteria":["Does not prevent placing orders","Makes ordering harder for one customer","Blocks one payment method for many customers","Blocks every ordering method for all customers"]},"human_review":{"type":"noul","instructions":"Under the stated policy, this request requires urgent human review."}},"expected":{"team":"engineering","impact":{"min":1.5,"max":2.5},"human_review":true}}
{"id":"support-routing-variant","state":"An online store customer reports: “The order has already been paid for. The parcel is still in transit, and I would like to check the delivery date. New orders and all payment methods are working normally.” Routing policy: website failures go to engineering; questions about charges and invoices go to finance; delivery times go to logistics. Only a payment-method outage affecting many customers requires urgent human review. A routine question about the delivery date does not require it.","questions":{"team":{"type":"choice","instructions":"Which team should handle this request? Choose based on the main cause of the problem.","criteria":{"engineering":"Engineering: website or integration failure","finance":"Finance: invoices and refunds of charged payments","logistics":"Logistics: shipping and delivery"}},"impact":{"type":"score","instructions":"Assess how the described issue affects placing orders.","criteria":["Does not prevent placing orders","Makes ordering harder for one customer","Blocks one payment method for many customers","Blocks every ordering method for all customers"]},"human_review":{"type":"noul","instructions":"Under the stated policy, this request requires urgent human review."}},"expected":{"team":"logistics","impact":{"min":0,"max":0.5},"human_review":false}}
{"id":"return-review-original","state":"Internal policy of the fictional store Line: a standard return is allowed if no more than 14 days have passed since receipt, the item has not been used and all tags are intact. If the item has been used, refer the request to a manager for individual review. If information is missing, ask for clarification first. Order: backpack. Days since receipt: 6 (already calculated by the system). The customer writes: “I took the backpack on a hike over the weekend. There is a stain on the bottom, but I kept the tags. I would like to return it.” This is a fictional internal policy for a learning scenario, not a statement of law.","questions":{"route":{"type":"choice","instructions":"What is the next step under the stated internal policy?","criteria":{"standard_return":"Standard return procedure","manager_review":"Individual review by a manager","ask_for_details":"Ask for missing information"}},"completeness":{"type":"score","instructions":"How many of the three required facts are known: time since receipt, use and presence of tags?","criteria":["None","One fact","Two facts","All three facts"]},"meets_standard_policy":{"type":"noul","instructions":"The described request meets all conditions for a standard return under the stated internal policy."}},"expected":{"route":"manager_review","completeness":{"min":2.5,"max":3},"meets_standard_policy":false}}
{"id":"return-review-variant","state":"Internal policy of the fictional store Line: a standard return is allowed if no more than 14 days have passed since receipt, the item has not been used and all tags are intact. If the item has been used, refer the request to a manager. If information is missing, ask for clarification. Order: backpack. Days since receipt: 6. The customer confirms that the item has not been used and all tags are intact. This is a fictional internal policy for a learning scenario, not a statement of law.","questions":{"route":{"type":"choice","instructions":"What is the next step under the stated internal policy?","criteria":{"standard_return":"Standard return procedure","manager_review":"Individual review by a manager","ask_for_details":"Ask for missing information"}},"completeness":{"type":"score","instructions":"How many of the three required facts are known: time since receipt, use and presence of tags?","criteria":["None","One fact","Two facts","All three facts"]},"meets_standard_policy":{"type":"noul","instructions":"The described request meets all conditions for a standard return under the stated internal policy."}},"expected":{"route":"standard_return","completeness":{"min":2.5,"max":3},"meets_standard_policy":true}}
{"id":"lead-routing-original","state":"An incoming inquiry to a fictional B2B service: “Hello! We have 80 support agents and around 120,000 inquiries per month. We use Zendesk. We want to route inquiries to teams automatically. Before a pilot, we need to discuss integration, private-network access and our security team's requirements. We will agree on commercial terms after the technical meeting.” Rules: send inquiries involving integration or security requirements to technical sales; send straightforward requests without these requirements to a self-service pilot; send existing customers' issues to support. This example does not confirm that any requested feature is available in the product.","questions":{"next_step":{"type":"choice","instructions":"Where should this inquiry go under the stated rules?","criteria":{"technical_sales":"Technical sales: clarify requirements and integration feasibility","self_service_trial":"A self-service pilot without a preliminary technical meeting","customer_support":"Support for an existing customer"}},"detail_level":{"type":"score","instructions":"How specifically is the task described?","criteria":["The task is not defined","Only general interest is expressed","A specific workflow or requirements are described","The workflow, scale and technical requirements are described"]},"security_review_requested":{"type":"noul","instructions":"The customer explicitly asks to discuss security requirements before starting the pilot."}},"expected":{"next_step":"technical_sales","detail_level":{"min":2.5,"max":3},"security_review_requested":true}}
{"id":"lead-routing-variant","state":"An incoming inquiry from a new customer to a fictional B2B service: “I would like to try routing inquiries to teams myself through your Playground. I do not need any integrations, and I have no specific security requirements.” Rules: send inquiries involving integration or security requirements to technical sales; send straightforward requests without these requirements to a self-service pilot; send existing customers’ issues to support.","questions":{"next_step":{"type":"choice","instructions":"Where should this inquiry go under the stated rules?","criteria":{"technical_sales":"Technical sales: clarify requirements and integration feasibility","self_service_trial":"A self-service pilot without a preliminary technical meeting","customer_support":"Support for an existing customer"}},"detail_level":{"type":"score","instructions":"How specifically is the task described?","criteria":["The task is not defined","Only general interest is expressed","A specific workflow or requirements are described","The workflow, scale and technical requirements are described"]},"security_review_requested":{"type":"noul","instructions":"The customer explicitly asks to discuss security requirements before starting the pilot."}},"expected":{"next_step":"self_service_trial","detail_level":{"min":1.5,"max":2.5},"security_review_requested":false}}
{"id":"answer-grounding-original","state":"The only source: “The Team plan of the fictional service Contour includes 3 workspaces and up to 50 members. An administrator can manually export a report as CSV. Automatic reports and PDF export are not available in the Team plan.” Customer question: “Can Team provide automatic PDF reports and unlimited workspaces?” Assistant's draft: “Yes, Team supports automatic PDF reports and an unlimited number of workspaces.” Assess only whether the draft is supported by the provided source.","questions":{"verdict":{"type":"choice","instructions":"How does the draft relate to the source?","criteria":{"supported":"All claims in the draft are supported","contradicted":"At least one claim directly contradicts the source","insufficient":"The source is insufficient to verify the claims, with no direct contradiction"}},"grounding":{"type":"score","instructions":"How many of the draft's two main claims (automatic PDF reports; unlimited workspaces) are supported by the source?","criteria":["Neither","One of the two","Both"]},"contains_contradiction":{"type":"noul","instructions":"The draft contains a claim that directly contradicts the provided source."}},"expected":{"verdict":"contradicted","grounding":{"min":0,"max":0.5},"contains_contradiction":true}}
{"id":"answer-grounding-variant","state":"The only source: “The Team plan of the fictional service Contour includes 3 workspaces and up to 50 members. An administrator can manually export a report as CSV. Automatic reports and PDF export are not available in the Team plan.” Customer question: “Can Team provide automatic PDF reports and unlimited workspaces?” Assistant’s draft: “Automatic PDF reports are not available in Team. The number of workspaces is limited to three.” Assess only whether the draft is supported by the provided source. The draft’s two main claims are: PDF reports are not available; there are no more than three workspaces.","questions":{"verdict":{"type":"choice","instructions":"How does the draft relate to the source?","criteria":{"supported":"All claims in the draft are supported","contradicted":"At least one claim directly contradicts the source","insufficient":"The source is insufficient to verify the claims, with no direct contradiction"}},"grounding":{"type":"score","instructions":"How many of the draft’s two main claims (automatic PDF reports are not available; there are no more than three workspaces) are supported by the source?","criteria":["Neither","One of the two","Both"]},"contains_contradiction":{"type":"noul","instructions":"The draft contains a claim that directly contradicts the provided source."}},"expected":{"verdict":"supported","grounding":{"min":1.5,"max":2},"contains_contradiction":false}}