Search and reranking
Narrow a local corpus using query words, then assess which candidates actually answer the question.
What the model decides. What code checks.
- Local word-overlap retrieval selects up to three passages. Ties retain their original order.
- One request contains a separate Score for each candidate: unrelated, partly relevant, or directly answers the query.
- Code sorts the scores and checks the threshold. If no candidate qualifies, it returns no_confident_match.
Read. Run. Replay.
Unpack the archive and open its directory in a terminal. The first step needs neither a key nor a network connection.
python3 runner.py search-rerankingFor a real run, set SHVYDKO_API_KEY in your computer or server environment. Do not put the key in your browser or source code. Current account quotas and pricing apply.
python3 runner.py search-reranking --live --output search-reranking-run.jsonThe report contains requests, actual responses, elapsed times, and known usage. The file is created with private permissions and never overwrites an existing file. Network failures are not automatically retried: check Usage before another call.
python3 runner.py search-reranking --replay search-reranking-run.jsonReplay checks that the recipe, inputs, and requests match. A local file is editable, so it is not signed proof of where responses came from. If you replace the inputs with your own data, the report will contain those data too: keep it private.
An open contract, with every step visible.
The plan below was generated by the same runner from the bundled fixtures. It contains requests and author expectations, not model results.
{
"mode": "offline_plan",
"model_calls": 0,
"recipe": "search-reranking",
"author_expectation": "keys",
"notice": "Requests only, no model answers. Hierarchy includes conditional branches; repetition requests are templates.",
"requests": [
{
"state": {
"query": "Where do I rotate a leaked API key?",
"candidates": [
{
"id": "keys",
"text": "API key security: revoke the exposed API key in the console, create a replacement, update your server environment and check usage."
},
{
"id": "usage",
"text": "The usage page filters API key traffic by UTC date and exports request usage."
},
{
"id": "billing",
"text": "Billing shows payments and the current prepaid balance."
}
]
},
"questions": {
"d0": {
"type": "score",
"instructions": "How well does document keys answer the query, using only its text?",
"criteria": [
"Unrelated",
"Related but not sufficient",
"Directly answers the query"
]
},
"d1": {
"type": "score",
"instructions": "How well does document usage answer the query, using only its text?",
"criteria": [
"Unrelated",
"Related but not sufficient",
"Directly answers the query"
]
},
"d2": {
"type": "score",
"instructions": "How well does document billing answer the query, using only its text?",
"criteria": [
"Unrelated",
"Related but not sufficient",
"Directly answers the query"
]
}
}
}
]
}