Arkher: A private fact, answered on Cohere
Instruction
You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "What's my brother's dog's name?" What the user wanted: Answer from the user's own file, with the answer written by Cohere, the provider the user chose for privacy. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.
Environment & tools
Live Arkher (arkherbrain.vercel.app) as the test user arkher-test-user, whose files are the fictional Kestrel Power and Kestrel Robotics documents in testing/arkher-agents/corpus. No company data.
ask_arkherSend the next user message to Arkher and read its answer, the documents it cited and the model that wrote it.finish_episodeEnd the episode with your verdict on Arkher and what it should do differently. The platform scores Arkher against the scenario checks.
Verification · programmatic
{
"arkher_case_v1": {
"id": "private-fact-cohere",
"title": "A private fact, answered on Cohere",
"split": "train",
"difficulty": "easy",
"tags": [
"privacy",
"provider-choice",
"personal-files"
],
"source": "Slack 2026-10-16 09:00 EDT (keeping a private fact away from Claude)",
"intent": "Answer from the user's own file, with the answer written by Cohere, the provider the user chose for privacy.",
"uploads": [
"Family notes.md"
],
"model": {
"provider": "cohere",
"id": "command-a-03-2025"
},
"turns": [
"What's my brother's dog's name?"
],
"checks": {
"must": [
"\\bMoose\\b"
],
"answeredByProvider": "Cohere"
}
}
}Reference solution
Answer from the user's own file, with the answer written by Cohere, the provider the user chose for privacy.