Scenario library

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "How many robots does Kestrel have deployed, and what margin do those contracts make?" What the user wanted: Same as wrong-company, in other words. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "Which of Kestrel Power's PPM deals are actually under signed contract today?" What the user wanted: Same as contracts-signed, in other words. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "An investor asked how Kestrel Power's revenue has developed over 2026 up to the end of September. As our strategy head, answer that for them without asking me anything first." What the user wanted: Same as revenue-trend, in other words. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "Acting as Kestrel Power's strategy head with a VC background: an investor wants to understand how we charge customers and what our per-unit economics look like. Answer them directly, and don't come back to me with questions first." What the user wanted: Same as pricing-answer, in other words. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "What does Kestrel Power charge per MW for PPM, and what can a site reach with the upside items?" 2. "So for a 150 MWdc site, what's the annual difference between the base and the upside price?" What the user wanted: The second answer uses the first: it keeps the figures already given and works out the new ask instead of starting over or asking what 'that' means. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "In two or three paragraphs of persuasive prose, with no bullet points, explain why Kestrel Power's PPM contract is worth paying double legacy O&M for." What the user wanted: Two or three paragraphs of compelling prose, no bullet lists, still grounded and cited. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "What contribution margin does Kestrel run at, and how big is the fleet?" What the user wanted: Kestrel Robotics is a separate company. Its 70% contribution margin and robot fleet must not be given as Kestrel Power's. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "How many PPM contracts has Kestrel Power signed?" What the user wanted: None of the four PPM contracts is signed: two pending final signature, two verbally awarded (Q3 board deck). The March overview's 'signed four' is out of date. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "What's my brother's dog's name?" What the user wanted: Answer from the user's own file, with the answer written by Cohere, the provider the user chose for privacy. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "What are Kestrel Power's revenue projections for the rest of 2026?" What the user wanted: Use the Q3 board deck's FY2026 forecast ($2,874K) and Q4 months, not the June deck's $5.2M plan. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "yes" The episode starts mid-conversation: Arkher has already answered the earlier turn shown in its history. What the user wanted: Arkher offered a quarterly plan; the user said yes. Arkher asked a clarifying question instead and claimed its previous answer had been cut off, which it had not. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're head of strategy and CHRO, a US solar O&M and software industry veteran and a former VC principal. List the hires Kestrel Power is planning, and include an HR Admin who knows technician licensing and reporting and can set up a 401k, a Health and Safety lead, and an O&M General Manager. The listing is in the latest Q3 2026 board deck. Also advise me on other hires to consider." 2. "The O&M GM must be good at legal review and contract negotiation. That's a priority over the CCO, which goes last; we'll likely hire a VP of Sales instead of a CCO. Compliance is already underway. Wait on sales and marketing. Add developers one at a time. Recruiter definitely. The HR lead comes after the O&M GM and just before the CCO. Procurement and supply chain go before the O&M GM because it's cheap. Ignore field techs and ROC roles for now; I'm planning overhead." What the user wanted: Turn 1: the planned overhead hires from the board deck appendix plus the three roles the user names. Turn 2: the user's order (procurement before the O&M GM, HR lead after the O&M GM, a VP of Sales instead of a CCO and last), a recruiter included, field technicians and ROC roles left out. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're the head of strategy at Kestrel Power and a former VC fund principal. An investor asked a question and I need you to answer it. Don't ask me any questions. The question is: \"What has the trend in revenue been in 2026 to the end of September?\"" What the user wanted: Two answers are technically true: monthly revenue fell over the summer, and annualized revenue grew 5x in twelve months. The user wants the annualized trend, with the summer dip explained as seasonal. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're the head of strategy at Kestrel Power and a former VC fund principal. A potential investor asked a question and I need you to draft the email in response. Don't ask me any questions; proceed as best you can. The question is: \"Help me understand your pricing model and unit economics.\"" What the user wanted: A ready-to-send email. Arkher once wrote instructions for another model (DELIVERABLE AND AUDIENCE, CONTEXT) instead of the email. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're the head of strategy at Kestrel Power. You've been a successful founder several times over and a top VC fund principal. A potential investor asked a question and I need you to answer it. Don't ask me any questions this time; proceed as best you can. The question is: \"Help me understand your pricing model and unit economics.\"" What the user wanted: Answer from the Q3 board deck: 80% contribution margin, not the March overview's 85% to 90%; contracts pending or verbally awarded, not signed; 4 technicians, not 12. The user said not to ask questions. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're our head of strategy. You've been a successful founder many times over and know differentiation and moats well. You're also a former top McKinsey consultant and a former VC fund principal. A VC fund deciding whether to join our Series A asked what Kestrel Power's moat and defensible differentiation are. Give a detailed, comprehensive answer. Compare us to the incumbent software vendor (Brightline Analytics), the O&M contractor (SunGuard Services) and the high-voltage firm (VoltWorks)." What the user wanted: A detailed answer a VC fund can read, comparing the company with the incumbent software vendor, the O&M contractor and the high-voltage firm by name. Scott preferred Arkher here because Claude turned it into a discussion. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.