Scenario library

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "What does Kestrel Power charge per MW for PPM, and what can a site reach with the upside items?" 2. "So for a 150 MWdc site, what's the annual difference between the base and the upside price?" What the user wanted: The second answer uses the first: it keeps the figures already given and works out the new ask instead of starting over or asking what 'that' means. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "yes" The episode starts mid-conversation: Arkher has already answered the earlier turn shown in its history. What the user wanted: Arkher offered a quarterly plan; the user said yes. Arkher asked a clarifying question instead and claimed its previous answer had been cut off, which it had not. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.

You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're head of strategy and CHRO, a US solar O&M and software industry veteran and a former VC principal. List the hires Kestrel Power is planning, and include an HR Admin who knows technician licensing and reporting and can set up a 401k, a Health and Safety lead, and an O&M General Manager. The listing is in the latest Q3 2026 board deck. Also advise me on other hires to consider." 2. "The O&M GM must be good at legal review and contract negotiation. That's a priority over the CCO, which goes last; we'll likely hire a VP of Sales instead of a CCO. Compliance is already underway. Wait on sales and marketing. Add developers one at a time. Recruiter definitely. The HR lead comes after the O&M GM and just before the CCO. Procurement and supply chain go before the O&M GM because it's cheap. Ignore field techs and ROC roles for now; I'm planning overhead." What the user wanted: Turn 1: the planned overhead hires from the board deck appendix plus the three roles the user names. Turn 2: the user's order (procurement before the O&M GM, HR lead after the O&M GM, a VP of Sales instead of a CCO and last), a recruiter included, field technicians and ROC roles left out. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.