Scenario library
Tasks, environments and tools that grow self-learning agents. Agents attempt scenarios, get verified, reflect, and keep the lessons that help.
- v4arkher
Arkher: Write it as persuasive prose when asked
You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "In two or three paragraphs of persuasive prose, with no bullet points, explain why Kestrel Power's PPM contract is worth paying double legacy O&M for." What the user wanted: Two or three paragraphs of compelling prose, no bullet lists, still grounded and cited. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.
mediumtrain2 toolsprogrammatic2 attempts · 100% pass - v4arkher
Arkher: Planned hires, then re-ordered to the user's priorities
You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're head of strategy and CHRO, a US solar O&M and software industry veteran and a former VC principal. List the hires Kestrel Power is planning, and include an HR Admin who knows technician licensing and reporting and can set up a 401k, a Health and Safety lead, and an O&M General Manager. The listing is in the latest Q3 2026 board deck. Also advise me on other hires to consider." 2. "The O&M GM must be good at legal review and contract negotiation. That's a priority over the CCO, which goes last; we'll likely hire a VP of Sales instead of a CCO. Compliance is already underway. Wait on sales and marketing. Add developers one at a time. Recruiter definitely. The HR lead comes after the O&M GM and just before the CCO. Procurement and supply chain go before the O&M GM because it's cheap. Ignore field techs and ROC roles for now; I'm planning overhead." What the user wanted: Turn 1: the planned overhead hires from the board deck appendix plus the three roles the user names. Turn 2: the user's order (procurement before the O&M GM, HR lead after the O&M GM, a VP of Sales instead of a CCO and last), a recruiter included, field technicians and ROC roles left out. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.
hardtrain2 toolsprogrammatic2 attempts · 100% pass