Arkher: Planned hires, then re-ordered to the user's priorities

Environment & tools

Live Arkher (arkherbrain.vercel.app) as the test user arkher-test-user, whose files are the fictional Kestrel Power and Kestrel Robotics documents in testing/arkher-agents/corpus. No company data.

• ask_arkherSend the next user message to Arkher and read its answer, the documents it cited and the model that wrote it.
• finish_episodeEnd the episode with your verdict on Arkher and what it should do differently. The platform scores Arkher against the scenario checks.

Verification · programmatic

{
  "arkher_case_v1": {
    "id": "new-hires-followup",
    "title": "Planned hires, then re-ordered to the user's priorities",
    "split": "train",
    "difficulty": "hard",
    "tags": [
      "multi-turn",
      "follow-up",
      "ordering",
      "instruction-following"
    ],
    "source": "Slack thread 1792412306.662169 (2026-10-19); Belenus_new_hires_comparison.docx",
    "intent": "Turn 1: the planned overhead hires from the board deck appendix plus the three roles the user names. Turn 2: the user's order (procurement before the O&M GM, HR lead after the O&M GM, a VP of Sales instead of a CCO and last), a recruiter included, field technicians and ROC roles left out.",
    "turns": [
      "You're head of strategy and CHRO, a US solar O&M and software industry veteran and a former VC principal. List the hires Kestrel Power is planning, and include an HR Admin who knows technician licensing and reporting and can set up a 401k, a Health and Safety lead, and an O&M General Manager. The listing is in the latest Q3 2026 board deck. Also advise me on other hires to consider.",
      "The O&M GM must be good at legal review and contract negotiation. That's a priority over the CCO, which goes last; we'll likely hire a VP of Sales instead of a CCO. Compliance is already underway. Wait on sales and marketing. Add developers one at a time. Recruiter definitely. The HR lead comes after the O&M GM and just before the CCO. Procurement and supply chain go before the O&M GM because it's cheap. Ignore field techs and ROC roles for now; I'm planning overhead."
    ],
    "checksPerTurn": [
      {
        "must": [
          "HR Admin|HR administrator",
          "Health (?:and|&) Safety",
          "O&M (?:GM|General Manager)|(?:GM|General Manager)\\b[^\\n]{0,20}O&M"
        ],
        "mustCiteAny": [
          "Board Deck Appendix"
        ]
      },
      {
        "order": [
          "Procurement|supply chain",
          "O&M (?:GM|General Manager)|(?:GM|General Manager)\\b[^\\n]{0,20}O&M",
          "HR (?:lead|Lead|Manager|Director|Admin)",
          "\\bCCO\\b|Chief Commercial|VP (?:of )?Sales"
        ],
        "mustNot": [
          "^\\s*(?:\\d+\\.|-|\\|)\\s*\\**(?:Field )?Technician",
          "^\\s*(?:\\d+\\.|-|\\|)\\s*\\**ROC (?:Operator|operator)"
        ],
        "must": [
          "[Rr]ecruiter"
        ]
      }
    ]
  }
}