Arkher: Two companies with one name: keep Kestrel Robotics out

Environment & tools

Live Arkher (arkherbrain.vercel.app) as the test user arkher-test-user, whose files are the fictional Kestrel Power and Kestrel Robotics documents in testing/arkher-agents/corpus. No company data.

• ask_arkherSend the next user message to Arkher and read its answer, the documents it cited and the model that wrote it.
• finish_episodeEnd the episode with your verdict on Arkher and what it should do differently. The platform scores Arkher against the scenario checks.

Verification · programmatic

{
  "arkher_case_v1": {
    "id": "wrong-company",
    "title": "Two companies with one name: keep Kestrel Robotics out",
    "split": "train",
    "difficulty": "medium",
    "tags": [
      "wrong-company",
      "entity-disambiguation"
    ],
    "source": "Renu Robotics files answering Belenus questions (Scott's eval, 2026-10-22); Slack 2026-10-16 09:25 EDT (Renu folder should not be ingested)",
    "intent": "Kestrel Robotics is a separate company. Its 70% contribution margin and robot fleet must not be given as Kestrel Power's.",
    "turns": [
      "What contribution margin does Kestrel run at, and how big is the fleet?"
    ],
    "checks": {
      "must": [
        "Kestrel Robotics",
        "separate|different compan|two compan|which (?:one|company)"
      ],
      "notAttributed": [
        {
          "subject": "Kestrel Power",
          "pattern": "\\b140 robots\\b|\\b70\\s?%",
          "unless": "Kestrel Robotics"
        }
      ],
      "onlyAsSuperseded": [
        "85\\s?%?\\s?(?:-|–|to)\\s?90\\s?%"
      ]
    }
  }
}