Arkher: Revenue trend (reworded)
Instruction
You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "An investor asked how Kestrel Power's revenue has developed over 2026 up to the end of September. As our strategy head, answer that for them without asking me anything first." What the user wanted: Same as revenue-trend, in other words. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.
Environment & tools
Live Arkher (arkherbrain.vercel.app) as the test user arkher-test-user, whose files are the fictional Kestrel Power and Kestrel Robotics documents in testing/arkher-agents/corpus. No company data.
ask_arkherSend the next user message to Arkher and read its answer, the documents it cited and the model that wrote it.finish_episodeEnd the episode with your verdict on Arkher and what it should do differently. The platform scores Arkher against the scenario checks.
Verification · programmatic
{
"arkher_case_v1": {
"id": "revenue-trend-reworded",
"title": "Revenue trend (reworded)",
"split": "heldout",
"difficulty": "hard",
"tags": [
"nuance",
"multiple-measures",
"no-questions",
"reworded"
],
"source": "Held-out twin of revenue-trend",
"intent": "Same as revenue-trend, in other words.",
"turns": [
"An investor asked how Kestrel Power's revenue has developed over 2026 up to the end of September. As our strategy head, answer that for them without asking me anything first."
],
"checks": {
"must": [
"annuali[sz]ed|run[- ]rate",
"5\\s?x|five[- ]fold|fivefold",
"seasonal|summer|outage"
],
"mustCiteAny": [
"Board Deck Appendix"
],
"noQuestionToUser": true
}
}
}Reference solution
Same as revenue-trend, in other words.