Arkher: Revenue trend: the annualized growth as well as the summer dip
Instruction
You are testing Arkher, a company document assistant, on behalf of a user. Play the user: send Arkher these messages with ask_arkher, in order, word for word, one per call, reading each answer before sending the next: 1. "You're the head of strategy at Kestrel Power and a former VC fund principal. An investor asked a question and I need you to answer it. Don't ask me any questions. The question is: \"What has the trend in revenue been in 2026 to the end of September?\"" What the user wanted: Two answers are technically true: monthly revenue fell over the summer, and annualized revenue grew 5x in twelve months. The user wants the annualized trend, with the summer dip explained as seasonal. Then call finish_episode. In `verdict`, say whether Arkher gave the user what they wanted, quoting the sentences that show it. In `reflection`, say what Arkher should do differently next time, as a rule that would also hold for other questions.
Environment & tools
Live Arkher (arkherbrain.vercel.app) as the test user arkher-test-user, whose files are the fictional Kestrel Power and Kestrel Robotics documents in testing/arkher-agents/corpus. No company data.
ask_arkherSend the next user message to Arkher and read its answer, the documents it cited and the model that wrote it.finish_episodeEnd the episode with your verdict on Arkher and what it should do differently. The platform scores Arkher against the scenario checks.
Verification · programmatic
{
"arkher_case_v1": {
"id": "revenue-trend",
"title": "Revenue trend: the annualized growth as well as the summer dip",
"split": "train",
"difficulty": "hard",
"tags": [
"nuance",
"multiple-measures",
"no-questions"
],
"source": "Slack thread 1791903326.351069 (2026-10-13); 'Immediate Arkher Dev Steps 2026-10-09' (the nuance example)",
"intent": "Two answers are technically true: monthly revenue fell over the summer, and annualized revenue grew 5x in twelve months. The user wants the annualized trend, with the summer dip explained as seasonal.",
"turns": [
"You're the head of strategy at Kestrel Power and a former VC fund principal. An investor asked a question and I need you to answer it. Don't ask me any questions. The question is: \"What has the trend in revenue been in 2026 to the end of September?\""
],
"checks": {
"must": [
"annuali[sz]ed|run[- ]rate",
"5\\s?x|five[- ]fold|fivefold",
"seasonal|summer|outage"
],
"mustCiteAny": [
"Board Deck Appendix"
],
"noQuestionToUser": true
}
}
}Reference solution
Two answers are technically true: monthly revenue fell over the summer, and annualized revenue grew 5x in twelve months. The user wants the annualized trend, with the summer dip explained as seasonal.