#1 · Retry chat calls the gateway refused before the function started

measured

infra change to arkher · by claude-code · recorded Oct 29, 05:46 PM · updated Oct 29, 06:15 PM

Before

75%

benchmark #2, 2 no answer

After

88%

benchmark #3, 0 no answer

Change

▲ +13 pts

Feedback

none yet

Diagnosis

Symptom

In two full runs of the 16 Kestrel scenarios, the same two scenarios (moat-vc, pricing-email) failed after 10 s with 503 {"code":"BOOT_ERROR","message":"Function failed to start (please check logs)"}. Both reran clean. A burst of 3 simultaneous hi requests reproduced it (200, 200, 503); 1 and 5 requests did not.

What the logs show

Edge gateway: every POST | 503 | /functions/v1/chat in the last 24 h (16:56, 17:19 twice, 17:36 UTC) is one of these test bursts, each against a cold function. 17:19:51 was 32 s after the deploy of chat v157.
Function logs at the same moments: only booted (time: 35-931ms) and shutdown. No error, no request handled for the refused calls. The gateway refused them before any worker ran.
chat/index.ts has no top-level await; boot work is imports only, so there is nothing in the function's own start to fix.

Conclusion

A cold chat/ hit by several requests at once can have some refused at the gateway. Nothing was searched, answered or written for a refused request, so sending it again is safe. Every other 503 (no working query embedder) must stay unretried.

Change

lib/chat-fetch.ts fetchChat: retry only 503 + code: BOOT_ERROR, up to 3 tries, 0.8 s then 1.6 s apart; abort stops the wait. Used by the chat page (useChat({ fetch })) and /api/agent/ask. The SyntaxTual harness (testing/arkher-agents/arkher.mjs) asks Arkher as the chat page does, so it gets the same retry once the page ships it.

How it is measured

Benchmark before (current production, harness without retry) and after (page and harness with retry), both started against a cold function. Expected: the no answer runs go to 0; answer-check results unchanged. Retries are logged by the harness so a pass that needed one is visible.

What the engineer read

logSupabase edge gateway, functions/v1/chat, last 24 h

2026-10-29T17:36:26 POST | 503 | /functions/v1/chat
2026-10-29T17:19:51 POST | 503 | /functions/v1/chat (x2)
2026-10-29T16:56:54 POST | 503 | /functions/v1/chat (x2)

logSupabase function logs, chat, 17:23-17:39 UTC

booted (time: 35ms ... 931ms), shutdown; no error line for the refused requests

noteburst reproduction

1 concurrent: 200 | 3 concurrent: 200 200 503 BOOT_ERROR | 5 concurrent: 200 x5

codeBrain supabase/functions/chat/index.ts

no top-level await; Deno.serve at line 1077; imports only at start

codeBrain app/chat/page.tsx:575, app/api/agent/ask/route.ts:77

the two production callers of functions/v1/chat that users and Hermes go through

benchmark2

Before (cold start, no retry): 12/16. No answer 2 (Signed or not reworded, Revenue trend reworded: HTTP 503 after 12 s). Answer failed 2 (pricing 85-90% check, moat missing competitor names).

benchmark3

After (cold start, with retry): 14/16. No answer 0. The same two scenarios that got 503 before needed 1 retry each and passed. Answer failed 2, on different scenarios (Two companies reworded gave 85-90% as current; back-and-forth turn 2 asked a question).

notereading the change

The fix removes the no-answer failures (2 -> 0). The answer failures moved between scenarios from run to run, so they are model variance, not an effect of this change; one run each cannot show more. Answer-quality work is a separate improvement.

The edit

Repository: InteCorr-ai/Brain
Branch: main
Commit: ad899a2
Files: lib/chat-fetch.tslib/__tests__/chat-fetch.test.tsapp/chat/page.tsxapp/api/agent/ask/route.ts
Summary: Merged to Brain main as ad899a2 (merge of d6b085d) and deployed to Vercel production at 13:47 EDT. fetchChat retries only the gateway's BOOT_ERROR 503. Brain: 1891 unit tests pass, build exit 0. The SyntaxTual harness asks Arkher as the chat page does, so it got the same retry (be79ae6) and records retries per turn.

Before and after, by scenario

passedanswer failed a checkno answer (HTTP error)

Scenario	Before #2	After #3
"yes" accepts the offer Arkher just made	passed	passed
A private fact, answered on Cohere	passed	passed
A productive back-and-forth: the follow-up builds on the first answer	passed	failed
Draft the investor email itself, not a prompt	passed	passed
Moat for a VC, against three named competitors	failed	passed
Planned hires, then re-ordered to the user's priorities	passed	passed
Pricing and unit economics from the newest deck, no questions back	failed	passed
Projections from the newest deck, not the June plan	passed	passed
Revenue trend: the annualized growth as well as the summer dip	passed	passed
Signed or not: pending and verbal awards are not signed	passed	passed
Two companies with one name: keep Kestrel Robotics out	passed	passed
Write it as persuasive prose when asked	passed	passed
Pricing and unit economics (reworded)held-out	passed	passed
Revenue trend (reworded)held-out	no answer	passed
Signed or not (reworded)held-out	no answer	passed
Two companies with one name (reworded)held-out	passed	failed