Overview
Each item is a development task in a real codebase, asked the way developers ask a coding agent in their editor: a short request, often incomplete, whose intent has to be recovered from the code, for a change that usually spans several files. An item has two parts: a re-runnable task environment (the codebase and its full toolchain in an offline Docker image, the request, a hidden grader of tests and a rubric, a reference solution, and wrong and alternative solutions with their grades), usable for RL; and a full trajectory (Claude Opus 5.5 at max effort doing the task in Claude Code: every model request, reasoning summaries, every tool call and result, the final change and its grade), usable for SFT.
Task types
Each sample is a different task type. The middle column gives the closest problem category CursorBench lists. A sample number jumps to its card.
| Task type | Closest CursorBench category | Sample |
|---|---|---|
| Production investigation and fix | investigation | 1 |
| Design-doc implementation | design adherence | 2 |
| Feature iteration | edit | 3 |
| Intent understanding | intent understanding | 4 |
| Code review | code review, bugfinding | 5 |
| Codebase Q&A | codebase understanding | 6 |
| Implementation planning | planning | 7 |
| Bug fix | bugfix | 8 |
What you get
<item>/
├── task/
│ ├── task.json # task type, language, capability tags, difficulty, request style
│ ├── prompt.md # the developer's request: the agent's only input
│ ├── workspace/ # files that come with the task: logs, a design doc, a pending change
│ ├── env/ # Dockerfile, the codebase at its starting commit, test runner
│ ├── grader/ # hidden: rubric, grading-agent prompt, tests, reference solution
│ └── validation/ # wrong and alternative solutions, and how the grader judged them
└── trajectory/
├── traj/ # request-level API log: full context and response of each call
├── steps.jsonl # per step: reasoning summary, text, tool calls and results
├── final.patch # the agent's change
├── final_message.md # the agent's report to the developer
├── result.json # steps, tool calls, time, tokens
└── grade.json # test results, verdict and evidence per rubric item, quality scores
base/ # base image every environment builds on
index.json # item overview
README.md # format, building, running and grading
- Two kinds of task: in code tasks the agent changes the code, and hidden fail-to-pass and regression tests run next to the rubric; in review, Q&A and planning tasks the agent answers without editing, and the rubric judges the answer against a reference.
- Rubric: the required items, together with the hidden tests in code tasks, decide the result; approaches the rubric lists as acceptable are not penalized. The grading agent also scores correctness, code quality, scope and communication from 1 to 5.
- Isolation: the environment runs offline, and the agent never sees the grader or the reference.
Samples
Eight tasks, each a different task type in a different language, at an estimated difficulty of 3 to 5 on a 1–5 scale. The trajectories run from 10 to 106 agent steps (4 to 75 minutes). Each card gives the developer’s request verbatim, what the hidden grader checks, the key moments of the trajectory and how it ended; the full rubric, the agent’s final message and its change are folded underneath.
Sample 1 of 8
Stop cold h2 bursts from opening one TLS connection per request
undici: a JavaScript HTTP/1.1 and HTTP/2 client for Node.js, with Client, Pool, Agent and BalancedPool dispatchers and fetch.
The developer hands over canary logs showing that, after a quiet spell, undici opens one TLS connection per request to an HTTP/2-over-TLS origin, and asks for the root cause and a fix in undici. A correct result serves every cold burst through Pool, Agent, BalancedPool or fetch on one h2 session (any body, also across event-loop ticks), keeps one connection per request for HTTP/1.1 origins, never strands or mass-fails queued requests when the first connection fails, and keeps the Dispatcher contract (‘drain’ after a false dispatch(), no pool left flagged busy).
Why it is hard
The logs point at the symptom quickly, but the cause sits in the interplay of Pool, PoolBase’s drain bookkeeping and Client’s busy state, and the fix has many ways to be almost right. The obvious Client-side shortcut serializes HTTP/1.1 origins; gating only until ALPN is first known breaks the second cold burst; releasing at the end of the TLS handshake still lets a request that arrives before the h2 SETTINGS open a second connection; holding requests outside the pool queue strands them when the first connection fails. A wrong busy flag or 'drain' hangs BalancedPool or makes Agent keep failed origins, which only one of about 3,300 existing tests notices.
The developer’s request, verbatim
Handing off: canary edge-gw (undici main) → ledger, h2 over TLS. Every reconcile run after a quiet spell opens one TLS connection per request instead of multiplexing on one session; ledger's ingress then resets us. Logs from both sides: @logs/. Find the root cause in undici and fix it here. Bare Pool does it too, also for fetch POSTs and for requests a few ms apart, not just same-tick GETs. Keep fan-out for HTTP/1.1 upstreams (pricing).
Files that come with the task
undici/
└── logs/
├── edge-gw-canary.log # 2,371 lines
└── ledger-ingress.log # 578 lines
What the hidden grader checks
- A cold burst through Pool, Agent, BalancedPool or fetch to an h2 origin shares one session, for any body and across event-loop ticks.
- After the session closes, the next cold burst opens one connection, for repeated Agent bursts (fetch POSTs too) and same-tick bursts on one long-lived Pool.
- HTTP/1.1 origins keep their fan-out: concurrent requests get separate connections, up to the connections cap, and are not serialized.
- If the first connection resets, the rest still complete; if all connections fail, every request rejects; close() waits for queued requests, destroy() rejects them.
- 'drain' always follows a false dispatch(), and a pool whose connection failed or idled out is not left busy, so Agent removes it.
- The final message names the cause (a still-connecting Client reports busy, so Pool opens one connection per request); existing tests pass, none weakened.
- Fail-to-pass tests
- 14
- Regression tests
- 3,300
- Rubric items
- 11
- Wrong solutions rejected
- 11
- Alternative solutions accepted
- 2
Key moments of the trajectory
Steps 2–4 Counting message types in both logs, the agent found ledger’s ingress rate-limits new connections at 20, dropping the rest, and that run 4412, starting after the ledger session idled out, opened 26 connections.
Steps 6–8, 10 The agent traced the fan-out in code: Pool opens a new Client when existing ones report busy, and a connecting Client with one queued request stays busy until ALPN completes (for h2, until SETTINGS).
Steps 20, 31–32, 34 A local repro opened 26 TLS connections for a same-tick h2 burst; the agent also found kMaxConcurrentStreams was never set from the option, leaving the h2 limit effectively 1 before SETTINGS.
Steps 36–40 The agent made Pool hold new requests while a connection that may negotiate h2 is opening; in the rerun repro, every h2 case used one connection and HTTP/1.1 still opened 26.
Steps 49, 52–53, 62–63 The agent’s 9 new tests in test/http2-pool-fan-out.js all passed with the fix; on an untouched copy of the code, the 6 h2 tests failed and the 3 HTTP/1.1/failure tests passed.
Steps 57, 64–65, 67–69 An existing Socks5 test exposed a regression where a failed connection left the per-origin pool flagged busy, so it was never removed; the agent fixed it and showed a new Agent test catches it.
Steps 78–79 An incident replay with a 20-connections-per-second limiter reproduced every failing run of the canary log on the unpatched code (exactly 20 successes each); with the fix, each run used one connection and no request failed.
Step 95 The final message explains the root cause, notes which test failures stem from the sandbox, flags pricing’s extra handshake after idle as a decision for the developer, and lists side findings like edge-gw’s content-length mismatch.
Result
- Agent steps
- 95
- Tool calls
- 109
- Minutes
- 75
- Files changed
- 8
- Lines added / removed
- +416 / −33
The agent changed undici’s Pool to hold new requests while a connection that may negotiate h2 is still opening, then send them to the h2 session or fan them out on HTTP/1.1 or failure; it also fixed two Client issues that made an h2 connection report busy before SETTINGS. The grader found the root cause in the final message correct and no false claims. It noted one gap, outside what the task requires and not mentioned in the message: on a long-lived bare Pool, one failed connection attempt switches the holding off until a later connection succeeds.
Quality scores from the grader (1–5)
- Correctness
- 4
- Code quality
- 4
- Scope
- 5
- Communication
- 4
Rubric items (11)
Required (8)
- C1 · A burst of concurrent requests sent through a Pool that has no established connection (and therefore through Agent / fetch and through a BalancedPool upstream), to an https origin that negotiates HTTP/2 via ALPN, is served over exactly one TLS connection and one h2 session with all requests multiplexed concurrently. This holds with the default unlimited connections and with a cap greater than 1; for requests without a body and for requests whose body is a string, Buffer, stream or (async) iterable, which includes every fetch() body; when the requests are dispatched on different event-loop turns while the first connection of a fresh Pool is still being set up (TLS handshake and the HTTP/2 SETTINGS exchange), not only in one tick; and for a caller that honours dispatch()'s return value.
- C2 · The fix holds for every cold burst, not only the first one: after the h2 session has gone away (idle timeout, disconnect), the next burst again opens exactly one connection. This is required for repeated bursts through an Agent (any kind of request, including fetch POSTs) and for repeated same-tick bursts on one long-lived Pool object. Knowing that the origin spoke h2 last time must not switch the protection off, because a reconnecting Client is busy in exactly the same way as a new one.
- C3 · HTTP/1.1 origins keep their connection fan-out: N concurrent requests to an https origin that negotiates http/1.1 (or with allowH2: false) are still spread over N connections and are in flight at the same time, bounded by connections; they are not serialized on a single socket and do not wait for the first response. This holds through Pool, Agent / fetch (with and without bodies), BalancedPool, and for a caller that honours dispatch()'s return value.
- C4 · Requests that were held back while the first connection was being established are never stranded and do not all fail with it: if that connection is reset, the requests that were not on it are dispatched on another connection and complete; if the origin never lets a connection through, every request is rejected with an error; close() still waits for them and destroy() still rejects them.
- C5 · The final message identifies the actual root cause, which is in undici's Pool and not in the upstream: a Client whose connection is still being established counts as busy (its dispatch() returns false, because Client[kBusy] only stops treating pending requests as busy once an h2 context exists, i.e. after ALPN, and for h2 after the peer's SETTINGS), so Pool (kGetDispatcher in lib/dispatcher/pool.js, reached from PoolBase[kDispatch]) creates a new Client, and with it a new TLS connection, for every further request that arrives before the first connection is usable, up to connections. An established h2 session multiplexes fine, which is why only bursts that start without a connection (after the idle timeout closed the session) fan out.
- C6 · Existing behaviour keeps working: the repository's existing test suite still passes, and no existing test was deleted, skipped or weakened to get there.
- C7 · While it waits for a connection the pool keeps honouring the Dispatcher contract that the rest of the library builds on: (a) whenever dispatch() returned false, a 'drain' event follows as soon as the pool can take requests again: when the h2 session is usable, as soon as the connection turns out to be HTTP/1.1, and when the connection attempt failed; (b) a pool with nothing left to do is not left flagged busy, so an Agent (and Socks5ProxyAgent) still drops an origin whose connection attempt failed or whose connections idled out, and its maxOrigins slot is released; (c) a BalancedPool over https upstreams keeps working: no request hangs, and HTTP/1.1 upstreams keep their fan-out.
- C11 · The final message accurately describes what was changed and what was verified, and claims nothing that is not true of the final tree.
Quality (3)
- C8 · No unrelated changes or leftovers. In particular the things the logs show that are not the cause are left alone: the RequestContentLengthMismatchError warning (edge-gw sends a content-length that does not match its body), GOAWAY handling, retrying connections reset during the handshake, keep-alive / idle timeout defaults, the TLS session cache. No repro scripts, captured output, debug prints or edited log files are left in the tree.
- C9 · The change fits the codebase: the logic lives in Pool / PoolBase (where clients are created) rather than in Agent or in special cases inside Client[kBusy]; private state is kept in symbol-keyed fields like the existing kClients / kNeedDrain / kOptions; no new public option is needed to get the behaviour; requests are only held back when h2 is actually possible (https with allowH2 not disabled, or useH2c) and only for real Client instances, not for plain http origins or custom factories; the changed files pass the repository's eslint (standard) configuration.
- C10 · The notes in docs/docs/api/Pool.md and docs/docs/api/Agent.md that describe the old behaviour (the default pool opens a client per concurrent dispatch, which defeats HTTP/2 multiplexing; cap connections, for example connections: 1) are updated to describe the new behaviour.
The agent’s final message
Every reconcile run after a quiet spell now uses one h2 connection, and pricing (HTTP/1.1) still fans out. Replaying the incident locally on unpatched main reproduces your log exactly, and with the fix every run succeeds. Nothing is committed yet; the changes are in the working tree on main.
Root cause. The ledger session closes after 4 s idle (keepAliveTimeout: 4000), and the Agent then drops ledger's Pool. The next reconcile burst lands on a client that hasn't connected yet. Until ALPN finishes, that client calls itself "busy" as soon as one request is queued. Pool reads busy as "open another connection", so every request got its own TLS connection. Ledger's limiter drops connections past 20 per second, which is why every failing run shows exactly ok: 20. Run 4416 succeeded only because its session happened to still be open.
The same busy signal fires for fetch POSTs, for requests arriving while the first handshake is in flight, and on a bare Pool. A previous change (commit …) hit this, backed out an attempt that made h1 serialize, and left the Pool-level fix for later. This is that fix.
Two smaller bugs widened the window:
- The client's
maxConcurrentStreamswas never set from the option. Until the server's SETTINGS frame arrived, the h2 limit was effectively 1, which contradicts the Client docs. - The rule "open only one stream until SETTINGS arrive" also made the client report busy, so Pool fanned out in that window too.
Fix
lib/dispatcher/pool.js: while a connection that might negotiate h2 is still opening, Pool holds new requests in its own queue instead of opening more connections.- It holds only as many as that connection could take (
maxConcurrentStreams), so bursts over 100, ormaxConcurrentStreams: 1, still open connections in parallel. - If the connection comes back h2, the held requests go onto its session.
- If it comes back HTTP/1.1, they fan out right away, and Pool remembers the origin is h1 so later bursts don't wait.
- If the connection attempt fails, each held request gets its own connection attempt.
- It holds only as many as that connection could take (
lib/dispatcher/pool-base.js: a small routine that sends queued requests through normal client selection; the h1 fan-out needs it.lib/dispatcher/client.jsandclient-h2.js: setmaxConcurrentStreamsfrom the option, and the one-stream-until-SETTINGS rule now only limits writes.- Docs: updated the Pool.md and Agent.md notes that told users to cap
connectionsto get h2 multiplexing.
Verification
- Incident replay: an h2 server with a 20-connections-per-second limiter, edge-gw's Agent options, and 8 runs of 24–40 GETs plus a journal POST after idle.
- On
main: 20 ok per run, 87 failures in total. - With the fix: 1 connection per run, 0 failures.
- On
- New tests in
test/http2-pool-fan-out.js(10):- h2 scenarios: cold burst, requests 10 ms apart, burst after idle, Agent with fetch POSTs.
- h1 fan-out, a failed first connection, the capacity limit, and Agent pool cleanup.
- The 6 h2 tests fail on
mainand pass with the fix.
- Existing suites: unit 1530 pass, node-test 232 pass, fetch-h2 and interceptors 344 pass, lint clean.
- The failures I saw also happen on
mainin this sandbox, which has no DNS or IPv6:connect-timeout.jshangs, plus 4 autoSelectFamily tests and 11 DNS-interceptor tests fail.
- The failures I saw also happen on
- A leak I introduced and fixed along the way: an early version left a pool marked busy after a failed connection, so Agent kept one dead pool per failing origin. An existing Socks5 test caught it; I fixed it and added an Agent test that fails without the fix.
Decision for you: pricing's first burst after idle. A pool that doesn't yet know an origin's protocol waits for one handshake before fanning out. Once it knows the origin is h1, it fans out with the same timing as before. But Agent drops idle pools, so pricing pays that one handshake (about 3 ms in your logs) on its first burst after each quiet spell. Avoiding it would mean Agent remembering each origin's protocol after dropping its pool; I left that out to keep the change contained.
Found but not changed
- The
UND_ERR_REQ_CONTENT_LENGTH_MISMATCHwarning on everyPOST /v2/journalcomes from edge-gw itself: it sends acontent-lengthheader that doesn't match the body. It's unrelated to the resets. h2Options.maxConcurrentStreamsthrows unlessconnectionWindowSizeis also set, because the validation atclient.js:241checks the wrong field.- RoundRobinPool still opens one connection per request on a cold h2 burst.
The agent’s change (excerpt)
diff --git a/lib/dispatcher/client-h2.js b/lib/dispatcher/client-h2.js
--- a/lib/dispatcher/client-h2.js
+++ b/lib/dispatcher/client-h2.js
@@ -351,7 +351,11 @@ function connectH2 (client, socket) {
* @returns {boolean}
*/
busy (request) {
- if (session[kRemoteSettings] === false && client[kRunning] > 0) {
+ // Until the server's SETTINGS arrive its stream limit is unknown, so only
+ // one stream is opened. This gates writing a request, not Client[kBusy]
+ // (busy(null)): the Client keeps accepting requests meanwhile, so a Pool
+ // multiplexes them onto this session instead of opening another one.
+ if (request != null && session[kRemoteSettings] === false && client[kRunning] > 0) {
return true
}
diff --git a/lib/dispatcher/client.js b/lib/dispatcher/client.js
--- a/lib/dispatcher/client.js
+++ b/lib/dispatcher/client.js
@@ -85,10 +85,10 @@ function emitH2OptionsNamespaceWarning (optName) {
}
// Protocol-aware dispatch ceiling. h1 RFC7230 pipelining is unrelated to h2
-// stream multiplexing — over h2 the ceiling is the (server-confirmed)
-// maxConcurrentStreams. Before a context is attached we use the h1
-// pipelining factor; once h2 attaches the queued requests can drain in
-// one batch up to maxConcurrentStreams.
+// stream multiplexing — over h2 the ceiling is maxConcurrentStreams (the
+// configured value until the server's SETTINGS replace it). Before a context
+// is attached we use the h1 pipelining factor; once h2 attaches the queued
+// requests can drain in one batch up to maxConcurrentStreams.
function getMaxConcurrent (client) {
if (client[kHTTPContext]?.version === 'h2') {
return client[kMaxConcurrentStreams]
@@ -335,6 +335,8 @@ class Client extends DispatcherBase {
initialWindowSize: h2Options?.initialWindowSize ?? initialWindowSize ?? 262144
}
}
+ // The h2 dispatch ceiling until the server's SETTINGS frame replaces it.
+ this[kMaxConcurrentStreams] = this[kHTTP2Options].maxConcurrentStreams
// kQueue is built up of 3 sections separated by
// the kRunningIdx and kPendingIdx indices.
diff --git a/lib/dispatcher/pool-base.js b/lib/dispatcher/pool-base.js
--- a/lib/dispatcher/pool-base.js
+++ b/lib/dispatcher/pool-base.js
@@ -17,6 +17,7 @@ const kGetDispatcher = Symbol('get dispatcher')
const kHasDispatcher = Symbol('has dispatcher')
const kAddClient = Symbol('add client')
const kRemoveClient = Symbol('remove client')
+const kDispatchQueued = Symbol('dispatch queued')
class PoolBase extends DispatcherBase {
[kQueue] = new FixedQueue();
@@ -169,6 +170,44 @@ class PoolBase extends DispatcherBase {
return !this[kNeedDrain]
}
+ // Hand queued requests to whatever dispatchers [kGetDispatcher] offers now,
+ // which may mean new ones, rather than waiting for existing ones to drain.
+ [kDispatchQueued] () {
+ if (!this[kNeedDrain]) {
+ return
+ }
+
+ const queue = this[kQueue]
+
+ // Dispatchers created below get their requests from this loop, so keep
+ // kAddClient from scheduling a drain for each of them. This also makes a
+ // nested call, e.g. from a connector failing synchronously, return early.
+ this[kNeedDrain] = false
+
+ while (!queue.isEmpty()) {
+ const dispatcher = this[kGetDispatcher]()
+ if (!dispatcher) {
+ break
+ }
+
+ const item = queue.shift()
+ this[kQueued]--
+ if (!dispatcher.dispatch(item.opts, item.handler)) {
+ dispatcher[kNeedDrain] = true
+ }
+ }
+
+ this[kNeedDrain] = !queue.isEmpty() || (!this.closed && !this[kHasDispatcher]())
+
+ if (!this[kNeedDrain]) {
+ this.emit('drain', this[kUrl], [this])
+ }
+
+ if (this[kClosedResolve] && queue.isEmpty()) {
+ return this[kClose]().then(this[kClosedResolve])
+ }
+ }
+
[kHasDispatcher] () {
for (let i = 0; i < this[kClients].length; i++) {
const dispatcher = this[kClients][i]
@@ -228,5 +267,6 @@ module.exports = {
kAddClient,
kRemoveClient,
kGetDispatcher,
- kHasDispatcher
+ kHasDispatcher,
+ kDispatchQueued
}
diff --git a/lib/dispatcher/pool.js b/lib/dispatcher/pool.js
--- a/lib/dispatcher/pool.js
+++ b/lib/dispatcher/pool.js
@@ -7,24 +7,51 @@ const {
kAddClient,
kGetDispatcher,
kHasDispatcher,
- kRemoveClient
+ kRemoveClient,
+ kDispatchQueued
} = require('./pool-base')
const Client = require('./client')
const {
InvalidArgumentError
} = require('../core/errors')
const util = require('../core/util')
-const { kUrl } = require('../core/symbols')
+const { kUrl, kConnected, kSize, kQueued, kHTTPContext, kMaxConcurrentStreams } = require('../core/symbols')
const buildConnector = require('../core/connect')
const kOptions = Symbol('options')
const kConnections = Symbol('connections')
const kFactory = Symbol('factory')
… 498 more lines
Sample 2 of 8
Let axe agents use tools from remote MCP servers, per an agreed spec
axe: a Go command-line tool that runs LLM agents defined in TOML files, with built-in tools, sub-agents and Anthropic, OpenAI and Ollama providers.
The developer asks the agent to implement an already-agreed spec for using tools from remote MCP servers, keeping the team’s decisions even where it would choose differently. A correct result validates [[mcp_servers]] config, connects via go-sdk v1.4.0 (sse or streamable-http) with ${VAR} header expansion before the first LLM call, offers the tools under their own names with flattened schemas (built-ins win), and routes calls in every dispatch path, including isolated sub-agent sessions, all within --timeout.
Why it is hard
The change spans axe’s agent config, a new MCP client layer, the CLI and the tool layer, against a go-sdk v1.4.0 API that has to be learned from the module cache with no network. The spec is full of details that are easy to miss: call_agent counts as a built-in only with sub_agents, built-ins an axe agent has not enabled don’t shadow MCP tools, arguments are converted only for declared integer, number and boolean parameters, the --timeout context must exist before connecting and reach every MCP call, sessions close on every exit path, and sub-agents never share the parent’s sessions.
The developer’s request, verbatim
can you implement @docs/plans/027_mcp_tool_support_spec.md? the team already agreed on the decisions in there before i joined, so pls stick to them even where youd have done it differently
Files that come with the task
Design doc: 027_mcp_tool_support_spec.md (261 lines)
Specification: MCP Tool Support (client side)
Status: Draft Version: 1.0 Created: 2026-03-05 Scope: Let agents use tools exposed by remote MCP servers, next to the built-in tools
1. Purpose
Right now an agent can only call axe's built-in tools (read_file, run_command, url_fetch, ...).
A lot of useful tooling ships as MCP (Model Context Protocol) servers these days, so this
milestone turns axe into an MCP client:
- an agent declares MCP servers in its TOML,
- at the start of
axe runaxe connects to each one and asks for its tools (tools/list), - those tools are offered to the model alongside the built-ins,
- when the model calls one, axe routes the call to the server that owns it (
tools/call) and hands the result back to the model.
Axe is a client only. Exposing axe's own tools as an MCP server is not part of this.
2. Design Decisions
Agreed in planning, binding for the implementation:
-
Official SDK, pinned. Use the official Go SDK,
github.com/modelcontextprotocol/go-sdk(packagemcp), pinned to v1.4.0 ingo.mod(the version we tried it with). This is the only new dependency the milestone may add. No hand-rolled JSON-RPC / SSE code. -
Two transports.
transport = "sse"uses the SDK'sSSEClientTransport,transport = "streamable-http"usesStreamableClientTransport. Nothing else. In particular nostdio: axe does not launch server processes (executor, not process manager). -
Connect eagerly, once per run. All declared servers are connected, and their tools listed (
tools/list, once — no refresh), before the first LLM call. One session per server per run. Sessions stay open for the whole conversation loop and are closed when the run ends — on every exit path, including a startup that fails halfway (close whatever was already opened). -
Fail fast. If any declared server can't be used — connection refused, DNS/TLS error, handshake rejected,
tools/listfailing, or the deadline running out — the run stops with exit code 3 before the LLM is ever called. The error names the server. No retries. -
One deadline. MCP connect, discovery and tool calls run under the same
--timeoutdeadline as the rest of the run. There is no per-server timeout setting. -
No name prefixing. MCP tools are offered to the model under the exact name the server gives them, with the server's description and a flattened schema (§3.4). We talked about
<server>__<tool>style names and decided against it — collisions are handled by 7 and 8. -
Built-ins win. If an MCP tool has the same name as a tool axe already gives this agent — its enabled built-ins from
tools, pluscall_agentwhen it hassub_agents— the MCP tool is dropped (logged in verbose mode). A built-in the agent has not enabled doesn't shadow anything: an MCP server'sread_fileis fine on an agent withoutread_fileintools. -
MCP vs MCP collision = config error. Two servers of the same agent exposing the same tool name → exit code 2 at startup; the error names the tool and both servers.
-
toolsstays built-in only. MCP tools are controlled by[[mcp_servers]]alone. They are not added totoolname.ValidNames(), and an MCP tool name intoolsis still an "unknown tool" config error. -
Sub-agents are isolated. A sub-agent with its own
[[mcp_servers]]opens its own sessions, is offered only its own MCP tools, and closes its sessions when it finishes. The parent's sessions and MCP tools are never handed to sub-agents — if parent and sub-agent declare the same server, that's two separate sessions. If a sub-agent's MCP setup fails, thatcall_agentcall returns an error result to the parent (like any other sub-agent failure) and the parent run continues. -
Dry-run doesn't connect.
--dry-runshows the configured servers (§3.6) and never touches the network or the header environment variables. -
Output. MCP calls show up in
--json(tool_calls,tool_call_details) exactly like built-in calls — same fields, no MCP marker. Verbose logging goes to stderr with an[mcp]prefix (§3.7). Stdout stays clean and pipeable.
3. Details
3.1 Config
[[mcp_servers]] array of tables in the agent TOML:
[[mcp_servers]]
name = "my-tools"
url = "https://my-mcp-server.example.com/sse"
transport = "sse"
[[mcp_servers]]
name = "pipedream"
url = "https://remote.mcp.pipedream.net"
transport = "streamable-http"
headers = { Authorization = "Bearer ${PIPEDREAM_TOKEN}" }
Go side (internal/agent/agent.go): new struct MCPServerConfig with Name, URL,
Transport (strings) and Headers map[string]string (TOML keys name, url, transport,
headers), and AgentConfig.MCPServers []MCPServerConfig with tag toml:"mcp_servers".
No [[mcp_servers]] (or mcp_servers = []) means no MCP at all — nothing changes for
existing agents.
3.2 Validation
In agent.Validate, after the existing checks — so a bad entry is a normal config error
(exit 2, --dry-run included, and agents list skips the file like any other invalid agent).
For each entry:
namerequired (not blank);urlrequired, and must be an absolutehttp://orhttps://URL with a host;transportrequired and exactlysseorstreamable-http. No default — we don't want to guess the transport;- names unique within the agent.
Per-entry errors start with mcp_servers[<i>] (0-based index), e.g.
mcp_servers[1].url is required; the duplicate-name error names the duplicated server.
Header values and environment variables are not checked at load time — that happens at run
time (§3.3), so agents list/show and --dry-run work without the secrets set.
3.3 Headers
- Header values may reference environment variables as
${VAR}. They are expanded at run time, right before connecting to that server. Several references in one value are fine ("${USER}:${PASS}"). Only the braced form counts —$VARis sent literally. Values without references are sent as-is. - Referenced variable unset or empty → exit code 2, before anything is sent to that server. The error names the variable and the server.
- The expanded headers go on every HTTP request to that server (handshake, SSE stream,
tools/list,tools/call, session close). The SDK transports take an*http.Client, so a smallRoundTripperthat sets the headers is enough. - Don't mutate the agent config while expanding.
3.4 Tool conversion (MCP tool → provider.Tool)
Name,Description: as reported by the server.- Parameters: from the input schema's
properties;Requiredfrom the top-levelrequiredarray. Noproperties→ no parameters. - Types
string,integer,number,booleanpass through unchanged. - Anything else —
object,array,null, a missing type,oneOf/anyOf, ... — becomesstring, and the original type is put in front of the parameter description so the model knows what to produce, e.g.Type array: filter list(Type unknown: ...when the schema doesn't say). Keep the original description text.
3.5 Calling MCP tools
- Arguments. Axe passes tool arguments around as
map[string]string. Before sending a call, convert the value of every parameter the tool declared asinteger,numberorboolean(per itstools/listschema) when the string parses as that type ("3"→3,"1.5"→1.5,"true"→true). Everything else is sent as the original string: string params (even"7"), values that don't parse, params the schema doesn't declare, and the flattened object/array params. - Results. Join the text of all text content items with
"\n", in order; other content (images, embedded resources) is dropped.- no text content at all → error result
MCP tool returned no text content - server-side
isError: true→ error result with the joined text - protocol/transport error on
tools/call→ error result with the error message - None of these abort the run: the model gets an error tool result, same as a failing built-in.
- no text content at all → error result
- Routing must work in both the sequential and the parallel tool-call paths, and in the sub-agent conversation loop.
3.6 Dry-run
New section after --- Tools --- and before --- Sub-Agents ---:
--- MCP Servers ---
github: https://api.example.com/mcp (streamable-http)
files: http://localhost:3000/sse (sse)
One line per server, <name>: <url> (<transport>), in config order; (none) when there are
no servers. Headers are not shown (they're usually credentials). The dry-run golden files
under cmd/testdata/golden/dry-run/ need the new section.
3.7 Verbose (--verbose, stderr)
Roughly:
[mcp] Connecting to "github" at https://api.example.com/mcp (streamable-http)
[mcp] "github" discovered 12 tool(s), registered 11
[mcp] Skipping "read_file" from "github" (conflicts with built-in tool)
[mcp] Routing tool "search_issues" to server "github"
Exact wording doesn't matter — keep the [mcp] prefix and the server/tool names in the lines.
Nothing MCP-related is printed without --verbose.
3.8 Scaffold
axe agents init template gets a commented-out example after the [sub_agents_config] block
and before [memory]:
# MCP server connections (optional)
# [[mcp_servers]]
# name = "my-tools"
# url = "https://my-mcp-server.example.com/sse"
# transport = "sse"
# headers = { Authorization = "Bearer ${MY_TOKEN}" }
4. Exit codes
| Situation | Result |
|---|---|
invalid [[mcp_servers]] entry (§3.2) |
exit 2 |
${VAR} in headers unset or empty |
exit 2 |
| two MCP servers expose the same tool name | exit 2 |
server unreachable, handshake rejected, tools/list fails, deadline hit while connecting |
exit 3 |
| MCP tool call fails during the conversation | not an exit — error tool result to the model |
| sub-agent's MCP setup fails | not an exit — call_agent error result to the parent |
5. Constraints
- No changes to the types in
internal/provider/provider.go(Tool,ToolCall,ToolResult,Message, ...). - No changes to
internal/toolname. call_agentstays special-cased outside the registry.- MCP connections are never shared across agents (parent/sub-agent included).
- Only new dependency: go-sdk v1.4.0 (plus whatever it pulls in indirectly).
6. Out of Scope
- axe as an MCP server
stdiotransport- resources, prompts, sampling, roots, server logging, progress/cancellation,
listChanged - OAuth or any auth beyond static headers with
${VAR} - retries, reconnects/session resumption, connection reuse across runs, caching tool lists
- per-server timeouts
- choosing a subset of a server's tools (all discovered tools are used, minus collisions)
- image / embedded-resource results (dropped, see §3.5)
- showing MCP tool names in
--dry-run(would need a connection) - README/docs pass — separate PR once this lands
7. Notes
- Why fail fast: the agent author declared the server as a dependency. Running without its tools degrades silently and is hard to debug; a clear exit 3 is better for cron/hooks.
- Why built-ins win: known, tested behaviour. If someone really wants the MCP version they can
drop the built-in from
tools. - Why no prefixing: tool names are what the model sees and what people write skills against;
github__search_issuesis noise. Collisions between two servers are rare and loud (exit 2). - Schema flattening works because the model reads the descriptions;
Typeis a hint. - The argument conversion exists because typed MCP servers validate their input — a string
"3"for an integer param gets rejected. We still don't touch the provider abstraction (map[string]string), that's a bigger change for another day.
What the hidden grader checks
- [[mcp_servers]] entries are validated in agent.Validate: non-blank name, absolute http(s) URL, transport exactly sse or streamable-http, unique names; bad entries exit 2.
- Servers are connected and tools listed once, before the first LLM call; unset ${VAR} header variables exit 2, connect or discovery failures exit 3.
- Connect, tools/list and tools/call all run under the --timeout deadline; header expansion leaves the config untouched; opened sessions are always closed, even after partial failure.
- MCP tools keep their server-given names with flattened schemas; enabled built-ins (and call_agent with sub_agents) win; one tool name on two servers exits 2.
- Calls are routed in sequential and parallel paths with typed argument conversion and text-only results; sub-agents open, use and close only their own sessions.
- --dry-run lists servers without connecting, [mcp] logs go to stderr only with --verbose, the scaffold gets an example, and the 421 existing tests still pass.
- Fail-to-pass tests
- 29
- Regression tests
- 421
- Rubric items
- 16
- Wrong solutions rejected
- 21
- Alternative solutions accepted
- 2
Key moments of the trajectory
Steps 1–2, 13–14, 43 With no network, the agent found go-sdk v1.4.0 in the module cache and added it with checksum lookups off, later checking the new go.sum lines against the SDK’s own go.sum.
Steps 16, 18–19 Reading the SDK source, the agent found the streamable-http client makes up to five reconnect attempts by default, which the spec rules out, and confirmed MaxRetries: -1 turns them off.
Steps 28–29 A quick Go probe showed the providers’ %v formatting turns 12345678 into “1.2345678e+07”, so the agent made integer conversion also accept whole numbers in exponent form instead of sending them as strings.
Steps 36–37, 39–40, 47–48 The agent added [[mcp_servers]] validation to agent.Validate, a new internal/mcpclient package for sessions, schema flattening and header expansion, sub-agent setup in tool.go, and run.go wiring that connects after the --timeout context is created.
Steps 62, 64–67 The new mcpclient tests hung; a goroutine dump traced it to TestConnect_DeadlineWhileConnecting, whose stalling test server never noticed the client disconnect, and after fixing that test the package passed under -race -count=5.
Steps 78, 82–83, 86–87 The agent ran the built binary against a real MCP server and a fake Anthropic API; after tracing a hang to piped stdin, it confirmed clean stdout, [mcp] verbose lines and expected exit codes.
Steps 88, 91–92, 95–96 Chasing a two-line timeout error, the agent found SDK Connect still blocked after 8s with a 1s deadline when requests weren’t tied to the run context; disabling that binding made the tests hang.
Steps 105–106 The final message listed the checks run (full -race suite green as a non-root user), the SDK deadline finding, each spec ambiguity with the choice made, and flagged the offline-generated go.sum for review.
Result
- Agent steps
- 106
- Tool calls
- 153
- Minutes
- 63
- Files changed
- 27
- Lines added / removed
- +3,173 / −20
The agent added an internal/mcpclient package on go-sdk v1.4.0, [[mcp_servers]] validation in agent.Validate, wiring in cmd/run.go and the sub-agent path, updated dry-run goldens, and tests against real SDK-built MCP servers on both transports. The change keeps one --timeout deadline over connect, discovery and calls, leaves the config untouched when it expands headers, and closes opened sessions when setup fails partway. The final message lists each judgment call on the spec, and the grader found that each one matches the code.
Quality scores from the grader (1–5)
- Correctness
- 5
- Code quality
- 5
- Scope
- 5
- Communication
- 5
Rubric items (16)
Required (13)
- C1 · Agent TOML supports [[mcp_servers]] (name, url, transport, headers) as agent.MCPServerConfig / AgentConfig.MCPServers, validated in agent.Validate: name non-blank, url an absolute http(s) URL with a host, transport exactly "sse" or "streamable-http" (no default), names unique within the agent. An invalid entry is a config error like any other invalid agent config: exit 2 (also for --dry-run), and agents list skips the file. Per-entry messages start with mcp_servers[<i>] (0-based); the duplicate-name error names the duplicated server. Header values and their variables are not checked at load time, so agents list and --dry-run work with them unset.
- C2 · At run start (never in --dry-run) every declared server is connected once, with the SDK transport matching its transport value, and its tools are listed once (tools/list), all before the first LLM call. ${VAR} references in header values are expanded from the environment at run time, before connecting: several per value, bare $VAR left literal, unset or empty variable -> exit 2 naming the variable and the server, with nothing sent to that server. The expanded headers are sent on every HTTP request to that server. Unreachable server, rejected handshake, failing tools/list or the deadline running out while connecting -> exit 3 naming the server, with the LLM never called. Every session that was opened is closed when the run ends, including when startup fails partway. That covers sessions opened before the failure and, with concurrent setup, sessions whose handshake completed around it.
- C15 · One deadline (decision 5): MCP connect, tools/list and every MCP tools/call run under the run's --timeout deadline. They use the context created from --timeout, which therefore exists before MCP setup, or a context derived from it that adds no deadline of its own. Inside call_agent they use the contexts ExecuteCallAgent already has, which derive from the parent's. None of these calls uses context.Background()/TODO(), context.WithoutCancel or a separate fixed timeout, so a server that stalls on any of them cannot keep the run going past --timeout.
- C16 · Header expansion leaves the agent config untouched (spec 3.3: "Don't mutate the agent config while expanding"). The expanded values go into a newly allocated map, or are applied per request in the HTTP layer. After setup, the loaded config's MCPServers[i].Headers still hold the original ${VAR} text.
- C3 · MCP tools are offered to the model under the server's own tool name (no prefixing) with the server's description and a flattened schema: string/integer/number/boolean pass through; any other or missing type becomes string with the original type put in front of the parameter description; required comes from the top-level required array. A tool already offered to this agent (its enabled built-ins, plus call_agent when it has sub_agents) wins and the MCP tool is dropped (logged with --verbose); a built-in name the agent has not enabled does not shadow the MCP tool. Two servers exposing the same tool name -> exit 2 naming the tool and both servers. The tools field keeps accepting built-in names only.
- C4 · MCP tool calls are routed to the owning server in the sequential and the parallel tool-call paths. Arguments for parameters declared integer/number/boolean are converted when the string parses; everything else (string params, unparseable values, undeclared params, flattened object/array params) is sent as the original string. Results join all text items with "\n" and drop non-text content; no text -> error result "MCP tool returned no text content"; server isError -> error result with the text; protocol/transport errors -> error result. None of these abort the run. MCP calls appear in --json tool_calls/tool_call_details exactly like built-in calls: same fields, no MCP marker (such as a server or source field).
- C5 · Sub-agents are isolated. A sub-agent with its own [[mcp_servers]] opens its own sessions and is offered (and can call) only its own MCP tools. It closes those sessions when it finishes, on every path: after its conversation ends normally or with an error, and when its MCP setup fails partway. In that last case, the sessions it already opened for earlier servers are closed before the call_agent error result goes back. The parent's sessions and tools are never handed to sub-agents, so the same server declared by both means two sessions. A sub-agent MCP setup failure becomes an error call_agent result and the parent run continues.
- C6 · --dry-run prints a '--- MCP Servers ---' section between '--- Tools ---' and '--- Sub-Agents ---' with one '<name>: <url> (<transport>)' line per server in config order or '(none)', never shows headers, never connects or reads header variables. --verbose logs [mcp]-prefixed lines on stderr naming the server (connect, discovery), skipped tools and routed calls (tool + server); nothing MCP-related is printed without --verbose and stdout stays clean. The agents init scaffold has a commented-out [[mcp_servers]] example (name, url, transport, headers) between [sub_agents_config] and [memory].
- C7 · Existing behaviour keeps working and existing tests are not deleted, skipped or weakened. The dry-run golden files under cmd/testdata/golden/dry-run/ are updated only by adding the new MCP Servers section.
- C8 · The MCP client is built on the official SDK github.com/modelcontextprotocol/go-sdk (package mcp) pinned to v1.4.0, which is the only new direct dependency in go.mod (its own dependencies appear only as // indirect); there is no hand-rolled JSON-RPC/SSE/streamable-HTTP protocol code.
- C9 · Spec constraints hold: no changes to the types in internal/provider/provider.go (Tool, ToolParameter, ToolCall, ToolResult, Message, Request, Response...), no changes to internal/toolname (MCP tool names are not added to ValidNames), and call_agent stays special-cased outside the tool registry.
- C10 · Nothing from the spec's out-of-scope list is added: no stdio transport; no axe-level retry, backoff, reconnect or session resumption around MCP connect, tools/list or tools/call; no per-server timeout setting; no option to pick a subset of a server's tools; no OAuth or auth beyond static headers; no refreshing or caching of tool lists and no reuse of connections across runs; no handling of resources, prompts, sampling, roots, server logging, progress or listChanged; no MCP server mode.
- C14 · The final message accurately says what was implemented and how it was verified, and does not claim anything that is not true of the final tree (files, tests, docs or behaviours that are not there); deviations from the spec, if any, are called out.
Quality (3)
- C11 · Follows the codebase's conventions: ExitError for exit codes, errors that name the server/variable and wrap causes, verbose output only through cmd.ErrOrStderr()/ExecuteOptions.Stderr, no package-level mutable state (AGENTS.md: no global state), dependencies passed explicitly, MCP logic in its own internal package rather than piled into cmd/run.go, gofmt/go vet clean.
- C12 · Adds focused tests for the new code in the codebase's style (table-driven, real go-sdk server via httptest rather than mocks, t.Setenv for environment variables).
- C13 · No unrelated changes or leftovers: no debug prints, stray files or unrelated refactors; the spec file is not rewritten; README/docs are not reworked (the spec defers that to a separate PR).
The agent’s final message
I've implemented the MCP spec and stuck to all 12 design decisions. Everything is uncommitted in the working tree, alongside the spec file, which was already untracked.
Verification
- Tests: the full suite passes under
go test -race, including about 130 new tests across both transports. They run against real MCP servers built with the SDK, not mocks. - Root-only failures: run as root, three existing permission tests fail. They fail identically on a clean
main, and pass when the suite runs as a normal user. - Real binary: I built
axeand ran it against a real MCP server and a fake Anthropic API. stdout stays clean, the[mcp]verbose lines appear, and exit codes are 3 / 2 / 2 / 2 for unreachable server / unset header variable / tool collision / invalid entry. A server that accepts connections but never answers gives exit 3 at exactly--timeout 2on both transports. Dry-run works without the env vars and without connecting. - Cross-compile: every release target in
.goreleaser.ymlbuilds.
One finding worth knowing
The SDK sends a couple of requests that ignore the caller's deadline: the session-close request, and a "call cancelled" notice to the server. Against a server that never answers, the SDK's Connect was still stuck 8 seconds after a 1-second deadline. So every MCP HTTP request is now tied to the run's --timeout (internal/mcpclient/headers.go). Disabling that makes the tests hang, so they cover it.
Where the spec was ambiguous, and what I chose
- Collisions: "built-ins win" is applied before the server-vs-server check. If two servers both expose
read_fileand the agent has the built-in enabled, both copies are dropped and it's not an exit 2. call_agent: following the spec text, it's reserved whenever the agent hassub_agents, even at max depth, where axe doesn't actually offer it.- Integers: axe's providers turn the number 12345678 into the string
"1.2345678e+07"before it reaches MCP. So whole numbers in exponent form are also converted forintegerparams; otherwise any integer of a million or more would go out as a string.NaN/Infstay strings. - No reconnects: the streamable-http transport has reconnects disabled, and its optional server→client stream is off. Without retries, that stream dropping would break the session, and it only carries features the spec puts out of scope.
- Smaller calls:
- Tool listing follows pagination, still once per run.
- Anything inside
${…}counts as a variable name, matching how path expansion already treats the braced form. - A tool listed twice by one server keeps the first copy.
Please double-check
- go.sum: there was no network, so I generated it from the local module cache without checksum-database verification. 18 of the 20 new lines match the SDK's own committed go.sum. The SDK's own two lines can't be checked offline; CI's fresh download will verify them.
- Lint: golangci-lint isn't installed here, so I reviewed against its checks by hand and
go vetis clean. CI will run the real thing.
Possible follow-ups (outside the spec)
- Object/array params go to the server as strings, as decided in the spec, so servers that validate their input will likely reject them.
- MCP results have no size cap, unlike
url_fetch. agents showdoesn't list MCP servers.
New code is in internal/mcpclient/ and internal/testutil/mcpserver.go, wired into cmd/run.go and internal/tool/tool.go. I added docs/plans/027_mcp_tool_support_implement.md to match the existing spec/implement pairs.
The agent’s change (excerpt)
diff --git a/cmd/run.go b/cmd/run.go
--- a/cmd/run.go
+++ b/cmd/run.go
@@ -291,6 +292,27 @@ func runAgent(cmd *cobra.Command, args []string) error {
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(timeout)*time.Second)
defer cancel()
+ // Step 17b: Connect to MCP servers and offer their tools. The sessions run
+ // under the same deadline and stay open for the whole conversation; closing
+ // them is deferred after cancel so it runs first.
+ var mcpTools *mcpclient.Set
+ if len(cfg.MCPServers) > 0 {
+ mcpTools, err = mcpclient.Connect(ctx, cfg.MCPServers, mcpclient.Options{
+ Reserved: tool.ReservedToolNames(cfg),
+ Verbose: verbose,
+ Stderr: cmd.ErrOrStderr(),
+ })
+ if err != nil {
+ var cfgErr *mcpclient.ConfigError
+ if errors.As(err, &cfgErr) {
+ return &ExitError{Code: 2, Err: err}
+ }
+ return &ExitError{Code: 3, Err: err}
+ }
+ defer mcpTools.Close()
+ req.Tools = append(req.Tools, mcpTools.Tools()...)
+ }
+
// Step 18: Call provider (conversation loop when tools are present)
start := time.Now()
@@ -370,7 +392,7 @@ func runAgent(cmd *cobra.Command, args []string) error {
req.Messages = append(req.Messages, assistantMsg)
// Execute tool calls
- results := executeToolCalls(ctx, resp.ToolCalls, cfg, globalCfg, registry, depth, effectiveMaxDepth, parallel, verbose, cmd.ErrOrStderr(), workdir)
+ results := executeToolCalls(ctx, resp.ToolCalls, cfg, globalCfg, registry, mcpTools, depth, effectiveMaxDepth, parallel, verbose, cmd.ErrOrStderr(), workdir)
totalToolCalls += len(resp.ToolCalls)
if jsonOutput {
@@ -512,6 +534,17 @@ func printDryRun(cmd *cobra.Command, cfg *agent.AgentConfig, provName, modelName
_, _ = fmt.Fprintln(out, "(none)")
}
+ _, _ = fmt.Fprintln(out)
+ _, _ = fmt.Fprintln(out, "--- MCP Servers ---")
+ if len(cfg.MCPServers) > 0 {
+ // Headers are not shown: they usually hold credentials.
+ for _, srv := range cfg.MCPServers {
+ _, _ = fmt.Fprintf(out, "%s: %s (%s)\n", srv.Name, srv.URL, srv.Transport)
+ }
+ } else {
+ _, _ = fmt.Fprintln(out, "(none)")
+ }
+
_, _ = fmt.Fprintln(out)
_, _ = fmt.Fprintln(out, "--- Sub-Agents ---")
if len(cfg.SubAgents) > 0 {
@@ -537,7 +570,8 @@ func printDryRun(cmd *cobra.Command, cfg *agent.AgentConfig, provName, modelName
// executeToolCalls dispatches tool calls and returns results.
// When parallel is true and there are multiple calls, they run concurrently.
-func executeToolCalls(ctx context.Context, toolCalls []provider.ToolCall, cfg *agent.AgentConfig, globalCfg *config.GlobalConfig, registry *tool.Registry, depth, maxDepth int, parallel, verbose bool, stderr io.Writer, workdir string) []provider.ToolResult {
+// Tools from the agent's MCP servers are routed first; mcpTools may be nil.
+func executeToolCalls(ctx context.Context, toolCalls []provider.ToolCall, cfg *agent.AgentConfig, globalCfg *config.GlobalConfig, registry *tool.Registry, mcpTools *mcpclient.Set, depth, maxDepth int, parallel, verbose bool, stderr io.Writer, workdir string) []provider.ToolResult {
results := make([]provider.ToolResult, len(toolCalls))
execOpts := tool.ExecuteOptions{
@@ -554,7 +588,9 @@ func executeToolCalls(ctx context.Context, toolCalls []provider.ToolCall, cfg *a
if len(toolCalls) == 1 || !parallel {
// Sequential execution (also used for single call)
for i, tc := range toolCalls {
- if tc.Name == tool.CallAgentToolName {
+ if mcpTools.Has(tc.Name) {
+ results[i] = mcpTools.Call(ctx, tc)
+ } else if tc.Name == tool.CallAgentToolName {
results[i] = tool.ExecuteCallAgent(ctx, tc, execOpts)
} else {
result, dispatchErr := registry.Dispatch(ctx, tc, tool.ExecContext{Workdir: workdir, Stderr: stderr, Verbose: verbose})
@@ -579,7 +615,9 @@ func executeToolCalls(ctx context.Context, toolCalls []provider.ToolCall, cfg *a
for i, tc := range toolCalls {
go func(idx int, call provider.ToolCall) {
var res provider.ToolResult
- if call.Name == tool.CallAgentToolName {
+ if mcpTools.Has(call.Name) {
+ res = mcpTools.Call(ctx, call)
+ } else if call.Name == tool.CallAgentToolName {
res = tool.ExecuteCallAgent(ctx, call, execOpts)
} else {
result, dispatchErr := registry.Dispatch(ctx, call, tool.ExecContext{Workdir: workdir, Stderr: stderr, Verbose: verbose})
diff --git a/internal/agent/agent.go b/internal/agent/agent.go
--- a/internal/agent/agent.go
+++ b/internal/agent/agent.go
@@ -12,6 +13,12 @@ import (
"github.com/jrswab/axe/internal/xdg"
)
+// MCP transport names accepted in an mcp_servers entry.
+const (
+ MCPTransportSSE = "sse"
+ MCPTransportStreamableHTTP = "streamable-http"
+)
+
// MemoryConfig holds memory sub-configuration for an agent.
type MemoryConfig struct {
Enabled bool `toml:"enabled"`
@@ -33,20 +40,31 @@ type SubAgentsConfig struct {
Timeout int `toml:"timeout"`
}
+// MCPServerConfig holds the connection settings for one MCP server an agent uses.
+// Header values may reference environment variables as ${VAR}; they are expanded
+// at run time, not at load time.
+type MCPServerConfig struct {
+ Name string `toml:"name"`
+ URL string `toml:"url"`
+ Transport string `toml:"transport"`
+ Headers map[string]string `toml:"headers"`
+}
+
// AgentConfig represents a parsed agent TOML configuration file.
type AgentConfig struct {
- Name string `toml:"name"`
- Description string `toml:"description"`
- Model string `toml:"model"`
- SystemPrompt string `toml:"system_prompt"`
- Skill string `toml:"skill"`
- Files []string `toml:"files"`
- Workdir string `toml:"workdir"`
- Tools []string `toml:"tools"`
… 3,335 more lines
Sample 3 of 8
Undeprecate useStateAction with useAction parity and a formAction
next-safe-action (TypeScript, pnpm monorepo): type-safe, validated Next.js Server Actions with React client hooks, react-hook-form and TanStack Query adapters, and a docs site.
The developer wants the deprecated useStateAction made a first-class hook exported from next-safe-action/hooks, with existing next-safe-action/stateful-hooks imports still working; the hook takes useAction’s options plus initResult and returns useAction’s return value plus a formAction for <form action>. A correct result handles redirect()/notFound() like useAction (hasNavigated, onNavigation, throwOnNavigation), adds executeAsync and a reset() after which the next run starts from initResult and an in-flight result never resurfaces, makes status and callbacks work for form submits, and removes the deprecation from types and docs.
Why it is hard
The library change sits in one package, with useAction as the template; the hard part is React 19’s semantics. useActionState has no reset and rethrows errors from the action, so navigation errors must be caught before React sees them, executeAsync needs a promise out of a void dispatcher, and reset() must survive an in-flight run. Form submissions run inside React’s own transition, so naive state handling leaves the form path stuck at 'idle' or fires callbacks twice.
The developer’s request, verbatim
useStateAction's been deprecated since v8, i want it back as a first class hook. right now it's half baked in @packages/next-safe-action/src/stateful-hooks.ts: no onNavigation/throwOnNavigation, redirect()/notFound() in the action blow up to the error boundary, no executeAsync or reset. export it from next-safe-action/hooks with the other two hooks, but exisiting next-safe-action/stateful-hooks imports have to keep working. same options and return value as useAction (plus initResult), and a `formAction` you can pass straight to <form action>, with status/callbacks working for form submits too. reset() = clean slate: the next run gets initResult as prevResult again, and if a run is still in flight when you reset its late result shouldnt come back. undeprecate it in the types and docs too
What the hidden grader checks
- useStateAction is exported from next-safe-action/hooks, and the built next-safe-action/stateful-hooks entry exposes the same new hook at runtime and in types.
- Takes initResult plus useAction's options; redirect/notFound set hasNavigated and call onNavigation (or hit the error boundary with throwOnNavigation: true); onSuccess/onError/onSettled fire once per result.
- Returns useAction's full object plus formAction, with matching public types; results chain as prevResult from initResult; executeAsync resolves with results, rejects on navigation.
- Real form submits: formAction passes FormData to the action as input, status shows executing, results fire onSuccess/onError and onSettled once, sharing execute's prevResult chain.
- reset() returns to idle and the next run starts from initResult; no pre-reset or in-flight result reappears, even while that run is pending.
- No @deprecated on the hook or its public types, deprecation notes gone from all five docs pages, and useAction/useOptimisticAction behave as before.
- Fail-to-pass tests
- 75
- Regression tests
- 382
- Rubric items
- 12
- Wrong solutions rejected
- 14
- Alternative solutions accepted
- 4
Key moments of the trajectory
Steps 2, 6–7 Reading the hook internals, the agent found useStateAction was a thin useActionState wrapper with navigation and error handling disabled, and noticed useAction’s reset() doesn’t discard requests still in flight.
Steps 19, 21–22 To design formAction, the agent ran a throwaway form test: React held back state updates made synchronously in an async form action until it finished, but yielding one tick first exposed ‘executing’.
Steps 26–27 The agent rewrote the shared useActionBase so execute delegates to executeAsync, built useStateAction on it with a formAction that yields one tick before running, and reduced stateful-hooks.ts to a re-export.
Steps 35–37 One new form test failed only when run with the others; the agent traced it to an earlier test’s never-resolved form submission leaking through React’s global async-action entanglement, and fixed the test.
Steps 38–41 The agent checked its new tests by removing the one-tick yield and, separately, reset’s in-flight discard (the matching tests failed each time) and by running them against the original hook, where 14 failed.
Steps 42–44 A package build confirmed dist/hooks.mjs and dist/stateful-hooks.mjs both keep “use client” and re-export useStateAction from one shared chunk, and that no built .mjs file references useActionState anymore.
Steps 46–47, 60–62 The agent updated the useStateAction API reference and turned the hooks guide’s deprecated accordion into a real section; since docs lint couldn’t fully run here, it compiled all 10 changed MDX pages with @mdx-js/mdx.
Step 70 The final message flagged decisions to revisit (permalink kept but ignored, prevResult semantics, thrown errors now handled like useAction) and noted, without fixing it, that useAction’s reset() still lets in-flight results through.
Result
- Agent steps
- 70
- Tool calls
- 80
- Minutes
- 28
- Files changed
- 21
- Lines added / removed
- +1,113 / −203
The agent rebuilt useStateAction on the shared useActionBase and exported it from next-safe-action/hooks, with next-safe-action/stateful-hooks re-exporting the same function. It added executeAsync, a reset() that drops in-flight runs and a formAction that drives status and callbacks on form submits, and removed the deprecation from types and docs. useAction and useOptimisticAction behave as before, thrown errors now set hasErrored instead of reaching the error boundary, and the grader found the claims in the final message accurate.
Quality scores from the grader (1–5)
- Correctness
- 5
- Code quality
- 5
- Scope
- 5
- Communication
- 5
Rubric items (12)
Required (7)
- C1 · useStateAction is exported from the next-safe-action/hooks entry point alongside useAction and useOptimisticAction, and the published next-safe-action/stateful-hooks entry point (package.json exports/typesVersions resolving to the built dist) keeps working and exposes the same new full-featured hook, at runtime and in its types: executeAsync, reset, formAction and useAction-style navigation handling, not the old implementation.
- C2 · useStateAction accepts initResult plus the same options as useAction (onExecute, onSuccess, onError, onNavigation, onSettled, throwOnNavigation) with the same typing rules as HookBaseOptions (onNavigation/onSettled not allowed together with throwOnNavigation: true). Navigation errors thrown by the action (redirect, notFound, forbidden, unauthorized) no longer crash the component: status becomes hasNavigated and onNavigation/onSettled receive the navigationKind; with throwOnNavigation: true the navigation error reaches the nearest error boundary instead. Success/error results fire onSuccess/onError/onSettled exactly once per execution with the same payloads as useAction.
- C3 · useStateAction returns everything useAction returns (execute, executeAsync, reset, input, result, status and the shorthand flags isIdle/isExecuting/isTransitioning/isPending/hasSucceeded/hasErrored/hasNavigated) plus formAction, and the public types say so (UseStateActionHookReturn and InferUseStateActionHookReturn include executeAsync, reset and formAction). The action is called with (prevResult, input), results chain as prevResult across sequential executions starting from initResult (or {}), and executeAsync resolves with the action result (also when it carries serverError/validationErrors) and rejects with the navigation error when the action navigates.
- C4 · formAction works with <form action={formAction}>: a submission calls the stateful action with the submitted FormData as input and the previous result as prevResult; status is 'executing' (isPending true) while it is in flight and then hasSucceeded/hasErrored/hasNavigated; result and input are updated; for a submission that succeeds or returns an error result, onSuccess or onError and onSettled fire exactly once (also on repeated submissions); a submission whose action navigates sets hasNavigated and calls onNavigation (how many times the navigation callbacks fire follows the C2 allowance); and form submissions share the same result/prevResult chain as execute.
- C5 · reset() is a clean slate: status goes back to idle with no error/navigation flags, input becomes undefined and the result is cleared; the next execution receives initResult (or {} when there is none) as prevResult rather than the last result; the cleared result stays shown until that next execution produces a new result, so the pre-reset result does not reappear while it is pending (as with useAction, on both the execute and the form path); and when reset() is called while an execution is in flight, its late result never comes back: not when it arrives, not while the next execution is pending, and the next execution again starts from initResult.
- C6 · useStateAction is no longer presented as deprecated, and its API reference is current. (1) Code: the hook as exported from next-safe-action/hooks and its public types (UseStateActionHookReturn, InferUseStateActionHookReturn) carry no @deprecated JSDoc tag. (2) Docs: no page under apps/docs/content/docs outside migrations/ calls the hook or its types deprecated, or tells readers to use React's useActionState instead of it. In the starting tree such statements are on five pages, and all of them must be gone: api/hooks-api.mdx (useStateAction heading and warning callout, the 'except deprecated useStateAction' note on HookBaseOptions), guides/hooks.mdx (the 'Deprecated: useStateAction' accordion), concepts/how-it-works.mdx (the next-safe-action/stateful-hooks row of the entry-point table), api/type-utilities.mdx (the '(deprecated)' on the InferUseStateActionHookReturn heading) and api/safe-action-client.mdx (the '(deprecated)' in the .stateAction() description). (3) The hooks API reference (api/hooks-api.mdx) documents the hook as imported from next-safe-action/hooks, with initResult plus the useAction options (including onNavigation/throwOnNavigation) and the useAction return plus formAction (including executeAsync and reset).
- C7 · Existing behaviour keeps working: useAction and useOptimisticAction behave as before, the rest of the package and the adapter packages are unaffected, and existing tests are not deleted, skipped or weakened (the only expected test change is the old type assertion that UseStateActionHookReturn has no executeAsync/reset).
Quality (5)
- C8 · Changes stay on the request: the hook, its types, entry points, docs (and optionally the playground demo/changeset). No unrelated refactors of the server-side action builder or adapters, no debug output, no stray files or committed build output.
- C9 · Follows the codebase's existing building blocks and conventions: options typed with HookBaseOptions (plus initResult) and return type built from UseActionHookReturn, status via getActionStatus/getActionShorthandStatusObject, callbacks via useActionCallbacks (or by reusing useActionBase), navigation detection via FrameworkErrorHandler.isNavigationError, render-phase throw for throwOnNavigation like useActionBase, 'use client' kept on client entry points, a changeset for next-safe-action (minor, new feature) as CLAUDE.md asks, and the library still passes pnpm run lint:lib (tsc + oxlint) once the outdated type assertion is updated.
- C10 · The final message accurately describes what changed (entry points, options/return parity, formAction, reset semantics, docs/types undeprecated) and states real limitations honestly (e.g. no progressive enhancement without JS); it does not claim behaviour, tests or checks that are not true of the final tree.
- C11 · Thrown non-navigation errors (e.g. from actions with throwServerError/throwValidationErrors, or network failures) are handled like useAction: the hook reports status hasErrored, onError receives the error as thrownError and onSettled fires, executeAsync rejects with it, and the hook stays mounted and usable for the next execution, instead of the error escaping to the nearest error boundary.
- C12 · The hooks guide (apps/docs/content/docs/guides/hooks.mdx) covers useStateAction as a supported hook, in place of the old 'Deprecated: useStateAction' accordion: at least a short section or example with the next-safe-action/hooks import and formAction usage.
The agent’s final message
useStateAction is now a first-class hook. It's exported from next-safe-action/hooks, and next-safe-action/stateful-hooks re-exports the same function, so existing imports keep working. Core lint is clean, all 305 core tests pass (274 before), and both adapters' test suites pass. Nothing is committed.
How it works now
- It no longer wraps React's
useActionState. It runs on the same shared code asuseAction, so options, status, callbacks,onNavigation/throwOnNavigation,executeAsyncandresetbehave the same.redirect()/notFound()now sethasNavigatedinstead of hitting the error boundary. reset()goes back toidlewithinitResult, and the next run getsinitResultasprevResult. A run still in flight is discarded: its late result shows up neither inresultnor as the nextprevResult.formActionworks with<form action>. React holds back state updates made inside a form action until the whole action finishes, soexecutingnever showed andonExecutedidn't fire. I confirmed this with a quick experiment.formActionnow waits one tick before running, and its promise keepsuseFormStatuspending until the run finishes. It never rejects; errors go through status and callbacks.- Docs and types no longer mark it deprecated. There are new sections in the hooks API reference and hooks guide, a tab in the form-actions guide, and a minor changeset.
Decisions you may want to revisit
permalink: it can't do anything withoutuseActionState, so I kept it in the type as deprecated and ignored, so existing code still compiles. The hook also can't support forms that work before JavaScript loads; the docs point touseActionStatefor that.prevResult: it's the last result the action returned, includingserverError/validationErrors. Thrown errors and navigations leave it unchanged, thoughresultis still cleared, as inuseAction.- Behavior changes vs the old hook (also in the changeset): thrown errors now set
hasErroredand callonErrorinstead of hitting the error boundary. Runs are no longer queued: the latest one wins, as inuseAction. executerefactor: in the shared code,executenow callsexecuteAsync, so there's one code path. The existing hook tests pass unchanged.
How I checked the tests: I removed the one-tick wait, and separately the in-flight discard in reset; each time the matching tests failed. Against the original hook, 14 of the new tests fail. In the built package, both entries share one implementation and keep "use client". All 10 changed docs pages compile.
Things I noticed but didn't change
useAction/useOptimisticAction: afterreset(), a result still in flight shows up inresultwhile status staysidle. I only fixed this foruseStateAction; extending it to the others is a one-line change if you want it.fmt:checkalready fails at HEAD (6 files, now 5, none from my code). The docs lint fails here because a generated.sourcefolder is missing andoxlint-tsgolintisn't installed forapps/docs.- The playground still demos
useActionState; I didn't add auseStateActiondemo.
The agent’s change (excerpt)
diff --git a/packages/next-safe-action/src/hooks-shared.ts b/packages/next-safe-action/src/hooks-shared.ts
--- a/packages/next-safe-action/src/hooks-shared.ts
+++ b/packages/next-safe-action/src/hooks-shared.ts
@@ -2,22 +2,38 @@
import * as React from "react";
import { getActionShorthandStatusObject, getActionStatus, useActionCallbacks } from "./hooks-utils";
-import type { HookActionStatus, HookBaseOptions, HookCallbacks, SingleInputActionFn, HookShorthandStatus } from "./hooks.types";
+import type {
+ HookActionStatus,
+ HookBaseOptions,
+ HookCallbacks,
+ HookShorthandStatus,
+ SingleInputActionFn,
+ SingleInputStateActionFn,
+} from "./hooks.types";
import type { SafeActionResult } from "./index.types";
import { FrameworkErrorHandler } from "./next/errors";
import type { InferInputOrDefault, StandardSchemaV1 } from "./standard-schema";
/**
- * Shared base hook for `useAction` and `useOptimisticAction`.
+ * Shared base hook for `useAction`, `useOptimisticAction` and `useStateAction`.
* Extracts common state management, execution logic, and callback wiring.
*
- * @param onTransitionStart Optional callback invoked inside `startTransition` before the action runs.
+ * @param baseOpts.onTransitionStart Optional callback invoked inside `startTransition` before the action runs.
* Used by `useOptimisticAction` to call `setOptimisticValue`.
+ * @param baseOpts.withState Whether `safeActionFn` is a stateful action, called with the previous result
+ * before the input. Used by `useStateAction`.
+ * @param baseOpts.initResult Result before the first execution and after `reset`. Defaults to `{}`.
*/
export function useActionBase<ServerError, Schema extends StandardSchemaV1 | undefined, ShapedErrors, Data>(
- safeActionFn: SingleInputActionFn<ServerError, Schema, ShapedErrors, Data>,
+ safeActionFn:
+ | SingleInputActionFn<ServerError, Schema, ShapedErrors, Data>
+ | SingleInputStateActionFn<ServerError, Schema, ShapedErrors, Data>,
opts: HookBaseOptions<ServerError, Schema, ShapedErrors, Data> | undefined,
- onTransitionStart?: (input: InferInputOrDefault<Schema, undefined>) => void
+ baseOpts?: {
+ onTransitionStart?: (input: InferInputOrDefault<Schema, undefined>) => void;
+ withState?: boolean;
+ initResult?: SafeActionResult<ServerError, Schema, ShapedErrors, Data>;
+ }
): {
isTransitioning: boolean;
result: SafeActionResult<ServerError, Schema, ShapedErrors, Data>;
@@ -28,8 +44,11 @@ export function useActionBase<ServerError, Schema extends StandardSchemaV1 | und
reset: () => void;
shorthandStatus: HookShorthandStatus;
} {
+ const withState = baseOpts?.withState ?? false;
+ const initResult = baseOpts?.initResult ?? {};
+
const [isTransitioning, startTransition] = React.useTransition();
- const [result, setResult] = React.useState<SafeActionResult<ServerError, Schema, ShapedErrors, Data>>({});
+ const [result, setResult] = React.useState<SafeActionResult<ServerError, Schema, ShapedErrors, Data>>(initResult);
const [clientInput, setClientInput] = React.useState<InferInputOrDefault<Schema, void>>();
const [isExecuting, setIsExecuting] = React.useState(false);
const [navigationError, setNavigationError] = React.useState<Error | null>(null);
@@ -40,9 +59,18 @@ export function useActionBase<ServerError, Schema extends StandardSchemaV1 | und
// This prevents stale responses from overwriting fresh state on rapid calls.
const requestIdRef = React.useRef(0);
+ // Previous result passed to stateful actions: the latest result returned by the action, or
+ // `initResult` before the first execution and after `reset`. Thrown errors and navigations don't
+ // produce a result, so they leave it unchanged.
+ const prevResultRef = React.useRef(initResult);
+
+ // Latest `initResult`, restored by `reset`.
+ const initResultRef = React.useRef(initResult);
+ initResultRef.current = initResult;
+
// Stable ref for the transition start callback to avoid destabilizing execute/executeAsync.
- const onTransitionStartRef = React.useRef(onTransitionStart);
- onTransitionStartRef.current = onTransitionStart;
+ const onTransitionStartRef = React.useRef(baseOpts?.onTransitionStart);
+ onTransitionStartRef.current = baseOpts?.onTransitionStart;
const status = getActionStatus<ServerError, Schema, ShapedErrors, Data>({
isExecuting,
@@ -52,56 +80,12 @@ export function useActionBase<ServerError, Schema extends StandardSchemaV1 | und
hasThrownError: thrownError !== null,
});
- const execute = React.useCallback(
- (input: InferInputOrDefault<Schema, void>) => {
- const thisRequestId = ++requestIdRef.current;
-
- // Set state synchronously before starting the transition.
- setIsIdle(false);
- setNavigationError(null);
- setThrownError(null);
- setClientInput(input);
- setIsExecuting(true);
-
- startTransition(() => {
- onTransitionStartRef.current?.(input as InferInputOrDefault<Schema, undefined>);
-
- safeActionFn(input as InferInputOrDefault<Schema, undefined>)
- .then((res) => {
- if (thisRequestId !== requestIdRef.current) return;
- setResult(res ?? {});
- })
- .catch((e) => {
- if (thisRequestId === requestIdRef.current) {
- setResult({});
-
- if (FrameworkErrorHandler.isNavigationError(e)) {
- setNavigationError(e);
- } else {
- setThrownError(e as Error);
- }
- }
-
- // Only re-throw non-navigation errors for React error boundary handling.
- // Navigation errors are handled via render-phase throw (throwOnNavigation).
- if (!FrameworkErrorHandler.isNavigationError(e)) {
- throw e;
- }
- })
- .finally(() => {
- if (thisRequestId !== requestIdRef.current) return;
- setIsExecuting(false);
- });
- });
- },
- [safeActionFn]
- );
-
… 1,575 more lines
Sample 4 of 8
Add vertical advances to GlyphMetrics, mirroring the h_advance path
fontations: Rust crates for reading OpenType fonts (read-fonts, skrifa); the task is in read-fonts’ experimental font API (Font, FontInstance, GlyphMetrics).
The developer wants v_advance, v_advance_exact and v_advance_batched on GlyphMetrics, built like the horizontal path, and in a font without vmtx (Type 1 included) every glyph should advance by the horizontal line’s ascender minus descender. A correct result caches vmtx and VVAR per font. At a location it varies heights by VVAR, or else by gvar’s top-minus-bottom phantom-point deltas, and it picks the fallback line as skrifa’s Metrics does, at the instance’s location with MVAR’s fractions kept.
Why it is hard
Most of it mirrors the horizontal path, but the short request only makes sense with that precedent in mind, and several details differ or are easy to get wrong: which vertical phantom points to use and their sign (top minus bottom, the reverse of the width); stopping at VVAR so gvar, glyf and loca stay unread; not copying the horizontal fallback; porting skrifa’s line choice with its guard for an all-zero hhea; and making the fallback follow the instance’s location.
The developer’s request, verbatim
GlyphMetrics needs the vertical side of h_advance (v_advance/_exact/_batched), same deal as the horizonal path, variations and all. no vmtx (type1 too) -> every glyph advances by ascender - descender of the horizontal line, picked like skrifa's Metrics
What the hidden grader checks
- v_advance/_exact/_batched mirror h_advance*. With vmtx, default-location heights equal vmtx's positive values, tail glyphs sharing the last long advance; past-end ids measure zero.
- At a location, VVAR answers if present, leaving gvar, glyf and loca unread; else gvar's top-minus-bottom phantom deltas; else vmtx unvaried; fractions kept.
- Without vmtx, Type 1 included, every glyph gets the same height: the horizontal line's ascender minus descender, without the line gap.
- The line follows skrifa's Metrics: typographic under USE_TYPO_METRICS, else hhea; a zeroed hhea falls back to typographic, then the Windows pair.
- Through a FontInstance the line is the one at that location, not the default: MVAR moves it, and its deltas are not rounded.
- vmtx and VVAR are parsed at most once per font; measuring at the default location reads no variation table; existing behaviour and tests hold.
- Fail-to-pass tests
- 20
- Regression tests
- 73
- Rubric items
- 8
- Wrong solutions rejected
- 11
- Alternative solutions accepted
- 2
Key moments of the trajectory
Steps 2–4, 6–7 The agent read skrifa’s metrics.rs, then read-fonts’ GlyphMetrics, Font and cache code and the vmtx/VVAR table modules, taking the horizontal h_advance path and its per-font hmtx/HVAR/gvar caches as the template.
Steps 8, 10, 13 It then settled two points: height deltas come from gvar’s phantom points at indices 2 and 3 (top minus bottom), and MVAR deltas stay unrounded, unlike in skrifa, since the font API keeps fractions.
Steps 16–17, 39, 41, 45–47 Finding no fixture with a VVAR that moves anything, or with both gvar and vmtx, it built one from trimmed Vazirmatn’s TTX, adding vhea, vmtx and VVAR, and checked the round trip with fontTools.
Steps 50–56, 58 It added Once cells for vmtx and VVAR to FontRepr (vmtx read only when a height is asked for), the three v_advance methods and RawGlyphMetrics::from_vmtx, then built on std and no_std+libm.
Steps 61–63, 65–68 It made grave’s vertical phantom points move, checked the expected heights with fontTools and wrote tests; once they passed, it planted bugs, such as adding the bottom phantom point, and the targeted tests failed.
Steps 70–72 A temporary skrifa test, later deleted, compared the no-vmtx fallback with skrifa’s Metrics over 118 font/location pairs; all matched apart from skrifa’s rounding of MVAR deltas (Amstelvar at 0.4: 2504.8 vs 2504).
Steps 74, 77, 79–82, 89–90 Without rustfmt or clippy, it hand-checked line lengths, built docs with -D warnings, ran read-fonts (755 default, 787 all-features) and skrifa tests, no_std and workspace checks, and matched the fixture to its TTX.
Step 92 The final message gives the VVAR-then-gvar order, flags unrounded MVAR deltas and the fallback for past-end ids as decisions to revisit, and asks for cargo fmt and clippy, but miscounts 19 new tests as 20.
Result
- Agent steps
- 92
- Tool calls
- 117
- Minutes
- 33
- Files changed
- 7
- Lines added / removed
- +1,149 / −12
GlyphMetrics gains v_advance, v_advance_exact and v_advance_batched, which read vmtx and VVAR from per-font caches and, at a location, vary heights by VVAR or else by gvar’s top-minus-bottom phantom deltas, with fractions kept. Without vmtx, Type 1 included, every glyph advances by the horizontal line’s ascender minus descender, with the line chosen as skrifa’s Metrics chooses it and moved at an instance by unrounded MVAR deltas. The grader noted two minor points: at a location the no-vmtx line height re-reads MVAR on every batch instead of using the instance’s cached metrics, and the final message counts 20 new tests where there are 19.
Quality scores from the grader (1–5)
- Correctness
- 5
- Code quality
- 4
- Scope
- 5
- Communication
- 4
Rubric items (8)
Required (5)
- C1 · GlyphMetrics gains public v_advance (f32), v_advance_exact (F48Dot16) and v_advance_batched(convert, glyphs), shaped like h_advance / h_advance_exact / h_advance_batched and reachable from both Font::glyph_metrics() and FontInstance::glyph_metrics(). At the default location a font with vmtx reports exactly the advance vmtx states: positive, with glyphs in the short-metric tail sharing the last long advance. In a font with vmtx, a glyph id at or past the glyph count measures zero at every location (fonts without vmtx: see C3). The batched form agrees with one-at-a-time, convert decides the output type, and v_advance equals v_advance_exact narrowed to f32.
- C2 · In a font with vmtx, away from the default location, heights vary the way widths do. If the font has VVAR, its advance-height deltas answer, and gvar, glyf and loca are then never read. Otherwise gvar's phantom-point deltas answer, using the vertical pair: top minus bottom, i.e. deltas[2].y - deltas[3].y of Gvar::phantom_point_deltas, which already follows USE_MY_METRICS. With neither table, vmtx is used unvaried. The exact height keeps the delta's fraction. At the default location, including all-zero coordinates, no variation table (VVAR, gvar, glyf, loca) is read.
- C3 · In a font without vmtx, Type 1 included, every glyph of the font gets the same height: ascender minus descender of the horizontal line, without the line gap. The line is chosen as skrifa's Metrics chooses it: (1) the typographic line if OS/2 sets USE_TYPO_METRICS; (2) else hhea; (3) if hhea is zeroed, the typographic line if it is not zeroed, else the Windows pair, winAscent + winDescent (the descent is stated as a positive depth). A Type 1 font's line is the bounding-box line GlobalMetrics::from_type1 provides. Measured through a FontInstance, the line is the one at that instance's location, so MVAR moves it, and its deltas keep their fraction as every measurement at a location does in this API. It is not the font's default-location line. No per-glyph VVAR or gvar delta is applied to it.
- C4 · Existing behaviour is unchanged: h_advance* results (hmtx, HVAR, gvar, Type 1 charstrings, half-em fallback), GlobalMetrics fields and FontInstance behaviour. No existing test is deleted, skipped or weakened.
- C5 · Follows the precedent's caching: vmtx and VVAR are each parsed at most once per font and kept on the font beside the existing hmtx/HVAR/gvar caches. That means Once<TableCache<...>> cells in FontRepr reached through Font accessors, or an equivalent per-font cache. They are not re-read from the tables on every measurement or every glyph_metrics() call.
Quality (3)
- C6 · Keeps the shape and the laziness the horizontal path documents (FontRepr's comment on its variation caches, font/cache.rs, the test taking_the_metrics_reads_hmtx_but_nothing_that_varies): what a location changes waits until something asks. VVAR, like HVAR, is read only when a height is asked for at a location, and gvar/glyf/loca only when VVAR is absent. Taking the metrics reads none of them. The varied vertical path is kept out of line (#[inline(never)]) and built on RawGlyphMetrics::run / run_varied. The new public methods carry doc comments in the module's style.
- C7 · The change stays within read-fonts' font API (src/model) plus any tests or test data the agent adds. No edits to skrifa, the table-level APIs, Cargo manifests or unrelated files, no debug output, no stray files.
- C8 · The final message says accurately what was done: which tables answer at a location and in what order, what fonts without vmtx (and Type 1 fonts) report, and how the line is chosen. It claims nothing untrue of the final tree, such as tests that do not exist or were not run.
The agent’s final message
GlyphMetrics now has v_advance, v_advance_exact and v_advance_batched, built the same way as the horizontal versions. All tests pass, but I couldn't run cargo fmt or clippy: neither is installed here and there's no network. Nothing is committed.
How it behaves
- With
vmtx: it returns the stored height, positive asvmtxstates it. Glyphs past the last long metric share its height, and a glyph past the end of the font gets 0, as across the line. - At a location: it uses
VVARfirst, then thegvarphantom points (top minus bottom), then the unvaried height. Like the horizontal path, the varied part is kept out of the fast path, and fractions are kept. - Table reads:
vmtxis only read once something asks for a height, so horizontal-only text never pays for it.VVARgets its own cached cell onFont, likeHVAR. - No
vmtx, including Type 1: every glyph advances by ascender minus descender, picked as skrifa'sMetricspicks them. That means the typographic pair ifUSE_TYPO_METRICSis set, otherwisehhea; ifhheais all zero, the typographic pair if it isn't zero, else the Windows pair with its descent negated. At a location the pair is moved byMVAR'shasc/hdsc, even when the Windows pair was picked, because skrifa and FreeType do that.
Decisions you may want to change
- Deltas are unrounded. skrifa rounds each
MVARdelta to a whole unit, so at fractional locations the results differ by less than a unit (Amstelvar at 0.4: 2504.81 here, 2504 in skrifa). This matchesh_advance_exactandGlobalMetrics. - The fallback applies to every glyph ID, including ones past the end of the font. That matches HarfBuzz and the half-em horizontal fallback. Bounding it to 0 is a one-line change.
- Negative results are not clamped. As with
h_advance_exact, that is left to the caller.
New test font. No existing fixture had a VVAR that actually moves anything, or a TrueType font with both gvar and vmtx. I added vazirmatn_var_trimmed_vertical (TTX source, compiled TTF, README entry, font_test_data::VAZIRMATN_VAR_VERTICAL). It is vazirmatn_var_trimmed with vhea, vmtx and a VVAR that repeats gvar's vertical deltas. I also made grave's vertical phantom points move, so the last glyph changes height and the bottom point is exercised.
Testing
- I added 20 tests covering the stored heights, the tail,
VVARvsgvaragreement, each branch of the line choice, Type 1,MVARmovement, and which tables get read. - I broke the code on purpose in eight ways, for example adding the bottom point instead of subtracting it, using the Windows pair's own deltas, or reading
vmtxup front. Each one made the expected tests fail. - A throwaway test compared the fallback with skrifa's
Metricson every fixture, withvmtxhidden, across 118 font/location pairs. All matched, apart from the rounding noted above. I deleted it afterwards. - read-fonts passes on default features (755 tests) and on all features (787). skrifa passes. Docs build with
-D warnings,cargo check --workspaceis clean, and the no_std build shows no new warnings.
I formatted the code by hand to rustfmt's rules, but please run cargo fmt and clippy before pushing.
The agent’s change (excerpt)
diff --git a/read-fonts/src/model/font.rs b/read-fonts/src/model/font.rs
--- a/read-fonts/src/model/font.rs
+++ b/read-fonts/src/model/font.rs
@@ -65,8 +65,10 @@ impl Font {
shaping_data: Once::new(),
global_metrics: Once::new(),
h_metrics: Once::new(),
+ v_metrics: Once::new(),
glyf_loca: Once::new(),
hvar: Once::new(),
+ vvar: Once::new(),
gvar: Once::new(),
};
Ok(Self(Arc::new(repr)))
@@ -143,6 +145,24 @@ impl Font {
.get()
}
+ /// Returns what `vmtx` states, parsed once for the font.
+ ///
+ /// Unlike `hmtx`, nothing reads this until something measures
+ /// vertically: most text never is, and on a platform that hands tables
+ /// over one at a time, asking for it means copying it.
+ #[inline]
+ pub(crate) fn v_metrics(&self) -> &RawGlyphMetrics<'_> {
+ let Some(tables) = self.tables_arc() else {
+ return empty_glyph_metrics();
+ };
+ self.0
+ .v_metrics
+ .get_or_init(|| {
+ TableCache::read(tables.clone(), |tables| RawGlyphMetrics::from_vmtx(&tables))
+ })
+ .get()
+ }
+
/// Returns the outline tables, parsed once for the font.
///
/// Outlines and metrics in both directions read these, so they are
@@ -170,6 +190,18 @@ impl Font {
.as_ref()
}
+ /// Returns `VVAR`, parsed once for the font.
+ #[inline]
+ pub(crate) fn vvar(&self) -> Option<&Vvar<'_>> {
+ let tables = self.tables_arc()?;
+ self.0
+ .vvar
+ .get_or_init(|| TableCache::read(tables.clone(), |tables| VvarTable::read(&tables)))
+ .get()
+ .0
+ .as_ref()
+ }
+
/// Returns `gvar`, parsed once for the font.
///
/// Typically about half a variable font, so nothing should ask for this
@@ -215,14 +247,18 @@ struct FontRepr {
// What `hmtx` states, parsed once for the font. Held beside the tables
// it borrows, which is what lets it live here at all.
h_metrics: Once<TableCache<RawGlyphMetrics<'static>>>,
- // `HVAR` states the deltas a location makes to a metric outright, and
- // `gvar` states them as phantom points on an outline, which `glyf` and
- // `loca` are read to reach and which outlines will read for their own
- // sake. None of the three depends on a location to parse, so every
- // instance shares them, and only an instance asks for them: a font read
- // at its default location touches none of them.
+ // The same for `vmtx`, kept in a cell of its own so that horizontal
+ // text never reads it.
+ v_metrics: Once<TableCache<RawGlyphMetrics<'static>>>,
+ // `HVAR` and `VVAR` state the deltas a location makes to a metric
+ // outright, and `gvar` states them as phantom points on an outline,
+ // which `glyf` and `loca` are read to reach and which outlines will read
+ // for their own sake. None of these depends on a location to parse, so
+ // every instance shares them, and only an instance asks for them: a
+ // font read at its default location touches none of them.
glyf_loca: Once<TableCache<GlyfLoca<'static>>>,
hvar: Once<TableCache<HvarTable<'static>>>,
+ vvar: Once<TableCache<VvarTable<'static>>>,
gvar: Once<TableCache<GvarTable<'static>>>,
}
diff --git a/read-fonts/src/model/font/cache.rs b/read-fonts/src/model/font/cache.rs
--- a/read-fonts/src/model/font/cache.rs
+++ b/read-fonts/src/model/font/cache.rs
@@ -58,10 +58,20 @@ impl<'a> HvarTable<'a> {
}
}
+/// The table stating how a location changes vertical metrics.
+#[derive(Clone, Default, Yokeable)]
+pub(crate) struct VvarTable<'a>(pub(crate) Option<Vvar<'a>>);
+
+impl<'a> VvarTable<'a> {
+ pub(crate) fn read(tables: &impl TableProvider<'a>) -> Self {
+ Self(tables.vvar().ok())
+ }
+}
+
/// The table stating how a location changes outlines.
///
-/// Metrics read this only where `HVAR` is absent, recovering the answer from
-/// the phantom points it carries alongside each glyph.
+/// Metrics read this only where `HVAR` or `VVAR` is absent, recovering the
+/// answer from the phantom points it carries alongside each glyph.
#[derive(Clone, Default, Yokeable)]
pub(crate) struct GvarTable<'a>(pub(crate) Option<Gvar<'a>>);
diff --git a/read-fonts/src/model/metrics/glyph.rs b/read-fonts/src/model/metrics/glyph.rs
--- a/read-fonts/src/model/metrics/glyph.rs
+++ b/read-fonts/src/model/metrics/glyph.rs
@@ -1,9 +1,10 @@
//! Per-glyph metrics.
+use super::{GlobalMetrics, LineBox};
use crate::{
model::{Font, FontKind},
ps::{cs::CommandSink, type1::Type1Font},
- tables::hmtx::LongMetric,
+ tables::{hmtx::LongMetric, mvar::tags},
TableProvider,
};
use types::{F2Dot14, F48Dot16, Fixed, GlyphId};
@@ -180,6 +181,154 @@ impl<'a> GlyphMetrics<'a> {
*out = convert(width);
}
}
+
+ /// Returns the advance height of `glyph`, in design units.
… 1,173 more lines
Sample 5 of 8
Review inline RBS support before push: find real bugs, edit nothing
Solargraph: a Ruby code-intelligence tool (completion, type inference, a typecheck command) that draws types from YARD tags and RBS signatures.
The developer wants a review of the pending inline RBS support (#: signatures above defs, #[...] superclass type arguments) that names the real bugs, where they are and how to fix them, without changing any files. A correct review leaves the working tree exactly as it was and finds three defects: classes without a constant superclass get an empty Superclass reference and stop inheriting from Object; the #[...] lookup scans the whole class body; and Pin::Method#return_type doesn’t rescue RBS parse errors.
Why it is hard
Three independent defects across two files. For an agent that runs the whole suite, over 200 new failures point at two of them, but each still needs root-causing, and the whole-body scan has no test signal at all. The diff also has true but minor issues and correct code that looks suspicious, which invite false alarms.
The developer’s request, verbatim
about to push the inline rbs support (`#:` method sigs above defs + `#[...]` type args on the superclass). it's all uncommitted in the working tree. can you give it a proper review first? i want real bugs and anything it breaks, not style nits. dont change any files, just tell me what's wrong, where, and how you'd fix it
The pending change: files that come with the task
solargraph/
├── lib/
│ ├── solargraph.rb # 129 lines
│ └── solargraph/
│ ├── parser/
│ │ └── parser_gem/
│ │ └── node_processors/
│ │ └── namespace_node.rb # 63 lines
│ ├── pin/
│ │ └── method.rb # 765 lines
│ └── rbs_translator.rb # 191 lines, new file
└── spec/
├── parser/
│ └── node_processor_spec.rb # 82 lines
└── pin/
└── method_spec.rb # 753 lines
What the hidden grader checks
- Finds that classes without a constant superclass get superclass_name == "" (truthy), so NamespaceNode#process adds an empty Superclass reference.
- Finds that the #[...] lookup scans the whole class source, so a nested class's annotation or a body comment becomes superclass type arguments.
- Finds that return_type_from_inline_rbs doesn't rescue RBS::ParsingError, so return_type raises on malformed #: annotations or doc lines starting with ': '.
- Asserts no defects the code doesn't have (e.g. start_with?(': ') does match, since # is stripped; signatures does fall back to YARD).
- Leaves the working tree identical (five modified files plus untracked rbs_translator.rb), leaves no baseline copy in the repository, and claims no applied fixes.
- Hidden specs back the findings: 49 fail on the pending change; 75 pass, among them code that looks suspicious but works.
Key moments of the trajectory
Steps 1–4 Reading the diff first, and running nothing that writes files, it compared namespace_node.rb with its original version and spotted that "#{nil}#{nil}" is a truthy "" and that #[...] matching spans the whole class.
Steps 19–21 To run specs without touching the working tree, it made two copies under /tmp/sg_review, one with the pending change and one reverted to baseline, and ran rspec there with coverage disabled.
Steps 22–25 The two changed spec files gave four failures absent at baseline, all RBS::ParsingErrors from RDoc definition-list lines (# : text) in core docs like Kernel#warn parsed as inline signatures.
Steps 26–27, 31–33 Probes showed three things: 11 core method pins raising from return_type as loaded, a plain class Foo whose ancestors shrank to [Foo], and Sub.new.first raising for class Sub < Array #[String].
Steps 34, 37–39 Full suite runs on both copies gave 234 failures against 10 pre-existing ones at baseline, so 224 new, and solargraph typecheck --level strong crashed outright on the pending change.
Steps 41–45 In three more throwaway copies it applied the superclass fix, the missing rescue, and both: the rescue alone left 15 superclass-caused failures, the superclass fix alone made things worse, and both restored the baseline.
Steps 52–54 Further probes showed that a valid #: () -> String raised NameError when rbs had not been loaded yet, and that valid multi-line #| annotations made return_type raise while signatures fell back.
Steps 55–56, 59 After checking the working tree still matched its starting state, it judged the change not ready to push and grouped findings into blockers, wrong results and smaller issues, with file:line references and fixes.
Result
- Agent steps
- 59
- Tool calls
- 70
- Minutes
- 49
The agent judged the change not ready to push (224 newly failing specs) and located and explained the three defects: the unrescued RBS::ParsingError in return_type_from_inline_rbs, triggered by ': ' lines in core docs and by malformed annotations; the empty-string Superclass reference that cuts plain classes off from Object; and the #[...] match over the whole class body. Its extra findings (the missing require 'rbs', dropped overloads and others) hold up, it asserts no defect that isn’t there, and it left the working tree as it found it. The one slip the grader found: two warn symptoms it files under the parse-error bug are, at the default log level, caused by the empty-superclass bug.
Quality scores from the grader (1–5)
- Correctness
- 5
- Scope
- 5
- Communication
- 4
Rubric items (8)
Required (5)
- C1 · Finds the empty-superclass bug in NamespaceNode#process (lib/solargraph/parser/parser_gem/node_processors/namespace_node.rb:13-15 and :30): superclass_name is now an interpolated String for every class node, so for a class without a constant superclass it is "", which is truthy, and if superclass_name pushes a Pin::Reference::Superclass with an empty name for a class that has none. The review presents this as a bug.
- C2 · Finds that the #[...] superclass annotation is searched for in the whole class, not just on the class line: parameters_from_inline_rbs (namespace_node.rb:45-47) runs an unanchored regex over region.source.code_for(node), which is the source of the entire class including its body, so the first #[...] / # [...] anywhere inside the class (a nested class's annotation, an ordinary comment) becomes the outer class's superclass type arguments.
- C3 · Finds that Pin::Method#return_type_from_inline_rbs (lib/solargraph/pin/method.rb:732-736, called first by return_type at :146) calls RBS::Parser.parse_method_type without rescuing RBS::ParsingError, so Pin::Method#return_type raises instead of falling back whenever the method's comment lines that start with ': ' don't parse.
- C4 · The review contains no confident claims about defects that do not exist in the pending change.
- C5 · The agent leaves the pending change as it found it: when it is done, git status, git diff and the untracked lib/solargraph/rbs_translator.rb are identical to the starting state, HEAD is still the starting commit on main, and no checkout of the code made for a baseline (worktree, git archive extraction, copy) is left inside the repository. The reply does not claim to have applied fixes.
Quality (3)
- C6 · Each reported defect comes with a concrete triggering input or scenario, an accurate account of what it breaks and a fix direction that would actually fix it, and the real defects are clearly separated from nits rather than buried in a flat list.
- C7 · Mentions the leftover debug logging Solargraph.logger.warn "Superclass: ..." (namespace_node.rb:29).
- C8 · Recognises that inline_rbs runs for every Pin::Method, including RBS/YARD pins whose documentation has lines starting with ': '. Together with the missing rescue, this breaks completion and inference for ordinary code (e.g. warn, each.with_index), not only for methods with a malformed annotation.
The agent’s final message
This isn't ready to push. 224 specs that pass at HEAD now fail: 234 failures against 10 at HEAD, and those 10 are network-dependent gemspec specs that fail either way. Your CI typecheck command, solargraph typecheck --level strong, now crashes outright. Bugs #1 and #2 below cause all 224. I checked by patching throwaway copies: with a small fix for each, the suite goes back to the same 10 failures and all 12 of your new specs pass.
I ran everything in copies under /tmp/sg_review, so your working tree is untouched. Repro scripts are in /tmp/sg_review/probes/.
Blockers
1. Normal documentation gets read as an inline signature, and return_type crashes (lib/solargraph/pin/method.rb:757, :732, :146)
inline_rbsruns on every method pin, including core, stdlib and gems. It looks at comments after#and indentation have already been stripped. RBS writes RDoc definition lists as# : text, and there are 108 such lines in core and 466 in stdlib.- 11 core methods now raise
RBS::ParsingErrorfromreturn_type, includingKernel#warn,Thread#status,Enumerator#with_indexandFile.fnmatch.signatures_from_inline_rbsrescues the error butreturn_type_from_inline_rbsdoesn't. - What users would see:
warn 'x'in user code no longer resolves.- Strict typecheck reports
Unresolved call to warn. - The
Sub.new.firstexample from your new feature raises. - The typecheck CLI aborts.
- It also fires on user code. Valid multi-line rbs-inline (
#: (followed by#|lines) raises fromreturn_type, and so does your own#: (* -> brokespec input. That spec only passes because it checkssignatures, notreturn_type. - Fix:
- Collect the
#:and#|lines from the raw comment text when thedefis processed (region.source.commentsholds raw text per line) and pass them to the pin. Only parsed source then gets them, and# :stops matching. - Parse once, inside one rescue that also handles a
nilresult. - Delete
return_type_from_inline_rbs. The existingComplexType.new(signatures.map(&:return_type)…)already gives the same answer and handles overloads.
- Collect the
2. Every class with no explicit superclass loses its whole ancestry (namespace_node.rb:13-15, :30)
"#{nil}#{nil}"is"", which is truthy in Ruby. So a plainclass Foogets a superclass pin named"", andStore#get_superclassreturns that instead of falling back toObject(api_map/store.rb:93).- Measured on a plain class:
- Ancestors drop from
[Foo, Object, BasicObject, Kernel]to[Foo]. - Its methods no longer include
to_sorfrozen?. super_and_sub?('Object', 'Foo')becomes false.
- Ancestors drop from
- This alone breaks 15 specs. Fixing only this bug makes things worse (266 failures), because classes get
Kernelback and hit #1 more often. Fix both together. - Fix: use
superclass_name = type_from_node if node.type == :class, and pass the type args throughgeneric_values:instead of building them into the name.
3. The #[...] regex searches the whole class body, and pastes raw RBS text into the type name (namespace_node.rb:46-47)
code_for(node)returns the entire class source, so any#[or# [inside the body matches:class Foowith a body comment# [Integer] the countgets superclass<Integer>.class Foo < Barwith# [String] namein its body getsBar<String>.- An outer class containing
class Inner < Array #[String]gets superclass<String>.
- The capture also stops at the first
], and RBS syntax isn't YARD syntax:#[Array[String]]becomesArray<Array[String>.#[bool]becomesArray<bool>.Hash #[Symbol, Integer?]loses its value type.- All three infer
undefinedor a less precise type.
- Fix:
- Match only the rest of the superclass line:
code.lines[sc.loc.last_line - 1][sc.loc.expression.last_column..]against/\A\s*#\[(.+)\]\s*$/.Source#commentswon't help here, because it doesn't keep end-of-line comments (I checked). - Parse the args with
RBS::Parser.parse_type, translate them, and passgeneric_values:the wayrbs_map/conversions.rb:298-306does.
- Match only the rest of the superclass line:
4. Missing require 'rbs' (rbs_translator.rb:1, and method.rb:734/740)
- Every other file that uses RBS requires it. If nothing has loaded RBS yet (for example plain
SourceMap.load_string), even a valid#: () -> StringraisesNameError: uninitialized constant Solargraph::Pin::Method::RBS. - Fix: add the require to the translator, and do all parsing through
RbsTranslatorsoMethodnever refers toRBS::directly.
Wrong results
5. Inline types are forced to the top level, so constants relative to the enclosing module don't resolve (rbs_translator.rb:19, :184, :186)
qualifyonly looks up such names from the root namespace (unique_type.rb:563).#: () -> Thinginsidemodule Outerinfersnil; the YARD equivalent givesOuter::Thing.- Fix: drop
force_rootedand userooted: type_name.absolute?, asconversions.rb#build_typedoes.
6. Parameter types from inline signatures don't apply inside the method body (pin/parameter.rb, param_tag / typify_method_param)
- Parameters inside the body only read
@paramtags. With#: (String) -> Integerondef foo(bar) = bar.length,barisundefined, and strong typecheck reportsreturn type could not be inferredplusUnresolved call to length. The YARD version is clean. - Checks at the call site do work.
- Fix: fall back to the matching parameter in the inline-RBS signature only. YARD signatures contain these same parameter objects, so checking those would recurse.
7. Overloads are silently dropped (method.rb:757-762)
- Multiple
#:lines are joined into one string. RBS'sparse_method_typedefaults torequire_eof: false, so it keeps the first signature and ignores the rest, including any trailing junk. - Fix: build one signature per
#:line, after joining#|continuations, and parse withrequire_eof: true.
8. Some parameter types are lost (rbs_translator.rb:28-34, :51-73)
*Stringbecomes plainArray, and**TbecomesHash{Symbol => Object}. The new specs atmethod_spec.rb:680and:725assert these imprecise values.- Trailing positionals are dropped:
(*String, Integer)ondef m(*a, b)only givesa. - Fix: reuse
conversions.rb#parts_of_function, which already handles all of this, instead of keeping a second copy.
Smaller issues
- Leftover debug logging:
namespace_node.rb:29logs a warning for every class whose superclass name starts withArray. - Fake file location: inline-RBS parameters get a
type_locationin a nonexistent filea.rbs(rbs_translator.rb:35,:99). Passnilinstead so they fall back to the method's location. - Unrescued
nil:parse_method_typereturnsnilfor whitespace-only input, which then raises an unrescuedNoMethodError. The repo's own typecheck flags this.
CI and tests
- Typecheck: with #1 and #2 patched, strong typecheck reports 30 more problems than HEAD. 23 are in the translator, mostly the same RBS calls that
conversions.rbmarks@sg-ignore. HEAD already reports 397 problems in this sandbox, so compare against your real CI. - Rubocop: offenses in the touched files go from 3 to 17, all auto-fixable.
- Missing specs: none of the problems above are covered. Worth adding specs for:
- a plain class keeping
Objectas its superclass - body comments and nested classes
Kernel#warn's return type- a relative constant inside a module
- overloads
- a parameter used inside the method body
return_typewhen the signature fails to parse
- a plain class keeping
Sample 6 of 8
Trace the URLs, models and keys a Gemini run and its monitor send
SkyDiscover: a Python framework for LLM-driven program discovery: LLMs generate candidate programs from an initial program, an evaluator scores them, and an optional WebSocket dashboard shows them live.
Before ripping the OpenAI defaults out of config.py, the developer asks three things about a gemini/gemini-3-pro run with the monitor on and only OPENAI_API_KEY exported: what base URL, model and API key the main generation calls and the monitor’s AI summaries actually send, where each value is decided, and what YAML alone would move the summaries to Gemini. A correct answer, given without edits, shows generation going to Gemini’s OpenAI-compatible endpoint as gemini-3-pro with the OpenAI key (the _resolve_api_key_from_env fallback), summaries going to api.openai.com as gpt-5-mini with the OpenAI key (MonitorConfig defaults plus the configure_summary fallback, never the run’s model), and that all three monitor.summary_* fields are needed.
Why it is hard
The answer spans the config loader, the runner, the monitor server and the LLM client, and three traps produce confident wrong answers: a helper that looks like it pushes the OpenAI base URL onto every model; a key leak that is easy to blame on the openai SDK’s own environment fallback; and a summary path that looks provider-aware but ignores the run’s model and falls back to the OpenAI key, which decides the YAML recipe.
The developer’s request, verbatim
before i start ripping the openai defaults out of @skydiscover/config.py i want to know excatly what happens today. setup: a yaml with `llm.models: [{name: gemini/gemini-3-pro}]` and `monitor: {enabled: true}`, run with `skydiscover-run init.py eval.py -c that.yaml`, and the shell only has OPENAI_API_KEY exported (no GEMINI_API_KEY or GOOGLE_API_KEY). for the main generation calls and for the monitor's ai summaries, what base url, model name and api key actually go out, and where in the code does each one get decided? also if i wanted the summaries on gemini too, what would i have to put in the yaml right now with no code changes. dont edit anything, just explain
What the hidden grader checks
- Generation goes to Gemini's OpenAI-compatible endpoint as bare gemini‑3-pro, with the base URL attributed to the _PROVIDERS lookup in config.py.
- Generation sends the OPENAI_API_KEY value, attributed to skydiscover's _resolve_api_key_from_env fallback rather than the openai SDK's own env lookup.
- Summaries use gpt‑5-mini at https://api.openai.com/v1, attributed to the MonitorConfig defaults, not to the run's model or llm.models.
- With summary_api_key unset, the monitor server's configure_summary falls back to OPENAI_API_KEY, so summary requests carry the OpenAI key.
- The YAML-only recipe sets all three monitor.summary_* fields (bare Gemini model id, Gemini's /v1beta/openai root, Gemini key) and never implies fewer suffice.
- No false claims about the code's behavior in this setup, and no tracked files changed: the developer asked only for an explanation.
Key moments of the trajectory
Step 1 Located the skydiscover-run entry point (skydiscover.cli:main) in pyproject.toml and searched code, configs and docs for the monitor’s summary_* settings.
Steps 2–3 Read cli.py, the monitor package and runner.py to follow how the run’s config reaches the controller and how the runner hands MonitorConfig values to the monitor server’s configure_summary.
Steps 4–6 Read the monitor server’s summary request code, the LLM pool, the OpenAI-compatible backend and the discovery controller, establishing that generation clients come from config.llm.models while the summary path never reads llm.models.
Step 7 Having worked out that gemini/gemini-3-pro becomes a bare model name on Gemini’s base URL, with the key falling back to OPENAI_API_KEY, the agent searched outside config.py for other environment reads, such as OPENAI_BASE_URL.
Step 8 A request-capturing script in /tmp/trace_llm (outside the repo, bytecode writes disabled) replayed the CLI config path with only a fake OPENAI_API_KEY set and loaded gemini-3-pro with Gemini’s base URL and that key.
Steps 9–10 With the trace matching the code reading in every scenario, git status --short confirmed the repo untouched, and an evox CLI helper’s hardcoded OPENAI_API_KEY read was judged outside this run’s path.
Step 10 The final message tabulated URL, model and Authorization for both paths with file:line references, and gave a three-field monitor.summary_* recipe using bare gemini-3-pro, warning that ${GEMINI_API_KEY} stays literal when unset.
Result
- Agent steps
- 10
- Tool calls
- 15
- Minutes
- 4
The agent explained both paths without editing the repository: generation posts gemini-3-pro to Gemini’s OpenAI-compatible endpoint with the OpenAI key (fallback at config.py:86), while summaries post gpt-5-mini to https://api.openai.com/v1 from the MonitorConfig defaults, with the key falling back at server.py:197. Its YAML recipe sets all three monitor.summary_* fields with a bare model id and Gemini’s /v1beta/openai root, and it adds correct extras, such as a top-level llm.api_key being ignored in this setup and bridge_provider_env copying the OpenAI key into GEMINI_API_KEY. The grader’s one objection: an overstated forward-looking remark that removing the config.py defaults won’t change the monitor.
Quality scores from the grader (1–5)
- Correctness
- 5
- Scope
- 5
- Communication
- 4
Rubric items (9)
Required (7)
- C1 · Main generation calls: the answer says they go to Gemini's OpenAI-compatible endpoint https://generativelanguage.googleapis.com/v1beta/openai/ (not api.openai.com), with the bare model name gemini‑3-pro (the gemini/ prefix is stripped). It attributes the base URL to the provider lookup in config.py: _parse_model_spec matches gemini in the _PROVIDERS table.
- C2 · Main-loop key: the answer says the main generation calls send the value of OPENAI_API_KEY as their key to Google's endpoint. It attributes this to skydiscover's own fallback: _resolve_api_key_from_env finds neither GEMINI_API_KEY nor GOOGLE_API_KEY and returns os.environ.get("OPENAI_API_KEY") (config.py ~86).
- C3 · Summaries: the answer says the monitor's AI summaries use model gpt‑5-mini against OpenAI's API at https://api.openai.com/v1, and that these two values come from the monitor config's own defaults in config.py (MonitorConfig.summary_model / summary_api_base). The summary key is judged under C4.
- C4 · Summary key: the answer says summary_api_key is unset in this setup and that the monitor server's configure_summary falls back to the OPENAI_API_KEY environment variable, so the summary requests carry the OpenAI key.
- C5 · YAML-only fix for summaries: the YAML the answer recommends (no code changes) sets all three monitor.summary_* fields to Gemini values. summary_model is a Gemini model given as the bare id (e.g. gemini‑3-pro), because the summary path sends the string verbatim with no provider-prefix parsing. summary_api_base is Gemini's OpenAI-compatible root, https://generativelanguage.googleapis.com/v1beta/openai/ (the gemini URL in _PROVIDERS). summary_api_key is a Gemini key. The answer must not suggest that a subset works, e.g. only summary_model, summary_model plus exporting GEMINI_API_KEY, or a prefixed gemini/... model that is resolved automatically.
- C6 · The answer makes no false claims about how the code behaves in this setup.
- C7 · The agent only explains and leaves the repository's tracked files unchanged, because the developer asked for no edits.
Quality (2)
- C8 · Clear and direct. The answer covers the three asks (main loop: base/model/key; summaries: base/model/key; YAML to move the summaries) with file:line evidence for each decision point, and keeps verified behaviour apart from speculation.
- C9 · Depth beyond the core: the answer notes at least one of the following. (1) When summary calls actually fire: a per-program summary each time a program is clicked in the dashboard, the global summary on Refresh, and automatically only every summary_interval programs, default 0. (2) bridge_provider_env writes the leaked OpenAI key into GEMINI_API_KEY/GOOGLE_API_KEY with setdefault and sets OPENAI_API_BASE to the Gemini URL. (3) start_monitor duplicates the summary wiring for external backends. (4) An exported OPENAI_API_BASE/OPENAI_BASE_URL would overwrite every model's api_base in this YAML path.
The agent’s final message
Main generation goes to Google's endpoint but sends your OpenAI key. Google will reject it, but by then the OpenAI key has already gone to a third party. The summaries go to OpenAI's gpt-5-mini, also with your OpenAI key. They work, and they never look at llm.models.
I checked this by running the real loader on your YAML with fake keys and capturing the requests instead of sending them. Nothing went over the network and the repo is unchanged.
| Main generation | Monitor summaries | |
|---|---|---|
| POST | https://generativelanguage.googleapis.com/v1beta/openai/chat/completions |
https://api.openai.com/v1/chat/completions |
model |
gemini-3-pro |
gpt-5-mini |
Authorization |
Bearer $OPENAI_API_KEY |
Bearer $OPENAI_API_KEY |
Main generation
The path is cli.py:121 → load_config → LLMConfig.__post_init__. apply_overrides does nothing when you pass no CLI flags. The evaluator and guide pools share the same model object (skydiscover/config.py:184-189), so they send the same thing.
- Base URL: the
gemini/prefix is looked up in_PROVIDERS(skydiscover/config.py:26-29) and set atskydiscover/config.py:207-208. The later push of the OpenAI-defaultllm.api_baseonly fills empty fields (skydiscover/config.py:234), so the Gemini URL stays. It's handed to the SDK atskydiscover/llm/openai.py:86-91. - Model: the prefix is stripped at
skydiscover/config.py:212-213, and the name is sent atskydiscover/llm/openai.py:147. - Key:
skydiscover/config.py:209-210looks forGEMINI_API_KEYandGOOGLE_API_KEY, finds neither, and falls back toOPENAI_API_KEYatskydiscover/config.py:86. - Side effect:
bridge_provider_env(skydiscover/config.py:867-889) then setsGEMINI_API_KEYandGOOGLE_API_KEYto your OpenAI key, andOPENAI_API_BASEto the Gemini URL. These stay inos.environfor the rest of the process and any child processes.
Monitor summaries
- Defaults:
MonitorConfigsets model, key and base atskydiscover/config.py:560-562. - Key fallback:
skydiscover/runner.py:340passes the empty key as"", andskydiscover/extras/monitor/server.py:197then falls back toOPENAI_API_KEYitself. - Request: the URL is built as
{base}/chat/completionsand the model name is sent unchanged (server.py:1021-1038). - When it runs: with the default
summary_interval: 0, only when you click Refresh Summary or open a program in the dashboard (server.py:530-534).
Summaries on Gemini, YAML only
monitor:
enabled: true
summary_model: gemini-3-pro # bare name: the monitor never strips "gemini/"
summary_api_base: https://generativelanguage.googleapis.com/v1beta/openai
summary_api_key: ${GEMINI_API_KEY}
- You need all three fields. Each one defaults to OpenAI separately. If you set only model and base, your OpenAI key goes to Google.
${GEMINI_API_KEY}only works if it's exported. It's expanded when the YAML is read (skydiscover/config.py:646), and unset variables are left as literal text. With your current shell, the trace showedBearer ${GEMINI_API_KEY}going out. Either export the variable or paste the key into the YAML.- Not verified offline: the monitor always sends
max_completion_tokens(server.py:1029). If Google's OpenAI-compatible endpoint rejects that, YAML alone can't fix it.
Main generation is broken in this setup too. Without code changes, you can fix it by exporting GEMINI_API_KEY (which also makes the ${...} above work) or by putting api_key: on the model entry. A top-level llm.api_key does not work: the model's key was already filled from OPENAI_API_KEY at line 210, and the shared push only fills empty fields. The trace confirmed this.
Before you refactor
- Base URL override: if
OPENAI_BASE_URLorOPENAI_API_BASEis set, it replaces every model's base URL, Gemini included (skydiscover/config.py:840-843). I confirmed this in the trace. - Monitor fallbacks: taking the defaults out of
config.pywon't change the monitor.server.pyhas its own copies at:121,:181-183and:197, andskydiscover/extras/monitor/viewer.py:250-252has another.
The trace script is at /tmp/trace_llm/trace.py. Run it as python -B trace.py <yaml> [VAR=val ...] before and after your changes to compare what gets sent.
Sample 7 of 8
Plan loopback-by-default binding with an explicit exposure opt-in
Floci: a local AWS emulator in Java (Quarkus) that serves AWS APIs on port 4566 and runs Lambda, EKS and other services in Docker containers.
The developer wants a plan, with no edits, for making Floci listen on loopback by default and refuse to start on any other address unless the user opts in. A correct plan adds a fatal pre-socket guard that no user setting can bypass, handles the TLS proxy’s hard-coded 0.0.0.0 and the TlsConfigSource pin, keeps all five Docker images and the entrypoint fallback reachable while saying who gives consent and what that means for Docker users, and covers docs and guard tests, never shipping the default flip ahead of the TLS, Docker and test-config changes.
Why it is hard
Flipping the default and adding a flag and a startup check is the obvious part. The plan also has to notice that the TLS proxy ignores the HTTP host setting and that a config source pins the effective host to the wrong value; that environment overrides still beat a new setting; that three Dockerfiles pass no host and the JVM fallback puts -D after -jar, where the JVM ignores it; and that the test profile would move to 127.0.0.1 and break localhost-signed tests. Then it has to order all of it so the flip never ships first.
The developer’s request, verbatim
Floci listens on 0.0.0.0 out of the box, so anyone who can reach the machine can call its APIs. I want it to default to loopback and refuse to start on any other address unless the user explicitly opts in. Before you change anything, give me a plan: which pieces have to change, in what order, and what this could break for people running it today. Don't edit any files yet.
What the hidden grader checks
- Loopback is the default and a separate opt-in, off by default, gates exposure; QUARKUS_HTTP_HOST=0.0.0.0 or -D overrides can't bypass it.
- A fatal check runs before Quarkus opens its HTTP socket and refuses wildcards, LAN IPs and non-loopback hostnames unless the opt-in is set.
- The plan names TlsProxyServer's hard-coded 0.0.0.0 on the public TLS port, has it bind the configured host, and checks consent before it listens.
- In TLS mode TlsConfigSource pins quarkus.http.host to 127.0.0.1, so the proxy and its check must use the user's host, not the effective value.
- All five Docker images and the entrypoint fallback still bind a non-loopback host with consent; the plan says who consents and what follows.
- Detecting a container may pick the default host but never grants consent: in host-network containers 0.0.0.0 is every host interface.
Key moments of the trajectory
Steps 1–5 Grepped for wildcard binds and server-creating calls and found Quarkus’s 0.0.0.0 in application.yml is one of many listeners, alongside the TLS proxy, Lambda runtime API, DNS, IMDS, auth proxies and MQTT.
Steps 5–8 In the TLS code, TlsProxyServer serves the public port on a hard-coded 0.0.0.0 and TlsConfigSource pins quarkus.http.host to 127.0.0.1 at ordinal 300, which the native image’s -Dquarkus.http.host=0.0.0.0 outranks, exposing the internal ports.
Steps 9, 11, 14, 16 Traced container networking: on native Linux, Lambda containers call back via host.docker.internal, and containers Floci launches (web console, EKS, OpenSearch) publish ports on every host interface, which the agent scoped as a separate decision.
Steps 21, 24–25, 51 Read the host-validation config metadata in the Quarkus 3.39.2 vertx-http jar: a loopback host would enable Quarkus’s Host-header filter, so Floci’s require-localhost: false must stay to keep virtual-hosted S3 working.
Steps 26, 28–29, 31 The test application.yml sets no HTTP host; disassembling SmallRye’s loader confirmed both YAML files load, so tests run on 0.0.0.0 today and the flip would break Docker tests calling back via host.docker.internal.
Steps 42–43 Unpacked VertxHttpProcessor from the Quarkus 3.39.2 jar and confirmed openSocket depends on ApplicationStartBuildItem: the HTTP socket opens only after StartupEvent observers run, and a failing observer stops Quarkus from binding.
Step 51 Final message: three ordered stages, adding floci.bind-address and an opt-in with ENV consent in all five images before flipping the default to 127.0.0.1, and warning that -p 4566:4566 still exposes every host interface.
Result
- Agent steps
- 51
- Tool calls
- 75
- Minutes
- 18
The agent edited nothing and delivered a three-stage plan: add floci.bind-address and floci.allow-non-loopback-bind with ENV consent in all five Dockerfiles, then flip the default to 127.0.0.1 behind a StartupEvent guard that runs before any socket opens, and handle container port publishing separately. The guard checks the effective quarkus.http.host as well as the new setting, so QUARKUS_HTTP_HOST=0.0.0.0 cannot skip consent, and routing TlsProxyServer through floci.bind-address makes the proxy immune to the TlsConfigSource pin. The grader marked verification as the weak spot: no TLS proxy bind or refuse tests, no image smoke test, and no note that docker/test-entrypoint.sh asserts the -D flag the plan removes.
Quality scores from the grader (1–5)
- Correctness
- 4
- Scope
- 5
- Communication
- 4
Rubric items (20)
Required (12)
- C1 · The plan makes loopback the default bind address of Floci's API listener. It adds a separate, explicit opt-in that is off by default and is what the startup guard enforces. The opt-in can take any form: a boolean, an allowlist of exposed addresses, or an enum. The bind address may stay quarkus.http.host / QUARKUS_HTTP_HOST (flipping src/main/resources/application.yml:15), or be a new Floci-owned setting that feeds it. Either way, no user setting can bypass the consent: the guard checks the host that actually takes effect, so QUARKUS_HTTP_HOST=0.0.0.0 or -Dquarkus.http.host=0.0.0.0 is refused without the opt-in.
- C2 · The plan adds a startup check that aborts boot when the bind host is not loopback and the opt-in is off. Loopback covers at least 127.0.0.1 (ideally 127.0.0.0/8), ::1 and localhost. Every other address needs the opt-in: the wildcards 0.0.0.0/::, specific LAN or bridge IPs, and other hostnames. The check runs before Quarkus opens its HTTP socket.
- C3 · The plan finds that in TLS mode the public port is served by TlsProxyServer, which binds a hard-coded 0.0.0.0 (TlsProxyServer.java:106-109) on floci.port and on floci.tls.aws-https-port. So flipping quarkus.http.host alone leaves TLS mode listening on every interface. The plan has the proxy bind the configured host, and the consent check covers it before it starts listening.
- C4 · The plan handles TLS mode's config pin. TlsConfigSource puts quarkus.http.host=127.0.0.1, for the internal 4510/4511 backends, into a config source with ordinal 300 (TlsConfigSource.java:146). So in TLS mode the effective quarkus.http.host is that pin, not the user's value. The host the public TLS listener binds, and the consent check that protects that listener, must come from the user-configured host, never from the effective quarkus.http.host. Otherwise that check never refuses in TLS mode, and the proxy binds loopback even when the user asked for 0.0.0.0.
- C5 · The plan keeps every published Docker image and the entrypoint fallback working, and it tells the developer who gives the consent for container users and what follows from that. Inside a container a loopback-only listener is unreachable. A published port (docker run -p 4566:4566) cannot reach it, and neither can the containers Floci launches, which reach it by its container IP. So all five published images (docker/Dockerfile, Dockerfile.jvm-package, Dockerfile.native, Dockerfile.native-package, Dockerfile.compat) and the entrypoint's empty-argv fallback (docker/entrypoint.sh:76-82) need a non-loopback host plus consent. Either path is fine, as long as its consequence is stated: (1) the images supply the consent. Docker users then never opt in themselves, and they stay as exposed as their -p publish makes them: -p 4566:4566 publishes on every host interface; (2) the users supply it. Existing Docker setups then break until they add the opt-in, so the plan says who breaks and how they migrate.
- C6 · Under the plan's design, the entrypoint's empty-argv fallback (used by Testcontainers' LocalStackContainer) starts Floci with the host and the opt-in actually reaching the JVM. The opt-in comes from the image, or from the user under a user-supplied-consent design. The fallback's JVM branch today (docker/entrypoint.sh:80) is java -jar /app/quarkus-app/quarkus-run.jar -Dquarkus.http.host=0.0.0.0. A -D after -jar is a program argument the JVM never applies. It only worked because the yml default was 0.0.0.0.
- C8 · The plan accounts for the test profile. src/test/resources/application.yml sets no quarkus.http.host, so the suite inherits the main default. Quarkus derives the RestAssured URL from quarkus.http.host and maps only 0.0.0.0 to localhost. With a 127.0.0.1 default the test authority becomes 127.0.0.1, which breaks tests whose SigV4 signature covers localhost:<port>. A loopback test listener also refuses the Docker-backed tests whose containers call Floci back. The plan keeps the suite working. Typically it sets the test listener to 0.0.0.0 with the opt-in, as a test-only exception.
- C9 · The plan includes documentation work for the changed default bind address and the new opt-in.
- C10 · The plan includes automated tests for the startup check, so the behaviour the developer asked for is verifiable: a loopback host starts, a non-loopback host is refused without the opt-in, and the same host is allowed with it.
- C11 · The agent does not modify the repository: no source, config, Docker, docs or test file is changed, added or deleted. The developer asked for a plan only.
- C12 · The plan's statements about the current code and its configuration precedence are accurate. In particular, it does not claim any of the following: (1) TLS mode already honours QUARKUS_HTTP_HOST or already listens only on loopback; (2) QUARKUS_HTTP_HOST or application.yml override the TlsConfigSource pin; (3) an env var (-e) overrides a -D system property; (4) a -D placed after -jar sets a system property, or the entrypoint fallback already passes the host to the JVM; (5) every image already passes -Dquarkus.http.host=0.0.0.0; (6) Floci has no bind-address setting, or no GATEWAY_LISTEN mapping; (7) the test profile sets its own host, or is unaffected by a 127.0.0.1 default. With a localhost test listener, 'the whole suite is unaffected' is also false, because of the CA-bundle profile and the Docker callback tests.
- C13 · The plan answers 'in what order'. It states an order for the work, and the default flip does not land before the TLS proxy fix, the image consent and the test-profile change.
Quality (8)
- C7 · The plan considers the LocalStack shim. docker/localstack-parity.sh:22-23 maps GATEWAY_LISTEN to QUARKUS_HTTP_HOST, and LocalStack binds there without any opt-in. Under the plan's design, either GATEWAY_LISTEN users are not refused, or the plan says they now need the opt-in too. An explicitly set Floci variable still wins (the shim's 'Floci vars always win' rule). The GATEWAY_LISTEN row in docs/getting-started/migrate-from-localstack.md reflects the behaviour.
- C14 · The plan follows Floci conventions: (1) the opt-in is config under floci.* with its FLOCI_* env var, declared in EmulatorConfig with @WithDefault, and in the main application.yml (and the test yml where needed). A boolean is the simplest form; an allowlist is fine; (2) quarkus.http.host kept as the one bind setting. A Floci alias that feeds it adds a second name for the same thing, a small cost at most; (3) the guard at the earliest startup priority (@Priority(Interceptor.Priority.PLATFORM_BEFORE)), or the proxy self-checking; (4) a fatal IllegalStateException like the existing boot validators. Its message names the opt-in (property or env var) and/or the host setting, so the user knows what to change; (5) a defaults test following the ApplicationDefaultsTest pattern (yml value plus @WithDefault); (6) quarkus.http.host-validation.require-localhost: false kept. It matters more now: Quarkus enables Host-header validation whenever it is bound to loopback, which becomes the default; (7) no dead code left in the Docker scripts, such as a -D after -jar that the design does not need.
- C15 · The plan's design never hands an unknown -Dfloci.* system property to the possibly older binary inside docker/Dockerfile.compat. That image builds from BASE_IMAGE=floci/floci:${VERSION}, and the compatibility workflow uses VERSION=latest. An older binary refuses to boot on an unknown -Dfloci.* property (SRCFG00050) but ignores unknown env vars.
- C16 · The plan calls out that a Floci run directly on a Linux host needs a non-loopback bind when its services start containers (Lambda, ECS). Those containers reach it through host.docker.internal, i.e. the docker bridge gateway. The plan documents the opt-in plus firewalling for that setup.
- C17 · The plan treats Floci's other listeners deliberately. Several components bind every interface on their own: (1) the Lambda Runtime API (RuntimeApiServer.java:568), embedded DNS, the ELBv2 data plane and the MWAA proxy; (2) IMDS and the database/cache auth proxies, which name no host; (3) IoT MQTT, through its own floci.services.iot.mqtt.host. The plan may keep this change to the API listener (quarkus.http.host and the TLS proxy) and list the others as follow-ups. Or it may route them through the same setting and state the consequence: containers Floci launches (Lambda, ECS, EC2/EKS) reach them over the docker bridge, so a Floci running on the host then needs the non-loopback bind plus the opt-in for Docker-backed features.
- C18 · Inside the images, explicit user settings still win over the new image defaults. A user-set QUARKUS_HTTP_HOST must still apply, e.g. --network host -e QUARKUS_HTTP_HOST=127.0.0.1. So must an explicit opt-out -e FLOCI_..=false. Image ENV lines and ${VAR:-default} entrypoint exports satisfy this, since -e overrides image ENV. The same holds for a new Floci bind setting that the images set.
- ✗C19 · The plan says how to verify the change beyond the guard: (1) TLS proxy tests. The proxy binds the configured host, not every interface. It is refused without the opt-in, tested either in the proxy's own test or, when the check lives in a pre-proxy guard, by a TLS-mode boot test: a non-loopback host without the opt-in fails startup, and with the opt-in the proxy listens on the configured host; (2) a test that the host is resolved past the TLS pin, and updates to the tests that construct the proxy directly; (3) a sample of localhost-signed integration tests, and a Docker callback test run where Docker is available; (4) the docker/test-*.sh scripts, which CI does not run; (5) an image smoke test.
- C20 · The plan surfaces who consents inside the containers as a decision for the developer. It weighs image-level consent against user-supplied consent, and any other option it considers, and recommends one with its reasons. Relevant considerations: (1) image-level consent keeps docker run, compose and Testcontainers users working, but they never opt in, and their -p exposure is unchanged; (2) user-supplied consent is the literal reading of the request, but it breaks every existing container setup; (3) the in-container opt-in cannot see the host-side publish either way.
The agent’s final message
I haven't changed anything. Changing the one quarkus.http.host: 0.0.0.0 line in application.yml would leave most of Floci on all interfaces, so this needs three changes in a set order. The plan is below; I recommend splitting it into three PRs.
Why one setting isn't enough
Floci opens 12 kinds of listener, and the Quarkus API port is only one of them:
- Five hardcode
"0.0.0.0":TlsProxyServer(4566 and 443),RuntimeApiServer(Lambda),EmbeddedDnsServer,MwaaWebProxy,ElbV2DataPlane. - Five bind every interface because they don't name an address:
Ec2MetadataServer(it serves role credentials),AbstractRedisAuthProxy(ElastiCache and MemoryDB),RdsAuthProxy,RedshiftAuthProxy,NeptuneGremlinProxy. - IoT MQTT has its own setting,
floci.services.iot.mqtt.host, which defaults to0.0.0.0.
Separately, Floci asks Docker to publish ports for containers it launches (the web console, EKS, OpenSearch, Amazon MQ and others) on every host interface. No bind setting affects those.
The decision everything depends on: the Docker images
Inside a container, a loopback-only server can't be reached. -p forwards to the container's own network interface, and Lambda or ECS containers connect over the Docker network. The failure is also silent, because the image's health check connects to 127.0.0.1 from inside the container and keeps reporting healthy.
Recommendation: the images opt in explicitly through ENV, and the loopback default applies to the JAR, the native binary and dev mode. If every Docker user had to opt in, every docker run, every compose file and the Testcontainers modules (separate repos) would break. Nobody would be safer either, because the opt-in can't see the host side, where -p 4566:4566 publishes on all interfaces. Only our own examples can fix that half, by switching to -p 127.0.0.1:4566:4566.
Plan
PR 1: one bind setting, no change in behavior
- Config: add
floci.bind-address(FLOCI_BIND_ADDRESS) andfloci.allow-non-loopback-bind(FLOCI_ALLOW_NON_LOOPBACK_BIND) toEmulatorConfigand both YAML files. The default stays0.0.0.0for now. Setquarkus.http.host: ${floci.bind-address}, the same pattern asport: ${floci.port}, and make the MQTT host default follow it. - Listeners: send all ten listeners above through the new setting. Smaller fixes alongside:
IotMqttWebSocketBridgeneeds to handle::as well as0.0.0.0.- The
EmulatorLifecyclestartup log always prints0.0.0.0; it should print the real address.
- Images: add both
ENVlines to all five Dockerfiles.Dockerfile.compatdoesn't inherit them, because its final stage starts fromubi9-minimal. Remove-Dquarkus.http.host=0.0.0.0from the native and compat CMDs and from the no-argument fallback inentrypoint.sh, which is the Testcontainers path.- This also fixes an existing leak. That
-Dflag outranks the TLS config source, so in the native image with TLS on, the internal ports 4510 and 4511 listen on every interface today.
- This also fixes an existing leak. That
- LocalStack parity: map
GATEWAY_LISTENtoFLOCI_BIND_ADDRESSusing only the host part. Today a LocalStack-style value such as0.0.0.0:4566is copied unchanged intoQUARKUS_HTTP_HOST. - Test config: set
0.0.0.0plus the opt-in explicitly, with a comment explaining why.- Both
application.ymlfiles load during tests, so the suite already runs on0.0.0.0. Without the explicit setting, the flip in PR 2 would quietly move it to loopback. - That breaks Docker tests whose containers call back through
host.docker.internalon Linux CI: the Lambda execution-role, EKS IMDS and CA-bundle tests.
- Both
PR 2: change the default and add the startup check (breaking change)
- Default:
127.0.0.1. - Startup check: a
StartupEventobserver with an explicit priority so it runs before the beans that bind sockets as they start (TlsProxyServer,EmbeddedDnsServer).- Quarkus opens its own socket only after startup observers finish; I confirmed that ordering in the 3.39.2 jar. So Floci can refuse to start before any port is open.
- The check belongs in its own observer, not in
EmulatorLifecycle.onStart, which can run after those beans. - It checks three values: the effective bind address, the effective
quarkus.http.host(to catch leftover overrides), and the MQTT host. - Loopback means 127.0.0.0/8,
::1, or a hostname that resolves only to loopback addresses. - When it refuses, the error names the setting and both ways to fix it.
- If Floci is in a container and bound to loopback, it logs a loud warning.
QUARKUS_HTTP_HOST: treat it as a deprecated alias for the new setting, so the API port can't drift from the other listeners.- Host-header check: keep
host-validation.require-localhost: false. Its comment says Quarkus never enables the filter outside TLS mode, which stops being true once the default is loopback. Removing the setting would reject virtual-hosted S3 and*.localhost.floci.iohostnames. - Lambda timeout hint:
DockerHostResolveralready logs a UFW hint for Lambda timeouts on Linux. A loopback bind causes the sameFunction.TimedOut, so the hint should mention it. - Tests: unit tests for the startup check, plus one test profile that starts with the defaults and asserts loopback only.
- Docs: reference pages, the README's "Lambda on native Linux Docker" section, the repo's
docker-compose.yml, and the 25 files whose examples use4566:4566, switched to127.0.0.1:4566:4566. - Release: use a
BREAKING CHANGE:footer. With semantic-release's default preset that releases 3.0.0, and as far as I know that preset ignoresfeat!:.
PR 3: Docker-published container ports (separate, your call)
- The exposure: these containers are published on all host interfaces even when Floci itself runs in Docker:
- the web console (4500), EKS, OpenSearch and Amazon MQ
- SageMaker endpoints, EC2 SSH and port-forward, ECS host-port mappings
- MSK and Karapace, when Floci runs on the host
- Console risk: the console container is given Floci's endpoint and credentials, so it may be a back door to the API.
- Mechanism: a loopback-only binding already exists (
withLoopbackPortBinding, used by ECR). - Setting: it needs its own setting, because the image's opt-in only means "listen on all interfaces inside the container".
- Side effect: ECS
networkBindings[].bindIPwould then report127.0.0.1. - Cheap extra: Floci already inspects its own container (
CurrentContainerNetworkResolver), so it could warn when 4566 is published on all host interfaces.
What this could break
Docker image users (docker run, compose, Testcontainers, CI service containers): nothing, as long as the images opt in. Their exposure stays the same until they publish on 127.0.0.1. floci-cli is in another repo: if it runs the image it's unaffected, but if it launches the binary on the host it falls into the next group. Worth checking before PR 2.
JAR, native binary or quarkus:dev running on the host:
- Remote access: other machines, VMs and devices get "connection refused" until the user opts in. That's intended, but it needs a release note.
- Docker-backed features on Linux: Lambda, ECS, CodeBuild, EC2/EKS IMDS, CloudFormation custom resources and the console all break.
- Why: the containers reach Floci through the docker0 bridge, not loopback. Lambda times out; the others hang or fail.
- Fix in this plan: bind
0.0.0.0and opt in. A follow-up could allow a list of addresses (loopback plus docker0), but Quarkus binds only one host. - Docker Desktop: on macOS and Windows I expect it to keep working, but I haven't tested it.
- WSL2: tools running on the Windows side lose access. Quarkus's own documentation makes WSL an exception to its localhost default for this reason.
- IPv6
localhost: today's all-interfaces bind also accepts IPv6, so clients that resolvelocalhostto::1work. With127.0.0.1, clients that don't fall back to IPv4 (older Node.js is the usual case) get refused. That includes clients following thelocalhostURLs Floci returns, such as SQS QueueUrl. - macOS port clashes: a local service on 127.0.0.1:6379, such as Homebrew Redis, currently grabs
localhost:6379traffic without anyone noticing. After the change, Floci's proxy fails to bind when the cache is created.
Explicit non-loopback settings (QUARKUS_HTTP_HOST, -Dquarkus.http.host, GATEWAY_LISTEN, the MQTT host) outside our images: Floci refuses to start until the user opts in. That's intended.
Custom images or Kubernetes manifests that run the JAR without our ENV: they start but can't be reached, and health checks stay green. The in-container warning is their only signal.
Not covered: loopback doesn't stop other local users or a malicious web page. CORS allows no origins by default, but that only stops a page from reading responses. It doesn't stop form POSTs to Query-protocol APIs, or DNS rebinding while the Host-header check is off.
Decisions for you
- Docker images: opt in inside the image (my recommendation), or require every Docker user to opt in.
- PR 3: I'd do it right after PR 2, with its own setting.
- Setting names: the two above are a proposal. Whatever we pick has to stay clearly separate from
floci.hostname, which is the advertised name. - Version: release this as 3.0.0, or treat it as a minor version.
Sample 8 of 8
Make fork-safe local time match localtime_r across DST periods
ZLToolKit: a cross-platform C++ library of networking (sockets, event pollers, KCP), logging and utility code.
The developer wants src/Util/local_time.cpp to convert each timestamp with the rules of its own period instead of the current moment’s offset, to make %Z print the right abbreviation, and to keep getGMTOff() current after DST switches and after setenv TZ + local_time_init(). A correct fix matches localtime_r() on every field, exact to the switch second, from a year back to 20 years ahead, with no localtime/gmtime/mktime/tzset call on the conversion path (getLocalTime/getTimeStr/getGMTOff) after start-up; libc timezone calls are allowed only during initialisation (local_time_init(), including repeat calls after a TZ change, or an equivalent one-time build at start-up).
Why it is hard
Four separate root causes sit behind the reported symptoms: the current moment’s offset applied to every timestamp, a fixed one-hour DST, an unset tm_zone, and getGMTOff()'s own stale cache. The hot-path constraint rules out the easy localtime_r() fix, so the agent has to build a transition table exact to the second from libc probes at start-up, copy abbreviations instead of pointing into tzname[], and keep getGMTOff() on the same data. Getting any one of these wrong gives a plausible partial fix that the hidden tests reject.
The developer’s request, verbatim
@src/Util/local_time.cpp only gets local time right inside the current dst period: other-season timestamps are an hour off (30min on lord howe), %Z prints garbage, and getGMTOff() lags dst switches / keeps the old offset after setenv TZ + local_time_init(). should match localtime_r from a year back to 20y out, with no localtime/gmtime/mktime/tzset calls after startup (fork safty is the whole point)
What the hidden grader checks
- getLocalTime()/getTimeStr() match localtime_r() on every field, including tm_isdst, tm_gmtoff and the %Z abbreviation, from a year back to 20 years ahead.
- Conversions are exact at and just before every switch in 33 zones, including half-hour, two-hour, negative and zero-width DST and isdst- or abbreviation-only switches.
- getGMTOff() matches localtime_r() for the current time right after a DST switch and after setenv TZ + local_time_init(), with no separate cache.
- Once initialised, no path from getLocalTime(), getTimeStr() or getGMTOff() calls localtime/gmtime/mktime/tzset, even as the wall clock moves; no refresh threads or libc fallbacks.
- tm_zone points to library-owned storage; results stay unchanged when tzname[] is perturbed or other threads call localtime_r() concurrently.
- The util.h API keeps its meaning, local_time_init() still re-reads TZ, and existing logger, creator, ini and pollerDeferredDelete tests still pass.
- Fail-to-pass tests
- 3
- Regression tests
- 4
- Rubric items
- 8
- Wrong solutions rejected
- 10
- Alternative solutions accepted
- 3
Key moments of the trajectory
Steps 1–3, 6–7 Read local_time.h and getGMTOff() in util.cpp (a 60-second calibration cache with a documented fork caveat), how the logger uses local time, and the test gating list.
Steps 9–11 Built the unmodified library and a small repro: a winter New York timestamp got the summer offset, an hour off, and getGMTOff() reported 28800 (+8 h) in the New York run.
Steps 12–14 A 21-year daily localtime_r() sweep took 1.3–1.6 ms in New York and Lord Howe; zdump showed the shortest period in range is about 7 days (Asia/Gaza), so daily probes plus bisection suffice.
Steps 16–18, 20 Rewrote local_time.cpp so local_time_init() builds a period table (offset, isdst, abbreviation) behind an atomic pointer; getGMTOff() became a lookup at time(NULL), the 60-second cache was deleted and util.h’s docs updated.
Steps 22–23, 26, 28–29 Every field matched localtime_r() across 447 zones, but right/ zones failed (getGMTOff() gave -14427, not -14400); once libc’s tm_gmtoff was stored apart from the leap-second conversion shift, the plain, posix/ and right/ trees all matched.
Steps 32–33, 35–36, 38–41 Wrote tests/test_localTime.cpp, registered it in GATING_TESTS and confirmed it fails on the old code; when an injected localtime_r() call initially went undetected, the agent tightened the check that reloads libc with another TZ.
Steps 42–45 ASan/UBSan and ThreadSanitizer (about 180k reads during 40 re-inits) were clean; after a fork while another thread held glibc’s tz lock, a child using the library finished while one calling localtime_r() hung.
Steps 46, 53–54, 57 Conversions fell from 49 to 26 ns and getGMTOff() rose from 1.4 to 10 ns; the final message flagged the removed local_time_refresh(), untested Windows/32-bit builds and the logger’s remaining mktime() call.
Result
- Agent steps
- 57
- Tool calls
- 61
- Minutes
- 31
- Files changed
- 6
- Lines added / removed
- +562 / −243
The agent rewrote local_time.cpp so that local_time_init() probes localtime_r() daily from a year back to about 20 years ahead, bisects each change to the second and builds a table of periods (offset, isdst, copied abbreviation); getLocalTime() and getTimeStr() now use each timestamp’s own period and set tm_zone, and getGMTOff() looks up the current time instead of reading a 60-second cache. No libc timezone call is left on the POSIX hot path. The grader’s remaining notes are minor: date arithmetic rewritten rather than kept (slightly beyond the request), replaced tables deliberately never freed, and a final message that leaves the root cause implicit.
Quality scores from the grader (1–5)
- Correctness
- 5
- Code quality
- 4
- Scope
- 4
- Communication
- 4
Rubric items (8)
Required (5)
- C1 · For any timestamp from about a year back to twenty years ahead, getLocalTime()/getTimeStr() give exactly what localtime_r() gives under the TZ in effect: date and time fields, tm_wday/tm_yday, tm_isdst, tm_gmtoff and the abbreviation (tm_zone, %Z). This includes timestamps outside the current DST period, the exact second of every switch and the second before it, zones whose DST is not +1h (half-hour Lord Howe, two-hour Troll, negative Irish DST, zero-width), and switches that change only isdst or the abbreviation (e.g. America/Vancouver going to permanent MST in 2026).
- C2 · getGMTOff() returns the UTC offset in effect at the moment it is called, i.e. the same value as getLocalTime(time(nullptr)).tm_gmtoff and localtime_r(now). This holds immediately after a DST switch, for non‑1h DST, and immediately after TZ is changed (setenv + tzset) and local_time_init() is called. There is no separately cached or frozen copy that can lag behind either.
- C3 · After initialisation, no code path reachable from getLocalTime(), getTimeStr() or getGMTOff() calls libc's timezone functions (localtime/localtime_r, gmtime/gmtime_r, mktime/timegm/timelocal, tzset). That includes periodic or clock-triggered recalibration, background refresh threads, and fallbacks for some timestamps. no_locks_localtime() stays lock-free and fork-safe.
- C4 · Between re-initialisations, i.e. as long as local_time_init() is not called again, tm_zone in every struct tm returned by getLocalTime() points at a valid NUL-terminated copy of the right abbreviation in storage the library controls. It is never libc's mutable tzname[], never an uninitialised pointer, and never a per-call or per-thread buffer that a later getLocalTime()/getTimeStr() call overwrites. Returned results don't change when the host program calls localtime_r() on other timestamps, or when other threads do so concurrently. The requirement does NOT extend across local_time_init(): a struct tm obtained before a TZ re-initialisation is not required to keep a valid or unchanged tm_zone afterwards.
- C5 · Existing behaviour and API keep working. util.h still offers long getGMTOff(), struct tm getLocalTime(time_t) and std::string getTimeStr(const char *, time_t = 0) with the same meaning. local_time_init() still exists and re-reads TZ, rebuilding whatever the hot path uses. Logging (LogChannel::printTime, FileChannel file naming and cleanup) still works. No existing test is deleted, skipped or weakened.
Quality (3)
- C6 · The change follows the file's existing conventions. It keeps the Redis-derived lock-free arithmetic of no_locks_localtime() and updates the explanatory header comment. Non-obvious logic gets bilingual (Chinese + English) comments like the surrounding code. The doc comments in util.h/local_time.h that described the old behaviour are updated (60-second calibration, 'timestamps of another season are off by one hour', the fork caveat). The abbreviation is copied by value. The table does not depend on dynamic-initialisation order across translation units (constant-initialised placeholder or equivalent). The Windows branch stays coherent (struct tm has no tm_gmtoff/tm_zone there).
- C7 · No unrelated changes, debug output, leftover files or commented-out code.
- C8 · The final message explains what was wrong: the current-moment offset applied to every timestamp, fixed +1h DST, unset tm_zone, and the separately cached getGMTOff(). It says how this was fixed and states the coverage window and its limits: timestamps outside it, and that a TZ change at run time needs local_time_init() or a restart. It claims nothing that is not true of the final tree (e.g. tests that were not added or run).
The agent’s final message
Local time now matches localtime_r in every season, from a year back to 20 years out. %Z prints the right abbreviation, getGMTOff() is correct immediately, and nothing calls localtime/gmtime/mktime/tzset after local_time_init(). Nothing is committed.
How it works
- Startup:
local_time_init()runs once at startup, or when you call it after changingTZ. It askslocaltime_rfor the offset once per day across that window and pins down each change to the exact second. The result is a read-only table of periods (offset,isdst, abbreviation), published through an atomic pointer. - Conversion:
no_locks_localtime()looks up the period the timestamp itself falls in, so Lord Howe's 30-minute DST, Troll's 2-hour DST and Dublin's winter "DST" all come out right. It now setstm_zone, which fixes the%Zgarbage. It also closes a hidden lock path: with a nulltm_zone, glibc'sstrftimecallstzset()itself. getGMTOff(): it's now a table lookup at the current time. That removes the 60 s lag after a DST switch and the stale value aftersetenv TZ+local_time_init(). The cache and refresh code inutil.cppis gone.- Old tables: they are never freed, because other threads and earlier
tm_zonepointers may still use them.
Verification
- New gating test:
tests/test_localTime.cppis inGATING_TESTSand runs in about 1.6 s. It compares alltmfields in 16 zones, hourly and second by second around every switch. It also checksgetGMTOff()after re-init,%Z %z, and that results don't change when libc is switched to another TZ without re-init. That last check catches any hiddenlocaltime/tzsetcall. - The test catches the bugs: it fails on the old code. I also re-injected each bug into the new code one at a time, and it catches each one.
- All installed zones: all 1341 zones on this machine (plain,
posix/andright/) matchlocaltime_ron every field. - Sanitizers and suite: ASan/UBSan/LSan and ThreadSanitizer are clean, and the full ctest suite passes.
- Fork demo: I forked while another thread held glibc's tz lock. A child calling
getLocalTime,getTimeStr("%Z")andgetGMTOff()finished normally, while a child callinglocaltime_rhung. - Cost on this machine: init takes about 0.5–2 ms, a conversion about 26 ns (was 49 ns), and
getGMTOff()about 10 ns.
Things to know
- Removed API:
local_time_refresh()is gone fromlocal_time.h; its only job was to calllocaltimeperiodically. Any outside caller needs to drop it. - Windows:
getGMTOff()now uses the same table, built withlocaltime_s, instead ofGetTimeZoneInformationwith a 60 s cache. That keeps it consistent withgetLocalTime(). I couldn't compile for Windows or run a 32-bittime_tbuild here, so CI will be the first real check of those. - Limits:
- Periods shorter than a day would be missed. Only contrived TZ strings do that; the shortest real period in range is Gaza's 7 days in 2040.
- Outside the window, the nearest period's offset is used.
- The window is anchored at the clock at startup. If the clock is badly wrong then (for example a device booting at 1970 before NTP), times stay wrong until
local_time_init()is called again.
- Not touched:
getLogFileTime()inlogger.cppstill callsmktime()during log rotation, which is the same fork risk. I can replace it with the same date math if you want.
The agent’s change (excerpt)
diff --git a/src/Util/local_time.cpp b/src/Util/local_time.cpp
--- a/src/Util/local_time.cpp
+++ b/src/Util/local_time.cpp
@@ -31,11 +31,13 @@
* POSSIBILITY OF SUCH DAMAGE.
*/
+#include <algorithm>
#include <atomic>
#include <ctime>
-#ifndef _WIN32
-#include <sys/time.h>
-#endif
+#include <limits>
+#include <set>
+#include <string>
+#include <vector>
#include "local_time.h"
@@ -46,162 +48,243 @@
* in order to log something to the Redis log, it may deadlock: in the copy
* of the address space of the forked process the lock will never be released.
*
- * This function takes the timezone 'tz' as argument, and the 'dst' flag is
- * used to check if daylight saving time is currently in effect. The caller
- * of this function should obtain such information calling tzset() ASAP in the
- * main() function to obtain the timezone offset from the 'timezone' global
- * variable. To obtain the daylight information, if it is currently active or not,
- * one trick is to call localtime() in main() ASAP as well, and get the
- * information from the tm_isdst field of the tm structure. However the daylight
- * time may switch in the future for long running processes, so this information
- * should be refreshed at safe times.
- *
- * Note that this function does not work for dates < 1/1/1970, it is solely
- * designed to work with what time(NULL) may return, and to support Redis
- * logging of the dates, it's not really a complete implementation. */
+ * 时区规则只在local_time_init()中读取:用系统的localtime按天探测、再二分到秒,求出调用
+ * 时刻之前一年至之后二十年内,偏移量、夏令时标志与时区缩写都不变的各个时段,存为一张只读表。
+ * 此后的换算只查这张表,不再调用localtime/gmtime/mktime/tzset;每个时间戳都按它自己
+ * 所处的时段换算,所以夏令时切换无需刷新,夏令时偏移不是1小时的时区(30分钟、2小时)
+ * 也一样正确。表外的时间戳沿用离它最近的时段,不保证与localtime一致。
+ * The time zone rules are only read by local_time_init(): the system localtime is
+ * probed day by day, then bisected down to the second, which yields every period from
+ * a year before the call to twenty years after it over which the offset, the daylight
+ * saving flag and the abbreviation stay the same; those are kept in a read only table.
+ * Conversions afterwards only look the table up and never call localtime/gmtime/mktime/
+ * tzset again. Every timestamp is converted with the period it falls in, hence a
+ * daylight saving switch needs no refresh, and zones whose daylight saving shift is not
+ * one hour (30 minutes, 2 hours) come out right as well. A timestamp outside of the
+ * table uses the closest period and is not guaranteed to agree with localtime. */
namespace toolkit {
-/* 两者都由local_time_refresh()更新、被任意线程读取,故使用原子变量
- * Both are updated by local_time_refresh() and read by any thread, hence atomic. */
-static std::atomic<int> _daylight_active { 0 };
-static std::atomic<long> _current_timezone { 0 };
+static const time_t secs_day = 3600 * 24;
+/* 建表时的探测步长,短于它的时段可能被漏掉;时区数据库(2026c)在表的覆盖范围内
+ * 最短的时段约7天(Asia/Gaza,2040年)
+ * Probing step used to build the table, a shorter period may be missed; within the span
+ * of the table, the shortest one in the time zone database (2026c) lasts about 7 days
+ * (Asia/Gaza, 2040). */
+static const time_t probe_step = secs_day;
+/* 表覆盖local_time_init()之前一年至之后二十年
+ * The table covers a year before local_time_init() to twenty years after it. */
+static const long long table_past = 366LL * secs_day;
+static const long long table_future = 20 * 366LL * secs_day;
-int get_daylight_active() {
- return _daylight_active.load(std::memory_order_relaxed);
-}
+namespace {
+/* 偏移量、夏令时标志与时区缩写都不变的一个时段
+ * A period over which the offset, the daylight saving flag and the abbreviation
+ * stay the same. */
+struct local_time_period {
+ time_t start; /* 时段的第一秒 First second of the period. */
+ long gmtoff; /* 即tm_gmtoff,单位秒 That is tm_gmtoff, in seconds. */
+ /* 换算时叠加到UTC上的秒数,仅在计入闰秒的时区(right/...)中与gmtoff不同
+ * Seconds added to UTC when converting, only differs from gmtoff in leap second
+ * aware zones (right/...). */
+ long shift;
+ int isdst;
+ const char *zone; /* 时区缩写 Abbreviation. */
+};
-long get_local_gmtoff() {
- /* 夏令时期间本地时间比标准时间快1小时,偏移量也必须体现这1小时
- * Daylight saving time puts the local time one hour ahead of standard time,
- * so it must be reflected by the offset as well. */
- return -_current_timezone.load(std::memory_order_relaxed) + 3600 * get_daylight_active();
+struct local_time_table {
+ std::vector<local_time_period> periods; /* 按start升序,不为空 Sorted by start, never empty. */
+ std::set<std::string> zones; /* 各时段zone的存储 Storage of the zone of the periods. */
+ /* 被替换的表永不释放:其他线程可能还在查它,交出去的tm_zone也还指向它;
+ * 串成链表只是为了让它们保持可达,免得被内存检测工具当成泄漏
+ * A replaced table is never freed: other threads may still be looking it up, and
+ * the tm_zone handed out still points into it; chaining them merely keeps them
+ * reachable, so that leak checkers do not take them for leaks. */
+ const local_time_table *prev = nullptr;
+};
+} // namespace
+
+/* local_time_init()之前按UTC换算
+ * UTC is used until local_time_init() runs. */
+static const local_time_period _utc_period = { 0, 0, 0, 0, "UTC" };
+static std::atomic<const local_time_table *> _table { nullptr };
+
+/* 公历日期距1970-01-01的天数,1970年之前同样适用,算法见
+ * http://howardhinnant.github.io/date_algorithms.html
+ * Days from 1970-01-01 to a date of the proleptic Gregorian calendar, dates before
+ * 1970 included, see the link above for the algorithm. */
+static long long days_from_civil(long long year, int month, int mday) {
+ year -= month <= 2;
+ long long era = (year >= 0 ? year : year - 399) / 400;
+ long long yoe = year - era * 400; /* [0, 399] */
+ long long doy = (153 * (month > 2 ? month - 3 : month + 9) + 2) / 5 + mday - 1; /* [0, 365] */
+ long long doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; /* [0, 146096] */
+ return era * 146097 + doe - 719468;
}
-static int is_leap_year(time_t year) {
- if (year % 4)
- return 0; /* A year not divisible by 4 is not leap. */
- else if (year % 100)
- return 1; /* If div by 4 and not 100 is surely leap. */
- else if (year % 400)
- return 0; /* If div by 100 *and* not by 400 is not leap. */
- else
- return 1; /* If div by 100 and 400 is leap. */
… 783 more lines
Customization
Including, but not limited to, the following; anything else is open for discussion.
- Task types and mix: features, bug fixes, production investigations, refactors, design-doc implementation, intent understanding, code review, codebase Q&A, planning, long-running jobs.
- Languages and codebases: languages and stacks, codebase size (monorepos included), size of the change.
- Difficulty: the reference model and the pass-rate band to hit.
- Requests: length and tone, pasted logs, errors and file references; multi-turn sessions.
- Format: task environments, trajectories or both; the agent harness and model for the trajectories.
- Volume and schedule: how many and by when.