Web tools: URL reading + web search

Agents can read a web page the user links and search the web for current information — provider-agnostically, so the capability works on any configured model, not just those with a vendor-native web tool. Two tools: web_fetch (read a URL) and web_search (find sources). Web search is a swappable seam (WebSearch) with ≥2 implementations (Brave adapter + Null), so a missing key degrades gracefully instead of breaking. Both do network I/O in the bot process — not through the Executor — so the no-repo research agent uses them with no workspace.

• Code: src/tools/web.ts (the WebSearch seam, BraveWebSearch + NullWebSearch, makeWebCapability, SSRF guards assertUrlAllowed/ipInBlockedRange (sync literal guard, full IPv6 expansion) and makeSsrfLookup (the undici connect-time IP-pinning guard), webFetchTool, webSearchTool); src/tools/workspace.ts (ToolContext.web, TOOLSETS wiring); src/agents/registry.ts (the research agent + RESEARCH_SYSTEM, general prompt points web asks at it); src/core/dispatch/run.ts (webCapability injects makeWebCapability(process.env) into the tool context).
• Tests: src/tools/web.test.ts.

Behavior

1. URL reading is provider-agnostic and in-process (Option B): web_fetch fetches an http(s) URL and returns its text (HTML is stripped to readable text), running in the bot process via an injected web capability — no Executor, no workspace. A no-repo agent can use it.

2. web_fetch is SSRF-hardened, connect-time: only http/https schemes; literal internal addresses are refused synchronously — including every IPv6 form of an internal address (loopback, 10/8, 172.16/12, 192.168/16, link-local 169.254/16 incl. cloud metadata, ULA, and IPv4-mapped/-compat/NAT64 IPv6 like ::ffff:169.254.169.254, expanded fully rather than string-matched) — and internal hostnames (localhost, *.local, *.internal). For hostnames, the production fetch uses an undici dispatcher whose lookup resolves and refuses internal IPs at connect time and connects to exactly that address — so the validated IP is the connected IP (no DNS-rebinding TOCTOU). Redirect targets are re-validated on every hop (sync guard) and re-connect through the same dispatcher. Responses are size-capped (~1 MB) and time-bounded (~12 s); blocks/timeouts/HTTP errors return a clear message, never a crash.

   • Binary links reach the model as vision/document input (M1b): when the response content-type (parameters stripped, case-folded) is image/jpeg|png|gif|webp or application/pdf, web_fetch returns a parts list — a text header plus an image / document content part (base64) — instead of text, so the model sees the picture or reads the PDF. This is the second producer of image/document parts after Slack attachments (slack-channel.md), and the reason tool_result.content is string | ToolResultPart[] (src/providers/types.ts). Per-file caps match the attachment path (5 MiB image, 10 MiB PDF); an over-cap binary is refused with a message (never truncated — a cut image is garbage), and other image/* types (e.g. SVG) are named as unsupported rather than sent to a model that would reject them. Text URLs are unchanged. Provider mapping: Anthropic carries text+image inside the tool_result block and hoists the PDF to a sibling document block after all tool results (SDK 0.39 types don't admit documents inside tool results; the API requires tool results to lead the turn); OpenAI-compatible endpoints get a string role:"tool" message plus the image in the following user message (PDF → the existing text placeholder). Run-visibility summaries render binary parts as [image image/png, N bytes] — the base64 never enters the stream.

3. web_search is a seam with ≥2 implementations: WebSearch has a real BraveWebSearch adapter (keyed by BRAVE_SEARCH_API_KEY) and a NullWebSearch. makeWebCapability selects Brave when the key is present, Null otherwise. With no key, web_search returns a clear "not configured" message (and notes URL reading still works) — it never breaks the run.

4. Enablement by toolset: web_fetch (URL reading) is in the full (coding), readonly (review), and assistant (general) toolsets — broadly available; web_search is gated to the web toolset, held by the research-capable agent. general reads a linked URL but does not search (agent-general.md item 1).

5. A dedicated research agent (toolset: "web", resources.repo: "none") answers research/URL questions with search + fetch and no workspace; since github-tools.md item 6 its toolset also holds the GitHub READ tools, so a question about one of our repos is answered from the repo over the App credential (a github.com URL of ours is read with github_file/github_tree, not web_fetch), and its prompt forbids concluding a repo is inaccessible from a public-web 404. general directs web-research asks to agent:research.

6. The capability is injected, not global: the dispatcher builds web from process.env per run; when absent (e.g., a run that didn't inject it), the tools report themselves unavailable rather than throwing.

7. A page reaches the model in windows, never whole. web_fetch READS up to 1 MB (enough to strip a script-heavy HTML document to its text) but HANDS the model at most MAX_FETCH_TEXT_CHARS (40,000 characters, ~10k tokens) per call. A longer page's header says showing characters <from>–<to> of <total>; pass offset=<to> to continue, and the tool takes offset (a non-negative whole number; anything else is refused) to read the next window; the last window has no continue hint, an offset past the end says so with the page's length, and a page under the cap carries no paging note at all. A 1 MB read is still marked [truncated at 1 MB]. Before this, one ~1 MB page handed over whole was 307k tokens and killed a research run with prompt is too long before its second tool call. Behind every tool sits the runner's ceiling (run-loop.md item 13).

Validation criteria

Criterion	Evidence
A page is handed to the model at most MAX_FETCH_TEXT_CHARS at a time with a header naming the window and the next offset; offset pages through it (windows tile exactly), the last window has no continue hint, past-the-end and bad offsets are refused by name, a short page carries no paging note; the 1 MB read cap is still named	[unit] src/tools/web.test.ts::web_fetch tool::hands the model at most MAX_FETCH_TEXT_CHARS of a page and says how to read the rest…, ::offset pages through a long page; past the end says so; a bad offset is refused
web_fetch reads a public URL and returns its text; HTML is stripped	[unit] src/tools/web.test.ts::web_fetch tool::fetches a public URL and returns its text, ::strips HTML to readable text
SSRF: non-http(s) schemes and literal internal IPs/hosts (incl. IPv6-mapped) refused without fetching	[unit] ::web_fetch tool::refuses SSRF targets (incl. IPv6-mapped literals) without ever fetching; ::assertUrlAllowed::rejects non-http(s) schemes, ::rejects internal hosts and literal internal IPs (incl. IPv6-mapped); ::ipInBlockedRange::flags loopback/private/link-local/metadata/ULA (v4 + all IPv6 forms), ::allows public addresses
SSRF: DNS-rebinding closed by connect-time IP pinning (validated IP = connected IP)	[unit] ::makeSsrfLookup (connect-time SSRF guard)::*; ::web_fetch tool::surfaces a connect-time SSRF refusal (dispatcher guard) as a refusal
SSRF: redirect targets re-validated; internal redirect refused, public redirect followed	[unit] ::web_fetch tool::re-validates redirect targets and refuses an internal redirect, ::follows a redirect to an allowed URL
Binary link → model-visible block: image/* URL → image part; application/pdf → document part named after the path; content-type normalized; text URL unchanged	[unit] src/tools/web.test.ts::web_fetch tool: binary links become model-visible blocks (M1b)::returns an image/* URL as an image content part the model can see, ::normalizes the content-type…, ::returns an application/pdf URL as a document content part…, ::a text URL still returns plain text…
Binary caps: over-cap image/PDF refused (not truncated); unsupported image types named, not sent	[unit] ::refuses an oversize image/PDF with a message instead of a truncated block, ::names an image type the model cannot consume instead of sending it
SSRF guards (literal, connect-time, redirect) sit in front of binary fetches	[unit] ::keeps SSRF guards in front of binary fetches (literal + connect-time + redirect)
Parts-array tool results reach each provider correctly (Anthropic: image in-block, PDF hoisted after all tool_results; OpenAI-compat: string tool message + hoisted image, PDF placeholder) and the runner passes them through with a text-only summary	[unit] src/providers/anthropic.test.ts::maps an array tool_result…, ::hoisted document blocks land after ALL tool_result blocks…, ::string tool_result content is unchanged…; src/providers/openaiCompat.test.ts::array tool_result…; src/runner.test.ts::tool results carrying non-text parts (M1b); src/tools/web.test.ts::toolResultText renders a parts array…
Live: agent:research describes an image link / summarizes a PDF link end-to-end	[agent] In Slack: @switchboard agent:research what does https://<public>.png show? → the answer describes the picture; … summarize https://<public>.pdf → the answer cites the PDF's content.
Timeout/HTTP-error handled gracefully	[unit] ::web_fetch tool::reports a timeout gracefully, ::reports non-2xx status
web_search formats seam results; missing key degrades gracefully	[unit] ::web_search tool::formats results from the search seam, ::degrades gracefully when search is not configured, ::handles empty query and missing capability, ::reports a generic search failure
Brave adapter calls the API correctly and parses results; ≥2 impls selectable	[unit] ::BraveWebSearch adapter::calls the Brave API with the key header and parses results, ::throws on non-200; ::makeWebCapability::selects Brave when a key is present, Null otherwise, ::NullWebSearch throws WebSearchUnavailableError
Enablement: web_fetch broad (incl. assistant), web_search gated to research; general holds assistant without web_search or bash	[unit] src/tools/web.test.ts::toolset + agent wiring::*
Live: agent:research answers a question / summarizes a URL end-to-end	[agent] In Slack: @switchboard agent:research <a question needing current sources> → the answer cites sources found via web_search; … summarize <public URL> → the answer reflects that page's content. The web_search half requires BRAVE_SEARCH_API_KEY to be provisioned on the deployment; without it the run must still answer, saying search is not configured.