OpenSwitchboard

Appearance

Self-improvement proposals ​

Human overview with architecture diagrams: docs/explanation/how-switchboard-improves-itself.md. "Proposals" is literal: the shipped system files labeled issues for a person to triage; it does not open PRs — the scope was narrowed to proposals on purpose.

OpenSwitchboard turns the run-friction analysis (run-friction.md) into action: it keeps every finished run's diagnosis in a ledger, clusters the friction that recurs across recent runs, and files the top recurring patterns as labeled GitHub issues — each with the evidence (the friction data, the affected runs) and a concrete suggested fix — deduped against the proposals already open. This is the second step of the loop observe → diagnose → propose. It is human-gated by construction: the only side effect is opening a labeled issue for a person to triage. It never opens PRs and never merges anything.

Built on the analyzer, not beside it (locked design). Every pattern is made from the FrictionFindings analyzeRunFriction already emits; the proposer adds no per-run detection. The one signal it adds — long_run, a run whose wall time is ≥2× the median of recent runs — cannot exist per run by definition, and stands in for cost spikes until per-run token accounting exists (AGENTS.md known gap). The clustering, ranking, rendering, and dedupe step is a pure function of the ledger records (no clock, no I/O), so it is tested against recorded runs.

Where recent runs live. The live registry evicts a finished run 60s after it ends, so "friction across recent runs" needs a ledger — and the ledger is run history (run-history.md): every finished run's record carries its diagnosis, and RunStoreFrictionLedger serves recent() from RunStore.list (never get — the listing carries the diagnosis, so no event stream is loaded), projecting each row field-explicitly to { runId, label, agent, finishedAt, diagnosis } (message text and platform ids never reach a FrictionRunRecord or an issue body). Nothing is written twice: the dispatcher's history write is the one write path, and runHistory.retentionDays/maxRuns bound the friction population. selectFrictionLedger(store) yields no ledger without a run store, and friction report/propose then answer unavailable naming runHistory. The FrictionLedger seam has one production implementation and one test double (InMemoryFrictionLedger, seeded by record(); AGENTS.md invariant 2); the section's own fields are repo/label/minRuns/top and any other (worker, ledgerPath, ledgerMax) is an unknown field refused at load. The record carries the run id (not the view token), so a ledger row or an issue body can never grant live-view access.

Behavior ​

  1. Ledger. After every agent run (fast paths excluded — they never run an agent) the dispatcher computes analyzeRunFriction(events) over the run's own event copy (a ring of the newest 5000 events, so a pathological run's terminal notes are never the part dropped) and the diagnosis rides the run record, with the registry run id, the runs-index label, the agent name, and the finish time (run-history.md item 2). recent({ limit?, sinceMs? }) returns runs oldest-first as copies, the newest limit ?? 500 of what run history retains, paged from RunStore.list (never get); a run store that cannot be read rejects, and the friction command replies ⚠️ naming the cause. The test double (InMemoryFrictionLedger) pins the same ordering and trimming rules over bare records and upserts by run id when seeded. Whether friction report sees any run at all is runHistory's: without it there is no ledger and both commands answer unavailable naming it; a selfImprovement field other than repo/label/minRuns/top is refused at load as unknown.
  2. Signatures. Each finding maps to a stable key <kind>:<signature>. Tool findings are keyed shell-aware, because real agents chain the same work differently every run (cd /tmp/ws/repo && npm ci --silent 2>&1 | tail -2 && … vs cd ~/switchboard && npm ci --silent >/dev/null 2>&1; (npm test …) — the first real capture proved an exact-command key never recurs): the analyzer's label prefix and the → result tail are dropped, the command is split into shell segments (&&, ||, ;, |), and each segment reduces to its program [subcommand [script]] head (npm ci, git checkout, npm run typecheck, npx vitest) with navigation/echo/pipe-filter programs (cd, echo, tail, grep, …), redirections, env assignments, wrappers (sudo), quoted strings, and volatile tokens (URLs → <url>, hashes → <sha>, numbers → <n>) removed; the ordered, de-duplicated heads (≤4) form the signature (slow_tool:git checkout, npm test). setup_install keys on the install segment alone plus the flags that change what is installed (pnpm install --frozen-lockfile; --silent/--no-audit-style flags dropped), so every npm ci run is one pattern. An unknown-tool failure (tool misuse) keys on the tool name. Note findings key on their kind, never their free text: budget_hit:time|turns, wrap_up:wrap_up, infra_failure:sandbox_dead, infra_failure:mid-tool <cmd>, infra_failure:<cmd>.
  3. Clustering. A pattern is a key seen in ≥ minRuns distinct runs (default 2) — a finding repeated within one run counts once toward recurrence (recurring means a process problem, not one bad run) but every occurrence is counted and its time summed. Ranking: distinct runs ↓, peak severity ↓, attributed time ↓, key — a total order, so the output is deterministic. Each pattern keeps up to 5 examples, most recent first, each anchored to its run. long_run: among runs with runMs, those ≥ max(2× median, 10 min) are outliers, grouped per agent (long_run:<agent>); a fleet where every run is long has no outliers.
  4. Proposals. The top top patterns (default 3) become issue proposals: title [friction] <kind> recurs in N of M runs: <signature> (≤120 chars), body = the dedupe marker <!-- switchboard-friction-pattern: <key> -->, a "proposal for a human to triage" preamble, Pattern (kind, signature, recurrence with share, occurrences, attributed time, peak severity), Evidence (a table of the example runs: run id + label, finish time, the redacted finding, its time; plus the remaining run ids), Suggested fix (kind-specific and sharpened by the evidence — e.g. a failing --frozen-lockfile install → regenerate the lockfile / document the install command / onboard as a resident; command not found → add to the sandbox image; unknown tool → align the agent's toolset; budget_hit:turns → maxTurns or batch calls; sandbox_dead → container sizing), and Provenance. One label, selfImprovement.label (default self-improvement), created in the repo when missing.
  5. Dedupe. Open issues carrying the label are listed (PRs excluded, paginated) plus the 30 newest open issues unfiltered — GitHub's label-filtered list can lag creation by a few seconds, which would let an immediate re-run refile a pattern — and matched by marker, never title (titles change with run counts) or label. A matched proposal is reported as already open and not refiled; a closed proposal's pattern can be proposed again. When no pattern recurs, GitHub is never consulted.
  6. Filing and failure. Fresh proposals are filed one by one through the IssueTracker (REST, App installation token — App issues:write; never gh; bodies clipped to GitHub's limit). A create failure is reported per proposal and later proposals still file. dryRun computes and reports everything (including what would be filed) and files nothing; it still lists open issues for the dedupe. With no repo at all (the CLI without --repo) the pass is a pure dry run that never consults GitHub — no credential needed.
  7. Triggers. (a) The command registry (command-registry.md item 17): friction.report {sinceMs?, limit?, minRuns?} (friction:read, the Slack baseline) shows the ranked recurring patterns; friction.propose {dryRun?, top?, minRuns?, repo?} (friction:write — never a baseline: admins and its grantees, the same people who manage repos) files. Both are one registration reachable on every surface with the same JSON (SelfImprovementReport) and the same text (formatSelfImprovementReport): chat — inline, channel-agnostic, never a model turn — as friction report [--since-ms <n>] [--limit <n>] [--min-runs <n>] / friction propose [--dry-run] [--top <n>] [--min-runs <n>] [--repo owner/name] (the registry's derived grammar; an unknown flag or a non-positive number is a named usage reply that never echoes the value; the refusal is the shared 🚫 wording and an unset selfImprovement.repo is a hint); HTTP GET /api/friction.report?limit=5 / POST /api/friction.propose; MCP friction_report / friction_propose; CLI npx tsx src/cli.ts friction report --json. Machine callers hold no friction scope implicitly. WHAT either command analyzes is the authorization policy (authorization.md items 6–7): the runs its actor may read — predicateFor(actor, "runs:read", "run") handed to the ledger, which the run-store ledger pushes into store.list as visibleTo; an all-channels actor (an admin, the self-improvement schedule) analyzes the fleet, a token its granted channels (plus any run stamped public), an actor with no channel grants the public channels' runs and its own (a Slack channel's visibility comes from the Slack ChannelDirectory at dispatch — authorization.md item 7), and a ledger of bare records (the in-memory test double — no channel to check) contributes nothing under any predicate narrower than all. No channel is compared by hand. (b) CLI, no bot process: npx tsx src/cli.ts friction propose --dry-run [--top N] [--min-runs N] [--repo owner/name] — the same registration over the same ledger/run store the bot reads (a fresh process reads persisted history), and npx tsx src/cli.ts friction analyze <run.jsonl | capture.sse> [--slow-ms n] [--in-progress] for the read-only diagnosis of ONE saved stream (run-friction.md item 6). friction propose has no file-loading mode: recent runs live in the run store, not in files on a laptop. (c) Scheduled — as an ordinary run: the self-improvement entry of the schedule registry (src/core/schedules.ts, Mondays 14:00 UTC, 0 14 * * 1) is fired by the Worker shim (deploy/cloudflare/worker.ts) as a POST /ingress of the text friction propose with the cron identity's bearer — the {"subject":"cron","channel":"cron"} entry of SWITCHBOARD_INGRESS_TOKENS; no bespoke route and no dedicated secret. The dispatcher sees http:cron and answers it exactly like chat, so the firing is a run: a registry record (input → answer) on /runs, a receipt {run:{id,status}} in the ingress response, and a firing record (fired-at, run id, outcome) on the state Worker's ScheduleDO that the /runs "Scheduled" panel shows (live-view.md item 14). The cron identity's grants entry must hold friction:write — friction propose is behind that never-a-baseline grant, so without the grant the firing runs and finishes failed with the 🚫 reply as its answer (visible on the panel), and nothing is filed — and channels: all (authorization.md items 6–7): what the pass analyzes is the runs its actor may read, and an ingress token is granted no channel by default, so config.production.yaml's native grants block gives http:cron channels: all (the registry declares the same for the schedule:self-improvement actor the shim will fire as). Without the channel grant the pass runs, completes, and analyzes 0 runs — a misconfiguration, not the design. Fail-closed on identity: no cron entry in the token map → the shim sends nothing and records the firing as misconfigured; ingress 401/503/5xx → ingress-error. The per-minute keep-alive cron is the registry's internal healthz schedule: it never becomes a run and is not shown on the panel. The issues are the notification — no channel post.

Validation criteria ​

CriterionEvidence
Command normalization: prefix/tail stripped, volatile tokens blanked, length capped[unit] src/core/frictionProposals.test.ts::normalizeCommand::*
Shell-aware signatures: segment heads, noise/redirections/env/wrappers dropped, capped; the two real chainings of npm ci share one install signature; install flags kept minus quiet flags[unit] ::commandSignature::*, ::installSignature::*, ::patternSignature::clusters the SAME install chained differently across real runs…
Signatures: tool findings by category + command (label prefixes dropped), unknown tool by tool name, notes by kind, slow_model_turn by the single key model_turn (a slow think is the agent/model tier's property, not the command's)[unit] ::patternSignature::*
No patterns for no/clean runs; a one-off (one run) never becomes a pattern; minRuns honored[unit] ::clusterFriction::returns no patterns…, ::clusters the same finding across DISTINCT runs and ignores one-offs…, ::honors minRuns (red-verified: disabling the minRuns filter fails these)
Within-run repeats count once toward recurrence but all occurrences are counted[unit] ::counts a finding repeated within ONE run as one run…
Ranking total order (runs, severity, time, key)[unit] ::ranks by distinct runs, then peak severity, then attributed time
Examples most-recent-first, capped, run-anchored[unit] ::keeps the most recent examples first, capped, each anchored to its run
long_run outliers per agent; none when all long or only one[unit] ::detects cross-run duration outliers…, ::does not flag long runs when every run is long…
Deterministic; input not mutated[unit] ::is deterministic and does not mutate its input
Proposals: ≤ top, rank order, marker embedded; title/labels/evidence/fix/preamble rendered; a fix template for every kind[unit] ::proposeImprovements::*
Dedupe by marker, not title; closed → re-proposable; marker parsing tolerant[unit] src/core/frictionProposals.test.ts::dedupeProposals::*, src/core/selfImprovement.test.ts::dedupes only against OPEN issues… (red-verified: disabling the marker match fails these)
The seam's rules over bare records (the test double): empty, oldest-first, limit/since, bounded, upsert by run id when seeded, copies; a narrower predicate than all yields nothing[unit] src/core/frictionLedger.test.ts::InMemoryFrictionLedger — the seam's rules over bare records::*
Run-history-served ledger: recent() ordering identical to the in-memory ledger over the same fixture (differential); projects only { runId, label, agent, finishedAt, diagnosis } — no message text, no platform ids; the actor's predicate reaches store.list in wire form, none reads nothing; a run store that cannot be read rejects with its error; list bounded to 500 and get never called; selectFrictionLedger: run store → the ledger, none → no ledger[unit] src/core/frictionLedger.test.ts::RunStoreFrictionLedger::*
A run's diagnosis reaches the ledger through run history — the record's diagnosis, keyed by the registry run id, equal to analyzeRunFriction over the registry snapshot[unit] src/core/dispatcher.test.ts::self-improvement wiring …::every finished run's friction diagnosis reaches the ledger through run history…, src/core/dispatcher.test.ts::friction diagnosis reads the registry backlog …::the run record's diagnosis…
selfImprovement is repo/label/minRuns/top and nothing else: an unknown field (a ledger key such as worker or ledgerPath included) is refused at load naming it; the section loads with its four fields[unit] src/config.test.ts::selfImprovement::*
Given a fixture of runs with a recurring pattern, ONE deduped proposal is filed with the evidence; a second pass files nothing; dry run files nothing; no repo → pure dry run, GitHub never consulted; nothing recurring → GitHub untouched; a create failure is reported, not thrown; minRuns respected[unit] src/core/selfImprovement.test.ts::runSelfImprovement::*
Report text: runs analyzed, ranked patterns, filed / already open / failed / dry run[unit] src/core/selfImprovement.test.ts::formatSelfImprovementReport::*
GitHub tracker: label-filtered paginated list + newest unfiltered page, deduped, drops PRs; label ensured once (422 race tolerated); bearer + user-agent; body clipped; HTTP errors and missing credential thrown with detail[unit] src/execution/githubIssues.test.ts::GithubIssueTracker.*
In-memory tracker: numbering, open+label filtering, close, call log[unit] src/execution/githubIssues.test.ts::InMemoryIssueTracker
Registry commands friction.report / friction.propose: report open in chat, byte-identical pre-migration reply for admin and non-admin, GitHub-free; propose files for a repo manager, 403 for a plain user, dedupes on a second pass; dryRun real boolean; repo override / unset-repo unavailable; no ledger unavailable (the defence in depth: with run history off both commands are hidden on every surface, command-registry.md item 28); the actor's run-read predicate decides what is analyzed (a token granted one channel sees it, all-channels the fleet, no channel grants nothing; the schedule:self-improvement actor the fleet — authorization.md deliberate change (a)); machine scopes (dispatch-only 403, runs:write 403 on propose, friction:write runs)[unit] src/core/commands/friction.test.ts::*
Chat flags are the derived grammar: friction propose --dry-run --top 3 --min-runs 2 --repo o/n binds to {dryRun, top, minRuns, repo}; an unknown flag or a bad value is a usage reply that never reaches the registry; friction report --min-runs 2 and --limit 5 each invoke friction.report once[unit] src/core/commandChat.test.ts::parseChatCommand::*, src/core/commandSurface.test.ts::parseInvocation — the one grammar::*, src/core/dispatcher.test.ts::registry chat commands in the fast-path chain …::\friction report --min-runs 2` and `friction report --limit 5`…`
Dispatcher records every run to the ledger (id/agent/label/diagnosis); a ledger failure is a warning, never surfaced; friction commands are inline (no model, no executor), reach the registry exactly once, and are gated with the shared restricted wording[unit] src/core/dispatcher.test.ts::self-improvement wiring …::*, ::registry chat commands in the fast-path chain …::*
friction.report over HTTP, MCP, and CLI hands back the exact invoke JSON (same recurring pattern, no token)[unit] src/channels/commandContract.test.ts::adapter contract for migrated commands — $name::friction.report…
CLI: npx tsx src/cli.ts friction propose --dry-run --top 3 is the registry command over the same ledger (no file-loading mode remains — friction analyze diagnoses one saved stream, item 7b)[unit] src/cli.test.ts::buildCoreCommands…::phase 4b…, src/channels/commandContract.test.ts::adapter contract for migrated commands — cli::friction.report…
Over real recent OpenSwitchboard runs, the proposer surfaces recurring patterns and files actionable, deduped self-improvement issues[agent] Capture the /runs/:id/friction JSON of ≥3 finished production runs (live view, within the TTL) — with runHistory configured they are the run store's rows — then npx tsx src/cli.ts friction propose --dry-run --repo <owner/name> (expect a ranked pattern list), then without --dry-run → the top proposals open as labeled issues, and a second run reports them as already open.
friction report / friction propose from Slack on the deployed bot read the durable ledger[agent] In a channel the bot is in: @switchboard friction report → the report (ranked patterns, or "no recurring friction pattern"); @switchboard friction propose --dry-run → the same plus a "would file" list when patterns exist. Both must parse when the Slack plugin footer arrives on the same line as the command (see slack-channel.md).
The ledger survives a bot redeploy: runs finished before a container redeploy are still listed by friction report after it (run history is durable on the state Worker)[agent] With prod on runHistory.worker: note friction report's run count, redeploy the bot Worker (deploy/cloudflare), then friction report again → the count is unchanged or higher, never reset. Also wrangler tail switchboard-memory shows [runs/put] after each run, and a friction propose --dry-run produces a [runs/list] line there (the ledger pages the run store).
friction report / friction propose are runs: registry record with input + answer, finished, labeled friction · #channel · user · "…"; receipt completed when the step ran, failed when refused / misconfigured / thrown — in every case the channel reply is the record's answer (a thrown command's ⚠️ <error> reply included, so the run page explains the failed status); two concurrent commands → two distinct runs; http:cron granted friction:write may propose[unit] src/core/dispatcher.test.ts::inline command runs + run receipts …::*
The weekly schedule is registered once: registry entry self-improvement = 0 14 * * 1, friction propose, identity cron; the shim's plan is a POST /ingress of that text with the cron token, fail-closed without one; the ingress answer maps to the firing outcome (run status / no-run / ingress-error)[unit] src/core/schedules.test.ts::schedule registry::*, ::planScheduledFiring…::*, ::interpretIngressResponse…::*
Weekly cron fires the pass without a human, as a run, over the fleet's runs[agent] With the bot deployed with a cron entry in SWITCHBOARD_INGRESS_TOKENS, and a grants entry for http:cron holding friction:write with channels: all: on a firing (temporarily reschedule the registry entry + wrangler.jsonc to fire within the hour, deploy, revert after), wrangler tail switchboard shows [schedule] self-improvement → completed run <id> — 🔍 N runs analyzed … with N > 0; /runs lists the run friction · #cron · cron · "friction propose" while live, and its "Scheduled" panel row shows the fire time, completed, and the run link; the run page's Answer is the report; a [runs/list] line appears in wrangler tail switchboard-memory alongside [schedules/record] self-improvement completed run <id>. Negative path: remove the cron entry (or its grant) → the panel shows misconfigured — nothing ran (or failed with the 🚫 reply) and no issue is filed.