serf tunes its way to 82% on terminal-bench, pepper-x brings voice dictation to GNOME
A week defined by measurement. serf ran experiment after experiment against the terminal-bench 2.0 benchmark, tuning prompts and coordination until its pass rate climbed from the mid-50s to 82%. Simon Willison built a companion tool to analyze the public leaderboard, and a wholly different project — pepper-x, a GNOME voice-dictation app — landed 82 commits of its own.
The week in numbers
- 14 repos active (5 featured, 9 also shipped)
- 288 commits
- 10 merged PRs
- +61,626 / −11,067 lines
- 13 commit identities
- 2 tagged releases
Featured
prime-radiant-inc/serf
An agent framework in Go — this week, tuning its agents against a public benchmark.
126 commits, almost entirely a documented tuning campaign against terminal-bench 2.0. It started with a gpt-5.4-mini eval passing reliably around 54%, followed by a root-cause pass over every failing task and a series of controlled experiments: “write-early” (produce deliverables before verification), deeper coordinator verification (check file contents, not just existence), preferring standard defaults over custom configuration, and stopping the coordinator from overriding a correct implementer answer. Each round shipped the fixes that helped and rejected the ones that didn’t — ending at a final combined 72/87 reliable (82%) on terminal-bench 2.0. It’s a rare public look at the actual loop of making an agent measurably better.
126 commits +26,727 −7,055obra/pepper-x
A voice-dictation desktop app for GNOME — Rust and GTK, local transcription, text insertion into any app.
A new project, 82 commits from design doc to a working dictation loop. Built as a Rust workspace with a GTK app shell, a session state machine, and a GNOME 48 extension for modifier-only recording triggers, it transcribes locally via sherpa-onnx and then faces the hard part — getting text into whatever app has focus. That drove a layered insertion strategy with several fallbacks: accessibility APIs, terminal string injection, clipboard, and a uinput helper.
82 commits +27,326 −2,151prime-radiant-inc/terminal-bench-analysis
A Datasette-powered analysis of the terminal-bench 2.0 leaderboard.
A new repo from Simon Willison (12 commits): a GitHub Actions workflow mirrors result files from the terminal-bench-2 leaderboard, builds them into a SQLite database, and deploys it via Datasette Lite on GitHub Pages — with COG-powered SQL queries embedded right in the README and “Run query” links after each block. The data-fetch commits are automated. (Line counts are suppressed here: the bulk of the diff is mirrored leaderboard data, not authored code.)
19 commits 1 PRAlso shipped
prime-radiant-inc/engineering-notebook
A local tool that turns agent session transcripts into a searchable, summarized journal.
A quieter 7 commits, but the week’s best for outside participation: contributions from rodbegbie (2), peteror, gcarothers, and dimdasci.
7 commits 5 PRs +945 −135obra/episodic-memory
A memory system that gives agents recall across sessions.
4 commits, including one from outside contributor schuyler, plus two fixes co-authored by external contributors shahabzohaib (a SIGHUP/stdin-close handler for orphaned MCP processes) and Tapiocapioca (redirecting embedding-model output to stderr so it stops corrupting the MCP protocol stream). (Line counts are suppressed here: the bulk of the diff is a committed generated artifact, dist/mcp-server.js, not authored code.)
Rounding out the week: superpowers (17 commits), clearance (10), and sprout (10) all kept ticking; LLM.swift took a Qwen template fix; and github-triage, scribble, stockyard, and the superpowers-lab/marketplace repos saw small maintenance commits.