What you install and run

The ones that hold a terminal. All of these read files, run commands and edit code. They differ on who owns the model, how the sandbox works, and whether you can point them at a model you already pay for. Benchmark position moves every few weeks and is the least useful thing to choose on.

• Claude Codepick

  Anthropic's agent for the terminal, with hooks, subagents, skills and MCP client support.

  The deepest extension surface of the group — hooks fire on tool events, skills load as plain markdown, and subagents get their own context window. That last one is the reason to pick it for long tasks: research can be delegated without the findings landing in the main transcript.

  npm install -g @anthropic-ai/claude-code

  • agent
  • mcp-client
  • hooks
  • subagents
• Codex CLI

  OpenAI's terminal agent, written in Rust, with OS-level sandboxing on macOS and Linux.

  The sandbox is the differentiator: seatbelt on macOS, Landlock on Linux, enforced by the kernel rather than by the agent agreeing to behave. Reach for it when you are running something you have not read.

  npm install -g @openai/codex

  • agent
  • sandbox
  • rust
• Gemini CLI

  Google's open-source terminal agent, with the most generous free tier of the group.

  The free quota makes it the cheapest way to put an agent in CI or in a cron job where you do not want to meter a subscription. Large context window helps on whole-repo questions.

  npm install -g @google/gemini-cli

  • agent
  • free-tier
  • oss
• opencode

  Model-agnostic terminal agent with a client/server split, so the TUI and the session can be on different machines.

  The one to pick when you refuse to be tied to a vendor — it will drive whatever key you supply, including a local endpoint. The server split also means you can start a session on a workstation and attach from a laptop.

  curl -fsSL https://opencode.ai/install | bash

  • agent
  • oss
  • model-agnostic
  • byok
• Aider

  Pair programming in the terminal with automatic git commits for every change.

  The git integration is the point: every edit lands as a commit, so undo is `git revert` rather than hope. Reach for it on a repo where you want a reviewable trail more than you want autonomy.

  uv tool install aider-chat

  • agent
  • git
  • oss
  • python
• GitHub Copilot CLI

  Copilot in the terminal, billed through an existing Copilot seat.

  Worth it only if the seats are already bought. Its advantage is org policy and audit landing in the same place as the rest of your GitHub estate.

  npm install -g @github/copilot

  • agent
  • github
  • commercial
• goose

  Block's open-source agent, extension-based, runs on any model including local ones.

  Its extension model is MCP end to end, so anything you build for it is portable to other clients. Good default if you are standardising on MCP rather than on a vendor.

  • agent
  • oss
  • mcp-client
  • local-models
• Crush

  Charm's terminal agent, LSP-aware, model-agnostic, with session switching.

  The LSP integration means it reads your project the way your editor does — real symbol resolution instead of grep. Notable on large typed codebases.

  • agent
  • oss
  • lsp
  • model-agnostic

What the agent is actually calling. An agent is only as good as the commands available to it. Every one of these is faster or more structured than what the agent would otherwise reach for, and each saves tokens by returning less: a scoped grep instead of a file read, a diff instead of a file, a count instead of a dump.

• ripgreppick

  Recursive line search that respects .gitignore and is fast enough to be interactive.

  The single highest-leverage tool on this shelf. An agent that greps for a symbol reads a few hundred tokens; an agent that reads three candidate files reads twenty thousand. Make sure it is installed before you tune anything else.

  brew install ripgrep

  • search
  • rust
  • token-saver
• ast-grep

  Structural search and rewrite on the syntax tree rather than on lines.

  Where ripgrep finds the string, this finds the construct — every call with a particular argument shape, every unawaited promise. Reach for it on mechanical refactors that a regex would get subtly wrong.

  brew install ast-grep

  • search
  • refactor
  • rust
  • ast
• fd

  A find replacement with sane defaults and gitignore awareness.

  Mostly a quality-of-life win over `find`, but the gitignore default matters when an agent is walking a repo with a 400 MB node_modules in it.

  brew install fd

  • files
  • rust
• jqpick

  Command-line JSON processor.

  The correct answer to 'the API returned 4 MB of JSON'. Filtering server-side of the context window is the cheapest token optimisation available.

  brew install jq

  • json
  • token-saver
• yq

  jq for YAML, XML and TOML.

  Editing a Helm values file or a CI workflow in place, without an agent rewriting the whole document and reformatting half of it.

  brew install yq

  • yaml
  • config
• GitHub CLI

  GitHub from the terminal: PRs, issues, checks, releases, raw API.

  `gh api` is the part that matters for agents — it is an authenticated HTTP client for anything the web UI can do, which means no scraping and no personal access token in a config file.

  brew install gh

  • git
  • github
  • api
• delta

  Syntax-highlighted pager for git diffs with word-level highlighting.

  For humans reviewing what the agent changed. Word-level highlighting is what makes a 200-line diff scannable rather than a wall.

  brew install git-delta

  • git
  • diff
  • review
• difftastic

  Structural diff that compares syntax trees instead of lines.

  Reach for it when a reformat has buried a one-line semantic change. It will tell you the only real change was a renamed variable; a line diff will show you 300 lines.

  brew install difftastic

  • git
  • diff
  • ast
• uvpick

  Python package and project manager, resolver and installer, in one Rust binary.

  `uv run script.py` with inline dependency metadata means a throwaway script needs no virtualenv and no requirements file. That property is what makes it the right tool for agent-authored scripts.

  curl -LsSf https://astral.sh/uv/install.sh | sh

  • python
  • packaging
  • rust
• mise

  Polyglot runtime version manager and task runner, with per-directory environments.

  Replaces nvm, pyenv, rbenv and direnv with one config file. Worth it when an agent needs the right toolchain active without you remembering which shim is loaded.

  curl https://mise.run | sh

  • versions
  • env
  • rust
• just

  A command runner with make's ergonomics and none of its build semantics.

  The best way to give an agent a stable vocabulary: `just test`, `just lint`, `just deploy`. It stops the agent inventing a slightly wrong invocation every session.

  brew install just

  • tasks
  • rust
  • agent-interface
• watchexec

  Runs a command when files change.

  Useful as a feedback loop the agent does not have to poll — tests re-run on save and the agent reads a result file rather than repeatedly invoking the suite.

  brew install watchexec

  • watch
  • rust
  • feedback-loop
• hyperfine

  Benchmarking tool with warmup runs, statistical outlier detection and export.

  The answer to 'is this actually faster'. Reach for it before accepting any performance claim, including your own.

  brew install hyperfine

  • benchmark
  • rust
  • measurement
• sqlite-utils

  Build and query SQLite databases from the command line, including CSV and JSON import.

  The fastest route from a pile of JSON to something queryable. For exploratory data work it beats loading a dataframe, and the result is a single file you can hand to anything.

  uv tool install sqlite-utils

  • sqlite
  • data
  • python
• llm

  CLI for prompting models, with plugins, logged conversations and embedding support.

  The right shape for putting a model in a pipe. It logs every prompt and response to SQLite, which makes it the easiest way to keep a reproducible record of one-off model calls.

  uv tool install llm

  • llm
  • pipe
  • python
  • logging
• files-to-prompt

  Concatenates a directory of files into a single prompt-shaped blob.

  For the case where you want the whole thing in the window on purpose. Pair it with a token counter before you paste, not after.

  uv tool install files-to-prompt

  • context
  • python
• Repomix

  Packs a repository into one AI-friendly file, with token counting and secret redaction.

  Better than a naive concatenation because it reports the token cost per file, so you can see which directory is eating the window before you send it.

  npx repomix

  • context
  • token-accounting
• fzf

  General-purpose fuzzy finder that filters any list interactively.

  A human tool, not an agent one. It is on this shelf because the review step — picking which of the agent's twelve changed files to read first — is where your time actually goes.

  brew install fzf

  • fuzzy
  • interactive
  • review
• tmux

  Terminal multiplexer with detachable sessions.

  The reason it belongs here: an agent run that takes forty minutes should not die because you closed a laptop. Detach, reattach, keep the transcript.

  brew install tmux

  • sessions
  • long-running
• direnv

  Loads and unloads environment variables per directory.

  Keeps project credentials out of your shell profile, which matters more when an agent can read your shell profile.

  brew install direnv

  • env
  • secrets-hygiene
• ShellCheck

  Static analysis for shell scripts.

  Agents write shell with unquoted variables and unhandled failures. This catches both, and it catches them before the script runs against something you care about.

  brew install shellcheck

  • shell
  • lint
  • safety
• Pandoc

  Universal document converter across roughly forty markup formats.

  The unglamorous first step of most ingestion pipelines. Before reaching for a document-parsing service, check whether the corpus is DOCX and Pandoc solves it for nothing.

  brew install pandoc

  • documents
  • ingestion
  • conversion

When the weights stay on your machine. Two different jobs get confused here. Ollama, LM Studio and Jan are experience layers over llama.cpp or MLX, sized for one user on a laptop. They are not serving infrastructure — the moment you have concurrent users, the engine underneath is the thing you are choosing.

• Ollamapick

  One-line local model runner wrapping llama.cpp, and MLX on Apple silicon, with an OpenAI-compatible endpoint.

  The fastest path to a local model and the right default for one user on one machine. It is not a serving tier — concurrency is where it stops being the answer.

  brew install ollama

  • local
  • runner
  • openai-compatible
• LM Studio

  Desktop app for discovering, running and serving local models, with a built-in server.

  The most polished GUI of the group, and the one to hand to someone who is not going to use a terminal. Same engine underneath as Ollama; the difference is entirely the interface.

  • local
  • gui
  • desktop
• llama.cpp

  The C++ inference engine that most local runners are wrapping.

  Go direct when you need control over quantisation, thread count, batch size or memory mapping — the wrappers hide exactly the knobs that matter on constrained hardware.

  brew install llama.cpp

  • engine
  • quantisation
  • cpp
• MLX LM

  Apple's array framework and its language-model package, built for unified memory.

  On Apple silicon this is the native path — unified memory means no host-to-device copy, and large models fit where a discrete GPU of the same nominal size would not.

  uv tool install mlx-lm

  • apple-silicon
  • engine
  • python
• Jan

  Open-source, offline-first desktop assistant with a local API server.

  The pick when 'the data must not leave the machine' is a written requirement rather than a preference — it is designed to run fully offline and is auditable.

  • local
  • gui
  • oss
  • offline
• Open WebUI

  Self-hosted web interface for local and remote models, with users, RAG and tools.

  Where you land when several people need to share one local model. It brings accounts and permissions, which is the thing the single-user apps deliberately do not have.

  • self-host
  • multi-user
  • web
• LibreChat

  Self-hosted multi-provider chat interface with agents, MCP support and per-user keys.

  The closest open equivalent to a commercial chat product. Reach for it when the requirement is 'our own ChatGPT' with real user management rather than a single shared key.

  • self-host
  • multi-provider
  • mcp-client
• AnythingLLM

  All-in-one desktop and docker app bundling RAG, agents and a vector store.

  Useful for a demo or a small internal tool where you want document chat working this afternoon. The bundled retrieval is a starting point, not a design.

  • rag
  • desktop
  • batteries-included
• Zed

  Rust editor with native agent support, multibuffer edits and collaborative sessions.

  The multibuffer is the relevant feature: an agent's change set across nine files appears as one reviewable surface rather than nine tabs.

  • editor
  • rust
  • agent-native
• Cline

  Open-source autonomous coding agent inside VS Code, model-agnostic, with plan and act modes.

  The plan/act split is worth copying even if you use something else — an explicit read-only phase before an editing phase catches wrong premises while they are still cheap.

  • vscode
  • agent
  • oss
  • byok
• Continue

  Open-source IDE assistant for VS Code and JetBrains, configurable down to the model per task.

  Reach for it when you want autocomplete on a small local model and chat on a frontier one. Few tools let you split those two budgets.

  • vscode
  • jetbrains
  • oss
  • byok

Servers, registries, and what is no longer maintained. The reference repository now maintains seven servers. Everything else that used to live there — GitHub, Postgres, Slack, Google Drive — moved to an archive, and most roundups still link at the dead copies. Where an official first-party server exists, use that instead.

• Official MCP Registrypick

  The canonical, API-addressable index of MCP servers, backed by Anthropic, GitHub and Microsoft.

  Use it for programmatic discovery — it is the only source with a stable API and a verification story. Treat the community directories as search engines over it, not as substitutes.

  • registry
  • discovery
  • official
• Reference servers

  The seven servers still maintained upstream: everything, fetch, filesystem, git, memory, sequential-thinking, time.

  These are the ones to read when writing your own — they are the executable version of the specification. Filesystem and git are also genuinely useful in production configs.

  • reference
  • official
  • filesystem
  • git
• servers-archived

  Where the GitHub, GitLab, Postgres, SQLite, Slack, Redis and Google Drive servers went.

  Listed here as a warning, not a recommendation. Most 'best MCP servers' roundups still link at these paths; they are unmaintained and several have been superseded by first-party servers.

  • archived
  • deprecated
  • warning
• GitHub MCP Serverpick

  GitHub's own server, available as a remote OAuth endpoint or a local binary.

  The correct replacement for the archived community GitHub server. Prefer the remote endpoint — it removes token handling from your machine entirely.

  • github
  • official
  • remote
  • oauth
• Playwright MCPpick

  Microsoft's browser automation server, driving pages through the accessibility tree rather than screenshots.

  The accessibility-tree approach is why it works: structured page state costs a fraction of what a screenshot costs, and it is deterministic. The right tool for verifying a UI change actually rendered.

  npx @playwright/mcp@latest

  • browser
  • official
  • testing
  • accessibility-tree
• Chrome DevTools MCP

  Exposes the DevTools protocol — performance traces, network, console — to an agent.

  Where Playwright drives the page, this inspects it. Reach for it on 'why is this slow' rather than 'does this work'.

  • browser
  • performance
  • debugging
  • official
• Context7

  Serves current, version-pinned library documentation into the context window.

  Directly targets the failure where a model writes an API that was removed two versions ago. Worth it in proportion to how fast your dependencies move.

  • documentation
  • freshness
  • grounding
• MCP Inspector

  Interactive tool for testing and debugging servers without wiring them into a client.

  The first thing to run when a server 'does not work'. It separates a broken server from a broken client configuration, which is most of the debugging time.

  npx @modelcontextprotocol/inspector

  • debugging
  • official
  • development
• FastMCPpick

  Python framework for building servers and clients, now the official Python SDK lineage.

  The shortest path from a Python function to a tool an agent can call. If you are writing a server rather than installing one, start here.

  uv add fastmcp

  • sdk
  • python
  • authoring
• TypeScript SDK

  The official TypeScript implementation of client and server sides of the protocol.

  Use it when the server has to run in the same process as an existing Node service, or when you need the client side to embed MCP in your own product.

  npm install @modelcontextprotocol/sdk

  • sdk
  • typescript
  • authoring
  • official
• PulseMCP

  Hand-reviewed directory of servers with changelogs and use cases.

  The editorial counterweight to registries that index everything. Useful precisely because a person looked at each entry.

  • directory
  • curated
• Smithery

  Registry plus hosting: install servers locally by CLI or run them as hosted remotes.

  The hosting side is the interesting part — it removes 'every developer must install fourteen node processes' from the rollout plan.

  • directory
  • hosting
  • remote

What you measure against, and what it does not tell you. A public benchmark tells you a model generalises. It does not tell you it works on your corpus, and models released after 2023 have very likely seen these datasets. Use them to rule things out, then build a set from your own traffic to rule things in.

• MTEBpick

  Massive Text Embedding Benchmark: retrieval, clustering, classification, reranking and STS across many languages.

  Use the leaderboard to build a shortlist of three, never to pick one. Its retrieval half is dominated by general web corpora, so if your documents are contracts or scientific PDFs the ranking is directional at best.

  • embeddings
  • leaderboard
  • retrieval
• BEIR

  Heterogeneous zero-shot retrieval benchmark spanning eighteen datasets, now a subset of MTEB.

  The zero-shot framing is what makes it useful — it measures generalisation to unseen domains. The catch is contamination: models trained after 2023 have very likely seen these corpora.

  • retrieval
  • zero-shot
  • contamination-risk
• MS MARCO

  Large-scale passage and document ranking dataset built from real Bing queries.

  The training substrate under most open retrieval models. Worth knowing about mainly so you recognise that a model scoring well on it has probably trained on it.

  • retrieval
  • ranking
  • training-data
• HotpotQA

  Multi-hop question answering with supporting-fact supervision.

  The standard evidence for 'does the pipeline actually chain facts'. Reach for it before approving a GraphRAG proposal — it is the cheapest way to test whether multi-hop is really your bottleneck.

  • multi-hop
  • qa
  • graphrag-evidence
• MultiHop-RAG

  Retrieval benchmark where answers require evidence from several documents.

  Closer to enterprise reality than HotpotQA because the hops cross documents rather than paragraphs of the same encyclopaedia.

  • multi-hop
  • rag
  • evaluation
• RULERpick

  Synthetic long-context benchmark with configurable sequence lengths and task types.

  The tool for checking a claimed context window against an effective one. Advertised length and usable length routinely differ by an order of magnitude.

  • long-context
  • synthetic
  • window-claims
• LongBench

  Bilingual multi-task benchmark for long-context understanding on realistic documents.

  Pair it with RULER: RULER tells you where retrieval inside the window breaks, LongBench tells you whether comprehension survives on real text.

  • long-context
  • bilingual
  • evaluation
• SWE-bench

  Real GitHub issues with the tests that verify a fix, plus a human-validated subset.

  The Verified subset is the only one worth quoting; the full set contains under-specified issues. Note that it measures patch generation on Python repositories, not general engineering.

  • agents
  • coding
  • verified-subset
• Terminal-Bench

  Benchmark for agents operating in a real terminal against containerised tasks.

  Measures the thing the agents on the first shelf actually do — run commands, read output, recover. Closer to your workload than a patch-generation score.

  • agents
  • terminal
  • sandboxed
• τ-bench

  Tool-agent-user benchmark with domain policies and a simulated user.

  The rare benchmark that scores policy compliance rather than task completion — it asks whether the agent followed the rules it was given, which is the governance question.

  • agents
  • tool-use
  • policy-compliance
• Berkeley Function Calling Leaderboard

  Evaluates function and tool calling across single, parallel, multiple and irrelevance cases.

  The irrelevance category is the useful one: it measures whether a model correctly declines to call anything. That failure mode costs more in production than a wrong argument.

  • tool-use
  • function-calling
  • leaderboard
• GAIA

  General assistant benchmark of questions that are easy for humans and hard for models.

  Its value is the asymmetry — near-ceiling human performance means a low score is a real capability gap rather than an ambiguous question.

  • agents
  • general
  • tool-use
• FineWeb

  Cleaned and deduplicated web corpus derived from Common Crawl, with an educational subset.

  The reference for what serious web-scale filtering looks like. Read the filtering methodology even if you never touch the data — most in-house pipelines skip steps it proves matter.

  • pretraining
  • web-scale
  • filtering
• Common Crawl

  Petabyte-scale open web crawl, published monthly since 2008.

  Almost never the right thing to use raw. It is the upstream of nearly every open corpus, which makes it the place to check provenance claims.

  • web-scale
  • raw
  • provenance
• The Stack v2

  Permissively licensed source code across hundreds of languages, with opt-out honoured.

  The licence-aware option for code data. The opt-out mechanism is the part that matters if anyone in legal is going to ask where the training data came from.

  • code
  • licensing
  • opt-out
• PubMed Central OA

  Open-access subset of the biomedical literature, full text, bulk downloadable.

  The standard corpus for biomedical retrieval work, and a good stress test for parsing — dense tables, figures and references break naive chunkers immediately.

  • biomedical
  • domain-data
  • full-text
• SEC EDGAR

  Every filing by every US public company, with a documented full-text search API.

  The best free corpus for financial retrieval, and a realistic one: structured XBRL alongside prose, with genuine numeric questions that a vector search alone will get wrong.

  • financial
  • domain-data
  • structured
  • api
• CourtListener

  Millions of US court opinions and dockets with a free API, from the Free Law Project.

  Legal text is where citation grounding gets tested properly — the documents cite each other, so a hallucinated reference is checkable rather than plausible.

  • legal
  • domain-data
  • citations
  • api
• OpenAlex

  Open catalogue of scholarly works, authors and institutions, with a full snapshot and API.

  The citation graph is the draw. It is one of the few realistic public datasets where graph retrieval genuinely beats vector search, which makes it the honest place to test that claim.

  • scholarly
  • graph
  • domain-data
  • api
• Wikipedia dumps

  Complete, versioned exports of every Wikipedia language edition.

  The default demo corpus, and the reason so many retrieval demos look better than they are: it is clean, well-structured and almost certainly in the model's weights already.

  • general
  • demo-corpus
  • contamination-risk