Orchestration & Protocols

Assemble context, coordinate tools

What runs the loop, and how does it survive a crash?

28 entries · 4 categories · 3 equations

The arithmetic

Reliability compounds multiplicatively

P_{\text{success}} \;=\; \prod_{i=1}^{n} p_i \;=\; p^{\,n}

A 20-step agent whose every step is 99% reliable succeeds 81.8% of the time. At 95% per step it succeeds 35.8%. This single line is the entire argument for durable execution — you cannot reach acceptable end-to-end reliability by improving prompts.

Expected steps executed, with and without checkpoints

E[S]_{\text{retry-all}} = \frac{n}{p^{\,n}} \qquad E[S]_{\text{checkpoint}} = \frac{n}{p}

At n = 20 and p = 0.95, restarting the whole run costs ~56 step-executions; resuming from the last checkpoint costs ~21. The gap widens exponentially in n, which is why the durability substrate is the decision and the agent library is the replaceable part.

Grammar-constrained decoding

p'(t) \;=\; \frac{p(t)\,\mathbb{1}\!\left[t \in V_{\text{valid}}\right]}{\displaystyle\sum_{t' \in V_{\text{valid}}} p(t')}

Mask invalid tokens, renormalise. Schema conformance becomes 100% by construction — which is why JSON validity is a solved non-issue. The mask says nothing about whether the model picked the right tool or plausible-but-wrong arguments.

Agent frameworks

Runtimes that hold state across steps and failures.

LangGraph

Graph runtime with checkpointed state machines, durable execution and human-in-the-loop

durabledefault-choice

MITopen sourcewidely deployedboth

Temporal

General durable execution treating agents as replayable workflows

Infrastructure rather than an agent library. Often the right substrate underneath one.

durableinfrastructure

MITopen corewidely deployedboth

Claude Agent SDK

The harness behind Claude Code: subagents, hooks, compaction, MCP-native

vendor-sdkanthropicmcp

Anthropiccommercialproduction viableboth

Google ADK

Agent SDK with evaluation and a deployment path into Vertex Agent Engine

vendor-sdkgoogle

Apache-2.0open sourceproduction viableboth

LangChain 1.x

Integration layer, now a thin facade over the LangGraph runtime

Third significant rewrite. Adopt LangGraph directly and treat integrations as optional glue.

churn-risk

MITopen sourceproduction viableself-host

LlamaIndex

Ingestion, parsing, indexing and retrieval, with agent features added later

A retrieval library with agent features, not an orchestration spine.

rag

MITopen sourceproduction viableself-host

Mastra

TypeScript-native agents, workflows, RAG and evals

typescript

Apache-2.0open sourceproduction viableboth

Microsoft Agent Framework

AutoGen and Semantic Kernel merged; .NET and Python agents plus workflow graphs

microsoftdotnet

MITopen sourceproduction viableboth

OpenAI Agents SDK

Minimal agent loop with handoffs, guardrails, tracing and sessions

vendor-sdkopenai

MITopen sourceproduction viableboth

Pydantic AI

Type-safe agent layer with schema validation and dependency injection first-class

typedpython

MITopen sourceproduction viableself-host

DBOS

Durable execution using Postgres as the workflow source of truth

durablepostgres

MITopen coreearlyboth

AutoGen

Research multi-agent conversation framework, superseded by Agent Framework

Maintenance mode. Greenfield work here is building on a dead branch.

deprecatedmicrosoft

deprecatedMITopen sourceresearchself-host

Semantic Kernel

.NET-first agent and plugin SDK, folded into Agent Framework

Maintenance mode since Agent Framework GA in April 2026.

deprecatedmicrosoft

deprecatedMITopen sourceresearchself-host

CrewAI

Role-based multi-agent crews with LLM-driven task routing

LLM-driven routing makes cost and latency non-deterministic and failures hard to reproduce. Widely downloaded, widely abandoned at the pilot boundary.

multi-agentpilot-risk

MITopen corefragileboth

Prompt optimization

Compiling prompts against a metric instead of hand-tuning.

Ax

TypeScript equivalent of DSPy's signature-driven programs

typescript

Apache-2.0open sourceearlyself-host

GEPA

Reflective evolutionary prompt optimization, shipping inside DSPy

research-derived

MITopen sourceearlyself-host

DSPy

Declarative signatures and modules that compile prompts against a metric

Offline optimizers, Python-only, no gateway or observability story.

compilationoffline

MITopen sourceresearchself-host

TextGrad

Backpropagates natural-language gradients through LLM pipelines

A paper you can import.

research-only

MITopen sourceresearchself-host

Protocols & conventions

How tools, agents and repos describe themselves.

AGENTS.md

Tells coding agents how to build, test and style a repository

Read by Codex, Cursor, Copilot, Gemini CLI, Zed, Aider and goose. No ratified versioned spec yet.

conventionde-facto-standard

AAIFopen sourcewidely deployed

CLAUDE.md

Hierarchical project context with imports and per-directory scoping

conventionanthropic

Anthropiccommercialwidely deployed

MCP

Client/server protocol for exposing tools, resources and prompts to models

Donated by Anthropic to the Linux Foundation in December 2025, removing single-vendor risk. Tool poisoning remains an OWASP-catalogued attack class.

protocolwon-the-layersecurity-caveat

Linux Foundation AAIFopen sourcewidely deployed

A2A

Peer agent discovery via Agent Cards, plus task delegation

protocolagent-to-agent

Linux Foundationopen sourceproduction viable

AGNTCY

Directory, identity, messaging and observability for agent meshes

protocolmesh

Linux Foundationopen sourceearly

llms.txt

Site-level markdown index intended for LLM consumers

No major AI platform has confirmed reading it. Optional SEO hygiene, not context engineering.

conventionunproven

Proposalopen sourceearly

Structured output

Constrained decoding and schema enforcement.

Instructor

Pydantic validation plus a retry loop around any provider

validationretries

MITopen sourcewidely deployedself-host

XGrammar

Pushdown-automaton constrained decoding, the default in vLLM, SGLang and TRT-LLM

constrained-decodingdefault-choice

Apache-2.0open sourcewidely deployedself-host

llguidance

Low-latency grammar engine with the strongest measured JSON validity

constrained-decoding

MITopen sourceproduction viableself-host

Outlines

FSM-based structured generation, largely displaced by XGrammar

constrained-decoding

Apache-2.0open sourceproduction viableself-host

Field notes

Stars do not equal standardization. CrewAI and LangChain dominate GitHub metrics; production tends to be LangGraph, a vendor SDK, or a hand-rolled loop on Temporal.
The category is converging on durable execution, not on agent abstractions. Pick your durability substrate first — the agent library is the replaceable part.
LangChain's rewrite churn is a procurement risk. If adopting, adopt LangGraph directly and treat LangChain integrations as optional glue.
LlamaIndex is a retrieval library with agent features. Use it for parsing and indexing, not as the orchestration spine.
DSPy is intellectually correct and operationally awkward: offline optimizers, Python-only, no gateway or observability story. Treat it as a prompt-compilation step for narrow well-evaluated tasks, not a runtime.
MCP has won the tool layer and is now vendor-neutral under the Linux Foundation, which removes the main enterprise objection.
MCP's security model is the weakest link. Tool poisoning — malicious instructions in tool descriptions and parameter metadata that users never see — is an OWASP-catalogued attack class, with cross-tool poisoning and rug-pull server updates as variants.
Mandate a private MCP registry, pin server versions by digest, review tool metadata as code, run least-privilege, and gate write-capable tools behind human approval.
The protocol layer is consolidating rather than fragmenting: MCP for tools, A2A for agent-to-agent, both under one foundation.
Constrained decoding is solved; tool-calling reliability is not. Grammar-level JSON validity is a non-issue, but wrong tool selection and plausible-but-wrong arguments remain. Evaluate tool selection, not just schema conformance.