The stack I use, every day.
Thirty tools in three groups: applied AI, data and retrieval, product engineering. Tap a bead to see what it is and where I use it.
Claude Code.
Anthropic's coding agent in the terminal and editor.
Where I use it: Daily; Tokalator was built to budget its context
www.anthropic.com/claude-code ↗133 tools, per layer.
Beyond my own stack: the main tools in each of the six layers, with licence and how proven each is in production.
133 tools
Guardrails & Governance.
- Microsoft Presidio
NER and regex PII detection, redaction and anonymisation
- NeMo Guardrails
Colang dialog-flow rails and topic control that orchestrate other classifiers
- Guardrails AI
Output validators and schema enforcement via a validator hub
- Llama Guard / Prompt Guard
Hazard-taxonomy classifier plus a fast injection pre-filter
- LLM Guard
Scanner suite covering PII, toxicity and injection heuristics
- Lakera Guard
Commercial injection and jailbreak detection API
- Azure Prompt Shields
Managed moderation and prompt-injection filters
- Credo AI
Risk registers, control mappings and Annex IV technical documentation
- Holistic AI
AI risk assessment, bias auditing and governance reporting
- IBM watsonx.governance
Model inventory, risk workflows and compliance evidence inside the IBM stack
- Trustible
Regulatory mapping and AI use-case registry for compliance teams
- EU AI Act
Risk-tiered obligations with extraterritorial reach; Regulation (EU) 2024/1689
- Digital Omnibus on AI
First AI Act amendment; Regulation (EU) 2026/1744, in force 27 July 2026
- NIST AI RMF 1.0
Voluntary US risk framework with a Generative AI profile
- ISO/IEC 42001:2023
Certifiable AI management system standard
Evaluation & Observability.
- Langfuse
Tracing, prompt management, evals and datasets with the strongest self-host story
- Arize Phoenix
OTel-native tracing and eval workflow that runs fully local
- Comet Opik
Tracing, evaluation and guardrail hooks with no enterprise feature gating
- Braintrust
Eval-first workflow with scoring, prompt playground and a VPC data plane
- LangSmith
Trace and debug LangChain and LangGraph agents, with datasets and online evals
- W&B Weave
Trace and eval layer atop Weights & Biases
- MLflow Tracing
GenAI tracing inside an existing MLflow registry estate
- HoneyHive
Tracing, evaluation and dataset curation with VPC deployment
- Datadog LLM Observability
LLM spans inside existing APM, correlated with infrastructure telemetry
- Helicone
Proxy-based logging and caching with cache-hit-rate dashboards
- OTel GenAI conventions
Semantic conventions for gen_ai.* spans, metrics and events
- RAGAS
RAG-component metrics: faithfulness, context precision and recall, answer relevancy
- DeepEval
Pytest-style LLM unit tests covering RAG, conversational and agent metrics
- promptfoo
Declarative YAML eval matrices plus automated red-teaming
- Inspect AI
UK AI Security Institute harness with solvers and scorers, strong for safety evals
- OpenAI Evals
Registry-based eval templates, OpenAI-centric
Orchestration & Protocols.
- LangGraph
Graph runtime with checkpointed state machines, durable execution and human-in-the-loop
- Temporal
General durable execution treating agents as replayable workflows
- Microsoft Agent Framework
AutoGen and Semantic Kernel merged; .NET and Python agents plus workflow graphs
- OpenAI Agents SDK
Minimal agent loop with handoffs, guardrails, tracing and sessions
- Claude Agent SDK
The harness behind Claude Code: subagents, hooks, compaction, MCP-native
- Google ADK
Agent SDK with evaluation and a deployment path into Vertex Agent Engine
- Pydantic AI
Type-safe agent layer with schema validation and dependency injection first-class
- Mastra
TypeScript-native agents, workflows, RAG and evals
- LangChain 1.x
Integration layer, now a thin facade over the LangGraph runtime
- LlamaIndex
Ingestion, parsing, indexing and retrieval, with agent features added later
- CrewAI
Role-based multi-agent crews with LLM-driven task routing
- DBOS
Durable execution using Postgres as the workflow source of truth
- AutoGen
Research multi-agent conversation framework, superseded by Agent Framework
- Semantic Kernel
.NET-first agent and plugin SDK, folded into Agent Framework
- MCP
Client/server protocol for exposing tools, resources and prompts to models
- A2A
Peer agent discovery via Agent Cards, plus task delegation
- AGNTCY
Directory, identity, messaging and observability for agent meshes
- AGENTS.md
Tells coding agents how to build, test and style a repository
- CLAUDE.md
Hierarchical project context with imports and per-directory scoping
- llms.txt
Site-level markdown index intended for LLM consumers
- XGrammar
Pushdown-automaton constrained decoding, the default in vLLM, SGLang and TRT-LLM
- llguidance
Low-latency grammar engine with the strongest measured JSON validity
- Outlines
FSM-based structured generation, largely displaced by XGrammar
- Instructor
Pydantic validation plus a retry loop around any provider
Caching & Context Optimization.
- Anthropic prompt caching
Up to four cache_control breakpoints with 5-minute or 1-hour TTL, refreshed on read
- OpenAI automatic caching
Zero-config prefix cache with a 1,024-token minimum and no write surcharge
- Gemini explicit caching
Declared cache object with 60-minute default TTL and a large token minimum
- vLLM prefix caching
Block-level content-hashed KV reuse, on by default in V1
- SGLang RadixAttention
Radix tree over token sequences for longest-prefix KV match, on by default
- LMCache
Cross-instance KV cache sharing and offload to CPU or NVMe
- LLMLingua
Token-classification prompt compressor achieving 2-5x reduction
- Sub-agent context isolation
Spawn agents with private windows so only summaries reach the parent
- tiktoken
OpenAI BPE tokenizer giving exact counts for OpenAI models only
- Anthropic count_tokens
Free endpoint returning billing-accurate counts including system prompt and tools
- LiteLLM
Gateway normalizing token accounting and cache headers across providers
- Context Rot (Chroma)
Eighteen models degrade non-uniformly as input grows; includes a replication toolkit
- NoLiMa
Needle-in-haystack without lexical overlap; scores collapse where standard NIAH saturates
- Lost in the Middle
U-shaped positional accuracy, with over 30% drop for mid-context evidence
- LongFuncEval
Long-context degradation measured specifically for tool and function calling
Memory & State.
- Mem0
Extracts facts into a vector, graph and KV store scoped by user, session or agent
- Graphiti
Bi-temporal graph where edges carry valid-from and invalid-at, so facts expire
- Zep
Async summarization and entity extraction layered over Graphiti
- Cognee
ECL pipeline building an ontology-typed graph alongside a vector index
- MemOS
OS-style abstraction over plaintext, activation (KV) and parametric memory
- Hindsight
Retain, recall and reflect over memory banks with fact extraction and an MCP server
- Supermemory
Memory API and RAG over user context, with a router SDK and MCP
- Letta (ex-MemGPT)
LLM-as-OS paging between main context, recall and archival stores
- LangMem
Extracts and updates memories into a LangGraph store
- Anthropic memory tool
Claude issues CRUD against a /memories directory; you own the storage backend
- Anthropic context editing
Server-side clearing of stale tool results and thinking blocks before token counting
- OpenAI Responses API state
Server-stored response items chained via previous_response_id, 30-day retention
- Gemini context caching
Explicit cached-content handles or implicit auto-cache on repeated prefixes
Retrieval.
- pgvector
Postgres extension: HNSW/IVFFlat vector indexes in your existing database
- pgvectorscale
StreamingDiskANN index layered on pgvector for larger-than-memory workloads
- VectorChord
Postgres extension with disk-friendly IVF/RaBitQ; successor to pgvecto.rs
- Qdrant
Rust vector database with payload filtering, quantization and on-disk HNSW
- Weaviate
Native BM25 and dense hybrid search, multi-tenancy, module system
- Milvus / Zilliz
Distributed billion-scale vector database; an LF AI & Data project
- Chroma
Embedded single-node vector store with a Rust core
- LanceDB
Serverless vector database on the Lance columnar format, reads from object storage
- Turbopuffer
Object-storage-native search with SPFresh index, BM25 and sparse vectors
- Pinecone
Serverless managed vector database with hybrid search and namespaces
- Vespa
Search and ranking engine with tensor compute and ML ranking phases
- MongoDB Atlas Vector Search
Vector index inside MongoDB with native embedding and rerank APIs
- FAISS
ANN index library rather than a database; baseline for offline search
- Elasticsearch
BM25 plus dense and sparse retrieval, RRF hybrid, retrievers API
- OpenSearch
Apache-2.0 fork with k-NN, RRF hybrid and neural sparse retrieval
- ParadeDB
Postgres extension adding Tantivy-backed BM25 for in-database hybrid search
- Typesense
Typo-tolerant keyword search with vector fields and a simple ops model
- Meilisearch
Lightweight full-text and hybrid search for in-application search
- Tantivy
Rust full-text search library embedded in many higher-level tools
- Qwen3-Embedding / Reranker
Open-weight embedding and reranker models topping MTEB in 2026
- BGE (BAAI)
Open-weight embeddings; bge-reranker-v2-m3 is the common self-host default
- Cohere Embed v4 / Rerank 4
Multimodal embeddings with Matryoshka dims, plus a cross-encoder rerank API
- Voyage AI
Domain-tuned embeddings and rerankers for legal, code and finance
- Jina AI
Open-weight embeddings and multimodal rerankers that score page images
- ZeroEntropy
zerank reranker models at the top of several 2026 leaderboards
- Mixedbread
Open-weight embedding and reranker models plus a hosted API
- ColBERT / PLAID
Late-interaction retrieval with token-level MaxSim; higher index cost
- FlashRank
Small CPU-only reranker for latency-constrained pipelines
- Docling
IBM Research layout and table pipeline; local PDF-to-Markdown, no per-page cost
- Unstructured
Thirty-plus formats to normalized elements, with connectors and VPC deployment
- LlamaParse
VLM-based PDF parsing to Markdown; best-in-class on messy documents
- Reducto
Agentic OCR-correction parsing with on-prem, SOC 2 Type II and HIPAA
- Marker
PDF and EPUB to Markdown, GPU-accelerated and batch-oriented
- Chunkr
Parsing and semantic chunking API, self-hostable
- Chonkie
Chunking library covering token, semantic, recursive and late chunking
- Firecrawl
Crawls sites to clean Markdown or JSON and handles JS rendering
- Azure AI Search
Hybrid BM25, vector and semantic reranking with integrated vectorization
- Vertex AI Search
Managed retrieval and grounding with connectors and a ranking API
- Bedrock Knowledge Bases
Managed ingestion and retrieval over OpenSearch, Aurora or Neptune
- Ragie
Managed ingestion and retrieval API with connectors and citations
- Vectara
Managed RAG with HHEM hallucination scoring; enterprise-only since 2026
- Microsoft GraphRAG
Entity extraction, community detection and hierarchical summaries
- LightRAG
Flat dual-level graph index, far cheaper to build than GraphRAG
- Graphiti
Temporal knowledge graph with edge validity intervals and hybrid graph-vector search
- neo4j-graphrag-python
Official Neo4j retrievers and knowledge-graph construction pipeline
- Cognee
Pipeline turning documents into a queryable memory graph