03The AI-native architect / Tools

The stack I use, every day.

Thirty tools in three groups: applied AI, data and retrieval, product engineering. Tap a bead to see what it is and where I use it.

Applied AI
Data & retrieval
Product engineering
Applied AI

Claude Code.

Anthropic's coding agent in the terminal and editor.

Where I use it: Daily; Tokalator was built to budget its context

www.anthropic.com/claude-code ↗
The wider map

133 tools, per layer.

Beyond my own stack: the main tools in each of the six layers, with licence and how proven each is in production.

133 tools

Guardrails & Governance.

Guardrails & PII
  • Microsoft Presidio

    NER and regex PII detection, redaction and anonymisation

    MITwidely used
  • NeMo Guardrails

    Colang dialog-flow rails and topic control that orchestrate other classifiers

    Apache-2.0production-ready
  • Guardrails AI

    Output validators and schema enforcement via a validator hub

    Apache-2.0production-ready
  • Llama Guard / Prompt Guard

    Hazard-taxonomy classifier plus a fast injection pre-filter

    Meta communityproduction-ready
  • LLM Guard

    Scanner suite covering PII, toxicity and injection heuristics

    MITproduction-ready
  • Lakera Guard

    Commercial injection and jailbreak detection API

    Proprietaryproduction-ready
  • Azure Prompt Shields

    Managed moderation and prompt-injection filters

    Proprietaryproduction-ready
Governance tooling
  • Credo AI

    Risk registers, control mappings and Annex IV technical documentation

    Proprietaryproduction-ready
  • Holistic AI

    AI risk assessment, bias auditing and governance reporting

    Proprietaryproduction-ready
  • IBM watsonx.governance

    Model inventory, risk workflows and compliance evidence inside the IBM stack

    Proprietaryproduction-ready
  • Trustible

    Regulatory mapping and AI use-case registry for compliance teams

    Proprietaryearly
Regulation & frameworks
  • EU AI Act

    Risk-tiered obligations with extraterritorial reach; Regulation (EU) 2024/1689

    Regulation
  • Digital Omnibus on AI

    First AI Act amendment; Regulation (EU) 2026/1744, in force 27 July 2026

    Regulation
  • NIST AI RMF 1.0

    Voluntary US risk framework with a Generative AI profile

    Public
  • ISO/IEC 42001:2023

    Certifiable AI management system standard

    Standard

Evaluation & Observability.

Tracing
  • Langfuse

    Tracing, prompt management, evals and datasets with the strongest self-host story

    MIT core + EEwidely used
  • Arize Phoenix

    OTel-native tracing and eval workflow that runs fully local

    Elastic-2.0widely used
  • Comet Opik

    Tracing, evaluation and guardrail hooks with no enterprise feature gating

    Apache-2.0production-ready
  • Braintrust

    Eval-first workflow with scoring, prompt playground and a VPC data plane

    Proprietarywidely used
  • LangSmith

    Trace and debug LangChain and LangGraph agents, with datasets and online evals

    Proprietarywidely used
  • W&B Weave

    Trace and eval layer atop Weights & Biases

    Apache-2.0 SDKproduction-ready
  • MLflow Tracing

    GenAI tracing inside an existing MLflow registry estate

    Apache-2.0production-ready
  • HoneyHive

    Tracing, evaluation and dataset curation with VPC deployment

    Proprietaryproduction-ready
  • Datadog LLM Observability

    LLM spans inside existing APM, correlated with infrastructure telemetry

    Proprietarywidely used
  • Helicone

    Proxy-based logging and caching with cache-hit-rate dashboards

    Apache-2.0production-ready
  • OTel GenAI conventions

    Semantic conventions for gen_ai.* spans, metrics and events

    Apache-2.0early
Evaluation frameworks
  • RAGAS

    RAG-component metrics: faithfulness, context precision and recall, answer relevancy

    Apache-2.0widely used
  • DeepEval

    Pytest-style LLM unit tests covering RAG, conversational and agent metrics

    Apache-2.0production-ready
  • promptfoo

    Declarative YAML eval matrices plus automated red-teaming

    MITproduction-ready
  • Inspect AI

    UK AI Security Institute harness with solvers and scorers, strong for safety evals

    MITproduction-ready
  • OpenAI Evals

    Registry-based eval templates, OpenAI-centric

    MITproduction-ready

Orchestration & Protocols.

Agent frameworks
  • LangGraph

    Graph runtime with checkpointed state machines, durable execution and human-in-the-loop

    MITwidely used
  • Temporal

    General durable execution treating agents as replayable workflows

    MITwidely used
  • Microsoft Agent Framework

    AutoGen and Semantic Kernel merged; .NET and Python agents plus workflow graphs

    MITproduction-ready
  • OpenAI Agents SDK

    Minimal agent loop with handoffs, guardrails, tracing and sessions

    MITproduction-ready
  • Claude Agent SDK

    The harness behind Claude Code: subagents, hooks, compaction, MCP-native

    Anthropicproduction-ready
  • Google ADK

    Agent SDK with evaluation and a deployment path into Vertex Agent Engine

    Apache-2.0production-ready
  • Pydantic AI

    Type-safe agent layer with schema validation and dependency injection first-class

    MITproduction-ready
  • Mastra

    TypeScript-native agents, workflows, RAG and evals

    Apache-2.0production-ready
  • LangChain 1.x

    Integration layer, now a thin facade over the LangGraph runtime

    MITproduction-ready
  • LlamaIndex

    Ingestion, parsing, indexing and retrieval, with agent features added later

    MITproduction-ready
  • CrewAI

    Role-based multi-agent crews with LLM-driven task routing

    MITfragile
  • DBOS

    Durable execution using Postgres as the workflow source of truth

    MITearly
  • AutoGen

    Research multi-agent conversation framework, superseded by Agent Framework

    MITresearch
  • Semantic Kernel

    .NET-first agent and plugin SDK, folded into Agent Framework

    MITresearch
Prompt optimization
  • DSPy

    Declarative signatures and modules that compile prompts against a metric

    MITresearch
  • GEPA

    Reflective evolutionary prompt optimization, shipping inside DSPy

    MITearly
  • Ax

    TypeScript equivalent of DSPy's signature-driven programs

    Apache-2.0early
  • TextGrad

    Backpropagates natural-language gradients through LLM pipelines

    MITresearch
Protocols & conventions
  • MCP

    Client/server protocol for exposing tools, resources and prompts to models

    Linux Foundation AAIFwidely used
  • A2A

    Peer agent discovery via Agent Cards, plus task delegation

    Linux Foundationproduction-ready
  • AGNTCY

    Directory, identity, messaging and observability for agent meshes

    Linux Foundationearly
  • AGENTS.md

    Tells coding agents how to build, test and style a repository

    AAIFwidely used
  • CLAUDE.md

    Hierarchical project context with imports and per-directory scoping

    Anthropicwidely used
  • llms.txt

    Site-level markdown index intended for LLM consumers

    Proposalearly
Structured output
  • XGrammar

    Pushdown-automaton constrained decoding, the default in vLLM, SGLang and TRT-LLM

    Apache-2.0widely used
  • llguidance

    Low-latency grammar engine with the strongest measured JSON validity

    MITproduction-ready
  • Outlines

    FSM-based structured generation, largely displaced by XGrammar

    Apache-2.0production-ready
  • Instructor

    Pydantic validation plus a retry loop around any provider

    MITwidely used

Caching & Context Optimization.

Provider caching
  • Anthropic prompt caching

    Up to four cache_control breakpoints with 5-minute or 1-hour TTL, refreshed on read

    Commercialwidely used
  • OpenAI automatic caching

    Zero-config prefix cache with a 1,024-token minimum and no write surcharge

    Commercialwidely used
  • Gemini explicit caching

    Declared cache object with 60-minute default TTL and a large token minimum

    Commercialwidely used
Self-hosted KV cache
  • vLLM prefix caching

    Block-level content-hashed KV reuse, on by default in V1

    Apache-2.0widely used
  • SGLang RadixAttention

    Radix tree over token sequences for longest-prefix KV match, on by default

    Apache-2.0widely used
  • LMCache

    Cross-instance KV cache sharing and offload to CPU or NVMe

    Apache-2.0production-ready
Compression & pruning
  • LLMLingua

    Token-classification prompt compressor achieving 2-5x reduction

    MITresearch
  • Sub-agent context isolation

    Spawn agents with private windows so only summaries reach the parent

    Patternwidely used
Token accounting
  • tiktoken

    OpenAI BPE tokenizer giving exact counts for OpenAI models only

    MITwidely used
  • Anthropic count_tokens

    Free endpoint returning billing-accurate counts including system prompt and tools

    Commercial (free)widely used
  • LiteLLM

    Gateway normalizing token accounting and cache headers across providers

    MITwidely used
Long-context evidence
  • Context Rot (Chroma)

    Eighteen models degrade non-uniformly as input grows; includes a replication toolkit

    Research
  • NoLiMa

    Needle-in-haystack without lexical overlap; scores collapse where standard NIAH saturates

    Research
  • Lost in the Middle

    U-shaped positional accuracy, with over 30% drop for mid-context evidence

    Research
  • LongFuncEval

    Long-context degradation measured specifically for tool and function calling

    Research

Memory & State.

Memory systems
  • Mem0

    Extracts facts into a vector, graph and KV store scoped by user, session or agent

    Apache-2.0production-ready
  • Graphiti

    Bi-temporal graph where edges carry valid-from and invalid-at, so facts expire

    Apache-2.0production-ready
  • Zep

    Async summarization and entity extraction layered over Graphiti

    Apache-2.0production-ready
  • Cognee

    ECL pipeline building an ontology-typed graph alongside a vector index

    Apache-2.0early
  • MemOS

    OS-style abstraction over plaintext, activation (KV) and parametric memory

    Apache-2.0early
  • Hindsight

    Retain, recall and reflect over memory banks with fact extraction and an MCP server

    MITearly
  • Supermemory

    Memory API and RAG over user context, with a router SDK and MCP

    MIT clientearly
  • Letta (ex-MemGPT)

    LLM-as-OS paging between main context, recall and archival stores

    Apache-2.0early
  • LangMem

    Extracts and updates memories into a LangGraph store

    MITearly
Native platform state
  • Anthropic memory tool

    Claude issues CRUD against a /memories directory; you own the storage backend

    Commercialproduction-ready
  • Anthropic context editing

    Server-side clearing of stale tool results and thinking blocks before token counting

    Commercialproduction-ready
  • OpenAI Responses API state

    Server-stored response items chained via previous_response_id, 30-day retention

    Commercialproduction-ready
  • Gemini context caching

    Explicit cached-content handles or implicit auto-cache on repeated prefixes

    Commercialproduction-ready

Retrieval.

Vector databases
  • pgvector

    Postgres extension: HNSW/IVFFlat vector indexes in your existing database

    PostgreSQLwidely used
  • pgvectorscale

    StreamingDiskANN index layered on pgvector for larger-than-memory workloads

    PostgreSQLproduction-ready
  • VectorChord

    Postgres extension with disk-friendly IVF/RaBitQ; successor to pgvecto.rs

    AGPL-3.0 + commercialproduction-ready
  • Qdrant

    Rust vector database with payload filtering, quantization and on-disk HNSW

    Apache-2.0widely used
  • Weaviate

    Native BM25 and dense hybrid search, multi-tenancy, module system

    BSD-3widely used
  • Milvus / Zilliz

    Distributed billion-scale vector database; an LF AI & Data project

    Apache-2.0widely used
  • Chroma

    Embedded single-node vector store with a Rust core

    Apache-2.0production-ready
  • LanceDB

    Serverless vector database on the Lance columnar format, reads from object storage

    Apache-2.0production-ready
  • Turbopuffer

    Object-storage-native search with SPFresh index, BM25 and sparse vectors

    Commercialproduction-ready
  • Pinecone

    Serverless managed vector database with hybrid search and namespaces

    Commercialwidely used
  • Vespa

    Search and ranking engine with tensor compute and ML ranking phases

    Apache-2.0widely used
  • MongoDB Atlas Vector Search

    Vector index inside MongoDB with native embedding and rerank APIs

    Commercial / SSPL corewidely used
  • FAISS

    ANN index library rather than a database; baseline for offline search

    MITwidely used
Search & hybrid retrieval
  • Elasticsearch

    BM25 plus dense and sparse retrieval, RRF hybrid, retrievers API

    AGPL-3.0 / ELv2 / SSPLwidely used
  • OpenSearch

    Apache-2.0 fork with k-NN, RRF hybrid and neural sparse retrieval

    Apache-2.0widely used
  • ParadeDB

    Postgres extension adding Tantivy-backed BM25 for in-database hybrid search

    AGPL-3.0 + commercialproduction-ready
  • Typesense

    Typo-tolerant keyword search with vector fields and a simple ops model

    GPL-3.0production-ready
  • Meilisearch

    Lightweight full-text and hybrid search for in-application search

    MITproduction-ready
  • Tantivy

    Rust full-text search library embedded in many higher-level tools

    MITwidely used
Embeddings & rerankers
  • Qwen3-Embedding / Reranker

    Open-weight embedding and reranker models topping MTEB in 2026

    Apache-2.0production-ready
  • BGE (BAAI)

    Open-weight embeddings; bge-reranker-v2-m3 is the common self-host default

    MIT / Apache-2.0widely used
  • Cohere Embed v4 / Rerank 4

    Multimodal embeddings with Matryoshka dims, plus a cross-encoder rerank API

    Commercialwidely used
  • Voyage AI

    Domain-tuned embeddings and rerankers for legal, code and finance

    Commercialproduction-ready
  • Jina AI

    Open-weight embeddings and multimodal rerankers that score page images

    Mixed OSS + APIproduction-ready
  • ZeroEntropy

    zerank reranker models at the top of several 2026 leaderboards

    OSS weights + APIearly
  • Mixedbread

    Open-weight embedding and reranker models plus a hosted API

    Apache-2.0production-ready
  • ColBERT / PLAID

    Late-interaction retrieval with token-level MaxSim; higher index cost

    Apache-2.0production-ready
  • FlashRank

    Small CPU-only reranker for latency-constrained pipelines

    Apache-2.0production-ready
Parsing & ingestion
  • Docling

    IBM Research layout and table pipeline; local PDF-to-Markdown, no per-page cost

    MITproduction-ready
  • Unstructured

    Thirty-plus formats to normalized elements, with connectors and VPC deployment

    Apache-2.0 corewidely used
  • LlamaParse

    VLM-based PDF parsing to Markdown; best-in-class on messy documents

    Commercialwidely used
  • Reducto

    Agentic OCR-correction parsing with on-prem, SOC 2 Type II and HIPAA

    Commercialproduction-ready
  • Marker

    PDF and EPUB to Markdown, GPU-accelerated and batch-oriented

    GPL-3.0 + commercialproduction-ready
  • Chunkr

    Parsing and semantic chunking API, self-hostable

    AGPL-3.0early
  • Chonkie

    Chunking library covering token, semantic, recursive and late chunking

    MITproduction-ready
  • Firecrawl

    Crawls sites to clean Markdown or JSON and handles JS rendering

    AGPL-3.0 coreproduction-ready
Managed retrieval
  • Azure AI Search

    Hybrid BM25, vector and semantic reranking with integrated vectorization

    Commercialwidely used
  • Vertex AI Search

    Managed retrieval and grounding with connectors and a ranking API

    Commercialwidely used
  • Bedrock Knowledge Bases

    Managed ingestion and retrieval over OpenSearch, Aurora or Neptune

    Commercialwidely used
  • Ragie

    Managed ingestion and retrieval API with connectors and citations

    Commercialearly
  • Vectara

    Managed RAG with HHEM hallucination scoring; enterprise-only since 2026

    Commercialproduction-ready
Graph retrieval
  • Microsoft GraphRAG

    Entity extraction, community detection and hierarchical summaries

    MITproduction-ready
  • LightRAG

    Flat dual-level graph index, far cheaper to build than GraphRAG

    MITearly
  • Graphiti

    Temporal knowledge graph with edge validity intervals and hybrid graph-vector search

    Apache-2.0production-ready
  • neo4j-graphrag-python

    Official Neo4j retrievers and knowledge-graph construction pipeline

    Apache-2.0production-ready
  • Cognee

    Pipeline turning documents into a queryable memory graph

    Apache-2.0early