Evidence label: deterministic synthetic model. This page runs a small JavaScript model of tool routing. It generates a catalog of synthetic tool embeddings clustered by domain, generates requests near a target tool, runs an exact cosine top-k search (or a group-then-tool choice) to build the prompt, and simulates the model's choice as similarity plus noise that grows with the number of tools in its prompt. Nothing here calls a model or an embedding service; all numbers are calculated estimates.
Play the lab
How to play
Choose a routing strategy, the catalog size, the shortlist size k and how distinct the tool descriptions are. Every change rebuilds the same synthetic catalog and replays the same requests with the same seed, so differences come from your change. Tiles show how often the right tool is chosen, how often it was in the prompt at all, prompt tokens per call, input cost per thousand calls and p95 routing latency. New random seed re-rolls the catalog and requests; results move by a few points, which is a useful reminder to evaluate on more than one sample.
Missions unlock in order. Events add near-duplicate tools, make requests vaguer, and triple the catalog.
What the model assumes
Tools are 24-dimensional unit vectors around domain centres (about 15 tools per domain); distinctiveness spreads tools further from their centre and from their duplicates. A request is its target tool plus noise; the retriever sees a noisier version than the model, which reads intent better than an embedding does. The model picks the candidate with the highest similarity plus Gumbel noise of scale 0.01 + 0.012 · ln(n). Each definition costs 150 tokens on a 1,500-token system prompt at an illustrative $3 per million input tokens; latency is 0.6 seconds per model call plus 0.03 seconds per thousand prompt tokens, plus 30 ms for retrieval. The group-then-tool strategy makes two model calls.
What it cannot tell you
Real tool embeddings are not clean clusters, real requests often need several tools, and real models fail in ways that are not random noise: they follow misleading descriptions, prefer recently used tools and hallucinate arguments. The model ignores prompt caching, hybrid lexical search and the search_tools fallback. Use it to understand the recall and confusion trade-off, then measure your own router on labelled requests.
Read the full explanation
- Tool Selection at Scale: Why Hundreds of Tools Break Agents and How to Route (companion article)
- Supervisor or Swarm: When Multi-Agent Systems Actually Beat One Agent
- MCP Tool Poisoning: How One Malicious Description Hijacks an Agent
All AI in Production labs · Lab catalog
- Gorilla: Large Language Model Connected with Massive APIs
- ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs (ICLR 2024)
- Berkeley Function Calling Leaderboard
- Model Context Protocol specification: Tools
- OpenAI documentation: Function calling
- pgvector: open-source vector similarity search for Postgres
Views are my own and do not represent any employer, past or present.