Why architecture matters here
Tool use fails on serialization (tools that could run in parallel done sequentially), on error handling (tools throw and the agent stalls), and on latency budgets (tools without timeouts hang the whole turn). Each is architectural.
The architecture: every piece explained
The top strip is the flow. Agent turn plans multiple tool intents. Parallel dispatch issues independent tool calls concurrently. Streaming result ships partial data as tools produce. Aggregator combines returns for the next agent turn.
The middle row is the safety. Error contract defines typed errors. Retry + backoff handles transient. Timeout budget caps wall-clock. Fallback tool lets the agent degrade gracefully.
The lower rows are governance. Observability tracks per-tool metrics. Governance applies policy + audit. Ops covers cost, safety, and reproducibility.
End-to-end flow
End-to-end: agent plans three tool calls (search, calculator, translate). Runtime dispatches all in parallel. Search streams; calculator returns; translate errors. Runtime retries translate; success. Aggregator combines. Agent proceeds. Total turn time: max(single tool) + a bit, not sum. Metrics show tool usage patterns. Governance logs approved calls.