The four golden signals per tool

  • tool.calls — count.
  • tool.latency — histogram.
  • tool.errors — count, tagged by error_kind.
  • tool.cost — sum, for tools with $ cost.
Advertisement

Attach via wrapper

public class MeteredTool implements Tool {
  public Object call(Map args) {
    var sample = Timer.start();
    try {
      var r = inner.call(args);
      counter("tool.calls", tag("name", inner.name())).increment();
      return r;
    } catch (Exception e) {
      counter("tool.errors", tag("name", inner.name()), tag("kind", kind(e))).increment();
      throw e;
    } finally {
      sample.stop(timer("tool.latency", tag("name", inner.name())));
    }
  }
}
Advertisement

Correlation IDs

Every tool call carries a session_id + turn. Emit as tag / label. Correlate metrics + logs + traces.