StreamingTool interface

public interface StreamingTool extends Tool {
  Stream callStreaming(Map args);
}





















































































































































Advertisement

Runtime handles the stream

The runtime consumes deltas, feeds each into the model context, and the model can react before the tool finishes.

Advertisement

Bounded buffer

The stream is bounded so slow model consumption doesn't OOM your tool. Backpressure is natural.

Advertisement
Disclaimer  ·  Privacy  ·  Contact