Enable streaming

Stream stream = geminiLLM.generateStream(req);
stream.forEach(delta -> {
  if (delta.type() == DeltaType.TEXT) {
    sseSend(delta.text());
  }
});
Advertisement

Chunk sizes

Gemini emits chunks of ~10-50 characters. Not word-boundary aligned. Consumers should handle partial words.

Advertisement

Usage in final chunk

The last chunk carries token usage. Extract there, not from a separate call.