Extend BaseLLM
public class OllamaLLM implements BaseLLM {
private final HttpClient http;
private final String modelName;
private final String endpoint;
public OllamaLLM(String endpoint, String modelName) {
this.endpoint = endpoint;
this.modelName = modelName;
this.http = HttpClient.newHttpClient();
}
}Advertisement
Implement generate
public ModelResponse generate(GenerateRequest req) {
String body = toOllamaFormat(req);
HttpResponse resp = http.send(
HttpRequest.newBuilder(URI.create(endpoint + "/api/chat"))
.POST(HttpRequest.BodyPublishers.ofString(body))
.build(),
HttpResponse.BodyHandlers.ofString());
return fromOllamaFormat(resp.body());
} Advertisement
Implement info honestly
Return actual capabilities. If your model doesn't support tool calls, don't say it does. The runtime will call code paths that break.