LangChain4j, in depth: AI Services, tools and memory, agentic workflows and supervisors, and running LangChain4j inside ADK Java

AI Services: an interface becomes an agent

An AI Service is a proxy. LangChain4j reads the annotations on your interface at build time and, on each call, turns the arguments into chat messages, sends them with the tool specifications to the model, executes any tool calls the model asks for, sends the results back, and repeats until the model returns a final answer, which it converts to the method's return type. String, enums, records and POJOs, Result<T> for metadata such as token usage, and TokenStream for streaming are all supported return types.

interface SupportAgent {
    @SystemMessage("You are a support agent for Acme. Use tools for any order data. "
                 + "Never promise a refund above the policy limit.")
    String chat(@MemoryId String customerId, @UserMessage String message);
}

ChatModel model = OpenAiChatModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("gpt-4o-mini")
        .temperature(0.0)
        .build();

SupportAgent agent = AiServices.builder(SupportAgent.class)
        .chatModel(model)
        .tools(new OrderTools(orderRepository))
        .chatMemoryProvider(id -> MessageWindowChatMemory.withMaxMessages(20))
        .maxToolCallingRoundTrips(8)          // default is 100
        .build();

String reply = agent.chat("cust-42", "Where is order 1001?");

@SystemMessage sets the instructions; templates use {{name}} variables bound with @V. @MemoryId selects which conversation the call belongs to, and the chatMemoryProvider creates one memory per id. This is already an agent in the useful sense: a model in a loop with tools and state.

Tools and the tool loop

A tool is a Java method annotated with @Tool. LangChain4j derives a JSON schema from the parameters, with @P supplying descriptions, defaults and whether a parameter is required. The model sees the description and schema, never your code, so write descriptions as instructions.

class OrderTools {
    private final OrderRepository repo;
    OrderTools(OrderRepository repo) { this.repo = repo; }

    @Tool("Look up an order's status, total and delivery date by order id")
    OrderView lookupOrder(@P("numeric order id, e.g. 1001") long orderId) {
        return repo.find(orderId).map(OrderView::from)
                   .orElseThrow(() -> new IllegalArgumentException("no such order"));
    }

    @Tool("Open a refund request; returns a ticket id. Only for delivered orders.")
    String openRefund(long orderId, @P("reason, one sentence") String reason) {
        return repo.openRefund(orderId, reason);
    }
}

Each model response that contains tool calls is one round trip. maxToolCallingRoundTrips caps them (the default is 100; the older maxSequentialToolsInvocations name is deprecated since 1.15.0), and exceeding it throws. Set it low, because a confused model can call tools far longer than any real task needs. When the model asks for several tools at once, they run sequentially unless you call executeToolsConcurrently(). ReturnBehavior.IMMEDIATE on a tool returns its result to the caller without another model call, which saves a round trip for terminal actions. Declare the method's return type as Result<T> when you use it: the tool output then arrives in toolExecutions() with empty content, and other return types fail with an exception in several cases.

Errors need a policy, and the two defaults differ. A malformed argument, such as invalid JSON or a missing required parameter, throws and ends the invocation; toolArgumentsErrorHandler can send the problem back to the model instead, which the documentation recommends. An exception thrown inside a tool is, by default, sent to the model as the tool's result via its message. That lets the model recover, but it means exception text written for developers, such as table names, paths or customer data, flows into the prompt, chat history and provider logs. Configure a toolExecutionErrorHandler that returns a sanitised message, and use hallucinatedToolNameStrategy for calls to tools that do not exist.

Memory per conversation

Chat memory is the window of past messages sent with each call. MessageWindowChatMemory keeps the last N messages; a token-window variant keeps a token budget instead. Both are in-memory by default; plug in a ChatMemoryStore to persist conversations in a database so they survive restarts and work across replicas.

One sentence in the documentation deserves a bold box in your design notes: an AI Service should not be called concurrently for the same @MemoryId, and LangChain4j does not prevent it. Two simultaneous requests from the same user, such as a double-clicked send button or a retry racing the original, can interleave messages and corrupt the conversation. Serialise per id with a lock, a single-threaded queue per conversation, or idempotency keys at your HTTP layer.

Agentic workflows over a shared scope

The langchain4j-agentic module (published as 1.21.0-beta31 at the time of writing, and documented as experimental and subject to change) treats each agent as an AI Service with an @Agent annotation and an output key. Agents in a system share an AgenticScope, a map of named variables: an agent's arguments are read from the scope by name, and its result is written back under its output key. Workflow agents compose them: sequenceBuilder, loopBuilder with an exit predicate and maxIterations, parallelBuilder with an optional executor, and a conditional builder that routes on scope state. Each agent is built with AgenticServices.agentBuilder(Classifier.class).chatModel(model).build().

interface Classifier {
    @UserMessage("Classify this support ticket as BILLING, SHIPPING or OTHER. "
               + "Answer with one word. Ticket: {{ticket}}")
    @Agent(description = "Classifies a support ticket", outputKey = "category")
    String classify(@V("ticket") String ticket);
}

interface Drafter {
    @UserMessage("Draft a reply to this {{category}} ticket: {{ticket}}")
    @Agent(description = "Drafts a reply", outputKey = "reply")
    String draft(@V("category") String category, @V("ticket") String ticket);
}

interface Reviewer {
    @UserMessage("Score 0.0-1.0 how well this reply follows policy: {{reply}}")
    @Agent(description = "Scores a draft reply", outputKey = "score")
    double score(@V("reply") String reply);
}

UntypedAgent reviewLoop = AgenticServices.loopBuilder()
        .subAgents(drafter, reviewer)
        .maxIterations(3)
        .exitCondition(scope -> scope.readState("score", 0.0) >= 0.8)
        .build();

UntypedAgent triage = AgenticServices.sequenceBuilder()
        .subAgents(classifier, reviewLoop)
        .outputKey("reply")
        .build();

String reply = (String) triage.invoke(Map.of("ticket", ticketText));

The supervisorBuilder is the pure agentic option: an LLM planner receives the sub-agents' descriptions, decides which to call next with which arguments, and stops when it judges the task done. Its SupervisorResponseStrategy is LAST by default, returning the last sub-agent's answer, with SUMMARY and SCORED as alternatives. Prefer workflows when you know the steps; reach for a supervisor only when the order genuinely depends on the request, and cap it.

Workflows also support an errorHandler that can retry, substitute a result or rethrow, and an AgentListener notified before and after every agent invocation, which is where tracing belongs.

Combining LangChain4j with ADK Java

Pattern A: a LangChain4j model as an ADK model. ADK's contrib module provides com.google.adk.models.langchain4j.LangChain4j, a BaseLlm that translates ADK's LlmRequest into a LangChain4j ChatRequest and back. ADK still owns the agent loop, sessions, callbacks and tool execution; LangChain4j only reaches the provider. The class is abstract, so build it as its own integration tests do.

// Maven: com.google.adk:google-adk-langchain4j (1.11.0 on Maven Central)
ChatModel claude = AnthropicChatModel.builder()
        .apiKey(System.getenv("ANTHROPIC_API_KEY"))
        .modelName("claude-sonnet-5-5")
        .build();

LlmAgent support = LlmAgent.builder()
        .name("support")
        .description("Answers order questions")
        .instruction("Use the tools for order data. Be brief.")
        .model(LangChain4j.builder()             // abstract AutoValue class: use the builder
                .chatModel(claude)
                .modelName("claude-sonnet-5-5")
                .build())
        .tools(FunctionTool.create(OrderFunctions.class, "lookupOrder"))
        .build();

Know the adapter's edges before shipping. Streaming requires a StreamingChatModel; with only a ChatModel, a streaming request fails with "StreamingChatModel is not configured". connect() throws, so ADK's live bidirectional mode is unavailable. Tool parameter schemas must be objects. A forced function-calling mode is mapped to LangChain4j's REQUIRED tool choice, with a TODO in the source questioning the mapping. For which generation settings survive the translation, see LLM abstraction boundaries in ADK Java, and for what the runtime expects of any model, the BaseLlm interface.

Pattern B: a LangChain4j service behind an ADK tool. Keep ADK's model and agent tree, and expose an existing LangChain4j AI Service, perhaps one with a tuned retrieval pipeline, as a static method wrapped in a FunctionTool. ADK sees one tool call; the inner service runs its own loop. Pass the ADK session id as the inner @MemoryId so conversations line up.

Worked example: one ticket through the pipeline

Follow one ticket through the triage system above: "I was charged twice for order 1001." The caller's map seeds the scope with ticket. The classifier reads ticket and writes category = BILLING. The loop begins: the drafter reads category and ticket and writes a reply; the reviewer reads reply and writes score = 0.55. The exit predicate is false, so the drafter runs again, overwriting reply, and the reviewer writes 0.86. The predicate is checked after each sub-agent by default, so the loop exits immediately, and the sequence returns the reply output key.

That run cost five model calls: one classification and two draft-and-score passes. The worst case is one plus three times two, or seven, and you can state it before deploying. A supervisor doing the same job has no such bound unless you impose one. If each drafter also used tools with up to eight round trips, the worst case would multiply again, which is why budgets belong on every layer.

Failure modes

• Runaway loops. Default round-trip limits are generous. Set maxToolCallingRoundTrips and maxIterations from measured needs, and alert when they are hit.
• Shared-memory races. Concurrent calls on one memory id corrupt the history; serialise them.
• Scope key collisions. Two agents writing the same output key silently overwrite each other. That is intended in a draft loop and a bug elsewhere; name keys deliberately and log them through a listener.
• Missing inputs. An agent whose argument is absent from the scope fails at invocation time. Mark genuinely optional agents optional, or seed defaults in the input map.
• Experimental APIs. The agentic module may change between releases. Pin versions with the LangChain4j BOM and keep workflow wiring in one class.
• Mismatched features in Pattern A. Settings ADK passes may be ignored by the translation; write a conformance test per provider.

Operational guidance

Running LangChain4j agents in production is mostly about making the hidden loop visible and bounded.

1. Trace every step. Register beforeToolExecution and afterToolExecution callbacks on AI Services and an AgentListener on the root of each agentic system, and emit one span per model call and per tool call with the memory id. Without this, a slow answer is a black box.
2. Set timeouts and retries on the model client. Provider builders such as OpenAiChatModel accept a timeout and a retry count; choose them so that the worst-case loop still fits inside your HTTP request deadline.
3. Authorise inside tools. The model decides which tool to call and with what arguments, so every tool must check that the current user may act on that order id. Never rely on the system prompt for access control.
4. Test with a fake model. Implement ChatModel with scripted responses, including tool calls, to unit-test tool wiring, error handlers and loop limits without network calls or cost.
5. Count tokens per conversation. Memory windows grow prompts on every turn; track usage from Result<T> and alert on outliers. In ADK, the equivalent hooks are ADK Java callbacks.

Trade-offs

For supervisor-style designs inside ADK itself, compare the hierarchical supervisor pattern in ADK Java.

What to do next

1. Write one AI Service with two tools against your real data and a low round-trip cap; log token usage through Result<T>.
2. Back its memory with a ChatMemoryStore and add per-memory-id serialisation before any load test.
3. If you need fixed multi-step behaviour, rebuild it as an agentic sequence or loop, compute its worst-case call count, and attach an AgentListener for traces.
4. If you run ADK, decide between Pattern A and Pattern B per agent, and add a conformance test for each LangChain4j-backed model.
5. Pin langchain4j, langchain4j-agentic and google-adk-langchain4j versions, and re-read the release notes before each upgrade.
6. Read implementing a custom LLM in ADK Java if the contrib adapter does not cover a feature you need.