Postgres event log storage for ADK Java, in depth: schema, the appendEvent transaction and running it in production

Why store events rather than a session blob

The simplest design serialises the whole Session into one row and overwrites it after each turn. It fails in three ways. Every append rewrites an ever-growing value, so write cost grows with conversation length. Two concurrent writers silently overwrite each other's events. And you lose history: you cannot answer "what did the agent see before it called that tool?" once the blob has been replaced.

An append-only log fixes all three. Each event is an immutable row with a sequence number. State becomes a derived value you update in the same transaction as the insert. Audit, replay and debugging read the log directly. This is the same reasoning behind the transactional outbox pattern: write the fact and its consequences atomically, then let everything else read facts.

Where the store sits

The Runner calls the session service after each event an agent produces. Your implementation turns that call into one database transaction, and only after it commits does it update the in-memory Session object the runner is holding.

The interface is reactive. appendEvent returns Single<Event>, getSession returns Maybe<Session> and deleteSession returns Completable, all RxJava 3 types. JDBC is blocking, so wrap each database call in Single.fromCallable(...) and subscribe on Schedulers.io() rather than blocking whichever thread the runner happens to use. For how the session and its state reach the model, see the session context deep dive.

The schema

CREATE TABLE adk_sessions (
    app_name    text        NOT NULL,
    user_id     text        NOT NULL,
    session_id  text        NOT NULL,
    state       jsonb       NOT NULL DEFAULT '{}',
    last_seq    bigint      NOT NULL DEFAULT 0,
    updated_at  timestamptz NOT NULL DEFAULT now(),
    PRIMARY KEY (app_name, user_id, session_id)
);

CREATE TABLE adk_events (
    app_name      text   NOT NULL,
    user_id       text   NOT NULL,
    session_id    text   NOT NULL,
    seq           bigint NOT NULL,          -- position in this session's log, assigned under the row lock
    event_id      text   NOT NULL,
    invocation_id text,
    author        text   NOT NULL,
    event_ts      bigint NOT NULL,          -- Event.timestamp() as returned; do not assume a unit
    created_at    timestamptz NOT NULL DEFAULT now(),   -- server time, for retention and partitioning
    payload       jsonb  NOT NULL,          -- Event.toJson()
    PRIMARY KEY (app_name, user_id, session_id, seq),
    UNIQUE (app_name, user_id, session_id, event_id),
    FOREIGN KEY (app_name, user_id, session_id)
        REFERENCES adk_sessions ON DELETE CASCADE
);

CREATE TABLE adk_user_state (app_name text, user_id text, state jsonb NOT NULL DEFAULT '{}',
                             PRIMARY KEY (app_name, user_id));
CREATE TABLE adk_app_state  (app_name text PRIMARY KEY, state jsonb NOT NULL DEFAULT '{}');

A few choices are deliberate. seq is a per-session counter, not a global identity column, because the property you need is a gap-free order within a session; global identity values can commit out of order across transactions. The primary key (app_name, user_id, session_id, seq) doubles as the index for the main read path, a range scan of one session in order. The unique constraint on event_id makes a retried append fail loudly instead of duplicating an event. Treat that conflict as success on retry; the idempotency article covers retry-safe design more broadly.

payload holds Event.toJson() and is read back with Event.fromJson, so the stored format tracks the library's own serialisation rather than a mapping you maintain. jsonb costs a little on write but lets you query into events during an incident. Large values are compressed and moved out of line by Postgres's TOAST mechanism automatically. The few fields you filter on (author, invocation id, timestamp) are copied into real columns so queries never need to parse JSON.

event_ts stores whatever Event.timestamp() returns as a bigint. We have not confirmed its unit from the Java reference, so check it in your ADK version before comparing it with wall-clock values.

State scopes and the delta rules

ADK distinguishes state by key prefix. Keys without a prefix belong to the session. Keys with the user: prefix are shared by all of one user's sessions in an app. Keys with app: are shared by every user of the app. Keys with temp: are, in the documentation's words, "Not Persistent": they exist for the current invocation only. In Java, use the constants State.APP_PREFIX, State.USER_PREFIX and State.TEMP_PREFIX rather than string literals.

Deletion needs care. A delta can map a key to State.REMOVED, a sentinel object meaning "remove this entry". If you serialise the delta naively, that sentinel becomes some JSON value in your database and the key is never removed. Route removed keys to a separate delete set and apply them with the jsonb - text[] operator; apply the remaining keys with ||, which merges top-level keys.

The append transaction

// Excerpt: the other BaseSessionService methods and the SQL helpers are omitted.
public final class PostgresSessionService implements BaseSessionService {
  private final DataSource ds;

  @Override
  public Single<Event> appendEvent(Session session, Event event) {
    if (event.partial().orElse(false)) {          // design choice: streaming fragments are not logged
      return Single.just(event);
    }
    return Single.fromCallable(() -> { persist(session, event); return event; })
        .subscribeOn(Schedulers.io())
        // only after COMMIT succeeds do we update the caller's in-memory Session
        .flatMap(e -> BaseSessionService.super.appendEvent(session, e));
  }

  private void persist(Session s, Event e) throws Exception {
    Map<String, Object> sessionDelta = new HashMap<>(), userDelta = new HashMap<>(), appDelta = new HashMap<>();
    Set<String> sessionDel = new HashSet<>(), userDel = new HashSet<>(), appDel = new HashSet<>();
    for (Map.Entry<String, Object> d : e.actions().stateDelta().entrySet()) {
      String k = d.getKey();
      if (k.startsWith(State.TEMP_PREFIX)) continue;                 // never persisted
      boolean removed = d.getValue() == State.REMOVED;               // sentinel: delete, never serialize
      if (k.startsWith(State.APP_PREFIX)) { route(k, d.getValue(), removed, appDelta, appDel); }
      else if (k.startsWith(State.USER_PREFIX)) { route(k, d.getValue(), removed, userDelta, userDel); }
      else { route(k, d.getValue(), removed, sessionDelta, sessionDel); }
    }
    try (Connection c = ds.getConnection()) {
      c.setAutoCommit(false);
      try {
        long seq = lockAndNextSeq(c, s);            // SELECT last_seq ... FOR UPDATE; throws if session is gone
        insertEvent(c, s, e, seq);                  // INSERT INTO adk_events (..., payload) VALUES (..., ?::jsonb)
        mergeState(c, "adk_sessions", s, sessionDelta, sessionDel, seq);  // state = (state - ?::text[]) || ?::jsonb
        mergeState(c, "adk_user_state", s, userDelta, userDel, seq);      // INSERT ... ON CONFLICT DO UPDATE
        mergeState(c, "adk_app_state", s, appDelta, appDel, seq);
        c.commit();
      } catch (Exception ex) {
        c.rollback();
        throw ex;
      }
    }
  }

  private static void route(String k, Object v, boolean removed, Map<String, Object> put, Set<String> del) {
    if (removed) del.add(k); else put.put(k, v);
  }
}

The order of operations matters. Persist first, then call BaseSessionService.super.appendEvent(session, event), the interface's default method, which appends to the in-memory session and applies the delta there. (Because the default lives on an interface, plain super.appendEvent does not compile.) If you mutate memory first and the commit fails, the running agent reasons over an event that does not exist in the database, and the next instance to load the session sees a different history.

Partial events, the incremental fragments of a streamed model response, are skipped here as a design choice: storing every fragment multiplies write volume and the final event carries the complete content anyway. We did not confirm whether the default appendEvent already filters them, so the check is explicit.

Concurrency: one writer per session at a time

Two invocations can target the same session: a user double-submits, or a retry overlaps a slow original. SELECT last_seq FROM adk_sessions ... FOR UPDATE serialises appends per session while leaving other sessions untouched, and then seq = last_seq + 1 is safe. The primary key on seq is the backstop: if a bug ever assigns a sequence without the lock, the insert fails instead of forking history.

The lock does not solve a stale in-memory session. If instance A loaded the session at seq 10 and instance B has since appended seq 11 and 12, A's model call was built from an old view. A cheap guard is to carry the last_seq you loaded, compare it with the locked value, and reject the append with a retryable error when they differ. Whether to reject or accept is a product decision; for most chat agents, rejecting and re-running the turn is safer than appending a reply to a question the user already superseded.

Reads: getSession and GetSessionConfig

getSession takes an optional GetSessionConfig with numRecentEvents() and afterTimestamp(). Map them straight onto SQL so the database, not the JVM, discards what you do not need:

-- getSession: events for one session, honouring GetSessionConfig
SELECT payload::text
FROM (
    SELECT seq, payload
    FROM adk_events
    WHERE app_name = ? AND user_id = ? AND session_id = ?
      AND (?::bigint IS NULL OR event_ts > ?)     -- afterTimestamp, converted to the unit your events use
    ORDER BY seq DESC
    LIMIT ?                                        -- numRecentEvents; binding NULL means no limit
) recent
ORDER BY seq ASC;

Build the returned session with Session.builder(id), setting app name, user id, events and last update time, and a state map that merges three sources: the session row's state, the user row's keys and the app row's keys. Keep the prefixes on user and app keys when merging, so a later delta routes back to the correct table. Return Maybe.empty() for a missing session rather than an error; callers use that to decide whether to create one.

Growth, retention and compaction

An event log only grows. Estimate size as events per turn multiplied by turns per session, sessions per day and average payload; tool results that embed documents dominate payload size, so measure your own. Three levers control growth.

• Retention. Delete whole sessions past an age limit through the ON DELETE CASCADE key, in batches, off-peak. Deleting single events from the middle of a log breaks the gap-free sequence and your replay assumptions.
• Time partitioning. At high volume, range-partition adk_events by month of created_at so expiry becomes dropping a partition, which avoids the bloat and vacuum cost of mass deletes. Partitioned tables require the partition key in every unique constraint, so plan the key before the first migration.
• Compaction. EventActions has a compaction() accessor for summary events. If your agents emit them, you can load the latest compaction plus the events after it instead of the whole history; keep the raw rows if audit requires them.

Append-only tables used to be neglected by autovacuum because they produce no dead rows. Since PostgreSQL 13, inserts also trigger vacuum (the autovacuum_vacuum_insert_threshold settings), which keeps the visibility map current for index-only scans. Check it is enabled on large event tables.

Operating it

• Pool sizing. Each append holds a connection for a few statements plus network round trips. Size the pool for concurrent appends, not for concurrent sessions; if you run agents on virtual threads, the pool, not the thread count, becomes the real concurrency limit.
• Timeouts. Set statement_timeout and lock_timeout for the service's role so a stuck session lock surfaces as an error within seconds instead of piling up waiting invocations.
• Metrics. Record append latency, lock wait time, rows per session (a runaway agent loop shows up here first), rejected stale appends and payload size percentiles. Tag them with app name; the observability article covers trace trees and telemetry for ADK Java.
• Privacy. Events contain user messages and tool outputs verbatim. Encrypt at rest, restrict read access to the service role, and make session deletion reach backups within your retention policy.

Failure modes

What to do next

1. Confirm the BaseSessionService signatures and the Event.timestamp() unit for the ADK Java version you run.
2. Create the four tables with a migration tool and load-test appends against a copy of production-sized payloads.
3. Implement appendEvent with the lock, sequence, scoped deltas, REMOVED handling and persist-then-mutate order.
4. Write tests for concurrent appends to one session, a retried append, a removed key and a temp key.
5. Map GetSessionConfig onto SQL and measure getSession latency at 10, 100 and 1,000 events.
6. Decide retention and partitioning before launch, and dashboard append latency, lock waits and events per session.