Memory Eviction Policies in ADK Java, in depth: compaction, TTL, capacity caps and scored eviction for agent memory

Two problems: the prompt and the store

Start with what the framework does. The Runner appends every event to the session through your BaseSessionService. Each LLM request is built from those events. In current ADK Java releases (the classes ship in the 1.10 artifacts; check yours) an App can carry an EventsCompactionConfig. With compactionInterval and overlapSize set, the runner summarizes older invocations after a run; with tokenThreshold and eventRetentionSize both set, a request processor compacts within the invocation once the prompt passes the threshold, keeping the most recent events verbatim.

The important detail is what compaction does to storage: nothing. The compactor appends a new summary event whose actions carry the compacted time range, and request building substitutes that summary for the range. The original events remain in the session. Compaction is therefore prompt-level eviction. It bounds tokens per request but not bytes per session, and it is covered in depth in ADK Java context compression.

Storage-level eviction has a narrower toolbox. BaseSessionService offers listSessions(appName, userId) and deleteSession(appName, userId, sessionId), so you can drop whole sessions but not individual events. BaseMemoryService has exactly two methods, addSessionToMemory(Session) and searchMemory(appName, userId, query). There is no delete, no list and no expiry. A MemoryEntry carries only content(), author() and a string timestamp(): no id, no score, no access count. Every long-term eviction policy you want, you build.

Architecture

The design that follows has four parts. A record store holds each memory with eviction metadata the framework does not track. An EvictingMemoryService implements BaseMemoryService, so agents and the Runner use it unchanged. An EvictionPolicy decides which records die. A session sweeper retires idle sessions, ingesting them into memory first so nothing useful is lost with them.

The classic policies, applied to memory

Every cache textbook policy has an analogue in agent memory, but the costs differ. Evicting a cache line costs a reload. Evicting a memory can cost a wrong answer, because the agent no longer knows the user's allergy or their preferred delivery address, and it will not know that it forgot.

No single rule is safe on its own. Production systems combine a hard TTL for compliance, a pin flag for facts that must never be evicted automatically, and a capacity cap whose victims are chosen by a blended score. That blend is what the code below implements.

A blended score

The score of a record r at time now is a weighted sum of three terms, each scaled to the range zero to one:

recency(r)    = 0.5 ^ (hours since r.lastAccessed / halfLifeHours)
frequency(r)  = min(1, ln(1 + r.hits) / ln(1 + 100))
importance(r) = assigned at write time, 0..1

score(r) = 0.4 * recency + 0.2 * frequency + 0.4 * importance
victims  = expired records, then the lowest scores until count <= maxRecords
           and tokens <= maxTokens, skipping pinned records

Exponential recency means a record untouched for one half-life scores half as much on that term, with no cliff. The logarithm keeps frequency from dominating: the hundredth hit adds far less than the second, which limits the entrenchment problem of pure LFU. Importance gives new records a chance before they have any hits. Assign it with deterministic rules where you can, such as identity, health and commitments high and greetings low, and use an LLM classifier only for the remainder, caching its answer on the record so the score is reproducible.

The eviction wrapper in code

The record carries the entry plus the metadata eviction needs, including provenance so you can trace any memory back to its session:

public record MemoryRecord(
    String id, String appName, String userId, String sourceSessionId,
    MemoryEntry entry, Instant createdAt, Instant lastAccessedAt,
    long hits, double importance, boolean pinned, int tokens) {

  /** New records start one half-life "old", so creation does not count as a retrieval. */
  public static MemoryRecord fresh(String id, String app, String user, String sessionId,
      MemoryEntry entry, Instant now, Duration halfLife,
      double importance, boolean pinned, int tokens) {
    return new MemoryRecord(id, app, user, sessionId, entry, now, now.minus(halfLife),
        0, importance, pinned, tokens);
  }
}

public interface EvictionPolicy {
  boolean expired(MemoryRecord r, Instant now);
  List<String> victims(List<MemoryRecord> all, Instant now);   // ids to delete
}

public interface MemoryRecordStore {
  Completable upsert(List<MemoryRecord> records);
  Single<List<MemoryRecord>> search(String app, String user, String query, int limit);
  Single<List<MemoryRecord>> listForUser(String app, String user);
  Completable touch(List<String> ids, Instant at);   // lastAccessedAt = at, hits++
  Completable delete(List<String> ids);
}

The wrapper implements the real interface and enforces the policy on every write:

public final class EvictingMemoryService implements BaseMemoryService {
  private final MemoryRecordStore store;
  private final Function<Session, Single<List<MemoryRecord>>> extractor;
  private final EvictionPolicy policy;
  private final Clock clock;

  @Override
  public Completable addSessionToMemory(Session session) {
    return extractor.apply(session)
        .flatMapCompletable(store::upsert)
        .andThen(Completable.defer(() -> enforce(session.appName(), session.userId())));
  }

  @Override
  public Single<SearchMemoryResponse> searchMemory(String app, String user, String query) {
    Instant now = clock.instant();
    return store.search(app, user, query, 50)
        .map(rs -> rs.stream().filter(r -> !policy.expired(r, now)).limit(10).toList())
        .flatMap(rs -> store.touch(rs.stream().map(MemoryRecord::id).toList(), now)
            .onErrorComplete()   // a lost touch must never fail a user's turn
            .andThen(Single.just(SearchMemoryResponse.builder()
                .memories(rs.stream().map(MemoryRecord::entry).toList())
                .build())));
  }

  Completable enforce(String app, String user) {
    return store.listForUser(app, user)
        .map(rs -> policy.victims(rs, clock.instant()))
        .flatMapCompletable(ids -> ids.isEmpty() ? Completable.complete() : store.delete(ids));
  }
}

The policy is plain Java with no I/O, which makes it trivial to unit test with a fixed clock:

public final class HybridPolicy implements EvictionPolicy {
  private final Duration ttl = Duration.ofDays(365);
  private final double halfLifeHours = 24 * 30;
  private final int maxRecords = 500, maxTokens = 40_000;

  public boolean expired(MemoryRecord r, Instant now) {
    return !r.pinned() && r.createdAt().plus(ttl).isBefore(now);
  }

  double score(MemoryRecord r, Instant now) {
    double hours = Duration.between(r.lastAccessedAt(), now).toMinutes() / 60.0;
    double recency = Math.pow(0.5, hours / halfLifeHours);
    double freq = Math.min(1.0, Math.log1p(r.hits()) / Math.log1p(100));
    return 0.4 * recency + 0.2 * freq + 0.4 * r.importance();
  }

  public List<String> victims(List<MemoryRecord> all, Instant now) {
    List<String> out = new ArrayList<>();
    List<MemoryRecord> live = new ArrayList<>();
    for (MemoryRecord r : all) { if (expired(r, now)) out.add(r.id()); else live.add(r); }
    live.sort(Comparator.comparingDouble(r -> score(r, now)));
    int count = live.size();
    long tokens = live.stream().mapToLong(MemoryRecord::tokens).sum();
    for (MemoryRecord r : live) {
      if (count <= maxRecords && tokens <= maxTokens) break;
      if (r.pinned()) continue;
      out.add(r.id()); count--; tokens -= r.tokens();
    }
    return out;
  }
}

Two implementation notes matter. Touches are writes on the read path, so batch them or send them to a queue if your store is expensive to write; an approximate access time is fine for eviction. And enforce lists every record for the user, which is cheap at a 500-record cap but should become an indexed query on score if your caps are large.

Retiring idle sessions

Sessions need their own rule, because compaction never deletes events. A nightly sweeper lists each user's sessions, picks those idle past a threshold, fetches each one with its events, ingests it into memory, and only then deletes it. The order matters: listSessions responses may not include events, and a deleted session cannot be ingested.

Completable retireIdle(String app, String user, Duration idle) {
  Instant cutoff = clock.instant().minus(idle);
  return sessions.listSessions(app, user)
      .flatMapCompletable(resp -> Flowable.fromIterable(resp.sessions())
          .filter(s -> s.lastUpdateTime().isBefore(cutoff))
          .concatMapCompletable(s -> sessions
              .getSession(app, user, s.id(), Optional.empty())
              .flatMapCompletable(full -> memory.addSessionToMemory(full)
                  .andThen(sessions.deleteSession(app, user, s.id())))));
}

If a session has artifacts, delete them before the session or you lose the ability to enumerate them; the ordering problem is worked through in memory and privacy in ADK Java.

Worked example

A pharmacy support agent uses the policy above: 500 records or 40,000 tokens per user, a 30-day half-life, a one-year TTL. A long-standing customer has 498 records averaging 70 tokens, about 34,900 tokens. After a chat about a delayed order, the extractor produces six new records: the new delivery address (importance 0.8), a complaint about the courier (0.5), and four pleasantries it should have dropped (0.1 each). The user is now at 504 records, so four must go.

Scores at the bottom of the list: a note that the user once asked about opening hours, last read eight months ago, scores roughly 0.4 x 0.0 + 0.2 x 0.15 + 0.4 x 0.1, about 0.07. The four new pleasantries score 0.4 x 1.0 + 0 + 0.04, about 0.44, because they were just created. That is the flaw in naive recency: creation counts as access. The fix is to initialise lastAccessedAt for new records to the creation time minus one half-life, as MemoryRecord.fresh does, so a fresh record's recency starts at 0.5 and importance decides. With that change the pleasantries score 0.24, which still beats long-unread notes like the 0.07 one, so the four lowest-scoring old notes are evicted and the pleasantries survive until they age. That is why the extractor should drop small talk before it is ever stored. The customer's allergy record, pinned at creation, is never a candidate.

Failure modes

Trade-offs

Eviction trades recall for precision and cost. A small cap keeps retrieval sharp and cheap, but every evicted record is a question the agent can no longer answer. Summarization is the middle path: before evicting a cluster of low-score records, merge them into one summary record that keeps provenance to every source; memory summarization strategies covers how. It costs an LLM call and risks distortion, so reserve it for clusters rather than single records.

Eviction is also not erasure. A policy that removes records for relevance gives no guarantee that a specific user's data is gone from embeddings, summaries, logs or backups. Keep the two paths separate: eviction is tuned for quality and may change weekly, while erasure is a compliance process that must be complete, verified and auditable.

What to do next

1. Measure today: records and tokens per user, growth per week, and session event counts at the 99th percentile.
2. Configure EventsCompactionConfig on your App for prompt size, and stop expecting it to free storage.
3. Add a record store with id, provenance, timestamps, hits, importance, pin and token count.
4. Wrap it in an EvictingMemoryService and register it on the Runner.
5. Write deterministic importance and pin rules; unit test the policy with a fixed clock.
6. Start new records at half recency so creation does not count as use.
7. Ship the session sweeper with ingest-then-delete and a dry-run mode that logs victims first.
8. Track evictions per day and a forgot-something rate from user corrections, and tune the weights against it.