Agent and AI engineering · Practical technical guide · 19 October 2026
Codex added observation-only PreModelRequest and PostModelRequest hooks on 18 October 2026. What they carry, why observers cannot veto or alter a request, how sync and async hooks order against tool dispatch, and how to build per-attempt metrics on them.
Two kinds of hook, and why the distinction matters
Agent harnesses expose hooks so that operators can run their own code at points in the agentic loop. Hooks come in two kinds that are easy to confuse. Control hooks may change the run: a PreToolUse hook that denies a shell command, a hook that injects context, a stop hook that forces another turn. Observation hooks may only watch: their output is recorded, never interpreted. The two need different guarantees. A control hook must be waited for and its failure mode decided, because a timed-out gate that silently allows is a security bug. An observation hook must never be allowed to affect the run, because its only purpose is to measure it.
On 18 October 2026 the Codex repository merged a pair of observation hooks around model requests, a useful specimen because the commit had to answer every question this guide raises.
What the Codex model request events carry
The two events are PreModelRequest and PostModelRequest. Both fire around each model request attempt, including retries, so a turn that retries a rate-limited request produces two pre and two post events. The input every handler receives is a JSON object with the session id, the turn id, the working directory, the transcript path when one exists, the model slug, and, for a request made by a subagent, the agent id and type. The post event adds a status of completed, failed or cancelled, an optional error string, and a token usage object that is always present in the schema even when its value is null, because usage is the point of a post-request observer.
| Field | Pre | Post | Why an observer wants it |
|---|---|---|---|
| session_id, turn_id | yes | yes | Join events to a turn; count attempts per turn |
| model | yes | yes | Attribute latency and tokens to a model |
| agent_id, agent_type | if a subagent | if a subagent | Separate subagent traffic from the root session |
| cwd, transcript_path | yes | yes | Correlate with a project; sensitive, see below |
| status | no | completed, failed or cancelled | Failure and cancellation rates |
| error | no | when failed | Group failures by cause |
| token_usage | no | always present, may be null | Cost per attempt, including incomplete responses |
The status mapping in the turn loop is precise. If the response stream completed, the status is completed and the usage is the final usage. If the stream ended in an error other than a turn abort, the status is failed with the error text. Anything else, including a user interrupt, is cancelled. If the request failed before a stream was even opened, the post event fires with failed and no usage. An incomplete response that the provider cut short still reports the usage it managed to count.
Observation only: output is diagnostic, never control
The commit states the rule in a one-line module comment: output is logged, never interpreted as control flow. Concretely, a handler's standard output is written to a debug log even if it is JSON that looks like a decision directive; standard error becomes error entries attached to the hook run; a non-zero exit code marks the hook run as failed. None of these can veto the request, change the prompt, add context or alter the model's view of the conversation. The integration tests in the same commit cover exactly the cases a sceptic would ask about: a handler that fails, a request that is retried, control directives that must be ignored, cancellation, and token accounting after an interrupted turn is resumed.
Compare the three harnesses. Codex's own PreToolUse can block or rewrite a call, and its documentation warns that a callback error, timeout or malformed response fails the hook without blocking the tool. Gemini CLI runs hooks synchronously and gives BeforeModel and AfterModel the power to block the turn, mock a response or redact one; an exit code of 2 is a critical block and stderr is the reason. Claude Code has no model-request hook at all; its PreModelSwitch and PostModelSwitch fire around a model change, a hook exits with code 2 to block an action on events that can block, and an onFailure setting decides whether a failed hook blocks. Codex chose to make the model request events the one place where no such power exists.
Ordering: sync and async post hooks and the tool dispatch race
The interesting engineering is in when the loop waits. A model response often ends with tool calls, and Codex can start dispatching those tools while the response is still streaming. If a post-request observer is configured as synchronous, the loop defers tool dispatch until the hook has finished, so that an auditor can be sure the record of the request exists before any tool runs on its account. If the hook is asynchronous, streaming tool dispatch continues and the hook completes in the background. The loop sets this up before the request is sent: it creates the post-request task, marks whether dispatch must be deferred, and only then starts the sampling timer, so hook time is not counted as model latency. An interrupt cancels the wait, not the hook itself, which still runs to completion and records usage.
Two further rules keep observers out of the way: internal memory-consolidation sessions get a filtered hook set, so housekeeping is not counted as user traffic unless a handler was declared for it, and the pre hook runs before the request is attempted, so a failed connection still produces a matched pair.
Building metrics on an observation hook
With these guarantees an observer is a small program that reads one JSON object on standard input and emits a measurement. The example below is the shape of such a handler: it never writes to standard output except for its own diagnostics, it treats a missing field as a value to record rather than a reason to fail, and it joins pre and post events on the turn id.
Illustrative python, checked for syntax; adapt names and storage to your system.
import json, sys, time, pathlib
STATE = pathlib.Path.home() / ".cache" / "model-request-hook"
def main():
event = json.load(sys.stdin)
name = event["hook_event_name"]
turn = event["turn_id"]
STATE.mkdir(parents=True, exist_ok=True)
marker = STATE / f"{turn}.start"
if name == "PreModelRequest":
marker.write_text(str(time.time()))
return
started = float(marker.read_text()) if marker.exists() else None
usage = event.get("token_usage") or {}
record = {
"turn": turn, "model": event["model"], "status": event["status"],
"agent_type": event.get("agent_type", "root"),
"latency_s": None if started is None else round(time.time() - started, 3),
"input_tokens": usage.get("input_tokens"), "output_tokens": usage.get("output_tokens"),
"error": event.get("error"),
}
with open(STATE / "requests.jsonl", "a", encoding="utf-8") as out:
out.write(json.dumps(record) + "\n")
print(json.dumps(record), file=sys.stderr) # diagnostics go to stderr by convention
if __name__ == "__main__":
main()Register it under both events; in Codex that is a PreModelRequest and a PostModelRequest matcher group beside the existing PreToolUse and Stop groups. Then derive four metrics: attempts per turn, which exposes retry storms; latency per attempt by model; tokens per attempt, including incomplete ones; and the failed and cancelled rates. A runnable version with tests is in agent-decision-lab.
Worked example: a retry storm that the transcript hides
Take a hypothetical team whose token bill doubles while completed tasks stay flat. Transcripts look normal, because a transcript records the final response of each turn, not the attempts. An observer on the model request events shows the cause within a day: one project averages 3.4 attempts per turn against a fleet average of 1.1, every extra attempt ends failed with a context-length error, and the failed attempts still report large input token counts. A context file had grown past the model's window; each turn hit the limit, the harness retried after compaction, and only the last attempt succeeded. The fix was in the project, and no hook had to touch a request to find it.
Failure modes worth testing
- An observer that blocks. If the harness honours any output from an observation hook, a logging bug becomes a run-control bug. Test that a handler printing a deny directive changes nothing.
- Counting attempts as turns. Each retry fires the pair again. Join on the turn id and count attempts separately, or costs will look like extra work.
- Slow synchronous post hooks. A sync observer delays every tool dispatch by its own runtime. Use sync only for the audit case and keep the handler under a few hundred milliseconds.
- Hook latency inside model latency. Measure from the pre event, not from the turn start, and check that the harness starts its own sampling timer after the pre hook.
- Sensitive fields. The input carries the working directory and transcript path; a pipeline that ships those to a third party has leaked project names at least.
- Cancelled without usage. A cancelled attempt may carry null usage. Record it as unknown, not zero, or cost estimates drift.
Observation at the API layer instead
The provider API is the other place to observe. On 14 October 2026 the Claude Platform release notes announced OpenTelemetry export in beta for Messages API requests and Managed Agents sessions, with spans emitted by the SDKs and a destination configured in the console. That path sees every request from every client, but not the harness view of a turn, its retries and the tool dispatch that followed. The two are complementary: harness hooks for attempt-level loop behaviour, API-level tracing for fleet-wide accounting. The agent tracing guide covers span structure across a plan-execute-reflect loop.
What to do next
- Classify your harness hooks as control or observation and write the failure behaviour of each down: a control hook needs a timeout policy, an observation hook a guarantee of no effect.
- Install an observer on model request events and log attempts per turn for a week before changing anything; then add the four metrics to a dashboard by model and agent type.
- Review what fields leave the machine; drop the working directory and transcript path unless the pipeline is internal.
- If the harness has no model request hook, use API-level tracing and accept that retries are invisible there.
Sources and scope
Checked against these sources on 19 October 2026. Examples and figures are illustrative, not measurements.
Separate observation hooks from control hooks, join pre and post events on the turn id, count attempts not turns, and never let an observer output change the run.