REST API Design, in depth: HTTP method semantics, status codes, caching and validators, conditional writes, PATCH formats and long-running operations

What REST actually constrains

REST is the architectural style Roy Fielding described in his 2000 dissertation: client-server separation, stateless requests, cacheable responses, a uniform interface, a layered system and optional code-on-demand. Statelessness means each request carries everything needed to process it, so any server instance can handle it and any intermediary can reason about it. The uniform interface means a small fixed set of methods with agreed meanings instead of a new verb per operation.

The payoff is the layered system: components that know nothing about your business still do useful work. A cache serves a GET because GET is safe. A client resends a PUT after a dropped connection because PUT is idempotent. A gateway returns 429 with Retry-After and clients know how to wait. The rule that follows: never say something in HTTP that is not true, because something downstream will believe it.

Method semantics are a contract with intermediaries

RFC 9110 (HTTP Semantics, 2022) defines three properties per method. Safe means the client does not request a state change, so crawlers, link prefetchers and caches may issue the request freely. Idempotent means sending the request N times has the same intended effect as sending it once, so a client or proxy may resend it automatically after a connection failure. Cacheable means a response may be stored and reused.

Idempotency is about the intended effect, not the response: a retried DELETE that returns 404 is still idempotent. What breaks the contract is a PUT whose effect depends on current state, such as "append this line", or a GET that changes data. A classic incident: GET /orders/42/cancel behind a link in an email, which link scanners then follow, cancelling real orders.

When an operation is naturally not idempotent (create a payment, send a message), keep POST and make retries safe with an idempotency key that the server stores alongside the result. The full design of the key store is in Idempotency architecture.

Status codes clients can branch on

A status code is the one part of a response every client, proxy and dashboard understands. Treat each code as a branch the caller will take, and choose the code by what the caller should do next, not by how the failure felt on the server.

RFC 9110 now defines 422 as Unprocessable Content, so it is fine for validation errors. For the body, use Problem Details (RFC 9457, which replaced RFC 7807): application/problem+json with type (a URI naming the problem kind, the stable thing clients switch on), title, status, detail, instance and extension members such as invalid fields. The most common mistake is 200 with {"error": ...} in the body: every cache, retry policy and alert then treats the failure as success.

Caching and validators

HTTP caching has two parts: freshness (reuse without asking the origin) and validation (a cheap conditional request to check a stale copy). max-age=60 allows reuse for 60 seconds. s-maxage overrides that for shared caches only. private forbids shared caches from storing the response. no-cache allows storing but requires revalidation before every reuse, which is different from no-store, which forbids storing at all.

Validators make revalidation cheap. The origin sends an ETag (an opaque version identifier, quoted, such as "a1f3") or Last-Modified. The client later sends If-None-Match: "a1f3"; if nothing changed, the server answers 304 with no body. A weak ETag (W/"a1f3") claims only semantic equivalence; it is fine for revalidating a GET but useless for conditional writes, because If-Match requires strong comparison.

Two traps cause most caching incidents. A response that varies by a request header must say so with Vary (for example Vary: Accept-Language), or a cache serves the French body to an English client. And per-user responses must be private or no-store: RFC 9111 protects requests carrying an Authorization header, but not cookies, API keys in custom headers or CDN overrides. Cache layers and CDNs are covered in Caching: CDN, application and database.

import hashlib, json
from fastapi import FastAPI, HTTPException, Request, Response
from fastapi.responses import JSONResponse

app = FastAPI()
ITEMS = {"sku-42": {"sku": "sku-42", "name": "Kettle", "price": 3900, "stock": 12}}


def etag_of(item: dict) -> str:
    """Strong ETag: a hash of the canonical representation."""
    body = json.dumps(item, sort_keys=True, separators=(",", ":")).encode()
    return '"' + hashlib.sha256(body).hexdigest()[:20] + '"'


@app.get("/items/{sku}")
def get_item(sku: str, request: Request):
    item = ITEMS.get(sku)
    if item is None:
        raise HTTPException(status_code=404)
    tag = etag_of(item)
    headers = {"ETag": tag, "Cache-Control": "private, no-cache"}
    # If-None-Match uses weak comparison: ignore any W/ prefix.
    sent = [t.strip().removeprefix("W/") for t in request.headers.get("if-none-match", "").split(",")]
    if tag in sent or "*" in sent:
        return Response(status_code=304, headers=headers)
    return JSONResponse(item, headers=headers)

Hashing the representation is the simplest correct ETag; a version column incremented on every write is cheaper and also serves conditional writes.

Conditional writes stop lost updates

Two clients read sku-42 at version 7; one changes the price, the other the stock, and both PUT the full item. The second write silently restores the old price. HTTP's fix: the client sends the ETag it read in If-Match, and the server writes only if it still matches, otherwise 412 Precondition Failed. To force every client to do this, answer unconditional writes with 428 Precondition Required (RFC 6585).

The check and the write must be one atomic step. Comparing the ETag in the handler and then writing leaves a window in which another request can commit. Push the comparison into the database: UPDATE items SET price = ?, version = version + 1 WHERE sku = ? AND version = ?, and treat zero updated rows as 412.

PUT, JSON Merge Patch and JSON Patch

PUT replaces the whole representation: idempotent and simple, but the client must send every field, including ones it does not understand, or erase them. PATCH sends only a change, in a media type you choose and advertise with the Accept-Patch response header (RFC 5789). There are two standard formats.

JSON Merge Patch (RFC 7396, application/merge-patch+json) looks like the resource: fields present are set, null deletes a field, and arrays are replaced whole. It is easy to write by hand and idempotent. Its limits are that you cannot set a field to null (null means delete), and you cannot change one element of an array. JSON Patch (RFC 6902, application/json-patch+json) is a list of operations (add, remove, replace, move, copy, test) applied in order, all or nothing. It can edit array elements and its test operation acts as a field-level precondition, but positional array edits are not idempotent and are fragile if the array changed.

The same change in each format: Merge Patch {"price": 3500, "promo": null}; JSON Patch [{"op": "replace", "path": "/price", "value": 3500}, {"op": "remove", "path": "/promo"}].

def merge_patch(target, patch):
    """RFC 7396 apply algorithm."""
    if not isinstance(patch, dict):
        return patch
    result = dict(target) if isinstance(target, dict) else {}
    for key, value in patch.items():
        if value is None:
            result.pop(key, None)
        else:
            result[key] = merge_patch(result.get(key), value)
    return result


def problem(status, kind, title, headers=None, **extra):
    body = {"type": f"https://api.example.com/problems/{kind}", "title": title, "status": status, **extra}
    return JSONResponse(body, status_code=status, headers=headers, media_type="application/problem+json")


@app.patch("/items/{sku}")
async def patch_item(sku: str, request: Request):
    if request.headers.get("content-type", "").split(";")[0].strip() != "application/merge-patch+json":
        return problem(415, "unsupported-patch", "Use JSON Merge Patch",
                       headers={"Accept-Patch": "application/merge-patch+json"})
    item = ITEMS.get(sku)
    if item is None:
        return problem(404, "not-found", "No such item")
    sent = request.headers.get("if-match")
    if sent is None:
        return problem(428, "precondition-required", "Send If-Match with the ETag you read")
    if sent.strip() != etag_of(item):          # strong comparison
        return problem(412, "stale-etag", "Item changed since you read it")
    updated = merge_patch(item, await request.json())
    if updated.get("sku") != sku or not isinstance(updated.get("price"), int) or updated["price"] < 0:
        return problem(422, "invalid-item", "Patch produces an invalid item")
    ITEMS[sku] = updated   # production: UPDATE ... WHERE version = ? in one statement
    return JSONResponse(updated, headers={"ETag": etag_of(updated)})

Default to Merge Patch; add JSON Patch only when clients must edit inside large arrays. Validate the patched result, not the patch, because a valid patch can still produce an invalid resource.

Work that outlives a request

Some operations take minutes. Holding the connection open invites gateway timeouts, and a client that sees a 504 cannot tell whether the work happened. The HTTP pattern is to answer 202 Accepted with a Location header pointing at an operation resource, optionally with Retry-After as a polling hint. The client GETs the operation until it reports a terminal state. On success, the operation either embeds the result or links to it; answering the poll with 303 See Other and the result URL in Location is a common convention.

POST /catalog/reprice            Idempotency-Key: 5b0e...   {"category": "kettles", "pct": -10}
-> 202 Accepted   Location: /operations/op_7f3   Retry-After: 5

GET /operations/op_7f3
-> 200 {"id": "op_7f3", "status": "running", "done": 412, "total": 1000}

GET /operations/op_7f3
-> 200 {"id": "op_7f3", "status": "succeeded", "result": "/reprice-reports/r_19"}

Give the operation resource a state machine (queued, running, succeeded, failed, cancelled), a retention period and, if supported, a cancel action. Make the initial POST idempotent with a key, so a client that lost the 202 gets the same operation back instead of starting a second reprice.

Worked trace: one item, two editors, one cache

1. Alice's client sends GET /items/sku-42 and receives 200 with ETag: "9c1e". Later revalidations with If-None-Match: "9c1e" cost one round trip and a bodiless 304.
2. Bob reads the same ETag and sends a Merge Patch setting stock to 11 with If-Match: "9c1e". The update succeeds; the response carries ETag: "4d70".
3. Alice sends a Merge Patch setting price to 3500 with If-Match: "9c1e". The database updates zero rows, so she gets 412 with a Problem Details body of type stale-etag. Her client re-reads (200, ETag 4d70, stock 11), reapplies the price change and succeeds. Bob's stock change survives.
4. An admin starts a category reprice: 202 with Location /operations/op_7f3. Gateway timeouts no longer matter, because the client polls the operation.
5. Under load the gateway answers some polls with 429 and Retry-After, and the SDK waits. Limiter design is in Rate limiter architecture.

Failure modes

• State change on GET. Prefetchers, crawlers and link scanners trigger it. Move every mutation to POST, PUT, PATCH or DELETE.
• 200 with an error body. Retries, caches and alerts all see success. Return the real status and Problem Details.
• Missing Vary or private. A shared cache serves one user's response to another. Audit every cacheable response for the headers it varies on.
• Check-then-write ETags. The precondition is compared in application code and the write races past it. Compare and write in one database statement.
• Synchronous long work. Gateway 504s leave callers unsure whether the work ran. Use 202 with an operation resource.

What to do next

1. List every endpoint with its method and check it against the safe and idempotent table; move any mutating GET and any non-idempotent PUT.
2. Replace 200-with-error responses with real status codes and an application/problem+json body whose type clients can switch on.
3. Set an explicit Cache-Control on every response, add Vary where representations differ, and mark per-user data private.
4. Add a version column to every writable resource, return it as a strong ETag, and enforce If-Match with 412 and 428 in one atomic UPDATE.
5. Pick JSON Merge Patch as the default PATCH format, advertise it with Accept-Patch and return 415 for anything else.
6. Convert any endpoint that can take more than a few seconds into 202 plus an operation resource with an idempotent POST.