HMAC, in depth: the ipad and opad construction from scratch, RFC 4231 test vectors, key rules, constant-time verification, webhook signing and key rotation

Why not just hash the key and the message

The obvious construction is tag = H(key || message). With SHA-256 it is broken: SHA-256 is a Merkle-Damgard hash, so its output is its full internal state, and anyone who sees a tag can keep hashing from it to produce a valid tag for the message plus padding plus any suffix, without knowing the key. That is the length-extension attack, worked through in the SHA family article. The reverse, H(message || key), avoids extension but turns any collision in H into a forgery: if two equal-length messages collide in H, they share a tag, and the attacker can search for collisions offline without the key.

HMAC fixes both by hashing twice with two keys derived from one. The inner hash absorbs the message under one derived key; the outer hash compresses that digest under the other. The attacker never sees the inner state, so there is nothing to extend, and the security proof (Bellare, CRYPTO 2006) needs only that the compression function behaves as a pseudorandom function, not that the hash is collision resistant. That is why HMAC-MD5 and HMAC-SHA-1 were not broken by the collision attacks on MD5 and SHA-1, though no new design should use either.

The construction, built and tested

The full definition is HMAC(K, m) = H((K0 XOR opad) || H((K0 XOR ipad) || m)). K0 is the key adjusted to the hash's block size B, which is 64 bytes for SHA-256 and 128 bytes for SHA-512: a key longer than B is first hashed, and the result is padded with zero bytes to B. The constants ipad and opad are the bytes 0x36 and 0x5c repeated B times; they only need to differ in many bits so that the two derived keys are unrelated. The implementation below is nine lines and matches both the RFC 4231 test vector and Python's own hmac module.

import hashlib, hmac

def hmac_sha256(key: bytes, msg: bytes) -> bytes:
    B = 64                                    # SHA-256 block size in bytes
    if len(key) > B:
        key = hashlib.sha256(key).digest()    # long keys are hashed first
    key = key.ljust(B, b"\x00")              # then zero-padded to one block
    ipad = bytes(k ^ 0x36 for k in key)
    opad = bytes(k ^ 0x5C for k in key)
    inner = hashlib.sha256(ipad + msg).digest()
    return hashlib.sha256(opad + inner).digest()

# RFC 4231, test case 2
tag = hmac_sha256(b"Jefe", b"what do ya want for nothing?").hex()
assert tag == "5bdcc146bf60754e6a042426089575c75a003f089d2739839dec58b964ec3843"
assert tag == hmac.new(b"Jefe", b"what do ya want for nothing?",
                       hashlib.sha256).hexdigest()

Two costs follow from the structure. Every tag needs at least two extra compression-function calls beyond the message itself, which dominates for short messages. And because K0 XOR ipad and K0 XOR opad are each exactly one block, an implementation can hash them once, save the two intermediate states and clone them per message. Python exposes this as hmac.new(key, digestmod=...) followed by .copy(); the PBKDF2 article shows how much that matters when HMAC runs millions of times.

Keys: length, equivalences and separation

Key length. RFC 2104 discourages keys shorter than the hash output and notes that keys longer than the output add little, because the security is capped by the hash state. For HMAC-SHA-256, generate 32 random bytes from the operating system's CSPRNG (secrets.token_bytes(32) in Python) and store them as bytes, not as a password. If the only secret you have is a human password, run it through a password KDF first; HMAC itself does nothing to slow guessing.

Equivalent keys. The key adjustment has two consequences that surprise people, both confirmed by running the code above. A key longer than 64 bytes and its SHA-256 digest produce identical tags. A key and the same key with trailing zero bytes appended produce identical tags, because both pad to the same K0. Neither is a practical weakness for random keys, but they break any design that treats "different key bytes" as "different keys", for example deriving per-tenant keys by appending a tenant ID that may end in a zero byte. Derive sub-keys with HKDF instead.

One key, one purpose. Do not reuse the webhook signing key as an encryption key, a cookie key and a JWT key. If a single root secret must feed several uses, derive one key per purpose with an info label, so a tag produced in one context can never be valid in another.

Worked example: signing webhooks

Signing outbound webhooks is the most common way engineers meet HMAC, and a good scheme answers four questions: which bytes are signed, how the receiver rejects replays, how the tag is compared, and how keys rotate. GitHub's scheme is the simplest widely used one: the X-Hub-Signature-256 header carries sha256= followed by the hex HMAC-SHA-256 of the raw request body under the webhook secret, and GitHub's documentation tells receivers to compare it with a constant-time function. Stripe's scheme adds a timestamp: its signature header carries a Unix time and one or more tags computed over the timestamp, a period and the raw body, so a captured request stops verifying once it is old.

The sketch below follows the timestamped design. The sender signs the exact bytes it puts on the wire. The receiver accepts a list of keys so that rotation needs no downtime: publish a new key, sign with it, keep verifying with the old key until its last in-flight deliveries have expired, then drop it.

import hmac, hashlib, time

def sign(key: bytes, ts: int, body: bytes) -> str:
    msg = b"v1." + str(ts).encode() + b"." + body          # canonical signed bytes
    return "v1=" + hmac.new(key, msg, hashlib.sha256).hexdigest()

def verify(keys: list[bytes], header_ts: str, header_sig: str, body: bytes,
           now: float, tolerance: int = 300) -> bool:
    try:
        ts = int(header_ts)
    except ValueError:
        return False
    if abs(now - ts) > tolerance:                         # stale or future-dated
        return False
    ok = False
    for k in keys:                                        # current key, then previous
        expected = sign(k, ts, body)
        ok |= hmac.compare_digest(expected.encode(), header_sig.encode())
    return ok

Walk one delivery through it. The sender has key k = 32 random bytes, timestamp 1791441000 and body {"id":"evt_42","amount":500}. It signs the 42 bytes v1.1791441000.{"id":"evt_42","amount":500}, sends the body unchanged with headers X-Timestamp: 1791441000 and X-Signature: v1=<64 hex characters>. The receiver reads the raw body before any JSON parsing, checks the timestamp is within five minutes of its clock, recomputes the tag and compares. If an attacker changes the amount to 5000, the tag no longer matches. If they replay the original request an hour later, the timestamp check fails. If they replay it within the window, the tag is valid, which is why the receiver must also deduplicate on the event ID: the timestamp bounds how long the dedupe table must remember IDs.

The tolerance is a choice, not a constant. It must cover clock skew and retry delays; a few minutes is typical. If you integrate with a vendor, use their SDK or read their current documentation for the exact window rather than guessing.

Comparing, truncating and encoding tags

Constant-time comparison. A naive equality check returns at the first differing byte, so the response time leaks how many leading bytes of a guessed tag were right. Over a network the signal is small, but it is measurable with enough samples, and it turns a 2²⁵⁶ search into a byte-by-byte one. Use hmac.compare_digest in Python, crypto.timingSafeEqual in Node.js (which throws on unequal lengths, so check lengths first) or MessageDigest.isEqual in Java. The side-channel attacks article covers how such leaks are found and tested.

Truncation. Some protocols send only part of the tag. RFC 2104 recommends keeping at least half the hash output and at least 80 bits; RFC 4231 includes a test case truncated to 128 bits. A forger who guesses a t-bit tag succeeds with probability 2^-t per attempt, so truncation trades bandwidth against the number of online guesses you can tolerate before rate limiting kicks in. Compare truncated tags in constant time too, and never accept a tag shorter than the length your protocol fixes, or an attacker sends one hex digit.

Encodings. Hex and base64 both work. Compare in one canonical form: decode the received value and compare bytes, or compare lowercase hex to lowercase hex. Case-insensitive or prefix matching reopens the guessing attack.

Failure modes

• Signing parsed data. The receiver parses JSON, re-serialises it and verifies that. Key order, whitespace and number formatting differ, so valid requests fail, and teams then disable verification. Sign and verify raw bytes.
• Ambiguous concatenation. Signing timestamp + body without a delimiter lets "17" + "9..." equal "179" + "...". Use a fixed delimiter after fixed-format fields, or length-prefix every field.
• Unsigned context. If the HTTP method, path or tenant is not in the signed bytes, a valid tag for one endpoint is valid for all of them.
• Algorithm confusion in JWTs. A verifier that trusts the token's alg field can be handed an HS256 token whose HMAC key is the server's RSA public key. Pin the algorithm per key; the JWT validation article lists the checks.
• Old keys that never expire. A rotation that never retires the previous key leaves a leaked key valid forever; give every key in the list an end date.
• Treating HMAC as a signature. Both parties hold the key, so a tag proves nothing to a third party; the receiver could have made it.

Trade-offs and alternatives

HMAC-SHA-256 is the default for authenticating data between parties that share a key: it is in every standard library, needs no nonce, and its security rests on modest assumptions. Alternatives fit narrower cases. KMAC (NIST SP 800-185) is the SHA-3 native MAC; SHA-3 has no length-extension problem, so it needs only one pass. Poly1305 and GMAC are much faster on bulk data but require a fresh one-time key or unique nonce per message, which is why they live inside AEAD constructions such as AES-GCM and ChaCha20-Poly1305 rather than being used alone. If you also need confidentiality, use an AEAD and do not compose your own encrypt-then-HMAC unless a standard tells you exactly how. If you need third parties to verify, or verifiers must not be able to forge, use a digital signature such as Ed25519, at the cost of key-pair management and slower verification.

What to do next

1. Implement the nine-line HMAC above and check it against all RFC 4231 test cases, then delete it and use your standard library's implementation.
2. List every place your system computes or checks a tag; confirm each uses raw bytes, a fixed algorithm and a constant-time comparison.
3. For each webhook or signed request, write down exactly which fields are signed, the delimiter, the timestamp tolerance and the deduplication key.
4. Generate 32-byte keys from a CSPRNG, store them in a secret manager, derive per-purpose keys with HKDF and support at least two active keys for rotation.
5. Add tests that tamper with one body byte, the timestamp, the path and the tag length, and assert each is rejected; add one that replays a valid request.
6. Rehearse a rotation in staging: add a key, switch signing, retire the old key, and watch verification failure rates throughout.