Model Signing + Provenance, in depth: in-toto statements, SLSA provenance for training runs, unforgeable signing and deploy-time lineage policy

Why a signature is not enough

Think about what an attacker or a careless process can change in the path from data to deployment. They can swap the weight file after training, which a signature catches. They can also change the inputs and then legitimately train and sign the result: point the job at a poisoned dataset, a base model pulled from a look-alike repository, or a trainer image with a modified loss. The output is a real model produced by your real pipeline, and its signature is valid. Only a record of the inputs, checked against policy, catches it.

Provenance also makes incidents tractable. When a dataset is found to contain personal data or a base model is found to carry a backdoor, the question becomes: which deployed models descend from it? With provenance that is a query over attestations. Without it, it is archaeology. The attack paths that make this necessary, from malicious pickles to look-alike hub repositories, are described in supply chain attacks on ML.

The provenance format

The standard envelope is an in-toto Statement. It binds a list of subjects, identified by digest, to a typed predicate. For build provenance the predicate type is SLSA provenance v1, which splits the record into a build definition (what was asked for and what was consumed) and run details (who ran it).

{
  "_type": "https://in-toto.io/Statement/v1",
  "subject": [
    {"name": "config.json",        "digest": {"sha256": "44136f..."}},
    {"name": "model.safetensors",  "digest": {"sha256": "9a1c0e..."}},
    {"name": "tokenizer.json",     "digest": {"sha256": "7d2b51..."}}
  ],
  "predicateType": "https://slsa.dev/provenance/v1",
  "predicate": {
    "buildDefinition": {
      "buildType": "https://ml.example.com/train/finetune/v1",
      "externalParameters": {"config": {"lr": 2e-5, "epochs": 2}},
      "resolvedDependencies": [
        {"name": "base-model", "uri": "hf://org/base@rev", "digest": {"sha256": "bbbb..."}},
        {"name": "dataset:support-tickets-2026q3", "uri": "s3://data/t.parquet",
         "digest": {"sha256": "d1d1..."}},
        {"name": "trainer-code", "uri": "git+https://git.example.com/ml/trainer",
         "digest": {"gitCommit": "aaaa..."}},
        {"name": "trainer-image", "uri": "registry.example.com/trainer",
         "digest": {"sha256": "cccc..."}}
      ]
    },
    "runDetails": {
      "builder": {"id": "https://ml.example.com/builders/train-cluster-a@v3"},
      "metadata": {"invocationId": "run-8812"}
    }
  }
}

The subject lists every file a loader will read, not just the weights: tokenizer, configuration, chat template and any custom code. externalParameters holds what the user controlled, which the SLSA spec treats as untrusted input to be checked. resolvedDependencies holds every artifact consumed, each by digest. builder.id names the trusted platform that ran the build; the verifier decides which builder identities it trusts.

Recording datasets by content

Recording a dataset by digest sounds simple until the dataset is ten thousand Parquet shards in an object store. Hashing a listing or a manifest of object paths is not enough, because the objects behind the paths can be overwritten. Hash the content the job will actually read, then combine the shard digests in a fixed order so one digest names the whole snapshot:

import hashlib
def snapshot_digest(shards):            # shards: iterable of (relative_path, bytes_reader)
    leaves = []
    for rel, reader in sorted(shards, key=lambda s: s[0]):
        h = hashlib.sha256()
        for chunk in iter(lambda: reader.read(1 << 20), b""):
            h.update(chunk)
        leaves.append(f"{rel}\0{h.hexdigest()}")
    return hashlib.sha256("\n".join(leaves).encode()).hexdigest()

Store the per-shard list beside the attestation, so an auditor can later prove that a particular shard was or was not in the run. Then make the job read only that snapshot: copy it to a read-only volume, or use object versioning and record version identifiers with the digests. A digest of data the job did not actually consume is worse than no digest, because it is trusted.

Build levels and who signs

SLSA v1.0 defines Build levels by how hard the provenance is to forge. Build L1 means provenance exists. Build L2 means it is generated and signed by a hosted build platform. Build L3 means the platform is hardened so that the build's own steps cannot influence or forge the provenance or the signing key.

For model training that last point is the one that changes architecture. If the training script builds and signs its own attestation, then anyone who can change the script can claim any dataset they like. At L3 the orchestrator that launches the job, not the job, records the inputs it resolved and signs with an identity the job cannot use. In practice the controller pins inputs by digest, mounts them read-only, launches the run in an isolated workload, hashes the outputs it collects, and only then signs.

Signing and verifying attestations

Once the controller has the statement, it signs it. With Sigstore's cosign, a predicate file can be attested against a blob and written to a bundle that carries everything needed for verification:

# controller identity signs; the training job never sees these credentials
cosign attest-blob --predicate provenance-predicate.json \
  --type https://slsa.dev/provenance/v1 --bundle model.intoto.bundle manifest.json

# at the deploy gate: check the signer identity AND the predicate type
cosign verify-blob-attestation --bundle model.intoto.bundle \
  --type https://slsa.dev/provenance/v1 \
  --certificate-identity https://ml.example.com/builders/train-cluster-a@v3 \
  --certificate-oidc-issuer https://issuer.ml.example.com \
  manifest.json

Here the blob is a manifest of the model's file digests, so the attestation's subject is the manifest and the manifest lists every file; one attestation covers a multi-file model, and the policy below applies to the per-file view either way. Always pass the predicate type at verification so an attestation of a different kind cannot satisfy the check. If your fine-tunes run on GitHub-hosted runners, GitHub's actions/attest-build-provenance action produces signed SLSA provenance for files you name, and gh attestation verify checks it; most training runs on private clusters, where the same pattern uses your own workload identity. The OpenSSF model_signing tool covers the signature on the model files themselves.

A deploy-time policy

A verified signature only proves who made the claim. The deploy gate must then check what was claimed. The policy below is deliberately small: the subject must match the files on disk exactly, the builder must be trusted, the required inputs must all be recorded, the base model must be allowlisted, and every dataset must be approved.

def verify(stmt, files_on_disk, policy):
    errors = []
    if stmt.get("_type") != "https://in-toto.io/Statement/v1":
        errors.append("not an in-toto v1 statement")
    if stmt.get("predicateType") != "https://slsa.dev/provenance/v1":
        errors.append("not SLSA provenance v1")
    claimed = {s["name"]: s["digest"]["sha256"] for s in stmt["subject"]}
    actual = {n: hashlib.sha256(b).hexdigest() for n, b in files_on_disk.items()}
    if claimed != actual:                      # extra, missing or changed files all fail
        missing = sorted(set(actual) ^ set(claimed))
        changed = sorted(n for n in set(actual) & set(claimed) if actual[n] != claimed[n])
        errors.append(f"subject mismatch: changed={changed} unlisted_or_missing={missing}")
    pred = stmt["predicate"]
    if pred["runDetails"]["builder"]["id"] not in policy["trusted_builders"]:
        errors.append("untrusted builder " + pred["runDetails"]["builder"]["id"])
    deps = pred["buildDefinition"]["resolvedDependencies"]
    names = {d["name"] for d in deps}
    for req in sorted(policy["required_deps"] - names):   # an omitted input must not pass
        errors.append("missing dependency " + req)
    if not any(n.startswith("dataset:") for n in names):
        errors.append("no dataset recorded")
    for d in deps:
        if d["name"] == "base-model" and d["digest"].get("sha256") not in policy["allowed_base_models"]:
            errors.append("base model not on allowlist")
        if d["name"].startswith("dataset:") and d["digest"].get("sha256") not in policy["approved_datasets"]:
            errors.append("unapproved dataset " + d["name"])
    return errors          # empty list means admit

Run this after signature verification, against the exact files the server will load, in the same place that loads them. Verifying in CI and then copying files to a serving volume leaves a gap where the bytes can change.

Worked example: three verdicts

Take a fine-tune with three files and the statement shown earlier, and run the policy against four situations. The outputs below come from running the function above.

The third case is the reason provenance exists. Signature verification alone admits that model, because the trusted pipeline really did produce and sign it. Only a policy over the recorded inputs rejects it. The table also shows why the subject comparison must be an exact set match: a check that only confirms the listed files would miss an extra custom-code file dropped beside the weights. The fourth row shows why required inputs are checked by presence: a record that simply leaves out the base model would otherwise skip the allowlist entirely.

Lineage across derived models

Models are rarely built once. A base model is fine-tuned, merged, quantized and wrapped with adapters, and each step is a build with its own provenance. Treat the parent model as a resolved dependency of the child, by digest, and the attestations form a chain. The verifier walks it: verify the child's attestation, look up the attestation whose subject matches the parent digest, verify that one, and continue until it reaches a root on the allowlist. A LoRA adapter is its own subject with the base model as a dependency, and the server must check that the base it actually loads has that digest.

Store attestations next to the artifact and in a searchable index keyed by subject digest. The index answers the incident question from the start of this article in reverse: given a bad dataset digest, list every model whose chain contains it. The same discipline applied to serving images is covered in container supply chain for LLMs.

Failure modes

• Self-attested provenance. The training job writes and signs its own record, so it can claim anything. Sign from the orchestrator with an identity the job cannot reach.
• Mutable references. A dependency recorded as a branch name, a hub repository without a revision, or an object path without a digest proves nothing. Record digests.
• Partial subjects. Only the weights are attested, but the loader also reads a tokenizer, a chat template or custom code that an attacker can change freely.
• Type confusion. The verifier checks the signature but not the predicate type, so an unrelated attestation from the same identity passes.
• Dataset digests of the wrong thing. Hashing a manifest that points at mutable objects instead of the content consumed. Hash the materialised snapshot.
• Verify far from use. Checks run in CI, files are copied afterwards, and the server never verifies.
• Lost attestations. Records stored only in a build log that expires. Keep them with the artifact and in an index.

Trade-offs

Provenance costs engineering on the platform side: pinning every input by digest, an orchestrator that signs instead of the job, and an index to store and query records. Hashing large datasets and checkpoints adds time, though it can run in parallel with other work and only once per snapshot. Strict policies slow experimentation, so most teams run two tiers: research models may carry L1 provenance and stay off production paths, while anything that serves users requires platform-signed provenance and a policy pass. Keyless signing removes long-lived keys but ties verification to an identity provider and transparency log; a private key in an HSM is simpler to operate offline but must be rotated and protected. Neither replaces scanning the content itself.

What to do next

1. List every file your model servers load and make all of them subjects of the attestation.
2. Record the base model, datasets, trainer commit and trainer image in resolvedDependencies, each by digest.
3. Move statement creation and signing out of the training job into the orchestrator.
4. Sign with cosign or your platform's attestation service, and verify with the predicate type and signer identity pinned.
5. Write the deploy-gate policy: exact subject match, trusted builder, allowlisted parents, approved datasets.
6. Run the gate where the server loads the files, not only in CI.
7. Index attestations by subject digest and rehearse the query that finds every model descended from a bad input.