AI Credit Decisions, in depth: authority separation, reason codes from model attributions, decision records and attacks on the credit pipeline

The constraints that shape the design

Start from the constraints that shape the design, because they are stricter than most AI applications face. In the United States, the Equal Credit Opportunity Act and its implementing Regulation B require a creditor to notify an applicant of action taken within 30 days after receiving a completed application. When the action is adverse, the notice must give the specific reasons or tell the applicant how to request them. Section 1002.9 states that saying the applicant failed the creditor's internal standards or policies, or did not reach a qualifying score, is not enough. The official commentary adds that a scoring system's actual factors must be disclosed, and that more than four reasons is not likely to be helpful.

The CFPB had spelled out in Circular 2022-03 that this applies even when the model is a complex algorithm the creditor cannot easily explain. That circular was withdrawn on May 12, 2025, along with dozens of other guidance documents. The withdrawal removed guidance, not the regulation: the requirement for specific reasons is in Regulation B's text, so an architecture that cannot produce them has a compliance gap. If a consumer report contributed, the Fair Credit Reporting Act adds its own notice content.

In the European Union, the AI Act lists AI systems used to evaluate the creditworthiness of natural persons or establish their credit score as high-risk in Annex III point 5(b), with an exception for systems used to detect financial fraud, and Article 86 gives affected persons a right to an explanation of certain individual decisions taken on the basis of high-risk systems. Amendments to the high-risk timetable have been proposed, so check current application dates. The common thread: you must be able to say why, for each applicant, with evidence.

Reference architecture

The architecture that satisfies those constraints separates four things that naive systems blur: data collection, decision, explanation and record. The diagram shows the flow. Applications and verified third-party data feed a decision engine; uploaded documents go through an extraction step that treats them as hostile input; the engine is a signed, versioned model wrapped in deterministic policy rules; attributions become reason codes; any natural-language notice is drafted from the codes and validated; and an append-only decision record captures everything needed to reproduce the outcome.

The single most important rule is that authority lives in exactly one place. The decision engine returns approve, decline, refer or a price; nothing downstream can change that output, and nothing upstream can inject an instruction into it. A model score feeds policy rules such as sanctions screening, affordability and exposure limits, which are tested code, not prompts. An LLM may extract or draft, but its output is cross-checked data or validated text. If you cannot point to the one component that decides, make one first.

From attributions to specific reasons

Reason codes are where model risk meets the notice requirement. For a points-based scorecard the computation is simple: for each characteristic, the points the applicant lost relative to the maximum available are that characteristic's contribution, and the largest losses become the principal reasons. For tree ensembles and other nonlinear models, most teams compute per-feature attributions such as SHAP values against a documented reference population and rank the features that pushed the score toward decline.

Three engineering details decide whether the reasons are true. First, the reference population is a choice; attributions against all applicants differ from attributions against approved applicants, so pick one, document why and version it with the model. Second, correlated features split credit between them, so map features into reason groups before ranking: three utilisation features should produce one reason, revolving balances too high relative to limits, not three fragments. Third, only adverse contributions count, and a feature with a protected or prohibited basis must never be in the model or the reasons.

from dataclasses import dataclass

REASON_GROUPS = {          # feature -> reason code (reviewed by compliance)
    "util_revolving": "R01", "util_max_card": "R01", "bal_to_limit": "R01",
    "months_since_delinq": "R02", "num_delinq_24m": "R02",
    "age_oldest_trade": "R03",
    "dti_ratio": "R04",
    "num_inquiries_6m": "R05",
}
REASON_TEXT = {
    "R01": "Balances on revolving accounts are high compared with credit limits",
    "R02": "Recent or frequent delinquency on credit obligations",
    "R03": "Length of credit history is short",
    "R04": "Debt obligations are high relative to income",
    "R05": "Number of recent credit inquiries",
}

@dataclass(frozen=True)
class Reason:
    code: str
    weight: float

def principal_reasons(attributions: dict[str, float], k: int = 4) -> list[Reason]:
    """attributions: feature -> contribution to log-odds of decline (positive = adverse)."""
    grouped: dict[str, float] = {}
    for feat, contrib in attributions.items():
        code_ = REASON_GROUPS.get(feat)
        if code_ is None:
            raise KeyError(f"unmapped feature {feat}: every model feature needs a reason group")
        if contrib > 0:
            grouped[code_] = grouped.get(code_, 0.0) + contrib
    ranked = sorted(grouped.items(), key=lambda kv: kv[1], reverse=True)
    return [Reason(c_, w) for c_, w in ranked[:k]]

Note the hard failure on an unmapped feature: a retrain that adds a feature without a reason mapping should stop at deployment, not quietly omit the factor that drove the decline. Reason texts are reviewed language, versioned with the mapping.

The decision record

A decision record is the evidence that the pipeline did what the diagram says. Write it once, append-only, at the moment of decision, and include enough to reproduce the outcome a year later: application identifier, timestamps, the exact feature vector, references to the bureau pull and bank-data snapshot, the model artefact digest and version, the policy-rule version, the raw score, the rule outcomes, the reason codes with weights, the notice text or template identifier, and who or what made any override.

{
  "decision_id": "dec_01J9...",
  "application_id": "app_77341",
  "decided_at": "2026-10-04T09:14:22Z",
  "model": {"name": "unsecured_pd", "version": "2026.09.2", "sha256": "9f1c..."},
  "policy_version": "credit-policy-41",
  "features_sha256": "c03a...", "features_ref": "s3://decisions/2026/10/04/app_77341.parquet",
  "bureau_pull_id": "br_5521", "score": 0.214, "threshold": 0.18,
  "rules": {"sanctions": "pass", "affordability": "pass", "min_age": "pass"},
  "outcome": "decline",
  "reasons": [{"code": "R01", "w": 0.41}, {"code": "R04", "w": 0.22}, {"code": "R05", "w": 0.09}],
  "notice": {"template": "aan-v7", "sha256": "71be...", "sent_at": "2026-10-04T09:15:03Z"},
  "override": null,
  "prev_record_sha256": "e88d..."
}

Chaining each record to the previous record's hash, and shipping records to storage the decision service cannot rewrite, turns silent edits into detectable breaks. The general patterns are covered in audit logging for LLM systems; in credit the record is also what you hand an examiner, a complaints team or a court, so test that you can actually replay a decision from it.

Attacks on the credit pipeline

Attackers target credit pipelines for money, and their techniques map cleanly onto the architecture.

The pattern across the table is that the strongest controls do not try to detect cleverness in the input. They reduce the input's authority: verified data beats uploaded documents, typed fields beat free text, and deterministic rules beat model judgement for anything that must hold absolutely.

If an LLM drafts the notice

If you use an LLM to make adverse-action notices more readable, constrain it so tightly that it cannot affect substance. Give it only the reason codes and their approved texts, never the feature vector or the applicant's documents, and ask for plain-language wording in a fixed structure. Then validate the output mechanically before it is sent, and fall back to the template on any failure.

import re

def validate_notice(draft: str, codes: list[str], allowed_markers: dict[str, str]) -> bool:
    """Each reason must appear as its marker, and nothing else may be claimed.

    allowed_markers maps code -> marker the drafter must keep verbatim, e.g. "[R01]".
    """
    found = set(re.findall(r"\[R\d{2}\]", draft))
    expected = {allowed_markers[c_] for c_ in codes}
    if found != expected:
        return False                     # missing or invented reasons
    banned = ("guarantee", "approved if", "%", "score of")
    if any(b in draft.lower() for b in banned):
        return False                     # no promises, numbers or score talk
    return len(draft) < 1500

def notice_text(codes, drafter, template):
    draft = drafter(codes)               # LLM call with codes only
    if not validate_notice(draft, codes, {c_: f"[{c_}]" for c_ in codes}):
        return template(codes)           # deterministic fallback
    return re.sub(r"\[R\d{2}\]\s*", "", draft)   # strip markers before sending

Many teams decide that the reviewed template is clearer than any generated text and skip the LLM entirely, which is a legitimate outcome of this analysis. Where an LLM summarises a file for a human underwriter, the general controls in AI and finance LLMs apply: the summary is advisory, numbers come from systems of record, and the underwriter's decision is recorded with the model's suggestion so automation bias can be measured.

Worked example: one decline, end to end

Walk one application through. An applicant asks for an unsecured loan. Consented bank data and a bureau pull provide the features; the applicant also uploads a payslip, and the extraction step reads a monthly income that matches the salary deposits in the bank data within tolerance, so it is accepted. The model returns a probability of default of 0.214 against a decline threshold of 0.18, and every policy rule passes, so the outcome is decline.

Attributions against the approved-applicant reference show revolving utilisation features adding 0.41 to the log-odds of decline across three features, debt-to-income adding 0.22, recent inquiries 0.09, and a long credit history pulling the other way. After grouping, the principal reasons are R01, R04 and R05. The notice is generated from those three codes, passes validation, and the record above is written with the model digest and the hash of the notice text. When the applicant later complains, the team replays the stored features against the archived model and reproduces 0.214 and the same reasons.

Now change one fact: the uploaded payslip claims an income 40 percent higher than the deposits support. The cross-check fails, the document claim is not used, and the application is referred to a human with the discrepancy flagged. The model never sees the inflated figure, so it cannot be fooled by it.

Failure modes

• Reasons computed from a different model version than the one that scored, because attribution runs in a separate service with its own deployment cadence. Put the digest in both and refuse to emit reasons on a mismatch.
• Generic reasons such as insufficient score, which the regulation's text rules out.
• An LLM extractor whose output flows straight into features, making any instruction in a document a lever on the decision.
• Overrides without reasons, which destroy the evidence trail and hide bias in manual decisions.
• Drift unnoticed for months because monitoring watches accuracy on approved loans only; declined applicants never generate outcomes, so also monitor input distributions and approval rates by segment.
• Fairness tested once at launch. Whatever the current legal treatment of disparate impact, test outcomes by group on every model version and keep the results; check the rules that apply to you with counsel.

Trade-offs

Simpler models make reasons easier and audits shorter; complex models may rank risk better but cost explanation effort, validation time and some reason fidelity. Strict separation and validation make LLM features slower to ship, and that is the intended trade. Pulling verified data reduces fraud but adds consent flows and vendors. Record the reasoning in the model documentation.

What to do next

1. Draw your credit pipeline and mark the single component with decision authority; remove any path by which an LLM output can change an outcome.
2. Build the feature-to-reason-group mapping with compliance review and make an unmapped feature a deployment failure.
3. Document the attribution reference population and version it with the model.
4. Write append-only decision records with model digest, feature snapshot and notice hash, and rehearse replaying one.
5. Replace uploaded-document trust with cross-checks against consented source data where you can.
6. Add velocity and entity-linking controls in front of scoring and pre-qualification.
7. If an LLM drafts notices, feed it codes only, validate mechanically and keep the template fallback.
8. Schedule per-version group outcome testing and drift monitoring that includes declined applicants.