Account Takeover and Deepfake Requests: Identity Controls for LLM Support

Support conversations are an attractive recovery path

An attacker who cannot log in may try to persuade support to change an email address, reset a factor, or move a payment destination. An LLM assistant can make this channel faster and more scalable, but it can also apply exceptions inconsistently if the policy is expressed only as conversational guidance.

Treat account recovery as its own high-impact workflow. The assistant may collect information and explain the process, but it should not decide that a convincing story overrides identity requirements. A recovery policy needs evidence rules, escalation paths, rate limits, and a record of every authority-changing action.

Separate recognition from authentication

A name, voice, photograph, writing style, or personal detail can support a story without proving control of an account. The FBI's 2024 advisory describes synthetic media used for impersonation. Design verification around established authenticated channels and independent records rather than asking the assistant whether the person seems genuine.

Do not use contact information supplied in the suspicious request as the independent verification channel. If a caller says the old number is unavailable and supplies a new one, that new number is part of the claim being evaluated. Recovery may still be possible, but it needs a separate evidence process with an appropriate delay or review.

Protect factor changes as strongly as login

A strong login flow can be undermined by a weak factor-reset flow. Require a protected workflow for adding a new authenticator, changing a recovery address, or disabling an existing factor. Notify established channels when appropriate, and make suspicious changes visible to the legitimate account holder.

The assistant should receive only the minimum status needed to guide the user. It should not expose secret recovery values, full authentication histories, or details that help someone guess verification answers. Prefer purpose-built identity operations with clear outcomes over free-form access to an identity database.

Represent recovery evidence explicitly

Illustrative python; adapt the persistence and identity adapters to your service.

evidence = {
    "existing_authenticated_session": False,
    "verified_existing_channel": False,
    "manual_review_complete": False,
    "new_contact_supplied_in_chat": True,
}
def recovery_route(e):
    if e["existing_authenticated_session"] and e["verified_existing_channel"]:
        return "eligible_for_policy_checks"
    return "independent_recovery_review"
assert recovery_route(evidence) == "independent_recovery_review"

This is a conservative routing example, not a universal identity-proofing policy. Your product must define what evidence is sufficient, how accessibility needs are handled, and which changes require stronger verification. Keep those rules in the identity service. The model may explain the result but cannot set the evidence flags.

Bind evidence to the account and operation being reviewed. Verification for reading an order should not automatically authorize changing the account's recovery address. A verified session also does not make every requested action low risk; a stolen session can still be used for fraud.

Add friction where it interrupts the abuse path

Rate-limit recovery attempts by several relevant dimensions, with privacy and fairness safeguards. Account-only limits can let an attacker lock out a victim; IP-only limits can harm users behind shared networks. Use graduated controls and preserve a legitimate recovery path.

For sensitive changes, consider a hold period, additional authentication, or independent review based on the risk model. Avoid arbitrary thresholds presented as universal best practice. Measure how each control affects both confirmed takeover attempts and legitimate users who have lost access. A secure recovery system must remain recoverable.

Test the full recovery journey

Use synthetic accounts and media, and avoid collecting real biometric data for a convenience test. Verify the identity service's final state, notification behavior, and audit records. A chatbot refusal is not enough if a background tool still changed the account.

Include support-agent mistakes and compromised staff accounts in tabletop exercises. Human review is not automatically safe; reviewers need clear evidence and limited permissions. Separate duties for especially consequential changes when the product's risk warrants it.

Worked recovery case: urgency plus a new contact channel

A requester claims to be a customer whose phone was lost and asks the assistant to replace the recovery number immediately. The message contains accurate order details and a persuasive emergency. Those details may support investigation, but they do not prove control of the account. The new number is an asserted destination, not an independent verification channel.

The assistant creates a recovery case and explains the available process. The identity service determines which existing authenticated signals or independent review steps are required. The tool exposed to the assistant can submit evidence references and retrieve case status; it cannot directly overwrite the recovery factor. This keeps conversational urgency from becoming administrative authority.

Test a legitimate lost-device customer as well as a fraudulent claimant. The legitimate user must have a documented route to recovery, with accessible alternatives and clear status. A system that permanently blocks everyone who loses a factor is not a successful recovery design. Measure completion time and mistaken denials alongside confirmed takeover attempts.

After a recovery change is approved, apply the product's policy for subsequent sensitive actions. A newly recovered account immediately requesting a new payment destination may need additional verification or a hold. Do not let the assistant infer that successful recovery automatically approves every following transaction.

In a drill, revoke sessions and pause sensitive changes after a simulated compromise. Confirm that delegated agent credentials and queued actions are included. Reconstruct the timeline from recovery request through factor change to later operations. This case connects identity controls with AFP: the assistant can make support easier to navigate while the authority to change access remains tied to verified evidence and an auditable process.

Detect and recover from takeover

Provide rapid revocation of sessions and delegated credentials, a way to freeze sensitive changes, and a reconciliation process for actions taken during compromise. Preserve a restricted timeline linking recovery attempts, evidence decisions, factor changes, and subsequent transactions.

Measure confirmed takeover, time to containment, legitimate recovery completion, and appeal outcomes. Do not equate an anomaly score with guilt. The assistant's role is to help users navigate a trustworthy process; identity authority must remain with systems that can verify evidence and enforce consistent rules.

Practice and continue

Open the interactive companion lab to inspect the assumptions and outcomes with synthetic data. Then return to the 25-chapter AFP learning path.

• Refund Fraud in LLM Agents: Atomic Reservations, Approval Binding, and Replay
• LLM Abuse Velocity: Quotas, Campaign Signals, and Bounded Agent Work

Sources and implementation scope

Primary references checked October 15, 2026. Framework details follow the linked documentation; the Northstar cases, policy contracts, and test designs are original illustrative designs. Python snippets labeled illustrative are not complete production integrations. ADK package metadata checked for this series reported 2.12.0; pin and test the version used by your application.

• OWASP multifactor authentication
• FBI: generative AI and financial fraud (2024)
• OWASP authorization