AFP engineering series · Chapter 5 of 25
Trace how extracted text and synthetic media can influence an agent, and design modality-aware screening and verification.
The preview and the model input can differ
A user sees a polished PDF invoice, while an extraction service supplies text in a different reading order. An image contains text that OCR recognizes but a small preview makes hard to notice. A voice assistant receives a transcript whose punctuation and speaker assignment differ from the original audio. These transformations create opportunities for untrusted content to influence a model in ways the user did not intend.
The security question is which representation reaches which component. Store a traceable relationship between original media, extracted text, and model input. Do not assume that a screenshot reviewed by a human is identical to the text subsequently processed by the agent.
Inventory each modality and transformation
For a document workflow, enumerate file upload validation, malware scanning, parsing, OCR, text extraction, image interpretation, chunking, and summarization. For voice, include transcription, speaker attribution, interruption handling, and text-to-speech output. Identify third-party services and temporary storage at every stage.
Attach source and transformation identifiers to outputs. A transcript should indicate that it is a derived interpretation, not a verified statement by a particular person. A document parser should report unsupported or failed extraction rather than silently returning an empty document that the assistant mistakes for a clean scan. Different formats can need different handling without changing the application's underlying authorization rules.
Screen what the model actually receives
A text-only detector cannot evaluate visual content it never receives. An image detector may not inspect an embedded document, and a service that evaluates individual requests may not understand a conversation spread over several turns. Google Model Armor's documentation describes modality and encoding limitations; use the current supported-input matrix when integrating it instead of assuming every payload is covered.
Build an explicit coverage record: original file checked, extracted text checked, image checked, transcript checked, or unsupported. Unsupported does not mean safe. Choose whether to reject the format, route it to review, or allow a restricted read-only experience. Make that decision visible in telemetry so unsupported traffic does not disappear into the normal pass rate.
Keep identity separate from resemblance
A familiar voice or face can make a request persuasive without proving its origin. The FBI's 2024 fraud advisory discusses synthetic audio and video in impersonation schemes. In a business workflow, a video call should not replace the authenticated approval channel for changing a payment destination.
For a fictional executive-payment request, let the assistant summarize the request and identify the claimed approver. The payment service should verify approval through an established account and bind it to the exact amount and recipient. Do not take a phone number, callback URL, or approval link from the suspicious message as the independent verification channel.
Represent incomplete inspection honestly
Illustrative python; adapt the persistence and identity adapters to your service.
inspection = {
"original_sha256": "fixture-hash",
"text": "passed",
"images": "unsupported",
"audio": "not_present",
"parser_version": "fixture-parser-3",
}
def may_auto_execute(report):
required = (report["text"], report["images"])
return all(x in {"passed", "not_present"} for x in required)
assert not may_auto_execute(inspection)This fixture assumes both text and images are required for automatic execution. A real service should derive required checks from the detected file contents and the proposed action, not from client-supplied labels. A parser failure and a detector failure should have separate reason codes because they require different remediation.
The record should also distinguish “not present” from “not inspected.” An image that was omitted due to a size limit was present. Treating it as absent creates a coverage hole that looks like a successful scan in aggregate metrics.
Design safe multimodal experiments
| Synthetic fixture | Observation | Expected control |
|---|---|---|
| Invoice image with benign instruction marker | OCR text differs from visible layout | External text remains untrusted |
| PDF with unsupported image format | Coverage shows unsupported | No automatic payment |
| Voice request to change recipient | Transcript makes a claim | Independent authenticated approval |
| Legitimate diagram containing instructions | Useful analysis remains possible | No blanket content rejection |
Use invented names and accounts, and keep the test media in a sandbox. Evaluate the exact ingestion pipeline and supported formats used in production. Include low-quality scans, rotated pages, multiple languages, and accessibility descriptions, since ordinary variation can alter extraction behavior without any attacker involvement.
Compare both task quality and security outcomes. If a defense removes all image text, it may prevent one fixture while making invoice processing useless. A better design preserves useful content for analysis and restricts the actions that content can authorize.
Worked ingestion contract: scanned invoice with partial coverage
Northstar receives a three-page scanned invoice. Page one is clear, page two is rotated, and page three contains an embedded image that the configured screening method does not support. The extractor returns text for the first two pages. A naive adapter sees text plus a successful API response and marks the entire invoice inspected. The missing third page has disappeared from the decision record.
Instead, build a page-level manifest from the parser: three pages present, two text extractions completed, one image inspection unsupported. The overall coverage result is incomplete. The assistant may summarize the inspected pages with an explicit limitation, but the invoice cannot enter an automatic payment path that requires complete inspection. The policy is based on coverage, not on whether the model sounds confident about the invoice.
A reviewer opens the original document through a restricted viewer and compares it with extracted text. If the workflow permits manual review to complete the missing check, record the reviewer, page identifiers, decision, and expiry. Do not simply flip a global clean flag. Future processing needs to know which parts were machine-inspected and which were manually assessed.
Test the parser with synthetic rotated text, a blank page, a corrupt page, and an image-only page. Confirm that blank and unparsed are different states. Include an ordinary diagram containing instructions so the system does not reject every imperative phrase as malicious. The objective is accurate representation and bounded authority, not deleting all instruction-like content.
For a voice-assisted approval, add a separate authenticated confirmation channel. A transcript saying “yes” should not by itself authorize a payment, especially when speaker attribution is uncertain. The approval record should reference the exact proposal and the authenticated approver, while the audio remains optional supporting evidence under the retention policy.
This case demonstrates why modality support is part of security configuration. Coverage depends on file format, parser behavior, screening method, and endpoint capabilities. Re-test the manifest when any of them changes. A successful content check on a subset of the input is useful evidence, but it must not be reported as complete inspection of the whole artifact.
Plan for sensitive media and incident recovery
Images and audio can contain personal data beyond the text you intended to process. Minimize retention, restrict access to original media, and keep routine metrics separate from evidence storage. Do not place raw recordings into general-purpose debug logs merely because a guardrail failed.
When a suspicious file is discovered, locate its extracted text, thumbnails, embeddings, transcripts, and cached summaries. Quarantine derivatives using the original file identifier and transformation lineage. Then re-run the corrected pipeline against controlled fixtures before restoring automatic processing. The objective is a verifiable chain from media to decision, with no transformation silently granting authority.
Practice and continue
Open the interactive companion lab to inspect the assumptions and outcomes with synthetic data. Then return to the 25-chapter AFP learning path.
Sources and implementation scope
Primary references checked October 15, 2026. Framework details follow the linked documentation; the Northstar cases, policy contracts, and test designs are original illustrative designs. Python snippets labeled illustrative are not complete production integrations. ADK package metadata checked for this series reported 2.12.0; pin and test the version used by your application.