AFP engineering series · Chapter 12 of 25
Configure content filtering deliberately while keeping fraud, authorization, and action limits in application policy.
Content risk and transaction risk are different
A refund request can contain no harmful language and still be fraudulent. A legitimate incident report can contain disturbing language and still be appropriate for an analyst to review. Content-safety filtering and business authorization therefore need separate inputs, labels, and outcomes.
Gemini's API documentation describes category-based safety settings and feedback. It distinguishes the probability of unsafe content from the severity of a possible harm. Do not reinterpret a category score as the probability that a customer is committing fraud. The application must define its own transaction and identity controls.
Choose a policy for the product, not for a demo
A general writing assistant, a school product, and a security-analysis tool have different legitimate content. Start with a written use policy and a labeled evaluation set. For each category, define what should be allowed, refused, or routed to a specialized experience. Include allowed quotations and analytical discussion where relevant.
Pin the model and configuration used in evaluation. Defaults can differ across model families and change over time, so do not leave a critical setting implicit and assume it matches a screenshot from an older tutorial. Store the effective configuration with each release and test it again when the model changes.
Interpret blocked responses correctly
A provider may block a prompt or a candidate response. Your client must handle both without assuming a normal text part is always present. A missing answer should not trigger an automatic retry with weaker settings. That pattern converts a safety decision into an invitation to bypass the same decision.
Return a stable, useful message to the user and preserve an internal reason category for operations. Do not expose sensitive detector details or reproduce the blocked content in an error page. If the workflow permits review, create a review record rather than asking the model to invent an alternative policy.
Keep configurations and observations separate
Illustrative python; adapt the persistence and identity adapters to your service.
release = {
"model_id": "your-evaluated-model-id",
"safety_config_version": "support-content-4",
"business_policy_version": "refund-policy-7",
}
observation = {
"provider_prompt_blocked": False,
"provider_candidate_blocked": True,
"application_action_decision": "not_attempted",
}
assert observation["application_action_decision"] != "allow"These records intentionally use application-level fields. Map the provider's documented feedback into them and retain the original enum values in restricted diagnostics if needed. Do not collapse a provider block, an authorization denial, and an infrastructure error into the same boolean.
A model-generated explanation of why it refused is not a substitute for provider feedback or application policy telemetry. The prose can be incomplete or misleading. Use structured observations when calculating rates and investigating changes.
Build a threshold evaluation with business costs
Create a dataset that represents actual user tasks and review labels independently. Split it by language, task, and relevant content category. Compare false blocks, missed violations, completion rate, and latency at candidate configurations. If the evaluation set is deliberately attack-heavy, do not use its raw precision as an estimate for production traffic.
Estimate operational impact using a plausible traffic mix. A small false-positive rate on a very large legitimate population can create more review work than a high detection rate saves. Report the assumptions and show sensitivity to prevalence. The threshold lab uses explicitly synthetic rates so readers can see this arithmetic without mistaking it for a provider benchmark.
Avoid accidental configuration bypasses
Check every model invocation path: main agent, summarizer, fallback model, evaluator, and background task. A carefully configured root agent does not help if a helper invokes the provider with different defaults. Centralize configuration where practical and inventory exceptions with owners.
Treat retries and fallbacks as new policy paths. A fallback should preserve or deliberately strengthen required protections, not silently relax them to obtain an answer. If a model family supports different categories or settings, translate the policy explicitly and test the translated behavior.
Worked configuration audit: a fallback model uses different defaults
The main support agent has an explicit content configuration, but a fallback summarizer uses the provider defaults. A difficult request fails on the main path and is retried through the helper. The resulting answer has bypassed the intended configuration even though the root agent's settings are correct. Inventory every model call, including summarizers and evaluators, to find these hidden paths.
Create a configuration resolver that takes task type and evaluated model identifier and returns a reviewed policy bundle. Reject an unknown model-policy combination until it has been tested. Record the effective settings with the request. Do not assume that identically named categories or thresholds have identical behavior across different providers or model families.
Use a small labeled suite containing ordinary support requests, analytical quotations, and content the product should refuse. Run it through the primary and fallback paths with the same preprocessing. Compare provider feedback, application decisions, and user-visible results. A fallback that produces more answers is not automatically better if it also releases content the product intended to withhold.
Handle a blocked candidate without indexing blindly into a missing text field. The client should produce a stable outcome, preserve safe diagnostics, and avoid a retry that weakens settings. Add a regression test for a response with no ordinary text parts and for a prompt blocked before generation.
The audit also checks business actions. A clean content result does not allow a refund whose approval expired. Conversely, a content block does not imply that a customer account should be suspended for fraud. Keep those policies separate so a model configuration change cannot accidentally alter financial eligibility or account enforcement. The final release record lists all model invocation paths, their effective configurations, and the evidence used to accept each one.
Combine content controls with independent action controls
For Northstar, the content layer determines whether a response is suitable to show. The action service determines whether a refund is eligible and approved. The quota service determines whether the workload may consume more resources. Each can deny independently, and none should manufacture the evidence required by another.
Measure these outcomes separately after launch. A rise in content blocks may reflect changing user topics, while a rise in authorization denials may indicate a broken tool mapping or attempted fraud. Keeping the layers distinct makes the system easier to improve: tune the content policy for usefulness while preserving hard limits on data access and money movement.
Practice and continue
Open the interactive companion lab to inspect the assumptions and outcomes with synthetic data. Then return to the 25-chapter AFP learning path.
Sources and implementation scope
Primary references checked October 15, 2026. Framework details follow the linked documentation; the Northstar cases, policy contracts, and test designs are original illustrative designs. Python snippets labeled illustrative are not complete production integrations. ADK package metadata checked for this series reported 2.12.0; pin and test the version used by your application.