AFP detector thresholds and review-capacity lab

What this calculator measures

This lab calculates expected confusion-matrix counts from a population size, a positive-case prevalence, and one of three invented detector operating points. The rates are teaching assumptions, not measured performance of ADK, Gemini, Model Armor, or any other product. The calculator also converts flagged cases into reviewer-hours and compares them with the capacity you enter.

Expected counts can be fractional because they describe an average implied by assumed rates. An observed dataset would contain integer cases and uncertain estimates. Precision is the share of flagged cases that are truly positive; it changes with prevalence even when sensitivity and false-positive rate remain fixed.

Reproduce the baseline arithmetic

The default population has 10,000 cases and 1% positive prevalence. With 90% sensitivity and 2% false-positive rate, the model produces 90 true positives and 198 false positives. Precision is 31.25%. At five minutes per review, the 288 flagged cases require 24 reviewer-hours. With 20 hours available, the expected excess workload is four hours.

Change prevalence to ten percent while keeping the detector fixed and observe precision. Then return to one percent and choose the fewer-flags operating point. Review demand falls, but missed positive cases increase. The calculator deliberately exposes this tradeoff instead of recommending a universal threshold.

Use the controls and inspect assumptions

Cases in period and prevalence define the population. The operating-point selector changes both sensitivity and false-positive rate using the values printed in its label. Minutes per review and reviewer-hours available affect workload, not classification accuracy. All values have bounded inputs; invalid entries produce a visible message instead of an unexplained result.

The trace shows the population assumption, detector assumption, and capacity calculation. Start trace, Next step, Show all, and the step slider change only how much of that explanation is visible. Metrics always show the complete calculation. Reset restores the baseline. Export JSON downloads the inputs and results locally for a comparison or a classroom exercise.

What the model leaves out

The calculator does not infer intent, score real people, or recommend account restrictions. It assumes fixed rates and a fixed average review time. It omits correlated errors, delayed labels, heterogeneous case complexity, reviewer disagreement, and queue-arrival timing. Excess review hours are workload beyond the entered capacity, not a prediction of exact customer waiting time.

For production, estimate rates on representative labeled data, report uncertainty, and examine legitimate-task harm. Use a time-separated evaluation where later investigation facts cannot leak into the model input. Pair detection with deterministic action constraints and an appeal process. A detector score is one piece of evidence, while ownership, approval, and transaction limits remain independent controls.