AI in Water Management, in depth: securing leak detection, optimisers and operator assistants in water utilities

Why water is different

Three properties set water apart. Consequences are physical and delayed: under-dosed disinfectant or a pressure transient that draws contaminated water into a main may surface days later as illness. The operational technology (OT) underneath is old and often reachable in ways its owners do not know. And many utilities have a few operators and no security staff, so controls must survive without a specialist.

The baseline threat is not exotic. In late 2023, a joint advisory from CISA, the FBI, NSA, EPA and Israel's INCD (AA23-335A) described IRGC-affiliated actors calling themselves CyberAv3ngers compromising internet-exposed Unitronics Vision PLCs at US water facilities through default passwords. The best-known case hit Aliquippa's municipal water authority. No model was involved. The lesson for AI projects is that a new data pipeline or vendor cloud connection adds to an attack surface whose basics are often still unfinished, so the AI work must not open new paths into OT, and ideally should pay for closing old ones.

Reference architecture

The architecture that follows from this is layered, in the spirit of the ISA/IEC 62443 zones and conduits model. Sensors and PLCs sit at the bottom; SCADA and the historian sit in the OT zone; an OT DMZ separates them from enterprise IT and any cloud. Models live above the DMZ.

• Data leaves OT one way. The historian replicates to a read-only copy outside OT, ideally through a unidirectional gateway or at least a push-only connection. Models train on and score from the copy, never by querying PLCs.
• Decisions enter OT through one conduit. A model output is a proposal with its evidence, placed on an advisory queue. An operator approves it, and only the approved value crosses into OT.
• A guard owns every write. Inside OT, a small, boring program checks each approved change against hard limits before it reaches SCADA. The guard does not trust the model, the queue or the approver's screen.

Threat model

A threat model for this architecture is short enough to keep on one page. Notice that most rows are about inputs and outputs of models, not the models themselves.

Losing every model should be uneventful. If the plant cannot run without its optimiser, the optimiser has become a control system and must be secured as one.

Control 1: a setpoint guard owns every write

The guard is the most important control and the least clever. It enforces an allow-list of writable tags, hard limits from the permit and engineering design, a maximum step per change, a minimum interval between changes, and an approval count that rises with the size of the change. It logs every decision, including rejections, because a burst of rejected proposals is itself a signal.

from dataclasses import dataclass
from datetime import datetime, timedelta, timezone

@dataclass(frozen=True)
class Limits:
    lo: float              # hard floor from permit / engineering
    hi: float              # hard ceiling
    max_step: float        # largest change per approval
    min_interval: timedelta
    auto_band: float       # changes within this band need one approver; larger need two

class SetpointGuard:
    """Sits in the OT zone. Models propose; only this code writes."""
    def __init__(self, limits: dict[str, Limits], read_tag, write_tag, audit):
        self.limits, self.read, self.write, self.audit = limits, read_tag, write_tag, audit
        self.last_change: dict[str, datetime] = {}

    def apply(self, tag, proposed, approvers, source, now=None):
        now = now or datetime.now(timezone.utc)
        lim = self.limits.get(tag)
        if lim is None:
            return self._reject(tag, proposed, "tag not on allow-list", source)
        current = self.read(tag)
        if not (lim.lo <= proposed <= lim.hi):
            return self._reject(tag, proposed, f"outside [{lim.lo}, {lim.hi}]", source)
        if abs(proposed - current) > lim.max_step:
            return self._reject(tag, proposed, f"step {proposed - current:+.3f} exceeds {lim.max_step}", source)
        last = self.last_change.get(tag)
        if last and now - last < lim.min_interval:
            return self._reject(tag, proposed, "changed too recently", source)
        needed = 1 if abs(proposed - current) <= lim.auto_band else 2
        if len(set(approvers)) < needed:
            return self._reject(tag, proposed, f"needs {needed} distinct approvers", source)
        self.write(tag, proposed)
        self.last_change[tag] = now
        self.audit(dict(tag=tag, old=current, new=proposed, by=sorted(set(approvers)), source=source, ok=True))
        return True

    def _reject(self, tag, proposed, why, source):
        self.audit(dict(tag=tag, new=proposed, source=source, ok=False, why=why))
        return False

Fill the limits from documents, not from the model's training data. For disinfectant dosing, the ceiling comes from regulation (in the US, the maximum residual disinfectant level for chlorine is 4.0 mg/L) and the floor from your permit's minimum residual; pump limits come from the pump curves and the surge analysis. Run the guard on a host inside OT, keep its code small enough to review in an hour, and change its limits only through the same change control you use for PLC logic. It complements, and never replaces, interlocks in the PLC itself.

Control 2: physics checks on inputs

Anomaly detectors flag what looks unusual. An attacker who controls the data, or a failing sensor, can make trouble look usual. The defence is a second opinion the model cannot learn around: conservation of mass. In a district metered area (DMA), water in minus change in storage minus estimated legitimate use is the unexplained loss. Leaks raise it; a spoofed inflow meter that under-reads drives it below zero, which no real network can do for long.

def dma_check(inflow_m3, tank_delta_m3, billed_est_m3, night_flow_m3h, hist):
    """Physics cross-check for one district metered area (DMA) over one hour.

    inflow - storage change - estimated legitimate use = unexplained loss.
    hist: recent unexplained-loss values for the same hour of week.
    """
    unexplained = inflow_m3 - tank_delta_m3 - billed_est_m3
    mu = sum(hist) / len(hist)
    sd = (sum((h - mu) ** 2 for h in hist) / (len(hist) - 1)) ** 0.5
    z = (unexplained - mu) / max(sd, 0.5)
    flags = []
    if z > 4:
        flags.append("loss far above normal: leak, burst or meter fault")
    if z < -4:
        flags.append("loss far below normal: inflow meter under-reading or spoofed")
    if night_flow_m3h <= 0:
        flags.append("zero night flow is physically implausible for a live DMA")
    return unexplained, round(z, 1), flags

def replay_suspect(series, window=12):
    """True if the latest window exactly repeats an earlier one: a classic sign of
    recorded data being replayed to hide a change. Real sensors carry noise."""
    tail = tuple(series[-window:])
    for start in range(len(series) - 2 * window + 1):
        if tuple(series[start:start + window]) == tail:
            return True
    return False

The replay check catches a cheap and effective trick: recording an hour of normal readings and playing it back. Raw analog samples carry noise, so an exact 12-sample repeat is very unlikely; run it on raw values, not deadband-compressed historian tags. Pair both checks with plain redundancy, such as two pressure sensors on the same main or a tank level against the sum of its inlet and outlet flows, and alert when they disagree beyond their stated accuracy.

Control 3: training-data integrity

Leak and burst models learn from historian data labelled with confirmed events from work orders. Both halves can be poisoned. Someone with write access to the work order system can mark real bursts as sensor faults, teaching the model to ignore that signature; someone with historian access can inject events. Keep a frozen, reviewed evaluation set of confirmed events that no pipeline can modify, record the source and editor of every label, and require a model to hold its recall on the frozen set before it ships. When a retrain changes performance on one district much more than others, investigate the labels there first. The general patterns are covered in Data Poisoning.

Control 4: a bounded operator assistant

An operator assistant is useful: "what is the procedure for taking filter 3 offline", "when was this pump last serviced", "summarise last night's alarms". It is also the component most exposed to untrusted text, because work orders, contractor emails, vendor manuals and alarm descriptions all flow into its context. Indirect prompt injection in any of them can steer its answers. The design rules are strict:

• Give it read-only tools only: document search, historian queries against the replica, and deterministic calculators. It never has a tool that writes a setpoint, opens a work order for execution or sends a message outside the control room.
• Require a citation to an approved SOP section for any procedural answer, and show the cited text next to the answer so the operator reads the source, not the paraphrase.
• Route every dose or rate calculation through a deterministic calculator tool with unit checks, and display its inputs. The model may explain the number but must not produce it.
• Keep the SOP corpus under change control and separate it from free-text sources, so injected text in a work order cannot pose as procedure.

The same separation of model from actuator appears in other physical sectors; AI in Agriculture applies it to prescription maps and AI in Autonomous Vehicles to a driving stack with far tighter timing.

Worked example: a quiet night in DMA 7

Consider a utility with a pump-scheduling optimiser and a DMA leak model. On a normal night, DMA 7 takes about 30 m3/h through its inflow meter, its tank level holds steady, and estimated legitimate use is about 20 m3/h, so unexplained loss sits near 10 m3/h with a standard deviation around 1.5. At 02:00 the inflow meter starts reporting 18 m3/h. The leak model, which mostly looks at night flow, sees a quiet district and scores it low. The optimiser, seeing low demand, proposes cutting the booster pump setpoint from 4.2 to 3.0 bar.

Now the controls. The mass balance gives 18 - 0 - 20 = -2 m3/h of unexplained loss, a z-score of about -8: the district appears to consume more water than enters it, which is impossible with a steady tank, so the under-reading flag fires. The redundancy check agrees: the upstream bulk meter that feeds DMA 7 still reads 30 m3/h. And the last 12 inflow samples exactly repeat the 12 from the night before, so the replay check fires too. Separately, the guard rejects the pressure change: a 1.2 bar step exceeds its 0.3 bar maximum. Operators get one alert with three pieces of evidence, dispatch a technician, and find a tampered meter gateway. The model was fooled; the system around it was not.

Failure modes that are not attacks

Most bad outcomes will not be attacks. Plan for these as seriously:

Operational guidance

Fold these systems into the obligations you already have. In the US, the America's Water Infrastructure Act of 2018 requires community water systems serving more than 3,300 people to maintain risk and resilience assessments and emergency response plans; add every model, data link and vendor connection to them. In the EU, drinking water and waste water are sectors under NIS2. Operationally: inventory every AI component and its data paths; drill manual operation with all advisory systems off at least yearly; monitor the guard's rejection log daily; and include model and data compromise in your incident playbooks, following LLM Incident Response for the AI-specific steps.

Trade-offs

Every control costs something. Human approval slows response, which is why small changes need one approver and PLC interlocks, not people, handle sub-second events. A one-way data path makes cloud models staler and vendor support harder. Physics checks need meters many small systems lack, so the first AI budget is often better spent on instruments. And a read-only assistant demos worse than one that "just does it". That is the point.

What to do next

1. Change every default password on PLCs and HMIs and remove any internet exposure; check with an external scan.
2. Draw your data flows and confirm no model or vendor service can reach OT except through one approved conduit.
3. Deploy a setpoint guard with limits taken from your permit and engineering documents.
4. Implement the DMA mass balance and replay checks beside every anomaly model.
5. Freeze a reviewed set of confirmed events and gate model releases on it.
6. Strip write tools from any operator assistant and require SOP citations.
7. Add AI components to your risk assessment and run a manual-operation drill.