AI in Rail, in depth: vital versus advisory functions, the EN 50716 and TS 50701 landscape, the attack surface and a restrict-only safety envelope

Rail safety from first principles

Four ideas from railway signalling decide everything else.

• Fail-safe. A failure must lead to a more restrictive state. A broken signal shows red; a lost radio link brings the train to a stop. Any AI component has to fit this asymmetry.
• Vital and non-vital. Vital functions prevent collisions and derailments: the interlocking, which refuses to set conflicting routes, and automatic train protection (ATP), which enforces speed and movement authority and brakes the train if the driver or the automation does not. Everything else, including timetabling, passenger information and much of maintenance, is non-vital.
• Safety integrity levels. Rail standards grade safety functions from SIL 1 to SIL 4, with a lowest "basic integrity" tier (called SIL 0 in older software standards) for functions with no safety role. Interlocking and ATP are typically SIL 4.
• ATO is not ATP. Automatic train operation drives the train, controlling traction and braking for punctuality and energy, but it runs under ATP supervision and is usually developed to a lower integrity level. Grades of Automation (GoA 0 to 4, IEC 62290 for urban guided transport) describe how much of the driving and supervising is automated, not how safe the system is. A GoA 4 metro is driverless; its safety still rests on ATP and the interlocking.

That last point is the most common error in writing about AI in rail. The question is never "can ML drive the train" but "can ML take over any part of a vital function". Today the practical answer is no; ML supplies information and can trigger more restrictive behaviour.

Where AI is used today

Every row is non-vital or restrictive-only. That is not timidity: it is what the evidence base currently supports.

Reference architecture

The architecture follows a pattern older than ML: a lower-integrity function may only make the system more restrictive, and an independent higher-integrity function always has the last word. The envelope between the ML detector and the vital layer is small, deterministic and reviewable, which is exactly what the ML model is not. The LLM assistant has no network path to operational technology at all; it reads controlled documents and answers people.

Standards and the certification gap

The European standards family, widely used beyond Europe, is the reference point.

The gap is visible in the third row. Software standards prescribe techniques such as formal specification, structured design and coverage-based testing, none of which say how to argue that a neural network meets a failure-rate target. Until sector guidance closes that gap, the defensible options are to keep ML at basic integrity and wrap it, or to argue safety for the wrapper rather than for the model. Treat any claim of "SIL 4 AI" with suspicion.

The attack surface

Rail is a public, physical and long-lived system, so its attack surface is broad. Two real cases set the tone. In August 2023 trains in Poland were brought to emergency stops by the Radio-Stop signal, a sequence of tones on the analogue train radio that carries no authentication; arrests followed. In December 2023 researchers from the Dragon Sector group reported code in Newag Impuls trains that locked units after they spent time at independent repair workshops, which the manufacturer denied. Neither involved ML, but both show the two lessons that do apply: unauthenticated inputs get abused, and embedded logic needs independent integrity checks.

• Sensor attacks on perception. Adversarial patches, lasers or projected images aimed at obstacle detection. Because the envelope is restrict-only, the worst realistic outcome is a spurious stop, a denial of service rather than a collision, unless the attack suppresses a real detection.
• Positioning spoofing. GNSS jamming and spoofing affect any function that uses satellite position. Vital positioning in ETCS today rests on balises and odometry, not satellites.
• Data poisoning. Maintenance models trained on work orders and inspection labels can be skewed by a compromised contractor feed, hiding a class of defects.
• Model supply chain. A substituted model file on a train computer is a firmware attack. Sign models and verify them at load, under the same zones and conduits as other software.
• Prompt injection. An LLM assistant that ingests supplier PDFs or work orders can be steered by text inside them; see prompt injection via RAG.

A restrict-only safety envelope

The envelope turns ML detections into at most one action: a brake request to the vital layer. It must be simple enough to verify exhaustively, and it must fail towards safety when the ML component is stale, silent or contradicted.

from dataclasses import dataclass

@dataclass
class Detection:
    distance_m: float
    confidence: float
    lidar_confirmed: bool
    age_s: float

def stopping_distance(v_ms, decel_ms2, latency_s):
    return v_ms * latency_s + v_ms ** 2 / (2 * decel_ms2)

def envelope(v_ms, detections, ml_heartbeat_age_s, cfg):
    """Return 'none', 'service_brake' or 'emergency_brake'. Restrict-only by construction."""
    assert cfg.service_factor > 1
    if ml_heartbeat_age_s > cfg.max_heartbeat_s:
        return "service_brake"            # a silent detector never means a clear track
    need = stopping_distance(v_ms, cfg.emergency_decel, cfg.latency_s) * cfg.margin
    worst = "none"
    for d in detections:
        credible = d.lidar_confirmed or d.confidence >= cfg.camera_only_threshold
        if not credible:
            continue                      # threshold is a safety-case parameter
        if d.age_s > cfg.max_detection_age_s:
            if d.distance_m <= need * cfg.service_factor:
                worst = "service_brake"   # stale data degrades, it never clears
            continue
        if d.distance_m <= need:
            return "emergency_brake"
        if d.distance_m <= need * cfg.service_factor:
            worst = "service_brake"
    return worst
# There is deliberately no function that releases a brake: release belongs to the
# driver or to ATP, never to this module.

Three properties are worth testing exhaustively over a grid of distance, age, confidence, lidar flag and heartbeat age: the output never becomes less restrictive as a detection gets closer; stale data cannot produce "none" when fresh data would brake; and the module has no output that grants movement authority. Ignoring low-confidence camera-only detections is a real trade-off between spurious stops and missed obstacles, so the threshold belongs in the safety case, not in a config file someone can quietly change.

Worked example: how far must the detector see?

Consider a mainline train at 120 km/h, which is 33.3 m/s. Assume an end-to-end latency from image capture to brake application of 1.5 s and an emergency deceleration of 0.9 m/s²; both are illustrative and vary widely by rolling stock and rail conditions. Then:

• Distance covered during latency: 33.3 × 1.5 ≈ 50 m.
• Braking distance: 33.3² / (2 × 0.9) ≈ 617 m.
• Total ≈ 667 m; with a 1.2 margin, the envelope must act on credible detections within about 800 m.

Now the engineering problem is visible. A person 1.8 m tall at 800 m subtends about 2.25 milliradians. A camera with a 10° vertical field of view and 1,080 rows gives roughly 0.16 mrad per pixel, so the person is about 14 pixels tall, before rain, glare or curvature. Detection at that range needs a narrow telephoto camera, which loses curves, or fused lidar or radar. It also means one wet day with a lower adhesion figure pushes the requirement past what the sensor can deliver, so the safety case must state the conditions in which the function is valid and what happens outside them, typically a lower line speed.

LLM assistants for maintenance

LLM assistants for maintenance staff are the fastest-growing AI use in rail and the easiest to get wrong, because an answer that cites the wrong revision of a procedure can put a technician in danger. Treat the assistant like any document control system:

def answer(question, retriever, llm, doc_control):
    hits = [h for h in retriever.search(question, k=8)
            if doc_control.is_current(h.doc_id, h.revision)]       # stale revisions never reach the model
    if not hits:
        return Refusal("No current controlled document covers this. Ask your supervisor.")
    draft = llm.generate(question, sources=hits, rules="Quote procedures verbatim with doc id and revision.")
    for cite in draft.citations:
        if cite.quote not in doc_control.text(cite.doc_id, cite.revision):
            return Refusal("Could not verify the procedure text. Use the controlled manual.")
    if draft.classifies_as("safety_procedure_change"):
        return Refusal("Procedure changes require engineering approval.")
    return draft.with_footer(f"Sources: {[(c.doc_id, c.revision) for c in draft.citations]}")

The assistant has read access to documents only; it has no tools, no work-order write access and no connection to signalling or train systems. Ingested supplier documents are scanned for instruction-like text before indexing, and the assistant's logs feed the same review process as other safety reports.

Operating AI in a safety case

• Change control. Each retrain is a modification under the safety case: version the data, re-run the evidence suite, and get sign-off before deployment, as for any software change.
• Seasonal drift. Leaf fall, snow and low sun change both images and adhesion. Slice validation by season and route and monitor detection confidence per slice.
• Operator feedback. Every spurious brake request and every missed defect found by humans becomes a labelled case.
• Incident handling. Combine rail occurrence reporting with AI-specific triage, following LLM incident response.
• Comparable regimes. Medical devices face the same tension between learned behaviour and conformity assessment.

Failure modes

• Scope creep into vital functions because a model performs well in trials.
• Automation bias: dispatchers accept re-plans without checking, so an advisory tool becomes a de facto decision-maker.
• Unstated operating domain: a detector validated in daylight is used at night.
• Alarm fatigue from spurious brake requests, leading to the envelope being disabled.
• Stale documents in the assistant index after a procedure revision.

What to do next

1. Inventory every AI component and label it vital, restrictive-only or advisory; anything that could relax a restriction is a design defect.
2. For each restrictive component, write the envelope as a small deterministic module and test the three properties above exhaustively.
3. Write the operating domain for each perception model: speed, weather, lighting, route, and the fallback outside it.
4. Map AI components into CLC/TS 50701 zones and conduits, and sign and verify model files at load.
5. Put the LLM assistant behind document control with current-revision filtering and verbatim citation checks, and remove any write or network access it does not need.
6. Track spurious-brake and missed-detection rates per season and route, and feed them into the safety case review.