AI Ethics Board, in depth: forms, charter, independence, triage, decision records and what makes a board matter

Three forms of board

Boards come in three broad forms, and conflating them is the root of most design mistakes.

Many organizations run two layers: an internal committee that gates every high-impact launch and an external council consulted on the hardest cases and on policy. Meta's Oversight Board is the best-known example of the third form; it is funded through a separate trust and its decisions on individual content cases are binding on the company, while its policy recommendations are advisory and require a public response. That split, binding on narrow cases and advisory with a mandatory answer on broad ones, is a pattern worth copying at smaller scale.

Lessons from boards that failed

Google announced an external Advanced Technology External Advisory Council in March 2019 and dissolved it about a week later after controversy over its membership. It had not yet met. The lesson is that membership is part of the board's legitimacy and has to be designed and consulted on before launch, not announced.

Axon's AI ethics board shows both outcomes. In 2019 it recommended against putting face recognition on police body cameras, and the company followed that advice. In 2022 the board voted against even a limited pilot of taser-equipped drones; the company then announced a broader plan for such drones in schools without consulting the board, and nine members resigned. The lesson is that consultation must be an obligation with a defined sequence, so that the board sees a decision before it is announced, and that declining advice must be done openly with reasons rather than by going around the board.

A third, quieter failure is common: boards that meet quarterly, receive slide decks prepared by the teams being reviewed, and are never told about the launches that skipped them. Nothing fails publicly; the board simply has no effect.

The charter as an interface

The charter is the board's interface definition. Write it so that each clause can be checked. A minimal charter in machine-readable form, which can also drive the intake tooling:

board: ai-ethics-board
scope:
  in: [new AI features with tier >= 2, material model changes, new data sources about people,
       uses in employment, credit, health, education, policing, children]
  out: [internal tooling with no effect on external people]
decision_rights:
  tier_2: binding launch gate (internal committee)
  tier_3: recommendations; company must respond in writing within 30 days
  veto: [deployments in prohibited-use list]       # the only unconditional veto
information_rights:
  - full access to evaluation results, red-team reports and incident logs for any case
  - may interview engineers directly, without management present
  - notified of every tier >= 2 launch, including those that claim an exemption
independence:
  term_years: 3
  staggered: true
  removal: only for cause, by two-thirds of the board
  budget: fixed for 3 years, approved by audit committee, not by the product org
  compensation: flat fee, independent of decisions
publication:
  annual_report: required
  case_summaries: published unless legally restricted; restriction itself is reported
conflicts:
  register: required, updated yearly
  recusal: mandatory for any case touching a member's employer, funder or investment
quorum: 5 members including at least 2 external

Three clauses do most of the work. Information rights stop starvation, where the board judges only what it is shown. The duty to respond in writing turns advice into accountability without pretending the board runs the company. Removal only for cause, with a budget the product organization cannot cut, is what independence actually means in practice.

Membership and independence

Aim for seven to eleven members: enough to cover the domains and sustain quorum with recusals, few enough to deliberate. Cover technical machine learning, law, the domains where the company deploys (health, finance, education), and the communities most affected, including people with lived experience of the harms in scope, not only experts who study them. Stagger terms so a third of seats turn over each year. Pay members a flat fee and publish it; unpaid boards skew toward people who can afford to volunteer, and fees tied to outcomes undermine independence. Keep a conflicts register and apply recusal mechanically from it, so recusal is not a judgement call made under social pressure in the meeting.

Intake and triage

A board that reviews everything reviews nothing well. Triage routes each proposal to the lightest review that is still adequate. Score impact on people, not technical novelty, and make the scoring visible so teams cannot argue their way down a tier in private.

from dataclasses import dataclass

@dataclass
class Proposal:
    name: str
    affects_rights: bool        # employment, credit, housing, liberty, benefits, education
    vulnerable_users: bool      # children, patients, people in crisis
    scale: int                  # people affected per month
    reversibility: int          # 0 = easily reversed ... 3 = irreversible
    autonomy: int               # 0 = suggestion to a human ... 3 = acts with no review
    novel_data_about_people: bool
    on_prohibited_list: bool

def tier(p: Proposal) -> tuple[int, int]:
    if p.on_prohibited_list:
        return 4, 99                                  # goes to the veto path
    score = (4 if p.affects_rights else 0) + (3 if p.vulnerable_users else 0)
    score += 1 if p.scale > 10_000 else 0
    score += 2 if p.scale > 1_000_000 else 0
    score += p.reversibility + p.autonomy
    score += 2 if p.novel_data_about_people else 0
    t = 1 if score < 4 else 2 if score < 9 else 3
    if p.affects_rights and p.autonomy >= 2:
        t = 3                                         # floor: automated rights decisions
    return t, score

The floor rule matters. Teams under deadline pressure will split a feature into parts that each score low. Floors keyed on the combination that matters, here automated decisions about rights, close that gap, and a periodic audit that compares shipped features against intake records catches the rest.

Decision records and response tracking

Every tier 3 case ends in a decision record: the question asked, the evidence reviewed, each recommendation with its rationale, dissents, and a response deadline. The company's response, accept, modify or decline with reasons, is attached to the same record. Tracking those responses is the single most useful thing the board's secretariat can automate.

from datetime import date, timedelta

def overdue(records, today=None, response_days=30):
    today = today or date.today()
    late, unverified = [], []
    for r in records:
        for rec in r["recommendations"]:
            due = r["decided_on"] + timedelta(days=response_days)
            if rec.get("response") is None and today > due:
                late.append((r["case_id"], rec["id"], (today - due).days))
            elif rec.get("response") == "accept" and not rec.get("verified_on"):
                unverified.append((r["case_id"], rec["id"]))
    return sorted(late, key=lambda x: -x[2]), unverified

def adoption_rate(records):
    recs = [x for r in records for x in r["recommendations"] if x.get("response")]
    return sum(x["response"] in {"accept", "modify"} for x in recs) / max(len(recs), 1)

Note the second list: an accepted recommendation that nobody verified is a promise, not a control. Link each accepted recommendation to a concrete control in the AI risk register and to the evaluation that proves it works, so the board's output becomes part of the release evidence.

Worked example: voice cloning

A customer-service platform proposes voice cloning: a business uploads a few minutes of a staff member's voice and its support bot speaks in that voice. Triage: no rights decision, but novel biometric data about people (2), over a million calls a month (3), hard to reverse once a voice model leaks (2), fully automated speech (3). Score 10, tier 3.

The board reviews the red-team report, the consent flow and the abuse data from a pilot. It issues four recommendations: verified, revocable consent from the voice owner with liveness checks; disclosure to callers that the voice is synthetic; watermarking of generated audio; and deletion of the voice model on revocation within a defined period. One member dissents, arguing the feature should be limited to the owner's own business account. The company accepts three, modifies disclosure to a spoken notice at call start instead of per utterance, and publishes its reasons. The tracker flags watermarking as accepted but unverified until the evaluation team shows detection rates on compressed phone audio. That is the board working: specific, answered, verified, and visible.

Operations and metrics

Meet monthly with an urgent path that convenes a quorum within five working days. Circulate pre-reads a week ahead, written by the secretariat from primary evidence, such as evaluation results, model cards and incident logs, not by the proposing team alone. Track board health with a small set of numbers: share of tier 2 and 3 launches that went through review (measured against shipped features, not intake forms), median time from intake to decision, response rate within deadline, adoption rate, and the count of accepted recommendations not yet verified. An incident in a reviewed system should reopen its case, using the same intake as incident response, so the board learns from outcomes.

Failure modes

• Ethics washing. The board exists for the press release. Symptom: no published responses, no declined recommendations, no cases reopened.
• Information starvation. The board judges curated decks. Fix with charter information rights and direct engineer interviews.
• Late review. Cases arrive a week before launch, when the only options are rubber stamp or public fight. Require intake at design review.
• Scope evasion. Features split or relabelled to dodge tiers. Use floor rules and a shipped-versus-reviewed audit.
• Capture. Members drawn from the company's partners or funded projects. Use a conflicts register and external nominations.
• Overload and burnout. Unpaid members reviewing dozens of cases. Triage harder and pay properly.

Trade-offs

Binding authority gives the board teeth but makes the company cautious about what it brings, so scope shrinks; advisory authority widens scope but depends on the duty to respond. External members bring legitimacy and outside views but need time to learn the product; internal members are fast but conflicted. Publishing case summaries builds trust but may chill candour in deliberation, which is why many boards publish outcomes and reasons but not attributed discussion. A practical default is a binding internal committee for tier 2, an external board with a mandatory response for tier 3, an unconditional veto only for a short prohibited-use list, and published annual reports. The NIST AI RMF Govern function gives a vocabulary for documenting these choices.

What to do next

• Decide which of the three forms you are building, and write it in the first line of the charter.
• Draft the charter with information rights, a written-response duty and a deadline, removal only for cause, and a protected budget.
• Publish a short prohibited-use list; it is the only place an unconditional veto belongs.
• Implement intake and the tiering score with floor rules, and require intake at design review, not before launch.
• Stand up a decision-record store and an overdue-response tracker; report both monthly.
• Link every accepted recommendation to a control in the risk register and an evaluation that verifies it.
• Audit shipped features against intake records each quarter to measure how many launches skipped review.
• Publish an annual report with case summaries, response rates and declined recommendations with reasons.