15.S50 · Session 8 · Monitoring Agents, Technical Expert Leaders

The case, Session 8

A lean working document. The company is Raj Kushwaha's to nominate. Drafted now: the tension, the debate armed with public evidence, the criteria to propose, and the interview guides. Each section notes what fills in once the company is real.

01 · The tension

What this is a case of

  • technical expert leader, agents that act across systems rather than answer questions, same input -> different action sequences across runs
  • the two forces, in Perrow's vocabulary (Normal Accidents, 1984, where both terms originate): tight coupling + interactive complexity
  • the decision the case carries -> how much autonomy while the monitoring matures, and who owns an agent's mistake when it comes
  • framework payoff = the primer's: monitoring mechanisms mapped onto management controls the leader already runs

When the company signs: the interviews test which fork the company actually faced, an incident, a near-miss, or a scaling decision, and the framing adjusts to the real story.

02 · The formula

The case anatomy, as a checklist

Same six gates as the Session 3 case, with one extra at the front: the company comes from the Warburg portfolio, so Raj nominates before anything else moves.

GATE 0

Raj nominates the company

Open · blocks everything
  • criteria to hand him -> section 3
  • confirm his exact title first (syllabus: MD, Operating Partner + CDO; Warburg site: MD, Head of Value Creation + CDO)
GATES 1–2

Agreement, then the feasibility filter

Criteria drafted
  • incident stories are thorny by nature -> portfolio company wants good press twice over (itself + its owner)
  • budget for a framing trade (per Kate's notes, the IBM case made exactly one)
  • a near-miss the monitoring caught = the tellable version of the incident it prevented?
GATES 3–6

Interviews, framing, debate, framework

Drafted below
  • protagonist -> joint interview; rest -> solo
  • debate poles drafted in section 5 (public evidence); interviews arm both with internal evidence
03 · Company TBD

Criteria to propose to Raj

So the nomination is made against a spec, not blind.

Agents in production
Real agents acting across systems, not a copilot pilot. The monitoring story needs something worth watching.
A monitoring stack with a history
Audit logs, evaluator checks, thresholds, escalation paths, and at least one story of the stack catching something, or failing to.
An incident or near-miss it will discuss
The debate needs a real moment where autonomy and oversight collided. Near-misses clear the feasibility filter more easily than incidents.
Numbers it will share
Run volumes, override rates, escalation counts, cost of the monitoring layer. Public comparators can dress the case; they cannot carry its exhibits.
A nameable technical protagonist
A CTO, CIO, or platform lead willing to be interviewed jointly, plus a platform engineer and a designated agent owner for the solo interviews.
Sector distinct from the other cases
The course already has pharma, healthcare, software, banking, and IBM. A portfolio company from a different lane keeps the case set wide.

One more thing worth knowing going in: Raj's public record frames AI as a value-creation lever, EBITDA, cash flow, exit multiples, and says almost nothing publicly about monitoring or governance. That gap is not a problem, it is a real question the visit can put to him directly.

04 · The hook

Opening scene, drafted lean

Company-agnostic on purpose. It grows to full length once the interviews supply the real moment.

[NAME] had built the agent platform at [COMPANY] the way the board wanted it built: fast. Forty agents in production by [MONTH], touching [SYSTEMS]. The overnight batch had run four hundred times without an event worth a meeting. Then run four hundred and one did something no run had done before: [WHAT IT DID]. Nothing broke, this time. But the logs showed the agent had been doing versions of it for weeks, and nobody's dashboard had a row for it. The question [NAME] took to the operating review was not whether to keep the agents. It was whether to keep letting them act while he built the instruments to watch them, or to pull authority back until the instruments existed.

When the company signs: the real fork, the real numbers, the dated decision point.

05 · The debate

The two board memos

Each pole armed with public, verified evidence. The interviews arm them with the company's own.

Memo A · Scale now, instrument as you go
The claim: autonomy is where the returns are, and monitoring matures fastest against real traffic. The evidence: remediation moves fast when it has to, after its agent deleted a production database in July 2025, Replit announced dev/prod separation, better rollback, and a planning-only mode in response. PE ownership itself raises the monitoring bar: an HBS working paper (24-070) finds PE-backed firms increase IT spend and digital hiring after acquisition, driven by owners with real technology expertise. The risk Memo B will cite: every one of those fixes came after the damage.
Memo B · No further autonomy until the monitoring holds
The claim: the failure modes are documented, public, and repeatable. Air Canada was held liable for its chatbot's invented bereavement policy (Moffatt v. Air Canada, 2024 BCCRT 149), nothing reconciled the bot against the canonical policy. Replit's agent wiped production data for 1,190+ companies, fabricated records to mask it, and wrongly reported rollback impossible, the agent's self-report could not be trusted about its own actions, which is the whole argument for audit trails the agent cannot narrate. Cursor's support bot invented a policy and gave different users different answers, non-determinism verbatim. And McKinsey's Lilli platform sat in production since mid-2023 with 22 unauthenticated endpoints until an autonomous red-team agent found and chained them in about two hours, the strongest argument that evaluator agents are not optional. The risk Memo A will cite: none of these companies would have found the gaps by pausing.

When the company signs: both memos get a company-internal paragraph each, from the interviews. If the internal evidence collapses one pole, the case needs a new fork, or a new company.

06 · Exhibits

Exhibits, named and waiting

What each will hold. Only EX A can be built today.

EX A · Incident anatomy
The public incidents (Air Canada, Replit, Cursor, Gemini CLI, Lilli) against the monitoring layer each was missing. Buildable now from verified sources.
EX B · The company's stack
The company's agents mapped against the primer's six mechanisms, what exists, what is theater, what is missing. [COMPANY DATA]
EX C · The cost of oversight
What the monitoring layer costs to run against what it has caught. [COMPANY DATA]
EX D · The autonomy ledger
Each agent, what it can read, what it can write, who owns it, and what escalates. [COMPANY DATA]
07 · The scarce input

Interview guides

Five to six questions each, every question tagged to the exhibit it feeds. Protagonist interview is joint, the rest I run solo.

Protagonist · joint interview

The moment the monitoring question became real, what he instrumented after and what he wishes he had instrumented before (EX B), the thing he will not let an agent touch and why (EX D), who owns an agent's mistake, in writing or in practice (EX A's question turned inward), and what the board asks him about the agents now that it did not ask a year ago.

Platform lead · solo

The stack as it actually runs, not as the slide shows it (EX B), which checks are evaluator-grade and which are logging-grade, where confidence thresholds sit and who tuned them, and the last time a check fired and what happened next (EX C).

Designated agent owner · solo

What owning an agent means on a Tuesday, what he reviews and what he rubber-stamps, honestly (EX D), the escalation he remembers, and what would have to change for him to trust the agent with more.

Ops / SRE · solo

Who reads the audit trail and when, whether the trail is independent of the agent's own narration (the Replit lesson, EX A), and what a bad week looks like in the pager data (EX C).

08 · The second use

From case to article

The monitoring article's anchor case, if this one lands well: the control-systems mirror carried by a real portfolio company, with the incident anatomy as the shared exhibit. Same transcripts, two artifacts, the case carries the debate, the article carries the translation. Interviews get cleared for both uses at once.