THIS MONTH'S FIELD SIGNAL

 The Oversight Illusion 

There's a phrase that comes up in nearly every AI vendor conversation, and increasingly in internal conversations about AI governance: human in the loop.

It sounds like a safeguard. And in many organizations, it's being treated as one — the assurance that humans haven't been removed from consequential decisions, that someone is reviewing what AI produces before anything irreversible happens.

But over the past few months, I've noticed a pattern worth naming. In many of the organizations I speak with, "human in the loop" has quietly become a rubber stamp. A human is technically present. They're approving things. But they're approving based on a summary, with no real visibility into how the recommendation was generated — and often under enough time pressure that deep review isn't realistic anyway.

That's not oversight. That's the appearance of oversight.

The deeper issue is that "human in the loop" only tells you where the human is positioned in a process. It doesn't tell you who designed that process — or whose assumptions are already baked into the frame by the time the human arrives to review it.

Last month I spent three days at AI Field Day 8, listening to enterprise AI vendors present to a small group of independent practitioners. What struck me most wasn't the technology. It was how much each vendor's product decisions revealed about their philosophy on human control — and how much those philosophies varied beneath the surface of similar-sounding language.

It got me thinking about a distinction I find increasingly useful: the difference between a human who is in the loop and a human who is at the helm. The first is a checkpoint. The second is a starting point — someone who brings irreplaceable judgment, context, and strategic direction before AI shapes the thinking, not after.

For some tasks, a checkpoint is exactly right. For others, ceding the starting point to AI is ceding more than most leaders realize. I explore this distinction in depth — and what it reveals about the AI vendors you may already be working with — in a new piece on Medium.

FROM THE FIELD

A Real-World Example

A change leader at a mid-size professional services firm told me recently that her organization had implemented an AI-assisted process for drafting internal communications — reorganization announcements, policy updates, leadership transitions. The outputs were reviewed and approved by HR before going out.

On paper: human in the loop.

In practice: the AI was framing the message — the tone, the emphasis, what got acknowledged and what got minimized — and the review was happening under enough time pressure that substantive changes were rare. The humans were signing off on AI's version of a sensitive story rather than bringing their own read of the room to it first.

"We kept approving things that felt slightly off," she said. "But we couldn't always articulate why, and there was always pressure to move."

That slight-off feeling is often the signal that the helm moment has already passed.

QUESTIONS FROM THE TRAIL

Ask Yourself

1. Where in your AI workflows is a human technically present but not meaningfully engaged with the reasoning behind a recommendation?

    

2. For which decisions in your organization does the framing matter as much as — or more than — the answer? Are humans setting that frame, or letting AI define it?

    

3. If you asked your AI vendors to show you exactly what a human sees when they're "in the loop," what would you find?

TOOLS FROM THE TRAIL

Perplexity

If the oversight question resonates, one low-stakes place to start building the habit of asking "how did you get there?" is Perplexity. Unlike many AI tools, Perplexity surfaces its sources alongside every response — making the reasoning visible rather than just delivering a conclusion.

For leaders who want to develop their own instinct for interrogating AI output rather than accepting it, tools that make sourcing visible are a useful place to start. The habit of asking where did this come from? is worth building at the individual level before trying to design it into organizational workflows.

ONE SMALL STEP

A Resource to Help

If you're evaluating AI vendors — or revisiting ones you've already selected — I put together a short framework to help cut through the demo and get to the assumptions that actually matter.

It's called "Beyond the Demo: What to Ask AI Vendors About Human Control." It's a one-page guide organized around the questions that will tell you more about a vendor's philosophy on human oversight than any product walkthrough will.

A quick note: You’re receiving this email because we’ve connected at some point about AI, leadership, or communications/marketing, and I thought you might find these field notes useful. I’ll send this about once a month. If it’s not helpful, you can unsubscribe anytime using the link below—no hard feelings.