On a sailboat, there is always a Watchkeeper—even when the boat is on autopilot.
Be honest: are you actually reading everything AI produces?
- Code changes.
- Documents.
- Analysis.
- Agent actions.
- Workflow changes.
Or are you increasingly scanning, approving, and moving on? And then, eventually, turning on auto-accept so you don’t even have to scan anymore?
Why this happens
AI agents don’t make oversight easier. They can produce more output, more changes, and more requests for approval. And they present that information in the form most convenient for the AI—not the form most useful for the human. Eventually, the human becomes the auto-approval mechanism rather than the approver.
The agent asks, the human clicks, the agent continues, repeat.
That isn’t human oversight. It’s a mechanical approval loop that gradually trains the human to stop paying attention.
What watchkeeping actually means on a sailboat
A skipper doesn’t continuously inspect all components of the boat. She has an intuitive sense of:
- current conditions
- what’s normal
- what’s changing
- what’s unusual
- when she needs to take the helm
She can go below deck because she knows when she needs to be on deck. That is the critical distinction. A good skipper needs to know what matters and when she needs to intervene.
Autonomy without responsibility
With AI, what you get is:
- “Here’s 47 changes.”
- “Here’s 12,000 lines of output.”
- “Here’s my reasoning.”
- “Approve?”
That’s not watchkeeping. That’s information dumping.
The human cannot reliably distinguish between what is routine and what is critical or where the AI model may have made a mistake when everything arrives as giant blobs of output.
And the more autonomous the agent becomes, the worse this can get. More autonomy can mean more actions. More actions can mean more output. More output can mean less meaningful human attention. Until eventually, the human is no longer really keeping watch.
Structure is essential
Human oversight doesn’t work without structured reasoning. The system needs to make the important things visible in a structured form consumable both by humans and software tools.
The human shouldn’t have to read everything. AI should make it possible to know what needs attention. The goal isn’t to put a human in front of every AI action and ask for approval. The goal is to give humans enough structured information to recognize when intervention is necessary.
And that may not always be in the interest of the model or agent creators. The more an agent can do autonomously, the more valuable autonomy can appear to be. But an enterprise has a different objective: making good decisions while remaining accountable for them.
Make humans better watchkeepers
A skipper doesn’t keep watch by reading everything happening on the boat. She keeps watch because the boat gives her a structured view of what matters.
The future of AI isn’t simply about making agents more autonomous.
It should include how to make humans better watchkeepers.