Day 9 · How I Knew My AI Scrum Master Wasn’t Doing Its Job

WHAT I BUILT TODAY

Reworked Sprint 3's estimates using a shared story point rubric instead of vague capacity percentages. Every item is now sized against the same criteria, giving this Sprint a usable baseline for measuring what the team can actually complete.

WHAT BROKE OR SURPRISED ME

I asked for the first story point estimate of Sprint 3's backlog. Then, before letting it run, I asked one more question: What methodology are you using to choose the number? There wasn't one. The instruction was essentially, "Give each item a number from 1 to 13." No shared criteria. No definition of what separated a 3 from an 8. It would have produced the same confident but empty estimate I was trying to replace, only dressed up in Fibonacci numbers. Fixing that led somewhere much bigger than a sizing rubric. I think we may be approaching agentic AI backwards. We keep asking what the agents can do. Maybe the better question is: if these were employees, how would I design and manage the organization around them? How would I know whether the Scrum Master was doing a good job? Does the system understand what good execution looks like for the business it is coordinating? If it doesn't, how would I recognize when it is confidently directing everyone toward the wrong outcome? Automating the labor does not remove the need for expertise. It moves the question from "Can this get done?" to "Does whoever, or whatever, is directing the work know what good looks like?" There is a real cost hiding inside that gap. There is the rework, but there are also the tokens spent pursuing the wrong direction with total confidence. At some point, it may genuinely be cheaper to hire the expert than to keep paying to discover afterward that nobody involved knew what they were doing. It was never about how many agents or sub-agents I could stand up. It was about whether the operating architecture gave them the right expertise, constraints, feedback, and authority.

WHAT I'M TRYING TOMORROW

Sprint 3 will run to completion without changing the sizing model, so the story points can produce one clean first data point. I will still intervene if something materially goes wrong, but I will not adjust the scale halfway through and pretend the resulting velocity means anything. What I'm really building toward is a redesign of the Scrum Master itself. The idea started with an almost technical distinction: the difference between a reasoning loop and an operating loop. A prompt can make an agent reason in a loop. It cannot make the agent wake itself back up later. Something outside it must provide the heartbeat. That became the larger design principle: earn autonomy through loops, not through promotion. Instead of deciding upfront how much authority to hand over, the redesigned Scrum Master will begin by observing, reasoning out loud, and proposing actions without having the authority to execute them. It will advance only after its judgment has been checked against reality repeatedly. Trust will come from evidence, not confidence.

QUESTION I'M SITTING WITH

If the real bottleneck is not the number of agents, but whether the system directing them understands the business it is running, then adding more automation before fixing that understanding only scales the confidence, not the competence. I don't know yet whether a system can earn enough trust through repeated evidence to close that gap, or whether some part of the operating loop will always require a human who has actually done the work.