I am a tad bit surprised at how long the hype has lasted with GenAI and how quickly it poured over into Agentic AI. So much so that my LinkedIn is swamped with agent-this and agent-that, and people bragging about how quickly they built an agent over lunch. The surreal talk from frontier model vendors claiming singularity and curing all diseases probably isn’t helping matters as reality on the ground is quite a bit different.
Everyone would agree that AI is powerful and the vision of agentic AI having a positive impact on an organization is very real. However, harnessing that power is a different story altogether and it appears that the harnessing comes right down to the intelligence within your organization. AI, in this case, is very human-powered rather than machine-driven.
Unfortunately we find ourselves in a continuous, never-ending conundrum of test-it and fix-it, test-it and fix-it. Behind the slick demos and effortlessness on building agents another story is unfolding. It is the digital transformation that includes an army of forward deployed engineers, application teams, QA teams, security people and business users continuously getting the machine into shape and trying to keep it there.
There’s some hidden utility in this but also a lot of waste, and a lurking danger that we lose hard-won human skills while everyone races to learn a new set of AI ones.
I’ve wanted to use the image for this article for some time. It’s obviously AI generated and you’ve likely seen enough of these on social media to recognize it right away. However, I thought it was humorous enough to get the point across and I asked AI to generate a 1950s grindhouse horror poster for The Fixer and The Babysitter. And hopefully the image captured the mess. 🫠
The Babysitter
To be clear, I am a proponent of AI and it is powerful. But I’m less enthusiastic about how people perceive it, measure it and use it. I increasingly see people intervening heavily in AI-generated work that I question the productivity gain everyone is so proud of. In some cases, the gains may not exist at all or be quite the opposite.
Have we simply moved the work around, renamed it “AI-assisted,” and congratulated ourselves on the transformation. This is not a new phenomenon.
In fact, one enterprise recently mentioned that it is now spending three times more effort on testing and quality assurance than it ever did before AI entered the process.
Three times more.
That was both shocking to hear but also illuminating in several ways. It confirmed an underlying suspicion that the productivity gain wasn’t real in all cases, and it also highlighted just how much of a gap exists between expectation and reality. In fact, I’d go as far as to say that AI is in very real danger of dulling the edge that an organization has. Hint, that edge is in its people.
Separately, this “dulling” goes to the heart of AI’s averaging problem and the algorithm but that’s a separate article that’s already been written.
But back on the more important point of 3X investment in testing. That point was backed up recently by another organization that stated there were not enough humans in the loop to keep up with what AI was producing. It was said in the context of more and more humans are needed to check the quality of results coming off the AI assembly line. Even regulators are demanding more and more humans in the loop when it comes to AI.
The Fixer
The requirement for so many humans is not a minor operational adjustment. It is a rather awkward side-effect for what is in reality, just technology, but being sold primarily on its intelligence, it’ singularity (that’s sarcasm). Which means that if you believe the hype, this tech is just as good as the humans.
And this gets us back to the original premise of the title and the drift in another full time occupation of “The Fixer.”
The Fixer spends an enormous amount of time repairing AI mistakes, correcting the time span and sequence, removing hallucinations, deleting invented facts, rewriting awkward passages and quietly swearing (this is more my reaction) while typing the fifth corrective prompt into the same little box on the screen. Even in coding where AI can shine better than it can in most domains, it’s a constant challenge to stay ahead of every little nuanced mistake generated at machine speed.
And also to be clear and transparent, I use AI to code. But I do not use it to design, architect, or even engineer. I just expect to spoon feed it specs and have it generate code that I watch like a hawk, and test the s*t out of. The hard stuff of architecting, designing, and engineering, I leave to the humans as I’ve seen AI get it blatantly wrong so many times. BTW … It’s far easier to see the gaps and errors when you are innovating—that’s highlighting the braintrust in humans.
BTW that em-dash was me not AI and I’m not sure why I feel the need to justify it.
If you take AI seriously, you have to ask yourself why are so many humans required if AI is supposed to be this good? Or better yet, what’s a better way to scale it if it’s so upside-down?
Articulation and Sounding Smart
We know by now that AI is articulate and that’s the hook, but also the facade. It can produce polished language with such a high degree of articulated confidence yet missing so much under the covers. When you need something targeted, precise, accurate and recognizably in your own voice, you have to consider how much prompting, re-prompting, editing and copy-and-paste work you are actually doing.
I’ve spent countless hours correcting and fine tuning AI output to a point that it’s tedious. I get tired of being The Fixer.
When switching gears to give AI something more consequential such as writing code, analyzing contracts, evaluating financial information or … operating inside an enterprise workflow. It is marvellous until it is not and that “not” shows up more than anyone would like.
AI can do some serious damage, even by missing an important detail (again, this happens more than you think). This is why we humans watch it like a hawk, and coincidently it’s also why I’m tired of being a glorified Babysitter.
This raises the lurking problem of selling our future short. Are humans gradually being relegated to fixing the machine and babysitting the assembly line? I might be overly dramatizing this but sometimes, AI gets mind numbing.
Creating, designing, building, innovating are human traits we have to be careful not to lose. The braintrust and tribal knowledge are mission critical to every organization and the actual intelligence is in the humans when using AI. So I posit that we as humans are just using this technology all wrong.
I don’t think the answer is more humans in the loop. And I definitely don’t think the answer is better prompting.
The answer starts to look much more like command and control.
Machines monitoring machines. Automated testing and qualification. Policy enforcement while work is happening. Telemetry that tells people where intervention is actually required instead of requiring people to inspect everything. We’ve learned this the hard way operating millions of complex workflows and billions of individual agentic tasks every month. The control plane has to keep the humans informed as the humans can’t possibly keep the control plane informed.
So The Fixer and The Babysitter just becomes a cult-classic and not just a bad movie.