The Second AI Didn't Find My Bugs. It Found a Gate That Looked Unnecessary.
Every AI service has blind spots. So I have a second one review the work, and the first one review the review.
Having one AI service check another isn't new. I did it expecting cosmetic notes, but the second time, it found a quality gate that looked like overkill.
I had another AI service review work I built with my go-to AI service. I'm not naming either one, and I'm not scoring one against the other. The point is the method.
Why I asked
My workflow gates are critical, and not every project needs the same gates. I ran a team review of them about a month ago, and another pipeline review is coming up. I keep an eye on them because I feel better when I do.
Even so, I didn't want to do it all in a silo. I wanted another perspective from an AI service that hadn't helped build the system.
What it found
The first round, on a website I built, brought mainly cosmetic changes.
The second round was on my QC team, the AI agents that review and test my work. I took the feedback back to my go-to AI service. It called some points overboard, but it also found gaps that needed to be filled. This time, the findings were more than cosmetic.
One was a gate that looked expendable. Looking at it again, I realized the gate itself was fine. The problem was how it was being applied. I run so many different projects with different requirements that the conditions for my gates had become convoluted over time. Some gates, like heavy compliance checks, belong on higher-risk work. Other projects don't need them.
Railroad switches
Forcing every gate on every project would eat up tokens. So I'm adding switches, like railroad switches, that send each project down the track it needs. I know where the gap is, and I'm working on it now.
This, too, is a known pattern: shared gates, with each project switched onto the track that fits its risk. Every project will keep the essential gates. The rest are chosen by the project's requirements and risk, not by its size. And if a project's track is unclear, the stricter one will win.
What I took from it
Don't depend on one AI service to validate its own work. Have another challenge it. Then take that feedback back to the first.
It's interesting what happens when the two disagree, especially when you make them explain why. AI services seem a bit less sensitive than people. They push back, then use the good feedback.
Neither gets the final vote. I do.
If you have a software project that's stalled, a development process that needs another set of eyes, or work you've won but can't staff, I'd be interested in hearing about it. Message me on LinkedIn or book a 20-minute call.