AI Doesn't Write Bad Code. It Writes Convincing Code.

Over the last month I’ve been putting Claude Code through its paces inside a structured mono-repo “Foundry” environment.

I’ll say it plainly: I’m impressed.

But I’m also uneasy.

I’m not using it as autocomplete on steroids. I’m using it inside a real system:

• A mono-repo with clear structure
• Specs and ADRs (Architecture Decision Records) committed to the repo
• Ubiquitous language documented
• Clear architectural boundaries
• Defined quality gates

The Controversial Thought

Here it is:

AI is not dangerous because it writes bad code.

AI is dangerous because it writes convincing code.

Bad code you catch. Convincing code slips through, because a rushed reviewer, a busy engineer, or a stakeholder who wants the ticket closed will read the AI output, nod, and merge it. The code looks plausible. It might even work for the happy path. But it embeds assumptions nobody thought through, patterns nobody validated, and edge cases nobody tested. This is a known failure mode: OWASP’s Top 10 for Large Language Model Applications catalogs several risk categories that boil down to “the output looked right, so nobody questioned it.”

What “Convincing But Wrong” Actually Looks Like

Two recent examples from my own work, both from a lending platform I consult on. Both AI-drafted. Both would have shipped if a rushed reviewer had signed off.

Example 1: The right shape, the wrong field.

I was building a corrective SQL script to handle a specific charge-off scenario for a small cohort of loans. Claude drafted the query that would extinguish the loans’ interest balance and mark them appropriately.

The output looked right. Correct table joins. Correct predicate structure. Arithmetic that reconciled. Comments where I’d expect them. If I’d been rushed, I’d have merged it.

But it was pulling the full pool balance instead of a specific deficit calculation. On 21 of 45 loans, that over-wrote what should have stayed on the books. $1,475 of legitimately-owed interest was extinguished: not lost, but marked as forgiven when it should have been collected.

An independent reviewer caught it during verification. Not through better technical skill, but through actually running the code against a snapshot of production state and diffing the result against what should have happened. The code review of the code itself would never have found it. The bug was in the shape of the question, not the shape of the answer.

Example 2: The confident correctness that ignored the world had changed.

Different project on the same platform, same class of failure. I asked the AI to build a corrective script that would apply amounts from a snapshot to fix a batch of loans that had been over-collected. The snapshot had been generated ninety minutes earlier during a manual review.

The script Claude produced was elegant. It read the snapshot, applied each amount to each loan inside a transaction, logged everything, handled errors gracefully. The kind of code that reads as obviously correct on first review.

What it didn’t do was re-verify state at fire-time. In the ninety minutes between snapshot generation and script execution, another process had already partially corrected some of those loans. Applying the snapshot amounts blindly over-collected $20,000 across 108 loans.

A rushed reviewer would have merged this. It has correct error handling, atomic transactions, defensive logging. Every code-review checkbox is checked. The defect isn’t in what the code does. It’s in what the code assumes about the world outside itself.

Both examples share a shape. The AI didn’t write buggy code. It wrote confident code that solved the wrong problem, with the right-looking syntax, the right-looking structure, the right-looking guardrails. The bug wasn’t visible in the diff. It was only visible if you replayed the code against real production state, or asked “what if the world changed between snapshot and execution?”

That’s what “convincing” means. Not that the AI is dishonest. That the output convinces a reviewer who’s only looking at the code, not at what the code is supposed to do.

The Working Hypothesis

Here’s what a month of real use has convinced me of:

AI performs dramatically better inside an opinionated, well-documented system. Without that structure, it amplifies chaos.

Give AI a codebase with clear boundaries, ADRs it can read, quality gates it must clear, and a ubiquitous language it can reason in, and it’s transformative. It compresses delivery time without cutting corners.

Give AI a codebase that’s a soup of inconsistent patterns, mystery abstractions, and undocumented decisions, and it will generate more of the same, faster. The output will look confident. The system will get worse.

This is why the Start with Good principle matters even more in an AI-augmented workflow. The structure that used to be optional for a small team becomes non-negotiable when you scale delivery with agents. The chaos gets scaled too.

What This Means If You’re Hiring a Developer

If you’re hiring a developer or agency that uses AI in delivery, the right questions aren’t “do you use AI?” or “is AI safe?” The right questions are:

• What structural conventions does your codebase enforce that AI has to comply with?
• Who reviews every AI-generated change, and against what?
• What quality gates run automatically before AI output can merge?
• Can you show me the ADRs or specs the AI is working from?

A vendor who can answer those specifically is using AI as a force multiplier for good work. A vendor who dodges or gives vague reassurances is scaling their chaos with your project as the vehicle.

AI is changing how I build. It’s not changing what I ship.