How to quality check AI-generated content before it ships

A few weeks back one of my own drafts got held at the door. Not edited, not softened. Held. The scanner flagged an em dash and a banned filler phrase, and the critic scored it in flag territory for being generic. Nobody overrode it by hand. The code just said no and told me why. That is the whole point of this guide: build a system that catches your own slop before a reader does.

The short answer

Run every AI draft through two separate gates before it publishes. The first is a plain script that scans for mechanical tells like banned punctuation and filler phrases. The second is an AI critic that reads the piece cold and scores how generic it is. A third piece of code, not the critic, makes the final publish or hold call.

Why this matters right now

Everyone is publishing AI-assisted content now, and most of it reads like nobody checked it. Google has a real, named policy for this. It calls out using generative tools to produce many pages without adding value for users, and it treats that as a spam problem. It means AI writing needs a gate before it ships, the same way code gets gated before a deploy.

The fix is a door: a checkpoint the content has to pass through, with rules a machine enforces every single time, no matter how tired the person publishing is.

The two-layer gate, at a glance

This is the pattern I run in production on my own content pipeline, described generically so you can build your own version regardless of what tools you use.

The reason this works is that the two layers do not overlap. The scanner cannot get fooled by good prose around a bad word. The critic cannot get fooled by a clean sentence with no point. Between them, most of the ways AI content goes wrong get caught.

Step 1: build the deterministic tell scanner

Start with the mechanical stuff. This is the cheap, boring layer, and it should stay that way. My own scanner runs a handful of pattern checks over the raw text and returns a list of findings, each one tagged with a severity: block or warn.

1. Ban the punctuation that reads as machine-written the fastest: em dashes and en dashes. Rewrite the sentence rather than swapping in a comma. A comma standing where a dash was almost always reads worse.
2. Ban emoji outright on anything serious.
3. Build a short list of generic filler phrases and flag them as warnings, not hard blocks, since one appearing once is a nitpick but a piece full of them is a pattern.
4. If you are also gating page design, add a couple of structural checks, like counting repeated visual patterns that show up when a template gets stamped out over and over.
5. Done looks like: you can paste in a paragraph you know is clean and get zero findings, and paste in a paragraph you know is bad and get the exact line number back.

Starter tell list, ready to copy into your own scanner. Hard blocks cap the score in block territory no matter what else passes:

• Em dash anywhere in the text.
• En dash anywhere in the text.
• Emoji anywhere in the text.

Warnings, advisory only, never block alone:

• leverage
• synergy
• cutting-edge
• revolutionary
• game-changer
• supercharge
• seamless
• ever-evolving
• elevate your
• take it to the next level
• more than just
• it's important to note
• when it comes to
• in conclusion

Step 2: wire up an independent critic

The scanner catches the obvious stuff. It cannot tell you if a piece is boring, generic, or says nothing a competitor could not say. That is a judgment call, and judgment calls need a second, independent set of eyes, even if that second set of eyes is also AI.

1. Give the critic a fixed job description and nothing else: read the content, score it, list findings, never rewrite it. It is a reviewer, not a co-author.
2. Feed it the scanner's findings as pre-computed input so it never misses a tell the mechanical layer already caught. Its own job is genericness, substance, and distinctiveness, on top of that.
3. Have it return a fixed score band: roughly 0 to 59 is block territory, 60 to 79 is a warn, 80 and up is clean. A single hard block tell caps the score in block territory no matter how good the prose reads.
4. Have it return specific findings, not vibes: a location, an issue, and a concrete direction to fix it, never a rewritten paragraph.
5. Done looks like: you can run a genuinely generic, templated paragraph through it and get a low score with a specific reason, and run a piece with a real, specific detail in it and watch the score go up.

Critic charter skeleton, ready to copy:

• Role: an independent, read-only reviewer. Never the writer.
• Judges: genericness, substance, distinctiveness.
• Never does: rewrite, edit, ship, or override the door.
• Output: a verdict of pass or flag, a numeric score, and for any flag a list of findings, each with a severity, a location, an issue, and a fix direction.
• Boundary: defers legal and claims review to a separate compliance pass, and defers final creative taste to a separate design pass. It only owns slop.

Step 3: build the actual hold state

This is the part most people skip, and it is the part that actually matters. A scanner and a critic that just print warnings to a log are decoration. The gate only works if there is a function standing between the draft and the publish button that a human cannot casually route around.

1. Write one function that takes the content, the scanner output, and the critic verdict, and returns exactly one of three outcomes: publish, hold for one revision, or hold indefinitely.
2. Make the precedence order fixed and boring. Use the exact order below, top to bottom, and never let a later rule override an earlier one.
3. Log every decision with the reason attached, so a held piece tells you exactly why it is sitting there instead of making you re-read the whole thing.

Done looks like: you can feed the function a piece with a banned dash in it and watch it refuse to publish even if every other check passed.

The one rule that makes the whole thing work

The critic recommends. The code decides. Keep those two roles separate and the system stays honest. The moment an AI critic is also the thing with publish authority, you have built a system that can talk itself into shipping anything, because a model under pressure to approve its own work can, over enough decisions, approve its own weak work. A deterministic function reading fixed rules cannot be talked into anything.

A closing discipline: it will hold your own work too

Building the gate is easy. Respecting it when it stops your own content is the actual discipline. On my own pipeline, drafts get held on a regular basis, mostly for banned punctuation and generic phrasing that slipped past a first pass. That is the system working exactly as designed. If your gate never catches anything, it is not doing its job.

None of this makes AI content risk-free. It lowers the odds that something generic, wrong, or mechanically obvious reaches a reader, and it forces a second look at anything that scores badly. It does not replace a human who actually reads the piece before it ships.

What this setup does not do

• It does not check facts. A confident, well-written, wrong statistic will sail through both layers unless a human sourced it first.
• It does not replace legal or claims review. Genericness and honesty are a different question from whether a claim is allowed to be made at all.
• It does not make content good. It filters out the obviously bad and the obviously generic. Getting from a passing score to something genuinely worth reading is still on you.

Questions people ask about gating AI content