AI Slop

AI Slop is everywhere. Technical debt always existed, but sure, slop will surpass it; somehow we managed to make things worse: Human-generated, handcrafted tech debt blended with synthetic AI generations. I won't say productivity is up because that's hard to measure; for sure, output is up, but so are bugs and incidents.

The less you look at the code, the less you pay attention, the more slop you get. Attention is all we have; it's precious. The catch is, to produce more, you need to look less.

It takes 5 minutes for the agent to produce the code and 5 hours for you to validate and understand. Multiply that by several PRs a day, and we have a big attention problem and slop proliferation.

AI Slop Definition

I think it would be fair to ask frontier LLM models what they define as ai slop. I had to ask to be max 2 lines; otherwise, it would be slop 🙂

GPT has the most interesting definition for me, specially because it mention judgement and quality. IMHO what we are also missing is that, you cannot judge what you don't understand so, we are missing something important which is learning. IF you don't learn you can't judge.

AI Slop has a set of problems like:

• Technical Debt
• Comprehension Debt
• Cognitive surrender
• Lack of good taste
• Poor Judgement (mix of lack of experience and lack of domain expertise)
• Codebase erosion (was bad before ai, now is worse)
• Lack of Critical Thinking
• Following the Flow, giving to the vibes or simple - not thinking.

LLM models in the context of coding agents(harness) they have something called trajectory, they then to do things in the same way. Humans also have that, we tend to go with the flow.

Deliver vs Understanding

Imagine there is a dial you can turn and either lean more or deliver more. Turn 100% into learning, you deliver nothing. Turn 100% into delivery you will have so much stop that the codebase will collapse in less than 6 months.

Companies expect engineers deliver much more than before. Talking to each other it's much more slower than asking an agent to get something done. There is a trend in Silicon valley where companies are having 1-2 engineers per product, same engineer is doing engineering and product. Sometimes called Product Engineer, obviously that has PROS like:

• You have the whole context
• You can move very fast
• You reduce the friction (meetings, convincing, explaining)

However there is also problems in such model like:

• You lost the team, you might not see everything and be biased, your taste is all you can have now.
• The shift forces you be good at engineering, Product/UX and domain expert which is hard.

• What happens if you left the company, how on-call looks like, how you get feedback from peers?

Naturally, nowadays companies are amping up much more towards delivery. Honestly it was always like that.

However today seems to be even more because:

• Coding agents/harness can output code much faster, so expectations to get things done is much higher and companies have much less patience.
• Like always competition is fierce, so the fear of a competitor doing faster than you also amp this.
• Finally this is a problem for the engineers that need to maintain the code. I would argue is a company problem but not always companies see this way.

Meat Proxy

Niklas Gruhn wrote a short blog post explain the issues out passing thought AI generated output. It's tempting to use claude/codex to write all your messages, all reply, all emails, all chats. However we need to think in who is reading, and the time to digest slop. This is a funny website built on this reality.

Perhaps this is just a side-effect of the cheer volume that AI created and yes, it's overwhelming. IMHO such affected happened before AI with stack overflow, and even google search but much less aggressive.

Now the problem it's not only throwing slop into others, it's interruptions! The one can firehose easily dozens of PRs a day, multiply that by the team size, you got another attention problem.

Most of times code can be obvious but not always. Requirements, users pains, hypothesis and decisions are not often obvious and just looking the code might not express it very well. You can't dial up all the way to heavy specs either, SDD do not work. But some of that need to be capture, even if in a simple list.

Prompt Requests

I'm not still not convinced of prompt requests (PRs) yeah how not name 2 things with the same name. Only carrying the prompt does not work, models still not ready for that however more and more I think we need to be sharing intent.

I believe it would be easier to review pull requests (PRs) that have:

• What is the problem
• What is being cover
• What it the tests cases you tought about
• What is your design:
• How API Contracts look like?
• What data structures and types will be used?

• What are important design decisions?

Immediately reviewers can check if all that holds and look for corner cases, gaps, design flaws and other problems. Looking the code helps with obvious things, deep analysis require investigation and that's what eat most of the time.

Tokens

The faster you want go, the more tokens you will burn. Sub-agents use a lot of tokens, running multiple agents in parallel speed up things considerable but also use much more tokens.

New models are all using more tokens. As reasoning keep improving, so token consumption keeps going up. The faster you go, less code you read, more slop is created. Slop can be trimmed out, but that require attention and fact-checking which is much much more slow in comparison.

Burning more tokens means, more things to review, more slop. Pretty sure we will have Lean AI very soon, where there is waste there is Lean. It's possible to use tokens to perform adversarial reviews and spend then towards polishing than just shipping output, but IMHO still produces slop maybe a different kind.

Becoming a Lawyer

I feel more and more that I'm becoming a lawyer, running adversarial review, fact checking everything AI says 3 times (at least). You cannot trust what AI "say". The more slop you have, more confuse AI get's, even with a perfect codebase AI hallucinate, AI produce slop answers that are wrong.

The more you do architecture and design, the more you are driving strategical decisions and let AI handle the implementation details the better. Easier to review, easier to explain and IMHO much less mental erosion. IF you let AI do architecture and decide it all for you, thats how you get tons of slop.

What code can you avoid reading?

None. 🙇🏻‍♂️

However there are parts of the code that can be read much faster, and others will require much more time. Our time is limited and more productivity is expected now a days. So there must be some level or prioritization, leverage and maybe even gambling.

IF you want to scale and deliver more, it's impossible to prioritize everything equally, some PRs are more critical than others, some files and some changes carry more criticality.

IMHO the most important parts are:

• The API Contracts
• The Data structures
• The Database Schema
• The Core logic and Invariants (what cannot never break?)
• Integrity and Consistency
• Tests (Unit, Property-based, Fuzzy, Evals, etc...)

Last one it's not obvious. AI loves to create new branches, and build new things, rather than re-use whats there. AI does not enforce contracts and integrity, you really really need to watching for this.

How to do Better?

Unfortunately ai slop is here to stay and we need to find ways to minimize and reduce as much as we can.

Here are some advices and things you can do to better handle your AI slop:

• Invest in proper guardrails (AI Agent Gateway like Portkey/LittleLLM/ByFrost etc..)
• Have good automated tests (Unit, Fuzzy, Property-based, Chaos, Stress, Integration, Evals) more testing diversity the better.
• Money needs go to the infra as well not only inference, companies are paying big bucks to llm providers, but need to increase cost of CI for instance, if tests taking a long time, this is slowing down all teams.
• Ask AI to be minimalist: change as little files as possible.
• Review Architecture and design with more attention, don't let AI do architecture.
• Communicate intention in PRs to make review easier, verifiable and scorable.
• Have proper API contracts, with proper SOA, it's possible to abstract services and at least centralize slop into one box you can refactor later.

We need to learn how to balance, delivery with understanding. Find ways to ship faster but also retain understanding of the systems, we are all doing much less code than before, for sure will be reading less in the future. We are all becoming F1 drivers so better learn how to think fast.