AI Slop Is a Choice

I’ve been using AI ever since ChatGPT 3, and as models have gotten better I’ve come to use them more. Now I’m at a point where I’m paying $200/mo for Claude and use it every day. It’s not perfect, but it’s really good.

One thing I’ve noticed in how I work with AI is that I do A LOT of iterations. And while I do give detailed prompts and ask Claude to interview me to develop an even clearer idea, I know AI is going to hallucinate extra crap regardless. And so iteration is just part of the process for me. The level of taste, specificity, and critique you bring to the output is what will decide the ultimate quality.

I’ve heard people claim that this process of iteration is essentially no different than if they had done it all themselves instead of using AI. I strongly disagree with that. In my experience, the work shifts from an act of generation to one of review. Perhaps it’s just how I use AI, but I find I get a lot done with it in a lot less time. I know how much time it would take me to do things by hand, and it’s probably 10X to 20X slower if I did, with a lot more discovery, learning, and refinement along the way. While I do care about the code, I care more about what the code enables. I don’t need to be the one writing it, but I do need to be the one specifying it.

Where I’ve seen people go wrong in their usage of AI, where what they produce is mediocre or needlessly embellished or complex, is that they stop after a first pass. I stop after maybe the hundredth, or thousandth. I keep going until it’s polished, and I bring a critical eye to everything it’s generated.

I don’t do much code writing anymore, and I do very little reading as well. But I do a lot more architecture, design, and QA. There are lots of things I don’t know, that I don’t need to or want to have to know: like how to integrate with a specific API, or performance improvements for a specific browser rendering. While I could dig into those specifics, I’ve been there/done that so many times in my career as a developer that I’d rather not learn more super-specific things only to never use them again. I care about the results, not the way. I do find I have to still emphasize good practices, composability, and performance, as Claude does not natively do those things unprompted.

I’m building my first game with an OSS game engine and am using Claude to do it, after having worked a lot with Unity doing more “productive” projects (I do not miss Unity’s quirks, though I did learn a lot from it). Claude’s done a surprisingly good job of generating art and music assets that I would not be able to generate myself. Not on the first pass, of course, but with a lot of back and forth it’s able to build on its previous generation and, with my guidance, arrive at something I like.

Iteration, in all of its forms and frequency, is my best advice for generating great results with Claude or any other LLM. I’ll write another post sometime about what those forms of iteration are, because in using Claude on a daily basis I’ve come to appreciate the limits of its knowledge/training and how to extend or work around them.

This isn’t to say that with enough iteration you’ll be able to achieve absolutely anything, like generating the next Skyrim or Google all by yourself. I mean, you’ll surely get to something, and it might be something worthwhile, but it won’t be like that. You must work in a scope where the specificity is manageable, knowable, and can be communicated/delegated to the LLM, or at the very least built up in the right layers or components. And this assumes that the LLM has enough training or abstract skill to work in the given domain.

So the next time you get AI to generate something for you, be wary of the early output. Be critical, demanding, and opinionated on what needs to be improved. An amateur considers the 3rd iteration good enough; a pro takes it to the 30th or 300th and probably starts over a few times.