I Built an AI Slop Machine

Slop Machine

Jul 12, 2026

I, a human, write every word of this blog in my own voice. Nonetheless, I built an AI Slop Machine.

Your browser does not support the video tag.

Background

(Boring! Skip to the pretty chart)

American and Chinese AI labs continue to make increasingly strong open-weights large language models (Qwen, Gemma, gpt-oss, and others). You can self-host these LLMs on consumer-attainable computers and GPUs. They’re pretty good at agentic knowledge work tasks, and when you use them, several of the things that people say are bad about AI don’t apply.

A tech company never handles your prompt or response data. It stays on your own computer.

You aren’t stuck on a ride of subscription-with-mystery-quota versus premium API pricing. Once you buy the hardware, it’s free to download the models and use them forever.

The ways that you access self-hosted LLMs are generally free of advertising or manipulation of your attention.

Once you have a copy of the model, its maker cannot withdraw your access to it.

You aren’t causing additional demand for AI data centers.

Yes, these models are trained at large computational expense, but the cake is done baking by the time you download a copy.

The electricity you’ll use is at most a few hundred watts while it’s actively working, similar to a video game console.

So, I’ve watched my friends with higher-spec Macbooks running increasingly-capable AI at usable speeds, bringing chunky power bricks to the cafe and using agents on hot, whirring laptops.

Meanwhile, my daily laptop (a 13th-gen Intel Framework 13) is not good at this. As a party trick, it fits decent LLMs in its 48 GB of system RAM, but the ~50 GB/s combined bandwidth of two DDR4 memory modules is a severe bottleneck. With an empty context window, I get single-digits of tokens per second from mixture-of-experts models (like Gemma 4 26B-A4B or Qwen 3.6 35B-A3B), and it slows down sharply as the context window grows. If I try a stronger dense model (like Qwen 3.6 27B or Gemma 4 31B), I measure throughput in seconds per token, unusably slow for interactive work. So, after leaving a job at which I deployed LLMs on a $15m pile of hardware, I’ve been using AI services hosted by companies for day-to-day tasks, mostly via Kagi Assistant and OpenRouter.

Also meanwhile, AI labs and neoclouds are supply-squeezing the RAM and GPU markets, making LLM-capable computers (nay, all computers) more expensive. This may not be an intentional strategy to make people more dependent on Codex and Claude subscriptions, but the economics sure point in that direction. If the AI bubble doesn’t pop then the shortage will continue raising hardware prices for years. So, if I wanted to do productive things with strong LLMs on my own computer, it was time to buy more computer.

LLM Selection

Given the pace of LLM evolution and obsolescence, it seems stupid to buy hardware sized to run one specific model, but I sorta did. Qwen 3.6 27B came out in April. It’s a Chinese model, with all the quirks and biases that entails. But if you believe LiveBench, Qwen 3.6 27B performs better than much larger models across several categories of task. Better than DeepSeek V3.2! To find an unambiguously more capable open-weights model on this benchmark, you need to reach for DeepSeek V4 Flash, which requires 10 times more VRAM (holding weight precision constant), and its overall score is only a little better. Other unpolluted, private benchmarks also show good things for this model. Also, I think it’s fair to say that the /r/localllama community really likes working with Qwen 3.6 27B. It behaves well in an agent harness, and malformed tool calls are rare in my experience.

Being a dense model, Qwen 3.6 27B is computationally intensive to run compared to mixture-of-experts models of similar overall size. But dense models get you the most ‘intelligence’ for a given amount of VRAM, and VRAM is the most limited aspect of ‘affordable’ GPU hardware these days. This model tolerates quantization down to ~4 bits per weight quite well, which makes it even more VRAM-friendly.

I already had good experiences harnessing cloud-hosted versions of this model. But if I want an alternative, Gemma 4 31B from Google is almost as good on benchmarks, and Google fixed the obnoxious corporate training manual personality that plagued Gemma 3.

I usually prefer slower-smarter tokens over faster-stupider tokens, because progress is often limited by me reading and understanding what the agent is doing, and ‘more code’ is often a liability. But if you want a faster model at similar size, you can use Qwen 3.6 35B-A3B or Gemma 4 26B-A4B, both sparse mixture-of-experts which accept an ‘intelligence’ penalty to use less computation per token.

So, any computer that will fit Qwen 3.6 27B will also fit good alternatives, with diversity of American + Chinese origin and sparse + dense architecture. And looking to the future, these are the worst models that we’ll ever be able to run at this size tier. If AI labs keep releasing better open-weights models (as they have for the past few years), the same computer will be able to run better LLMs as time goes on.

Hardware Selection

I hear you saying “just get a Mac”. I was prepared to! My shopping list was:

Enough VRAM to fit useful models without aggressive quantization blah blah Qwen 3.6 27B at ~4 bpw, MTP, vision support, deep-ish context window.

The highest-quality 4-bit GGUF (UD-Q4_K_XL) is 17.9 GB, so 24 GB of VRAM isn’t really enough but 32 GB probably is.

Enough memory bandwidth to generate text at comfy reading speed (20-30+ tokens per second).

Enough compute for fast context prefill

Especially important for agentic programming. Agents do a lot of reading.

Single computer or GPU, to avoid a mess of tradeoffs and spotty backend support for splitting model weights (tensor or pipline parallelism) across a PCIe or network link.

(RTX 3090 has faster NVLink but it’s discontinued.)

Reasonably mature ecosystem support for LLMs, i.e. there is already software that uses the hardware fully and efficiently to run the models that you want.

Right now, the market looks like this.

(Here is the code and data if you want to modify this chart. And yes, I made the chart with the slop machine.)

At time of writing, everything on the chart (except RTX 3090) is actually in-stock at a non-sketchy seller, for the prices indicated, to buy new with a warranty. (So there’s no discontinued 128+ GB Mac Studio or 64 GB Mac Mini.) Also, you can use all of this stuff in your house without obnoxious noise levels, 240 volt circuits, or needing to attach a separate fan + shroud. (So there’s no old datacenter GPUs like MI50 or V100.)

My takeaways from the chart:

High-spec Macs have a lot of VRAM and good memory bandwidth, but are weak on compute compared to the latest prosumer GPUs. (Note the logarithmic X-axis!) Agentic workloads do a lot of reading stuff, i.e. prefill, which is compute-bound while you wait for a response to start streaming. This tells me to only get a Mac if I need all that VRAM, which I don’t if my daily model is ~27B parameters.

DGX Spark is a much better value than Strix Halo at current prices! But even so, it’s very bandwidth-constrained compared to prosumer GPUs and Macs, so if you don’t need all that VRAM, skip it.

For $4k, NVIDIA will sell you either a lot of VRAM or a lot of memory bandwidth, but to get both you need to spend $12,500.

But I didn’t have this chart when I decided! My decision process was: buy the cheapest new 32 GB+ GPU that has reasonably good support in llama.cpp and vLLM. This was almost an Intel Arc Pro B70, but AMD’s maturer software ecosystem tipped me to a Radeon AI Pro R9700. I paid $1349 on June 6 for the ASRock version. And lo, my AI Slop Machine.

Yes, that’s an external GPU dock. I don’t have a desktop computer but I do have a spare old Thinkpad, and it was much cheaper to buy an eGPU dock for the T480 than to build a new desktop in these RAM-constrained times. The T480’s Thunderbolt 3 port only supports two lanes of PCIe 3.0. This was slow even by 2018 standards, but I don’t expect it to affect overall performance much. It takes an extra moment to load the model weights across the Thunderbolt link, but after it’s started up, all the bandwidth-intensive stuff stays local to the GPU. You’re passing prompt and response tokens over Thunderbolt, but that’s tiny data. If you allow your inference software to swap out prompt caches to system RAM, then if your agents are alternating between multiple loong prompts, those several-GB transfers each take a few seconds, but you’re still getting a big speedup compared to prefilling with an empty prompt cache.

Yes, this is not portable. The loaded dock and laptop weigh a combined ~20 pounds. But that’s fine, because I set it up as a server. I’m making API calls to it over an SSH tunnel from a different laptop, on the couch, cafe down the street, hotspot from my phone, etc. There are several ways to get secure remote network access to a computer like this, with varying levels of ease and robustness. One benefit of running your LLM on a separate server is that the laptop under your hands stays cool, quiet, and long-lasting on battery. The drawback, of course, is you need a reliable internet connection at both ends. It doesn’t need to be a fast connection, as you’re only passing text and the occasional image back and forth.

My agent harness sees the slop machine as an OpenAI-compatible API, just like you get from any other inference provider. The location, shape, and size of the physical server are totally abstracted away from the client.

Anyway, this works mostly great. Within a couple hours of opening the box, I had it running Qwen 3.6 27B via llama.cpp at speeds reflecting full GPU usage.

Configuration and Tuning

Given a GPU+LLM combination, the possibility space of how you run it is surprisingly large.

There are several choices of inference engine (the computer program that loads the LLM and manages the work on the GPU), with tradeoffs. For example, llama.cpp runs faster for a single request at a time, while vLLM accepts slower per-request generation in exchange for handling many parallel requests at much higher combined generation speed.

These programs have many config knobs to push performance in one direction or another, to fit your hardware or workload better. One recent example, you can enable multi-token prediction to get 50+% faster generation speed, at the cost of incrementally slower prefill and a little less VRAM available for the context window.

For some hardware, you get a choice of multiple GPU backends. ROCm or Vulkan? I dunno, try both and see which works better.

The less-popular combinations of GPU+LLM have various bugs and performance degradations. vLLM is especially finicky to get working on new or niche hardware/models. You end up collecting fixes to apply from various bug reports and pull requests, or maybe find someone else’s fork with the necessary set of patches baked in.

I wanted to explore this space somewhat systematically, and use my agent to do most of the work, which turned out to be learningful and a lot of fun. I made all of this happen in https://codeberg.org/cmart/slop-machine/, a workspace where I ‘collaborate’ with Pi coding agent. We discuss research questions and things to try, and put them in TODO.md. The agent works on todos and puts its findings in lab-notebook.md, where I review what it found and guide the follow-up work. Sometimes I’ll also write in the lab notebook, and each section indicates who authored it. These files help future agent sessions underst

[truncated for AI cost control]