待翻译：AI Home Lab for Beginners: The Deep Dive

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

Augmented Mind: Think with AI and Manolo Remiddi Aug 21, 2026 You can now run an almost frontier class model, locally, on your own hardware. I run one, and I can tell you: it is real, it is fast, and it changed how I work. And at the same time you can run a useful local model on a phone. Those two facts together mean the interesting question is no longer “is local AI possible?” The interesting question is “what do I actually need?” That is what this is. It is the deep dive that goes with the video, The AI Home Lab, and it covers the whole map: the three parts of a home lab, the memory trilemma, where the limits really are, what you can run today, and the software layer that most beginners never see, which is where the actual difference shows up. The shift to keep in mind while reading: the same hardware gets more capable over time, not less. The machine I bought months ago now runs better models, faster. My ASUS GX10, the DGX Spark equivalent, costs about $1,000 more today than when I bought it, and it does more than before. Your hardware is not getting old. The models that fit inside it are getting better. The Three Parts An AI home lab is not a GPU. It is a system with exactly three parts, and if you understand all three and how they constrain each other, you will not waste money. If you only understand one or two, you will build something that frustrates you and sits in a drawer. Hardware is the box. Memory and speed decide what the model can hold and how fast it can answer. Model is the brain, the weights, a probabilistic engine that works out of probabilistic networks. Bigger and newer usually means better, that is the trend, though it is not always the truth. Harness is the wrapper, the software around the model that lets it use tools, keep memory, follow instructions, and actually have a job description. Most beginners focus only on the model. They download a model, open a chat window, and assume the quality of the conversation tells them what local AI can do. It does not. A model on its own can generate an answer. A harness can let that model read a file, call a transcription tool, search a codebase, run a command, check the result, send a message, remember a preference, and route a difficult task to another model. The harness does not magically make the underlying model smarter. It changes the effective intelligence of the system by giving the model better context, reliable tools, defined procedures, and ways to verify its work. The model matters, but the system around the model determines whether it becomes useful. How It Fits Together The three parts stack in layers. The hardware at the bottom holds the model in memory and runs it. The model in the middle “thinks”. The harness on top is where the work happens: it feeds the model the right context, hands it tools, checks its output, and repeats until the job is done. Each layer needs its own research and its own decision. A powerful model inside a weak harness is a smart person locked in a room with no doors. A great harness on a model your hardware cannot run fast is a brilliant employee who takes ten minutes to reply to every sentence. You are sizing all three at once. Start With the Work, Not the Computer Choosing which hardware to buy is connected to what kind of work you need to do. Different workloads need very different levels of intelligence, memory, speed, and reliability. Before you look at any machine, look at this table and be honest about which row is you. This is why “which GPU should I buy” has no universal answer. A person building a private transcription and email assistant does not need the same machine as someone running several coding agents in parallel. The RAM Trilemma Every AI home lab runs into the same constraint, and it determines every decision you make after it: the trilemma. Memory has three properties: Size, how much model and context fits; Speed, how fast tokens come out, which is how many words per second you can see; and Cost, what you pay. You can pick two of the three. Not all three. Big, fast, cheap: always a trade off. Bigger models handle much more complex scenarios. Faster memory means you stay productive instead of frustrated. There is a minimum speed below which a model stops being usable, and once you are above it, faster simply means you can do more in parallel. Faster and bigger, together, always costs more. Two practical consequences. First, do not buy hardware from a “parameters fit into gigabytes” calculation. A model file fitting on paper does not guarantee that your desired context length, the runtime, and the tools will fit comfortably in practice. The weights are not the only thing consuming memory: the context window, the KV cache, the operating system, and everything else on the machine all need space too. Second, the trilemma resolves itself if you start from the right question: what is the biggest model I actually need to run, and what is the cheapest machine that holds it? Where the Model Runs: Fast Memory Wins Here is the misunderstanding most beginners have. You can run a model inside ordinary system RAM, no GPU required. You can also run it so slowly that you quit. The rule is: for anything you want to work with daily, you are forced into fast memory. System RAM at roughly 50 GB/s. GPU VRAM at roughly 1.8 TB/s. That gap is everything. That is why GPUs, with their HBM, are the default answer: the RTX 5090 sits at about 1,792 GB/s, a completely different planet from the ~50 GB/s of DDR5 system RAM. But “GPU” is not the only shape. Unified memory systems, Apple Silicon, the RTX Spark and equivalents, share the system memory between CPU and GPU. You get a decent speed and a decent size in one machine, and that is a legitimate lab architecture, not a compromise. VRAM Is a Budget Now the math that decides your real options. Take a 32 GB card, one RTX 5090, and the model I run, Qwen 3.8 27B at Q6 quantization. The weights are 25 GB. They load once, and every session shares them. What each session costs is only its context, the KV cache. The budget view: weights load once, each chat gets a KV slot. On paper that means one model, three chats at once, fits comfortably in 32 GB. And here is the part nobody tells you: this is exactly why multi session, multi agent work is possible on a single card at all. The model is not duplicated. Only the conversations are. Reality Check: What the 5090 Actually Does On paper, three chats. In reality, I make a choice, and the choice is the lesson. The quantization of Qwen 3.8 27B (Q6) I am happy running, plus the maximum context window, fills the card. When I do that, I get one session, not three. No parallel agents. Full context for the one conversation I am in. That is a trade, and it is a real trade: on this hardware I can run this model at about 131,000 tokens of context out of the 264,000 it supports, and I can only run one instance of it. Is 131K enough? For real work, yes. Is it the full context the model supports? No. That gap is the difference between “it runs” and “it runs the way the people who made it intended,” and it is the whole reason a 48 GB machine exists as a sweet spot (more on that below). What can you do if you want more than one instance? Quantize harder. Q4 files are smaller, they leave room for more KV cache, you get your parallel agents back. But here is the honest part: while the model is nominally the same, you start to lose capability. It is not “less intelligent” in some clean linear way. It feels less capable. It answers more directly, reaches less far. Quantization is a dial between “it fits and it is fast” and “it thinks the way it is supposed to think,” and the position of that dial depends on the hardware you have. The number to remember: on one 5090, Qwen 3.8 27B at Q6 runs at 100 to 110 tokens per second when the model is warm, 81 on a cold start. One word is roughly 2 to 3 tokens. That is what “local” feels like on serious hardware. Shrink the Weights: Two Dials You Did Not Know You Had Every model has two dials that change what fits on your machine and how fast it runs. Beginners miss both because the dials are invisible: the file size is just a number in the URL. Same 27B brain, three file sizes. And the MoE twist below. Dial one: quantization. The same 27B model exists as Q6_K_XL at 25 GB, closest to full precision, which is what I run. Q5_K_XL at 19 GB, the balance point. Q4_K_XL at 17 GB, for small cards. Lower bits means a smaller file, which fits smaller cards and leaves more room for context. The model is the same brain. Quality trades a little, and as we said above, it feels more than it should. Dial two: architecture. Qwen 3.8 27B is a dense model. Every one of its 27 billion parameters is activated for every single token it generates. The alternative is Mixture of Experts, or MoE. Qwen 3.6 35B A3B is a MoE model with 35 billion total parameters, but only 3 billion fire per token. 128 experts per layer, 8 active. You get 35B class knowledge at roughly 3B class speed. The catch with MoE: all 35B of the weights still have to sit in memory, even though only 3B do work per token. So a 35B MoE model at Q4 needs about 21 GB and can use 256K context. Quantization decides what fits. MoE decides how fast it runs. Q6 for quality, Q4 for small cards, A3B for speed. That is the whole sentence. Reasoning Effort: The Thinking Dial The third dial is not about the file at all. It is about how long the model thinks before it answers. For some models you can turn thinking off entirely. The model answers word by word, direct, fast, weakest. Turn it on and the behavior changes: the model spends tokens in the background, analyzing the problem, finding a solution, then answers. You can set a budget for how long it thinks, and on Qwen 3.8 27B the levels go off, low, medium, and xhigh, which is the default I run it at. The longer it thinks, the more complex the problems it can solve. The proof is in the numbers. Qwen 3.6 27B, the previous version, had thinking off and thinking on. Turning it on was a massive jump. Same model, same weights. The switch changed the class of work it could do. And Qwen 3.8 27B at xhigh, running on one 5090, reaches frontier class: in the same league as GPT 5.6 Luna at max effort, DeepSeek V4 Flash, and just behind GLM 5.2 at max. The cost is real: longer thinking uses more tokens, takes more time, and eats more context. This is why “set it to max and forget it” it can be wrong. Reasoning effort is a per task decision, the same way you would choose between a sketch and a full drawing. The Hardware Map With the trilemma, the budget, and the two model dials in hand, here is the actual map of machines you could build on. Read it as archetypes, not a catalogue. Pick the shape that fits your work, then let the numbers argue from there. Bandwidth is the speed axis, capacity is what fits. Dashed lines are machines coming soon. At the top, the discrete flagships: the Nvidia RTX 5090 at 32 GB and about 1,792 GB/s, and the RTX PRO 6000 at 96 GB, which is a different animal entirely, 96 GB of VRAM means full context windows and multiple instances of models that would not even load on a 5090. The PRO 6000 is also where the money gets absurd: it launched around $8,000 and is now closer to $16,000. For most people it is not in the argument. In the middle, the unified memory machines that matter: the DGX Spark and its equivalents, at about 273 GB/s with 128 GB of memory. The RTX Spark, coming soon, should land a little faster, maybe 280 to 290, not enough to change the decision. AMD Halo class systems, the Ryzen AI Max 395, sit at similar bandwidth. These are the machines where “128 GB” stops being a server spec and becomes a desktop spec. And the Apple line, which people misunderstand because the chip name, not the bandwidth, is what gets marketed. M5 base is 153 GB/s, that is low. M5 Pro is 307 GB/s, that [truncated for AI cost control]