An AI-friendly video-to-3D asset pipeline

3D for the AI era — GoeteiaBuilt in pure Scheme AI is already good at producing reference imagery — designs, turntables, motion video. It is still bad at producing what a game engine actually eats: a rigged, skinned, textured, animated character. That gap is the industry's chokepoint, and closing it does not need a bigger model. It needs a pipeline where every step is checkable by a machine — because an AI can only iterate where it can verify. That pipeline is what this stack is built for. 1 The loop is the machine Propose, project, compare, correct — every stage below is that same shape. The accuracy of the result comes from the loop, not from any single estimate. An initial guess only decides how many iterations it takes, and an iteration here is milliseconds of CPU math. Data all the way down. The mesh is bytes you can read, the skeleton is a tree you can walk, the shader is a list you can rewrite — and the rendered frame comes back out as bytes too. That is what makes the loop closable at all; it is the homoiconic argument pointed at geometry. No eyes required until the very end. A human is asked for the judgement a program genuinely cannot make, and for nothing before it. 2 Skeleton from video Give it a turntable or a few views. The camera comes from the silhouette. Rasterize the mesh's outline on the CPU, compare against the reference mask, search. No gradients, no GPU. The joints come from reprojection. 2D keypoints from any off-the-shelf pose model initialize the skeleton; then the loop refines it — hypothesize joints, project them into the view, measure the pixel residual, adjust. Anatomy is a regularizer, not a hope. Left/right symmetry and constant bone length are not assumptions to wish for; they collapse the search space. Depth ambiguity dies under multiple frames. One skeleton has to explain all of them at once. 3 Texture from reference Projection baking, and then the loop again. Baking sprays the reference views onto the mesh and down into UV space: the correspondence between a pixel in a view and a texel in the atlas is one perspective projection — pure arithmetic. So the correction runs backwards through the same projection. The left cheek is too dark becomes exactly which texels to change. A texture lint stands guard underneath. UV coverage, dead islands, seam color mismatch, whole-image misalignment — generated garbage is rejected by a program before any human looks at it. 4 Skinning under feedback Weights are bytes that can change between one frame and the next. No re-export, no reimport, no engine restart. Pose the character at a reference frame, compare the deformed silhouette, nudge the weights, watch the result in the same second. And behind the eye, validators. Bone lengths must not stretch, symmetry must hold, no vertex's normal may collapse. Wrong skinning is caught by an assertion, not by QA three weeks later. 5 Motion capture without the stage One video of a martial artist, instead of an actor in a marker suit. With the skeleton pinned, per-frame solving is the easy direction: bone lengths are locked, only rotations remain. Frames chain into clips, temporal smoothing eats the jitter, and validators check what matters — feet on the ground, bones constant, amplitude in range. The source library stops being actors we can hire and becomes every video ever shot. And the output is not frozen capture data. A clip here is editable text: compress the strike from six frames to three, exaggerate the windup — then measure that the timing changed the way the director asked. Optical capture delivers data you clean; this delivers material you keep working. 6 A family per line, not a model per month Macros generate assets the way they generate code. Game production never wants one model. It wants a family — every fence length, forty crates, a lamp post per district. Here a parameterized model is a function, and a macro stamps the family out at expansion time; the variation lives in arguments, not in an artist's afternoon. The asset is source code. A character is kilobytes of text, not hundreds of megabytes of binary — so git can diff it, a review can read it, a merge can reconcile two people's changes to the same prop. Validators are the gate. Generated variants pass through the same computable checks as everything else — degenerate triangles, collapsed normals, broken symmetry never reach a human eye. Generate wide, filter hard. Honest scope. This eats hard-surface work — props, kits, architecture, the bulk of any asset list. Sculpted hero organics stay in the sculptor's tools; fitting them is the pipeline above. 7 Light enough for a fleet Weight and concurrency are one argument: when everything is text and everything is small, agents multiply. The traditional 3D stack is physically hostile to concurrent AI work: one editor instance, a gigabyte install, binary assets git cannot merge. Ten agents on one project means nine waiting. Ten worktrees, one laptop. Scenes, shaders, meshes and animation logic are all text, so agents work in parallel git worktrees — each with second-scale compiles and headless assertions — and converge by ordinary merge. Nothing in the loop needs a window. The workbench is a browser tab. The whole compiler crosses the wire in under 60KB and runs in the page — a Chromebook or a tablet is enough to model, skin and iterate. Disposable 3D, at conversation scale. Generating, compiling and running all happen on one surface, so an AI can hand you an interactive 3D demo the way it hands you a paragraph — made to answer one question, then thrown away. Throughput in the AI era is bounded by how parallel your environment lets the agents be — this one was shaped for it. 8 Why this stack, specifically The loop has hard prerequisites. They are exactly what shipped. A complete glTF 2.0 skeletal pipeline that verifies headless. Joint matrices, vertex streams and animation samplers are all assertable from a terminal. Shaders are data. Lighting is written once as an s-expression; the skinned variant is derived from it mechanically, by a combinator that checks its own output — so a fitting loop can rewrite rendering the same way it rewrites geometry. Pixel readback. The machine can see the frame it just rendered — which is the half of the loop everything else depends on. Second-scale compilation, and an asset memory writable at runtime. Together they close the iteration to something an agent can drive thousands of times a day — with the solver arithmetic (inverse trig, constant-rate slerp) already in the box. The API detail lives in the manual; the point of this page is what the pieces add up to. The agent that drives them is 3d-builder. 9 Where it stands Not a roadmap slide. The camera-from-silhouette solver, the projection baker and the texture lint exist, and they hold themselves to round-trip fixtures: synthetic views baked back to the atlas and compared, recovered cameras checked against ground truth — every claim a computable assertion. The per-frame pose solver is in progress. The GLB writer, lighting-from-reference and the feedback protocol for vision models come next. The discipline, unchanged as it grows Nothing ships on the strength of looking right. It ships when a program agrees. ;; 3d.html — authored in Scheme, rendered to HTML by Goeteia. ;; A static page: the whole document is SXML data, so there is no ;; browser-side half and nothing to mount. (import (web html) (web css) (web component) (chrome)) (define body (list (header (div (@ (class "head-row")) (h1 "3D for the AI era") (span (@ (class "era")) "video" ,(raw " ") "in," ,(raw " ") "game" ,(raw " ") "asset" ,(raw " ") "out")) (p (@ (class "lede")) "A martial-arts clip instead of a motion-capture stage. A character " "turnaround instead of a month of sculpting and rigging.")) (section (p "AI is already good at producing reference imagery — designs, " "turntables, motion video. It is still bad at producing what a game " "engine actually eats: a rigged, skinned, textured, animated character. " "That gap is the industry's chokepoint, and closing it does not need a " "bigger model. It needs a pipeline where every step is checkable by a " "machine — because an AI can only iterate where it can " (em "verify") ". " "That pipeline is what this stack is built for.")) (section ,(layer "1" "The loop is the machine" '("Propose, project, compare, correct — every stage below is that " "same shape.")) (div (@ (class "layer-body")) (p "The accuracy of the result comes from the loop, not from any single " "estimate. An initial guess only decides how many iterations it takes, " "and an iteration here is milliseconds of CPU math.") (ul (@ (class "points")) (li (b "Data all the way down.") " The mesh is bytes you can read, the " "skeleton is a tree you can walk, the shader is a list you can " "rewrite — and the rendered frame comes back out as bytes too. " "That is what makes the loop closable at all; it is the " (a (@ (href "why.html")) "homoiconic argument") " pointed at " "geometry.") (li (b "No eyes required until the very end.") " A human is asked for " "the judgement a program genuinely cannot make, and for nothing " "before it.")))) (section ,(layer "2" "Skeleton from video" '("Give it a turntable or a few views.")) (div (@ (class "layer-body")) (ul (@ (class "points")) (li (b "The camera comes from the silhouette.") " Rasterize the mesh's " "outline on the CPU, compare against the reference mask, search. " "No gradients, no GPU.") (li (b "The joints come from reprojection.") " 2D keypoints from any " "off-the-shelf pose model initialize the skeleton; then the loop " "refines it — hypothesize joints, project them into the view, " "measure the pixel residual, adjust.") (li (b "Anatomy is a regularizer, not a hope.") " Left/right symmetry " "and constant bone length are not assumptions to wish for; they " "collapse the search space.") (li (b "Depth ambiguity dies under multiple frames.") " One skeleton " "has to explain all of them at once.")))) (section ,(layer "3" "Texture from reference" '("Projection baking, and then the loop again.")) (div (@ (class "layer-body")) (p "Baking sprays the reference views onto the mesh and down into UV " "space: the correspondence between a pixel in a view and a texel in " "the atlas is one perspective projection — pure arithmetic. So the " "correction runs backwards through the same projection. " (em "The left cheek is too dark") " becomes exactly which texels to " "change.") (ul (@ (class "points")) (li (b "A texture lint stands guard underneath.") " UV coverage, dead " "islands, seam color mismatch, whole-image misalignment — " "generated garbage is rejected by a program before any human looks " "at it.")))) (section ,(layer "4" "Skinning under feedback" '("Weights are bytes that can change between one frame and the " "next.")) (div (@ (class "layer-body")) (p "No re-export, no reimport, no engine restart. Pose the character at " "a reference frame, compare the deformed silhouette, nudge the " "weights, watch the result in the same second.") (ul (@ (class "points")) (li (b "And behind the eye, validators.") " Bone lengths must not " "stretch, symmetry must hold, no vertex's normal may collapse. " "Wrong skinning is caught by an assertion, not by QA three weeks " "later.")))) `(section ,(layer "5" "Motion capture without the stage" '("One video of a martial artist, instead of an actor in a marker " "suit.")) (div (@ (class "layer-body")) (p "With the skeleton pinned, per-frame solving is the easy direction: " "bone lengths are locked, only rotations remain. Frames chain into " "clips, temporal smoothing eats the jitter, and validators check what " "matters — feet on the ground, bones constant, amplitude in range. " "The source library stops being " (em "actors we can hire") " and " "becomes " (em "every video ever shot") ".") (p "And the output is not frozen [truncated for AI cost control]