跳到主要内容
AI News HubLIVE
来源内容 · 翻译待补全2 分钟阅读

待翻译:Getting video models to learn better, faster

文章摘要

AI 服务暂时不可用,以下为来源摘要,待恢复后补全翻译:Image and video models have gotten a lot better over the last few years, even though the internals of these models haven't changed much since Stable Diffusion 3.Of course, there have been small variants like the auto-re…

来源Hacker News AI作者: schopra909
待翻译:Getting video models to learn better, faster
报告错误

纠错通道尚未开通,可先复制下方文章信息留存。

查看更正说明
直接读正文

AI 服务暂时不可用,以下为来源正文,待恢复后补全翻译。

Image and video models have gotten a lot better over the last few years, even though the internals of these models haven't changed much since Stable Diffusion 3.Of course, there have been small variants like the auto-regressive diffusion that GPT-Image popularized. But at a high level, it's pretty much all flow matching with a transformer backbone and a v-prediction objective. In our experience, most of the gains are directly attributable to 3 flavors of data improvements:RL has driven some improvement, but it's only started truly working for image and video in the past few months. Data Filtering & Rebalancing: Remove noisy data and resample your data strategically so your model learns more effectively Data Annotation: Gather better annotations like richer captions, bounding boxes, and font details so that it's easier for your model to disambiguate visual conceptsLLMs have gotten infinitely better at image captioning in the last 12 months. Less so for video, but that's for another time. Synthetic Data Generation: Finetune an ensemble of existing generative models to create training data for which there is little-to-no naturally occurring data (e.g. image editing / reference-conditioning for Nano-Banana style models)Often a collection of brittle LoRAs trained for very specific tasks and an LLM finetuned to filter out bad synthetic data. A couple of years ago, the prevailing wisdom across all generative models (be it text, image, audio) was to aggregate as much data as humanly possible for pre-training. Luckily, the field has gotten a lot smarter about this. If you throw a bunch of low-quality data (e.g. heavily compressed JPEGs) into pre-training, your model is going to waste a significant amount of its capacity learning how to mimic this slice of data. If you filter your dataset well, your model will have a lot easier time learning what you want it to learn. We know this sounds obvious, but it's a lot harder to do in practice.Then again, all good advice should seem obvious in retrospect. Today we're going to walk you through how our approach to data filtering has evolved since 2024. And, hopefully we'll save you from a couple of headaches if you end up training your own generative models down the line. 2024Old-school CV on CPUs

展开要点与分析

文章情报

工程师进阶

要点

  • AI 服务暂时不可用,系统已先保留来源内容与降级元数据。
  • Image and video models have gotten a lot better over the last few years, even though the internals of these models haven't changed much since Stable Diffusion 3.Of course, there h…

要点与分析由自动化流程生成,可能有误,请结合原始来源核实。