Gemma 4 技术报告
Gemma 4 是 Gemma 系列的新一代开源多模态语言模型，采用密集和混合专家架构，参数规模从 2.3B 到 31B。该模型集成了改进的视觉和音频编码器，其中 12B 模型采用统一的无编码器架构，可直接处理原始音频和图像块。此外，Gemma 4 引入了思考模式，能在响应前生成推理轨迹，并在推理速度、内存和计算效率以及长上下文能力方面有所提升。在 STEM、多模态和长上下文基准测试中表现优异，可与更大的前沿开放模型相媲美。
来源arXiv Computational Linguistics作者: Gemma Team, Sherif El Abd, Vaibhav Aggarwal, Robin Algayres, Alek Andreev, Olivier Bachem, Ian Ballantyne, Cormac Brick, Victor C\u{a}rbune, Michelle Casbon, Mayank Chaturvedi, Victor Cotruta, Alice Coucke, Phil Culliton, Robert Dadashi, Lucas Dixon, Mohamed Elhawaty, Utku Evci, Cl\'ement Farabet, Johan Ferret, Filippo Galgani, Sertan Girgin, Jean-Bastien Grill, Maarten Grootendorst, Jiaxian Guo, Cassidy Hardin, Yanzhang He, Steven M. Hernandez, Omri Homburger, L\'eonard Hussenot, Juyeong Ji, Armand Joulin, Aishwarya Kamath, Parnian Kassraie, Olivier Lacombe, Preethi Lahoti, Ga\"el Liu, Gus Martins, Luciano Martins, Tatiana Matejovicova, Ramona Merhej, Nikola Momchev, Sneha Mondal, Ryan Mullins, Sindhu Raghuram Panyam, Shreya Pathak, Sarah Perrin, Andr\'e Susano Pinto, Etienne Pot, Ang\'eline Pouget, Alexandre Ram\'e, Sabela Ramos, Douglas Reid, David Rim, Morgane Rivi\`ere, Karsten Roth, Louis Rouillard, Omar Sanseviero, Pier Giuseppe Sessa, Shane Settle, Danila Sinopalnikov, Sara Smoot, Piotr Stanczyk, Andreas Steiner, Lawrence Stewart, Ilya Tolstikhin, Michael Tschannen, Anton Tsitsulin, Nino Vieillard, Renjie Wu, Pingmei Xu, Haichuan Yang, Edouard Yvinec, Li Zhang, Joe Zou, Nicolas Aagnes, Abdelrahman Abdelhamed, Shivani Agrawal, Shubham Agrawal, Ibrahim Alabdulmohsin, Jean Baptiste Alayrac, Uri Alon, Chandramouli Amarnath, Ankesh Anand, Chrysovalantis Anastasiou, Setareh Ariafar, Fran\c{c}ois-Xavier Aubet, Kyriakos Axiotis, Federico Barbero, Joelle Barral, Alexei Bendebury, Urs Bergmann, Stanley Bileschi, Kat Black, Mathieu Blondel, Sebastian Borgeaud, Arthur Bra\v{z}inskas, Ryan Burnell, Robert Busa-Fekete, Mu Cai, Glenn Cameron, Charlotte Caucheteux, Garima Chadha, Jetha Chan, Aditya Chawla, Blake Jianhang Chen, Jesse Chen, Lin Chen, Xu Chen, Derek Cheng, Tzu-hsiang Chien, Nikolai Chinaev, Yi Chou, Zhaohui Chu, Benjamin Coleman, Pooja Consul, Sam Conway-Rahman, Scott Crowell, Dylan Cutler, Vivek Dani, Samira Daruki, Anil Das, Daniel Deutsch, Nishanth Dikkala, Li Ding, Qiuhan Ding, Shenil Dodhia, Konstantin Donhauser, Tulsee Doshi, Anca Dragan, Alex Druinsky, Sahil Dua, Zoltan Egyed, Danielle Eisenbud, Daniel Eppens, Cindy Fan, Bahare Fatemi, Yassir Fathullah, Vlad Feinberg, Milen Ferev, Takumi Fujimoto, Isaac Galatzer-Levy, Jo\~ao Gante, Simon Geisler, Soham Ghosal, Antonious M. Girgis, Alec Go, Alhaad Gokhale, Alex Grills, Yiming Gu, Pramod Gupta, Guru Guruganesh, Raia Hadsell, Hamza Harkous, Jitendra Harlalka, Demis Hassabis, Anja Hauth, Joe Heyward, Arian Hosseini, Chih-Yang Hsia, I-Hung Hsu, Xiaopeng Huang, Yangsibo Huang, Kevin Hui, Adrian Hutter, Te I, Fotis Iliopoulos, Advait Jain, Ganesh Jawahar, Ziwei Ji, Qilin Jin, Melvin Johnson, Kandarp Joshi, Arun Kandoor, Wang-Cheng Kang, Koray Kavukcuoglu, Mehran Kazemi, Kathleen Kenealy, Amr Khalifa, Phoebe Kirk, Suraj Kothawade, Vitaly Kovalev, Neel Kovelamudi, Adam Kraft, Ravin Kumar, Harish Kuppam, Justin Lannin, Chen-Yu Lee, Seungji Lee, Dmitry Lepikhin, Dongdong Li, Qiujia Li, Valentin Li\'evin, Ethan Lin, Ziqian Lin, Casper Liu, Tianlin Liu, Tianqi Liu, Xin Liu, Mayank Lunayach, Min Ma, Gagan Madan, Andrii Maksai, Eric Malmi, Michal Matuszak, Daniel McDuff, Gaurav Menghani, Daniil Mirylenka, Karolis Misiunas, Vedant Misra, Andreea Mitran, Kareem Mohamed, Maksim Mukha, Eric Noland, James O'Donnell, Kate Olszewska, Bernett Orlando, Wanqiong Pan, Rina Panigrahy, Unnati Parekh, Chunjong Park, Eric Paskie, Liqian Peng, Bryce Petrini, Slav Petrov, Jonas Pfeiffer, Bilal Piot, Martyna Plomecka, Siim Poder, Octavio Ponce, Arijit Pramanik, David Racz, Anish Rajan, Michelle Ramanovich, Anand Rao, Marvin Ritter, Vitor Rodrigues, Evan Rosen, Miko{\l}aj Rybi\'nski, Noveen Sachdeva, Micha\"el E. Sander, Rohit Sathyanarayana, Sagar Savla, Samuel Schmidgall, Tal Schuster, Benoit Seguin, Andrew Sellergren, Aliaksei Severyn, Izhak Shafran, Dhruv Shah, Yuan Shangguan, Ashish Shenoy, Pradeep Shenoy, Rakesh Shivanna, Pauline Sho, Lucas Spangher, Wojciech Stokowiec, Tim Strother, Yao Su, Yinghao Sun, Mukund Sundararajan, Andrea Tacchetti, Mor Hazan Taege, Pouya Tafti, Chetan Tekur, Rahul Thapa, Madeleine Traverse, Lenart Treven, Tao Tu, Chien Te Tung, Petar Veli\v{c}kovi\'c, Malini Pooni Venkat, Sagar Gubbi Venkatesh, Vidya Venkiteswaran, Francesco Visin, Alex Vitvitskyi, Kiran Vodrahalli, Weiyi Wang, Xin Wang, Tris Warkentin, Jan Wassenberg, John Wieting, Lechao Xiao, Hao Xu, Yuhui Xu, Fuzhao Xue, Arun Yadav, Jun Yan, Antoine Yang, Lin Yang, Ming-Hsuan Yang, Ziyu Ying, Jae Hyeon Yoo, Sajjad Zafar, Fred Zhang, Jiageng Zhang, Jianyi Zhang, Xiaofan Zhang, Chao Zhao, David Zhou, Chen Zou
Gemma 4 是 Google 旗下 Gemma 系列的最新成员，代表新一代开源权重、原生多模态语言模型。该技术报告于 2026 年 7 月 2 日提交至 arXiv，由 Gemma 团队（包括 Sherif El Abd 等 300 位作者）完成，共 17 页、2 幅图。报告详细介绍了模型架构、训练策略和性能表现，展示了 Gemma 4 在计算效率和推理能力上的重大进步。
Gemma 4 模型套件涵盖了从 2.3B 到 31B 参数的不同规模，包括密集（dense）和混合专家（Mixture-of-Experts, MoE）两种架构。这种多样性旨在兼顾计算效率与推理能力，为不同应用场景提供灵活选择。所有尺寸的模型均配备了改进的视觉和音频编码器，能够更有效地处理多模态输入。特别值得注意的是，12B 参数模型采用了一种统一的无编码器架构，能够直接处理原始音频和图像块，无需单独的编码器模块。这种设计简化了模型结构，可能带来更高效的推理。
一个关键的创新是集成了“思考模式”（thinking mode）。该模式允许模型在生成最终响应之前，先产生一系列推理轨迹。这种机制有助于提高复杂问题的推理准确性，使模型能够进行更深入的思考，类似于人类在作答前的推理过程。
在工程优化方面，Gemma 4 通过关键设计选择提升了推理速度、内存占用和计算效率，同时扩展了长上下文处理能力。这些改进使得模型能够更高效地处理大规模数据，适用于需要长距离依赖的任务。具体来说，模型在推理时采用了更优的缓存机制和注意力算法，从而在保持性能的同时降低资源消耗。
性能上，Gemma 4 在 STEM（科学、技术、工程和数学）领域、多模态任务以及长上下文基准测试中均取得了显著进步。报告指出，该模型在人工评估任务中与更大的前沿开放模型不相上下，展现了其强大的竞争力。例如，在数学推理、代码生成和图像理解等任务上，Gemma 4 达到了与更大模型相当的水平，这表明其架构设计和训练策略的有效性。
这一技术报告的发布为开放模型社区提供了重要的技术参考，有望推动多模态 AI 研究的发展。Gemma 4 的多种尺寸和架构选择，加上其突出的性能，使其成为研究人员和工程师的有力工具。