CEO-Bench is a new benchmark that evaluates language model agents on long-horizon, uncertain tasks by simulating running a startup for 500 days. Even the most advanced models like Claude Opus 4.8 and GPT-5.5 barely finish above the $1M starting balance and fail to consistently turn a profit.
Live AI News Intelligence
Live monitoring
Live updates
Trusted sources, attribution, rights, and in-site reading distilled into a signal-first AI brief.
Live updates
2026-06-18
Using Collaborative Gym and DiscoveryBench tasks, this study examines human-AI team coordination in shared workspaces. Results show that adding collaborators without structure lowers performance, while scaffolding with shared group memory and simulated human-in-the-loop gates improves performance, especially in three-person teams.
CaVe-VLM-CoT is a modular reflection-based agentic-RAG framework that reduces hallucinations in Vision-Language Models via a five-stage closed-loop pipeline. It introduces a comprehensive 23-metric evaluation suite, anchored by CaVeScore, measuring accuracy, citation precision/recall, attribution, and evidence grounding. Without architectural changes, it achieves 87.1% accuracy on ScienceQA and 55.2% on MMMU.
This paper presents NAVI-Orbital, a software system on a LEO spacecraft that achieved the first in-orbit demonstration of a vision-language model performing autonomous multi-modal inference entirely onboard on April 16, 2026. Using Gemma 3 and LangGraph, it classifies scenes, generates descriptions, and responds to operator dialogue. Ground benchmark accuracy 88.16%, and it successfully processed uncorrected YAM-9 imagery onboard, demonstrating feasibility of semantic compression to reduce downlink bandwidth.
LoRA is the most popular parameter-efficient fine-tuning (PEFT) technique, but research shows other methods can outperform it on certain tasks. This article introduces Hugging Face's PEFT library and its benchmarks, discussing how to choose the right PEFT technique based on specific needs, and points out that LoRA is not always the best choice.
Most vector database prototypes fail at ingest. This guide covers server-side batching, error handling, blobHash data type, and multimodal ingestion for Weaviate, with code examples and production-ready patterns.
A new benchmark harness evaluates the entire process of AI agents using software libraries, using Hugging Face Transformers as a case study. By measuring token usage, time, and error rates across different models and tooling tiers, the authors uncover tradeoffs between ease of use and resource consumption, providing insights for library maintainers and agent users.
2026-06-17
Chinese AI lab Z.ai released GLM-5.2, a 753B parameter Mixture of Experts model with 1M token context, under MIT license. It leads the Artificial Analysis Intelligence Index among open weights models but is token-hungry. It also ranks 2nd on Code Arena WebDev. Despite strong performance on SVG generation, it shows inconsistency compared to its predecessor GLM-5.1.