Agent-EvalKit is an open-source toolkit (Apache 2.0) that makes this evaluation infrastructure available by integrating with AI coding assistants, including Claude Code, Kiro CLI, and Kilo Code. This post walks through how Agent-EvalKit works across its six evaluation phases, using a travel research agent built with the Strands Agents SDK and Amazon Bedrock as a running example.
Agent-EvalKit provides a six-phase evaluation workflow (Plan, Data, Trace, Run agent, Eval, Report) integrated with AI coding assistants.
It detects issues like hallucination when tools return empty results, as demonstrated with a travel research agent.
Databricks Marketplace now offers pre-built MCP servers from partners including Climb, Atropos Health, Kythera Labs, and Redox, covering biomedical, clinical evidence, medical semantics, and interoperability. These servers are centrally governed in the MCP Catalog with Unity AI Gateway, enabling rapid development of secure healthcare AI agents via low-code or custom coding.
Ready-to-use MCP servers on Databricks Marketplace lower the barrier for healthcare AI agent development.
Partner-provided MCP servers cover target-drug interactions, clinical trials, FDA labels, medical semantic translation, and data interoperability.
Ecolab leveraged Databricks and Anthropic's Claude to unify nine siloed data sources into a single retail intelligence platform, reducing compliance report compilation from two weeks to under two minutes.
Ecolab unified nine data sources on Databricks with Claude models
Compliance report time reduced from two weeks to under two minutes
As generative AI evolves into agentic AI, European enterprises face new challenges in data sovereignty, cost control, and infrastructure. This article argues that regional cloud providers like Vultr offer better compliance, performance, and cost efficiency than traditional hyperscalers for agentic workloads.
The agentic AI market is projected to reach $139.19 billion by 2034, with Europe growing at 42% CAGR.
European businesses must balance innovation with regulatory compliance, requiring localized cloud infrastructure.
OpenAI is considering cutting API token prices to win customers from Anthropic, according to the Wall Street Journal, signaling a potential price war in the AI industry.
OpenAI plans to lower token prices to attract Anthropic's customers
The move could trigger a broader price war in AI APIs
Datasette 1.0a33 is a significant alpha release extending the ?_extra= pattern to queries and rows, now documented. An AI-built API explorer demonstrates the feature.
Blueprint instruction optimization, a new feature of Amazon Bedrock Data Automation, automatically refines extraction instructions using 3-10 example documents and ground truth values, improving accuracy in minutes without model fine-tuning.
Provide 3-10 representative documents with ground truth values
BDA automatically analyzes discrepancies and refines natural language instructions
Employees at artificial intelligence companies are coming into gargantuan sums of money amid a boom in IPOs, driving home prices in the already expensive San Francisco Bay Area even higher. Experts say the trend may continue as companies like OpenAI, Anthropic, and SpaceX plan to go public.
AI employees are cashing in on IPOs, fueling a surge in Bay Area home prices.
OpenAI, Anthropic, and SpaceX are among the companies expected to go public, creating more wealth.
NVIDIA's GeForce NOW summer sale offers up to $70 off a 12-month Ultimate membership and $35 off a Performance membership. The cloud gaming service eliminates hardware barriers, provides instant access to high-performance RTX gaming across devices, and announces Guild Wars 3 coming to the platform with exclusive rewards for current Guild Wars titles.
GeForce NOW summer sale: $70 off Ultimate and $35 off Performance annual memberships for a limited time.
Cloud gaming removes hardware constraints, offering instant game access, automatic updates, and cross-device play.
The article argues that while AI executives promise efficiency will let us focus on healthier activities, history shows that labor-saving technologies often reduce physical activity and harm health. Past conveniences like drive-throughs, microwaves, and escalators have gradually chipped away at daily movement.
AI executives claim their products will free us to pursue healthier lifestyles, but past innovations suggest otherwise.
Technologies such as drive-throughs, microwaves, and escalators have replaced physical tasks, reducing daily movement over time.
This article is the second part of the PyTorch profiling series, delving into the internals of nn.Linear layers, including transpose operations, bias-fused epilogue techniques, and the impact of torch.compile on a single linear layer. It then dissects the performance characteristics of a Multilayer Perceptron (MLP) with GeGLU activation, showcasing the scheduling and execution of GPU kernels.
nn.Linear fuses bias addition into the matrix multiplication kernel via an epilogue, avoiding extra memory accesses.
torch.compile offers no significant speedup for a single nn.Linear layer but eliminates CPU dispatch overhead.
Isomorphic Labs, a Google DeepMind spinout, is using its novel AI system IsoDDE to discover hidden pockets on proteins for drug binding, going beyond AlphaFold. The system successfully predicted a cryptic pocket on cereblon, validating its ability to find novel drug targets.
IsoDDE goes beyond AlphaFold by predicting protein-ligand interactions, not just structure.
The system identified a cryptic pocket on cereblon, published in Nature, using only the protein sequence.
Visa has linked its payment infrastructure to ChatGPT, enabling AI agents to recommend retail products and execute financial transactions. The deployment removes human intervention from the final stages of the retail funnel. Autonomous agents will now process user prompts, evaluate merchant catalogues, and complete the checkout process using Visa’s payment rails at any supporting merchant.
Visa integrates with ChatGPT, allowing AI agents to autonomously complete retail purchases.
AI agents select products based on data rather than visual merchandising, requiring retailers to provide structured data.
Warren is a free AI financial planning assistant that creates a personalized plan via a single voice conversation in 10 minutes. The new version 3.0 features a transparent, editable financial model, shows two futures (inaction vs. action), tracks progress, and monitors economic changes. Over 3,000 UK users have built plans; 1 in 3 retirement planners may fall short by £258,000.
Free AI financial planning through a voice conversation, no forms or advisor fees
Version 3.0: fully transparent and editable financial model with explanations
This is the eighth article in a series on agentic engineering and AI-driven development. It addresses context loss in AI agents performing complex multistep tasks. The author introduces the Externalize-Recognize-Rehydrate (ERR) pattern: saving agent state to disk, detecting context degradation, and recovering from files. Historical analogies (640K memory limit) and a real Copilot crash example illustrate the problem. The article details externalizing two layers of state: execution continuity (current step) and task continuity (overall goals).
AI agents have limited context windows, causing information loss, akin to early memory constraints.
The ERR pattern: externalize state, recognize loss, rehydrate from files.
Xebia's global CTO Niels Zeilemaker emphasizes that AI agents need a solid data foundation, especially proper data cataloguing. The company's Agentic Data Foundation (ADF) and ACE framework help enterprises accelerate AI adoption while maintaining governance and quality.
AI agents require correct data catalogues and foundations; otherwise they misinterpret data.
Xebia's Agentic Data Foundation extends data platforms to host agents.
Nous Research has released a Profile Builder for Hermes Agent within its local web dashboard, replacing the multi-step CLI setup with a single guided flow. Users can define identity, select model/provider, toggle built-in skills, install skills from the hub, and attach MCP servers, all producing isolated profile directories for running multiple agents without state collision.
Hermes Agent's new Profile Builder consolidates multi-step CLI profile creation into a single browser-based guided flow.
Users configure agent identity, model/provider, built-in and hub skills, and MCP servers in one place.
The traditional enterprise data playbook, which assumes a slow, stable pace, breaks down under acquisition-driven growth and agentic AI consumption. Barry Panayi, Group CDO at Howden, advocates shifting from per-use-case data products to a services layer, moving data governance left, reducing insight lag, and embracing conversational analytics to keep pace with rapid business change.
The one-product-per-use-case model fails when growth comes through acquisitions and consumers include AI agents.
Shift data mastering and quality checks left to ingestion to cut integration cycles from months to weeks.
SmithDB supports full-text search and JSON filtering over agent traces with a median latency of 400 ms, despite large nested JSON documents in object storage. The article covers challenges, query shapes, inverted index basics, why Tantivy wasn't used, and the two design iterations.
SmithDB's inverted index is tailored for object storage and large agent trace payloads
Traditional search libraries like Tantivy are not suitable due to mmap and local disk assumptions
Most AI agent tools run on servers, limiting access to browser APIs, device capabilities, and frontend state. Discover how LangChain headless tools enable secure client-side tool execution for modern agent applications.
Most agent tools only see the backend, missing browser and device capabilities.
Headless tools bring client-side capabilities into the agent loop as first-class tools.
Cloudskill is a platform that governs AI skills, turning scattered skill files into a managed catalogue with version control, per-person access policies, and a full audit log. It integrates with agents like Claude, Cursor, and Copilot, ensuring every change is reviewed and approved, keeping skills safe and consistent.
Cloudskill transforms AI skill files into a managed catalogue with version control, access policies, and audit logs.
Supports various AI agents including Claude, Cursor, GitHub Copilot, and more.
A quiet day reflects on a great essay by Sarah Guo discussing open models, the difference between model labs and agent labs, and the untrainable aspects of AI. The article also covers Anthropic's Fable/Mythos rollout and the trust backlash, Fable 5's benchmark strength, Google's DiffusionGemma release, agent tooling progress, and technical updates in optimization, retrieval, and scientific modeling.
Sarah Guo's framework based on legibility explains the place of open models and the distinction between model labs and agent labs.
Anthropic's Fable/Mythos faced backlash for silently degrading AI research capabilities, damaging trust.
This article argues that despite fears, AI has not led to mass layoffs in software engineering. It presents evidence that layoffs attributed to AI are often financial in nature, and that AI compresses execution but not decision-making and delivery. The 'decide-execute-deliver sandwich' model explains why coding agents haven't displaced workers: the bottlenecks are deciding, verifying, and deep understanding.
AI-driven mass layoff stories are often 'AI washing' — layoffs are typically due to financial pressures.
Writing code is not the bottleneck; bottlenecks are deciding what to build, verifying delivery, and deep understanding.
Frontier teams are not just using AI to code faster. They’re redesigning how software gets built. The result is 4.5x productivity gains, in some cases more than 10x. This article details case studies from Amazon Bedrock, Prime Video, and others, outlining five key practices to become a frontier team, emphasizing that workflow transformation matters more than tools.
Frontier teams achieve 4.5x to over 10x productivity gains by redesigning workflows, not just adding AI tools.
Amazon Bedrock team completed a project in 76 days with 6 engineers, originally estimated for 30 over 12-18 months.
We implement an instrumented workflow for Microsoft SkillOpt end to end. We set up the repository, connect OpenAI-compatible model access, and configure the optimizer and target models. We evaluate the original seed skill as a baseline, then run a real optimization loop with rollout, reflection, aggregation, selection, updating, and validation-based gating. We inspect training history, visualize accuracy, edit-budget behavior, and token usage, then compare the evolved skill against the baseline.
Set up SkillOpt repository and connect OpenAI-compatible models, configure optimizer and target models
Evaluate initial seed skill as baseline to obtain hard and soft match scores
As robotaxi services expand globally, NVIDIA introduces Halos OS—a comprehensive safety system integrating certified OS, standardized interfaces, AI guardrails, and a validation framework to ensure safety is built into autonomous vehicles from the ground up.
Multiple robotaxi programs are launching worldwide using NVIDIA DRIVE Hyperion, including Uber/Autobrains in Munich, Foxconn in Taiwan, VinFast in Southeast Asia, and HUMAIN in Saudi Arabia.
NVIDIA Halos OS addresses four key safety challenges: a safety-certifiable operating system, safe interfaces, AI with verifiable guardrails, and validation at scale.
Onpilot creates specialized AI workers tailored to your systems, workflows, and processes. It monitors operations, identifies risks, uncovers opportunities, recommends actions, and automates work across 3,000+ integrations. Deploy in Slack, Teams, WhatsApp, your SaaS, or on-premises. The platform emphasizes trust with approval workflows, audit trails, and exception handling.
Onpilot is an AI workforce that customizes to a business's specific systems and processes, proactively monitoring operations for risks and opportunities.
It integrates with 3,000+ tools to automate tasks, with approval flows and exception handling for reliable operation.
Install and configure LSP servers for GitHub Copilot CLI, replacing brute-force grep/decompile with real code intelligence. The LSP Setup skill automates the process, supporting 14 languages. This post explains how it works and how to get started.
GitHub Copilot CLI previously relied on text search and binary extraction to understand code, which was inefficient and inaccurate.
The LSP Setup skill automates installation and configuration of LSP servers for 14 languages.
Google DeepMind's DiffusionGemma is an experimental open-weight model that uses diffusion to generate text blocks in parallel, offering faster local inference compared to traditional autoregressive models. Built on the Gemma 4 26B A4B MoE architecture, it trades some quality for speed, making it ideal for interactive and editing tasks. The article explains its architecture, how text diffusion works, benchmark results, and provides a step-by-step guide to run it locally using llama.cpp.
DiffusionGemma generates and refines blocks of tokens in parallel, reducing latency for local inference.
It uses bidirectional attention and a 256-token canvas with multiple denoising steps.
Anthropic publishes a sweeping essay and two policy frameworks. The company calls for binding audits of frontier models and paints a picture of AI as a strategic weapon wielded by nation-states.
Amodei uses a Lord of the Rings analogy to argue the political system is too slow to react to AI risks.
Anthropic calls for mandatory third-party audits of frontier models and government authority to block risky models.
Anthropic has apologized for stealthily throttling its new AI model, Claude Fable 5, with hidden guardrails that undermine both researchers and rivals using it to develop competing systems. The company says it is reversing course and will be more transparent about when the restrictions kick in, even if that means Fable refuses more queries.
Anthropic admitted to deploying invisible distillation-detection guardrails in Claude Fable.
Users triggered guardrails received degraded responses without notification.
Cohere has released its first developer-facing coding model, North Mini Code, a 30B total parameter mixture-of-experts model with only 3B active parameters per token. It runs on a single H100 GPU, supports 256K context length, and is optimized for code generation, agentic software engineering, and terminal tasks. The weights are open under Apache 2.0.
North Mini Code is Cohere’s first coding model, 30B total parameters with 3B active, supporting 256K context and 64K max output.
Runs on a single H100 at FP8; weights open under Apache 2.0 via Hugging Face, Cohere API, and more.
Anthropic changes its policy on Claude Fable 5 after backlash, making safeguards for frontier LLM development visible. Previously, the model would limit effectiveness without notifying users. Now flagged requests visibly fall back to Opus 4.8, and API returns reasons for refusal.
Anthropic reverses controversial policy after public outcry
Claude Fable 5 previously limited requests for frontier LLM development invisibly
Ollama's MLX engine has been updated to deliver its highest performance on Apple Silicon yet. By leaning more heavily on Apple's unified memory and the Metal-backed MLX framework, models output higher quality responses, respond faster, and use less memory. The update includes support for NVFP4 format, up to 20% faster output, and a snapshot system for agent workflows.
Ollama's MLX engine now supports NVFP4 format, halving quantization quality loss.
Output speed increased by up to 20% due to fused Metal kernels and optimized sampling.
Release of datasette-agent 0.2a0 with new user interaction and query saving capabilities.
Tools can now ask users questions mid-execution via `context.ask_user()`, supporting yes/no, multiple-choice, and free-text. Unanswered questions persist across server restarts.
New built-in `save_query` tool allows the agent to save SQL as a Datasette stored query, requiring human approval.
Google has released DiffusionGemma, a new open-weight model under Apache 2 license, available for free via NVIDIA's NIM cloud API. It delivers impressive generation speeds exceeding 500 tokens per second.
Google releases open-source DiffusionGemma model under Apache 2 license.
Oracle Cloud customers can now access OpenAI models and Codex using their existing cloud commitments, enabling AI development with enterprise security and governance.
Oracle Cloud integrates OpenAI models and Codex for enterprise AI development.
Customers can use existing Oracle Cloud commitments without additional cost.
Google released DiffusionGemma, a 26-billion-parameter model that generates text via diffusion, achieving 1,000 tokens per second on an H100 GPU—four times faster than autoregressive models, but with lower quality. It's currently experimental.
26-billion-parameter diffusion model for text generation
DiffusionGemma is Google DeepMind's experimental open text generation model that uses text diffusion instead of standard autoregressive decoding, achieving up to 4x faster generation on dedicated GPUs. The 26B MoE model (3.8B active parameters) is built on the Gemma 4 backbone, supports multimodal inputs (text, image, video), has a 256K context window, covers 140+ languages, and is released under Apache 2.0.
DiffusionGemma is a 26B Mixture of Experts (MoE) model with 3.8B active parameters that generates text in parallel via diffusion, not token-by-token.
It achieves 1000+ tokens/s on a single NVIDIA H100 and 700+ tokens/s on an RTX 5090, fitting in 18GB VRAM when quantized.
Anthropic released Claude Fable 5, its most powerful public AI model, but it refuses to answer basic biology questions like 'what are mitochondria' due to strict safety guardrails designed to prevent misuse for bioweapons. The company admits this is overly conservative but necessary for safe deployment.
Claude Fable 5 refuses basic biology questions, handing them to older model Opus 4.8.
Anthropic intentionally set conservative safety filters to mitigate bioweapons risks.
Anthropic released Claude Fable 5, its first Mythos-class AI model. Microsoft restricts internal use due to new data retention requirements that retain prompts and outputs for 30 days (up to two years for flagged content). Other Claude models remain available under zero data retention. Legal teams are evaluating.
Microsoft limits employee access to Claude Fable 5 over data retention concerns.
Claude Fable 5 requires 30-day data retention; flagged data may be kept for two years.
Google DeepMind released DiffusionGemma, an experimental open model for fast text generation using parallel token generation. NVIDIA optimized it to run faster on GeForce RTX, RTX PRO, and DGX Spark systems, achieving up to 1000 tokens/sec locally.
DiffusionGemma generates up to 256 tokens in parallel per step, unlike traditional autoregressive models. Based on Gemma 4 (26B parameters, MoE), activating only 3.8B per step. Up to 4x faster performance. Open source under Apache 2.0, runs locally with no cloud dependency.
Everything we hear about artificial intelligence is conflicting, and hearing about it feels inescapable. AI is terrible. AI is wonderful. It will break the world. It will transform the future. It’s essential to embrace it. It’s a moral imperative to abstain from using it. Already, AI is projected to generate nearly unfathomable amounts of revenue. In the last quarter of 2025, it represented nearly 60% of the growth in the US economy. Already, pundits and economists wring their hands about what calamity will befall us if and when the AI bubble bursts.
AI absolutism encompasses both extreme optimism and pessimism, distorting public perception
AI already drives significant economic growth, accounting for nearly 60% of US GDP growth in Q4 2025
Deezer will now scan your playlists on other streaming platforms to detect AI-generated music. Deezer was the first of the big streaming services to start labeling AI-generated music. It even offered its tech to other platforms, but it doesn't seem like it had many buyers. Qobuz launched its own detection tech, while Apple and Spotify have opted for a voluntary tagging system.
Deezer launches AI music detector that scans playlists on 20 streaming platforms.
Deezer was first to label AI-generated music, but others opted for voluntary tagging.
OpenAI supports the EU Code of Practice on AI content transparency, advancing provenance standards and tools to help people understand AI-generated content.
OpenAI backs EU Code of Practice on AI transparency
PixelForge is an AI-powered tool that converts a single photo into a recognizable RPG character with a 4-direction sprite pack (4x4 sheet, 16 transparent PNG frames, walk GIFs) ready for game engines like Godot and Unity. One-time $5 fee, no account or subscription required. Created by Bernard Huang, launched on Product Hunt.
Upload a photo to generate a stylized game character
New college graduates around the country have been booing and heckling commencement speakers who hype up AI. Microsoft would like everyone to talk it out. In a blog post, Brad Smith addressed the backlash, suggesting it's a wake-up call, but the substance echoes the same pro-AI arguments that sparked the booing.
Graduates are booing AI-hyping speakers, reflecting broader societal discontent
Microsoft's Brad Smith responds with conciliatory tone but similar pro-AI messaging
Google is making changes to how it saves your interactions with Search. In an email to users, Google says it will save images, files, audio, and video from Lens, Search Live, voice searches, and Translate under a new 'Search Services History' setting. Users can opt out and disable 'Save Media' if they prefer not to have these interactions stored. The data will be used to improve services, including AI models.
Google introduces 'Search Services History' to save search-related media.
Covers interactions with Lens, Search Live, voice search, and Translate.
In this interview, AAAI Fellow Tanya Berger-Wolf discusses her pioneering work at the intersection of AI and ecology, including the development of the BioCLIP foundation model for the Tree of Life, its applications in biodiversity monitoring and conservation, and future directions for AI in science.
Tanya Berger-Wolf is a professor leading the Imageomics Institute, applying AI to ecology and conservation.
Her team developed BioCLIP, a foundation model for the Tree of Life that can classify species and discover new traits.
Anthropic released two new models, Claude Mythos 5 and Claude Fable 5, showing significant coding improvements but limited progress in image understanding. Testing reveals Fable 5 and GPT-5.5 can solve many vision tasks that stumped last year's models, yet geometric reasoning remains at the level of young children, suggesting general AI is still far off.
Anthropic unveils Claude Mythos 5 and Claude Fable 5, both variants of a preview model from two months ago.
Mythos is restricted to select organizations; Fable is public but with safety filters that reroute dangerous requests to weaker models.
The Verge's Regulator newsletter returns to a chaotic Washington landscape, covering the Washington AI Network gala, Pope Leo XIV's AI encyclical, and the unpredictable nature of AI regulation under Trump. The piece highlights the industry's dilemma in navigating partisan politics and the upcoming midterm elections, where AI is becoming a key voter issue.
Pope Leo XIV's encyclical on AI, Magnifica Humanitas, is met with indifference in Washington despite public interest.
Trump's back-and-forth on AI executive orders illustrates the volatile regulatory environment for tech.
Google researchers introduce Regularized f-Divergence Kernel Tests to audit machine unlearning and privacy. The framework adaptively selects optimal divergence measures, improving detection of data leaks and unlearning failures while requiring fewer samples and less tuning.
Two-sample tests lose power for large models; new framework is more sensitive and adaptive.
Uses f-divergences (chi-squared, KL, hockey-stick) to detect both global and local data shifts.
A group of independent musicians is suing Google, claiming it illegally used songs uploaded to YouTube to train its Lyria 3 model. Google has filed a motion to dismiss, arguing that the Terms of Service grant a broad license to use uploaded content. While Google hasn't explicitly confirmed using YouTube uploads for Lyria, past statements suggest it does.
Independent musicians sue Google over using YouTube songs to train Lyria AI.
Google moves to dismiss, citing broad license in Terms of Service.
Discover how astrophysicist Chi-kwan Chan uses Codex to build black hole simulations, helping scientists study extreme physics and test Einstein’s theory of general relativity.
Astrophysicist Chi-kwan Chan uses Codex to build black hole simulations.
Simulations help study extreme physics and test general relativity.
Sam Altman told employees he expects an OpenAI IPO "within the next year," but a delay to 2027 is possible. He frames it as caution around self-improving AI, though Anthropic's stronger growth numbers and imminent IPO may be the real reason to wait.
Altman expects OpenAI IPO within a year, possibly by 2027