Red Hat Learns New AI Tricks at Summit 2026

IBM used its Red Hat Summit conference that’s taking place this week in Atlanta, Georgia as the location for the launch of Red Hat AI 3.4, a new release of its overarching product suite for building and deploying AI. Among the items in this new suite is a new service called Red Hat AI Inference on IBM Cloud, new AI developer tools, better security, Red Hat Hardened Images, and a new Dev Spaces framework, among other enhancements.

In the early years of the AI boom, most of the focus was on using big data to train large language models (LLMs) and other foundation models. But things have changed, and today it’s all about running AI against real world data, or AI inference.

AI inference has different requirements than AI training. There are real-time performance and latency requirements. There are a large number of AI agents to manage. Users also demand that each AI sessions is secure and that the AI models are well-governed.

These are all factors that IBM is taking into account with its Red Hat AI Inference Server, a shrink-wrapped offering that is based on a pair of open source libraries, including vLLM, which includes an AI inference server and AI inference engine, as well as llm-d, a Kubernetes-based framework for running LLMs in a distributed and disaggregated manner.

Red Hat says its AI Inference Server is “optimized for high throughput and low latency” of AI applications and agents. Its model catalog ships with IBM Granite 4.0 H Small, Mistral-Small-3.2-24B-Instruct, Llama 3.3 70B Instruct, GPT-OSS-120B, and Nemotron-3-Nano-30B-FP8, with more open models and custom models on the way.

Red Hat CEO and President Matt Hicks delivers the keynote address at Summit 2026

With the launch of Red Hat AI Inference on IBM Cloud, IBM is making it easy for IBM Cloud customers to get up and running with the Red Hat AI Inference stack, according to Jason McGee, CTO of IBM Cloud.

“Enterprises are eager to operationalize AI, but the gap between pilot and production may hold them back,” McGee stated. “With Red Hat AI Inference on IBM Cloud, we’re giving clients a managed platform that is built for real workloads, not just experiments. At the same time, our new virtualization offering on IBM Cloud is enabling enterprises to migrate to a resilient and security-focused virtualization environment while giving them the flexibility to adopt Red Hat OpenShift at their own pace for future AI workloads and containerization.”

The new service is in limited release, with general availability expected next month.

IBM also announced that Red Hat AI Inference can now run on other flavors of Kubernetes besides Red Hat OpenShift, including Kubernetes distributions hosted by CoreWeave and Microsoft Azure. This will give customers another option if they don’t want to run on IBM Cloud.

Red Hat AI 3.4 also includes several other new and enhanced capabilities, including:

A new governed model as a service (MaaS) capability, which governs the use of internal models as well as external APIs;

A centralized registry for developing and managing prompts;

Built-in evaluations for models and agents to replace fragmented testing of models;

Integrated security technology from Chatterbox Labs to screen models and agents for risks like jailbreaks, prompt injections, and bias;

Built-in observability via support for MLflow and OpenTelemetry to provide end-to-end tracing of LLM calls, reasoning steps, model responses, and token usage;

Support for cryptographic identity management via SPIFFE/SPIRE;

Built-in support for AutoRAG and AutoML that automate AI tasks;

And support for Nvidia Blackwell and AMD MI325X GPUs.

IBM is also improving tools for AI developers. Included in the new release of Red Hat Desktop is a build of Podman Desktop, which provides a foundation for developing containerized AI apps. IBM/Red Hat is also giving developers new tools for building isolated AI agent sandboxes, which will help developers test and iterate in a safe manner.

The updated Red Hat Advanced Developer Suite brings access to Red Hat Trusted Libraries, as well as security services aimed at preventing AI-driven exploits. IBM/Red Hat says it’s using AI “to determine if known vulnerabilities in generated code are relevant to a specific application runtime, allowing developers to prioritize remediation based on actual risk.”

“The transition to agentic AI expands the requirements for modern application development,” said James Labocki, senior director, product management, Red Hat. “We’re helping developers accelerate and own their AI strategy with the same rigor they apply to their core IT applications.”

Also announced is an update to Red Hat OpenShift Dev Spaces, which it describes as “an extensible framework that allows developers to integrate preferred AI-driven tools directly into their cloud-based IDE.” With this release, IBM/Red Hat is now incorporating the AWS Kiro coding assistant to go along with existing integrations for Microsoft Copilot, Anthropic Claude CLI, Cline, Continue, Roo, and others.

The company aims to bolster security via Red Hat Hardened Images, which is a collection of secure components for deploying AI. IBM/Red Hat says the Hardened Images are developed using its “trusted software pipeline” and are secure out of the box. It’s part of IBM/Red Hat’s strategy for developing a “Zero-CVE” environment, referring to the US Government’s Common Vulnerabilities and Exposure database.

“Our goal is to cut through the security noise and give developers a foundation where they can build and scale without having to patch or manage software that their applications do not actually need,” stated Gunnar Hellekson, vice president and general manager of Red Hat Enterprise Linux.

This article first appeared in HPCwire.

The post Red Hat Learns New AI Tricks at Summit 2026 appeared first on AIwire.