The AGI moment? Databricks’ new releases zero in on support and deployment of AI agents

Major enterprise platform companies are racing to build tools for a new class of users: artificial intelligence agents.

One example of this was apparent today with the latest releases from Databricks Inc. The company unveiled a new architecture – Lake Transactional/Analytical Processing – that enables AI agents to access operational and analytics workloads on a primary copy of data that resides in a data lake.

By placing this data in the same open format, Databricks believes that agents will have the capability to observe and reason across a multitude of production databases within an enterprise and take action accordingly. It’s an important milestone in the realization of artificial general intelligence or AGI, the ability of AI to match or exceed human capabilities, according to Databricks co-founder and Chief Executive Ali Ghodsi.

“We believe that AGI is already here,” Ghodsi (pictured) said during his keynote remarks at the Data + AI Summit in San Francisco today. “AI does not have an intelligence problem right now. It’s plenty smart. The problem is that AGI is not completely permeating our organizations. The question is: ‘How do we enable this at work?’”

Clearing a path for autonomous employees

Ghodsi’s belief in AGI’s arrival is being constrained by a set of factors encompassing context, cost and control. Databricks wants to build a platform that can remove these obstacles while serving a new class of autonomous employees that can launch numerous versions of an application at the same time while creating and discarding entire software environments in minutes.

Databricks’ launch of its real-time Lakehouse is powered by Reyden, a new compute engine designed to deliver millisecond query latency for tens of thousands of concurrent users and agents. The name originated from “Reynold’s Dream Engine,” a nod to the role of Databricks’ co-founder Reynold Xin in the new release’s creation.

Xin appeared during the keynote session to demonstrate Reyden’s features, which included consistent low-latency response times when thousands of AI agents hit the same query concurrently.

“None of the other existing systems can do that,” Xin told the conference gathering. “It is probably the single largest introduction we have done since the launch of Lakehouse.”

Enhancements for Genie AI

Databricks also addressed the challenge of agentic context with the introduction of a set of enhancements for its Genie AI platform. Today’s launch of Genie One is designed to help business teams automate work that is grounded in real business data.

In a session with the media following his keynote, Ghodsi described the differentiators that Databricks has built into its new agentic offering. “Genie One computes whereas other agents recite,” Ghodsi explained. “That’s a unique advantage that it has.”

Databricks is now powering its suite of AI co-workers with Genie Ontology, a live context layer that continuously learns from internal and external business data. Ghodsi likened Genie Ontology to page rank algorithms used by Google Search to identify the most relevant information.

“We’re doing the same thing for the enterprise,” Ghodsi said. “Genie Ontology searches behind the scenes. We think this is the missing puzzle piece for agents.”

OpenAI co-founder Greg Brockman