Meta’s Spark Muse 1.1 is now available on Databricks, fully governed by Unity AI Gateway

Meta’s Spark Muse 1.1 is now available on Databricks, fully governed by Unity AI Gateway | Databricks Blog

Skip to main content

• Get day-one access to Meta's new Muse Spark 1.1 (and any model) through Model Provider Services in Unity AI Gateway. Register providers once in Unity Catalog to eliminate API key sprawl and centralize access across every team.

• Teams can use familiar Unity Catalog permissions, rate limits, and guardrails to securely enable new models on day one, while keeping provider credentials encrypted and centrally managed.

• Every request is automatically tracked with token usage, latency, cost attribution, and audit logs, giving platform teams end-to-end observability and enabling governance, budgeting, and compliance across all model providers

Every new model release promises better reasoning, lower costs, or new capabilities, and developers want access on day one. But every provider also introduces another set of API keys, another integration, and another governance surface for platform teams to manage. Without a centralized way to govern all model providers, access becomes fragmented, API keys proliferate, and visibility into usage and spend disappears.

Today, we're announcing support for Meta's new Muse Spark 1.1 on Databricks through the new Model Provider Services (MPS) in Unity AI Gateway. MPS lets you connect and govern model providers, including OpenAI, Anthropic, Amazon Bedrock, and newly released models like Muse Spark 1.1, through Unity Catalog. Now you can register models once, control access using familiar Unity Catalog permissions, and let every team query through Unity AI Gateway with full governance and observability in place.

In this post, we'll use Muse Spark 1.1 to show how organizations can adopt a newly released model on day one without compromising governance or security.

The AI governance gap when new models launch

Say your team wants Muse Spark 1.1 the day it ships. One team creates a provider account and starts building with it. Another team requests its own API key. Soon, multiple copies of the same key are spread across notebooks, applications, and CI/CD pipelines, each managed independently. Access control is just as fragmented because there’s no consistent way to say “these three teams can use the premium model, everyone else uses the standard one.”

On the platform admin side, there is no unified view of spend, no token-level attribution, no record of which prompts left the building, and no place to enforce a rule before a request reaches a model provider. When finance asks why the Meta bill tripled, you can see the total in Meta's console, but not which team or workspace drove it in a single centralized report.

This pattern doesn't just apply to Muse Spark, as it is the same challenge organizations face every time they adopt a new model.

The solution: Model Provider Services in Unity AI Gateway

A Model Provider Service is a governed Unity Catalog securable that represents an external provider. It lives in a catalog and schema, and holds the provider's connection configuration and API key. Callers reference the service by name and authenticate with their own Databricks credentials; the gateway attaches the provider's API key at request time. The API key is stored via a Unity Catalog connection, encrypted with a platform-managed or customer-managed key, and never exposed directly to client consumers.

Once a model provider service is registered in Unity Catalog, your organization gets three things: Choice, Control, and Clarity.

Choice — adopt any model, then switch freely: Define a provider and its API key once and make it available to every team across every workspace. Access newly launched models on day one and switch between models within a provider. Quickly switch between OpenAI-compatible models across providers for evaluation and production use cases.

Control — govern access in one place: The service is first-class, so you govern permissions with standard GRANT and REVOKE: EXECUTE to query, READ_METADATA to view, and MANAGE to update the service. Attach rate limits and service policies to the model provider service, so throttling and guardrails apply to every model request.

Clarity — see and account for every request: Usage and spend tracking is on by default, so every request is metered with token counts, latency, and status codes for accurate cost attribution per user, team, or app. Attach inference tables to log full request and response payloads to a governed Delta table for troubleshooting or audit.

Figure 1. Register a provider once in Unity Catalog; every team and provider is governed through one gateway.

How it works

Let’s register the externally hosted Muse Spark 1.1 model as a Model Provider Service, lock down who can use it, turn on monitoring, and query it end-to-end.

Register the provider

In order to use Muse Spark 1.1 on Databricks, you need to first register it in Unity Catalog. Obtain your Muse Spark API key from Meta's Model API, which is currently in Public Preview. Because Muse Spark is compatible with the OpenAI Responses API, you can register it using the OpenAI provider type to connect directly to Meta's API.

Register it in the Unity Catalog UI: Catalog Explorer → Create → Create a service → Model provider service, choose OpenAI, paste the Meta key as the API key, https://api.meta.ai/v1 as the Base URL, add muse-spark-1.1 as the model and set the model’s API type to /openai/v1/responses.

With the provider registered, two key security controls are applied. First, the API key is stored encrypted within the Unity Catalog. Second, the model list strictly defines which models and API surfaces are exposed, so any request for an unlisted model is intercepted and rejected at the gateway before it reaches Meta.

Grant access

The service is a Unity Catalog securable, so you govern it using the same principles you apply to any other object. To query it, a caller needs EXECUTE on the service plus USE CATALOG and USE SCHEMA on its parent. The service creator or admins with MANAGE permission on the service can grant such privileges.

To grant access:

In Catalog Explorer, open the model provider service.

On the Permissions tab, click Grant.

Select the users or groups to grant access to, select the EXECUTE privilege, and click Grant.

Query the model

Figure 2. A single request: the gateway checks access, applies rate limits and policies, routes to Muse Spark, and logs usage.

Because Muse Spark speaks the OpenAI Responses API, point any OpenAI-compatible client at the gateway and set one header for querying:

The additional header allows Unity AI Gateway to identify the model provider service and validate the caller's privileges. Once the privileges are validated, the gateway will resolve the configuration and route the request to the external model.

Log, monitor, and guardrail

Unity AI Gateway meters every request routed through the service. Every usage is reported in system.ai_gateway.usage with input/output token counts, latency, and status codes. Spend info is recorded in system.ai_gateway.external_model_spend. Add a Databricks-Ai-Gateway-Request-Tags header to slice spend by project, and attach inference tables to log full request and response payloads to a governed Delta table for audit.

Attach a policy to the service to enforce guardrails for each request, including default guardrails for common risks such as PII, prompt injection, and unsafe content, and the ability to add custom policies for your own rules. Guardrails run centrally at the gateway, so any unsafe prompt is caught before it reaches Muse Spark, no matter who sent it. Per-service rate limits cap capacity and cost.

Getting started

Model Provider Services are available across AWS, Azure, and GCP. Account administrators can enable the preview on the account console's Previews page.

To go deeper, see the docs on Model Provider Services, governing access, and querying through the gateway.

Unity AI Gateway brings choice, control, and clarity so your team can use any model they want to. Enable the preview and try out Model Provider Services today!

Get the latest posts in your inbox

Subscribe to our blog and get the latest posts delivered to your inbox.

Sign up

View all blogs