AI News HubLIVE
站内改写6 分钟阅读

待翻译:Operationalizing Genie Ontology in Your Data Stack

AI 服务暂时不可用,以下为来源摘要,待恢复后补全翻译:Beyond the semantic model: Building shared business context for AI agentsLarge language...

AI 服务暂时不可用,以下为来源正文,待恢复后补全翻译。

Operationalizing Genie Ontology in Your Data Stack | Databricks Blog Skip to main content Genie Ontology works on day one, but achieving the highest possible accuracy depends on the underlying foundation. This guide shows you how to build that foundation on your data. Use the six layers as your progressive maturity path to improve the data foundation, enrich metadata, model critical business semantics, curate trusted assets, govern access, and evaluate and improve. Roll out one domain at a time rather than trying to boil the ocean. Every resolved entity, documented table, certified metric, and governed dataset improves answer quality, while evaluation keeps the system accurate as the business evolves. Beyond the semantic model: Building shared business context for AI agents Large language models know how to reason, but they don't know your business. Giving enterprise AI the business context it needs means more than connecting it to data. Agents also need to understand your definitions, relationships, business rules, authoritative sources, and permissions. Genie Ontology closes that gap by combining modeled business semantics with context learned from the governed tables, queries, dashboards, notebooks, and other supported assets your teams already use. Genie ranks that context by authority and relevance, applies permissions, and delivers the most useful context to Genie at answer time. External agents can also access Genie’s intelligence through MCP. A good semantic model provides an authoritative core. Semantic models capture the business concepts you deliberately define; an ontology extends that foundation with the broader relationships, knowledge, and context AI needs to understand how the business actually operates. In Databricks speak, Unity Catalog Semantics combine Metric Views, Pages, and Domains to establish your trusted business definitions. Genie Ontology then builds on that modeled core by incorporating inferred context from your existing assets, giving agents a much broader understanding of the business than a semantic model alone can provide. The key is to model the “head” and let Genie Ontology infer the “tail”. Genie works from what it can automatically learn on day one, while deliberate curation improves the critical definitions and sources that must be right. The following six layers are progressive practices for increasing trust over time, not prerequisites for Genie to begin delivering value. Let’s look at each layer to get a deeper understanding. Layer 0: Get your data foundation right for agents Before you describe or model anything, the underlying data has to be in a shape an agent can reason over. This layer is about the physical foundation: clean data tables, sound schemas, and one consistent identity per real-world entity. Logical business modeling comes later in Layer 2. This layer is easy to skip and expensive to fix later, because no amount of good metadata or semantic modeling can compensate for a broken physical foundation. Getting your data foundation right means focusing on two key areas: Model the durable gold layer around business processes. This includes identifying facts with a clear grain and reusable, conformed dimensions. A star schema or hybrid model gives you a reliable foundation of facts and dimensions without copying business logic into every downstream table. That does not mean agents should reason over raw dimensional tables. The interface an agent sees can be narrower, and often should be: a Metric View or a purpose-built view that pre-joins the common dimensions for one domain, exposes only the fields that matter, documents the grain, and defines its measures canonically. You will build exactly that in Layer 2. The point is to shape the consumption surface deliberately on top of a sound model, rather than throw an agent at a data dump. A wide table is not the problem. A wide table with mixed grains, duplicated business concepts, and no canonical metric definitions is an invitation to guess. Resolve entities into golden records. If "customer" means active accounts in Sales and every account ever in Support, an agent will not know which definition to trust. If the same customer has three different IDs across systems, it can also be double-counted. Reconcile the same real-world entity across sources so that one customer is one customer. Layer 1: Enrich your metadata Metadata is the descriptive foundation that helps both the semantic layer and context extraction understand your data. When a table is named fct_rev_daily, and a column is named rev_amt, an agent has to guess what they mean. When the same table carries a description that says "daily recognized revenue, net of refunds, by product" and the column has a comment that says "recognized revenue in USD," the agent has something real to reason with. Good descriptions are one of the highest-return, lowest-cost investments you can make, and they improve every downstream tool, not just Genie. Here are three essential steps to take. Add table descriptions and column comments in Unity Catalog. Write for a new analyst who does not know your schema: say what the data represents, what its business purpose is, and flag any known caveats. Concentrate the effort where it pays off, on the curated, business-ready tables that dashboards and agents actually query. Apply tags to classify and organize. Descriptions provide the narrative meaning; tags provide structured signals for classification, discovery, and governance. Use them to capture sensitivity (PII, PHI, PCI), ownership, business function, and other attributes that need to be consistently understood across the data estate. Governed tags let administrators define an approved set of keys and values so classification remains consistent rather than drifting team by team. These signals can also feed into access policies and other governance controls later. Automate the first pass where volume makes manual work impractical. The Databricks Solution Accelerator, dbxmetagen, uses large language models to generate descriptions, detect and tag sensitive data, and propose classifications. Nothing is written to Unity Catalog until a human reviews and approves it, so it accelerates the work rather than replacing the judgment. Layer 2: Model the business with a semantic layer Metadata explains individual tables. The semantic layer defines the business logic on top, so metrics that matter most mean the same thing everywhere they are used. Where Layer 0 established the physical foundation, this layer creates the logical model: measures, relationships, domains, and terms. Here are four important steps: Declare your relationships. Agents join tables to answer questions, and if they have to guess how tables connect, they will sometimes guess wrong. Declaring primary and foreign keys in Unity Catalog tells agents how tables relate, so they join correctly instead of inventing paths. These constraints are informational rather than enforced, so your governance process has to keep them accurate, but declaring them is one of the most direct ways to reduce join errors. Relationships are as much a part of the model as the metrics themselves. Build a semantic model using Metric Views. A Metric View is a Unity Catalog object that defines your measures (the aggregated numbers, like total revenue) and dimensions (the ways you slice them, like region or month) once, as governed code. Because the aggregation is resolved at query time rather than baked in, consumers who query the Metric View use the same governed definition. This is one of the most important steps for accuracy because it removes the ambiguity that causes agents to choose the wrong definition. Define your critical KPIs as Metric Views first: these are the numbers that absolutely cannot be wrong, like revenue, active customers, and core compliance measures. Add agent-facing metadata to your metrics. Metric Views can carry display names and synonyms, so natural language like "sales" maps to the right measure, format patterns for currency and dates, and example queries. This metadata flows into Genie Ontology, so the work you do to model a metric also makes it easier for an agent to find and use it correctly. Organize and document. Unity Catalog Domains and sub-domains group assets into business-aligned collections so that the context stays scoped. This improves the speed and accuracy of Genie, as it can focus its discovery on assets within relevant business areas rather than searching across the whole estate. Unity Catalog Pages capture the shared business terms, concepts, and definitions that business users and agents use to reason. Each Page lists the authoritative assets tied to that concept, so when the ontology resolves a term from a question, it already knows which tables, Metric Views, and queries to draw on rather than searching the estate and guessing. Owner review keeps these definitions trustworthy, and because they are human-asserted and reviewed, they carry more authority than inferred context when the ontology has to resolve a conflict. Automating Semantic Modeling with Genie Code Of course, the semantic modeling process does not have to be a purely manual effort. You can leverage Genie Code to create and maintain Metric Views using natural-language instructions. In the Genie Code prompt, describe the source tables, joins, fields, measures, and filters, and it generates the YAML for you to review before saving. You can also use the /importBI skill in Genie Code to import Tableau or Power BI semantic models, and it builds a Metric View, which you then promote to Unity Catalog for reuse, governance, lineage, and discoverability. Genie Code can also draft Pages. In the Page editor, select a domain, attach relevant files, links, Unity Catalog assets, or MCP-connected content, and Genie Code drafts the structured fields and rich-text Page body. Review the draft, add the appropriate Sources and Related assets, then save or publish it. For multiple concepts, use Bulk import pages. Genie Code extracts and deduplicates proposed Pages from your documents and sources, flags conflicts, duplicates, and low-confidence terms for review, and creates the approved Pages as drafts for subsequent editing and publication. Layer 3: Curate context-rich assets The inferred part of the ontology learns from the assets your teams already produce, like dashboards, notebooks, SQL queries, Genie Agents, and documentation. The richer and more trustworthy your data estate is, the more useful the inferred context becomes. Layer 3 is about making those assets worth learning from. Here are four essential steps: Build a deep, well-used asset base. A workspace with many well-documented, widely used dashboards, queries, and Genie Agents gives the ontology more material to learn from than a sparse one does. As those assets are used and improved, the signals available for context extraction become richer. Make your assets context-rich. The richer an asset, the more the ontology can learn from it. When you enrich a Genie Agent with definitions, examples, and instructions, you not only get a more performant agent, but you also give Genie Ontology stronger context to extract and rank across the estate. Do the same for your other assets: document notebooks with Markdown cells, saved SQL queries with comments, and AI/BI dashboards with descriptions and annotations. Together, these provide the rich, long-tail business knowledge that Genie Ontology draws on. Certify the assets you trust. Certification marks your data & AI assets like Metric Views, Genie Agents, or Notebooks as validated and approved and serves as a strong signal for determining which sources are authoritative. Certified and widely used assets carry more authority than unvetted ones, so certifying your trusted assets directly influences which context prevails in a conflict. Deprecating stale asse [truncated for AI cost control]