AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。
Transactions are Generally Available | Databricks Blog Skip to main content Multi-Statement Transactions are Generally Available, enabling organizations to run complex, multi-step workflows with atomic guarantees without needing separate warehouse systems. Data warehouses power some of an enterprise’s most important workloads, including data ingestion, transformation, business intelligence, applications, and operational analytics. Many of these workloads require changes across multiple tables to succeed or fail together. Historically, organizations relied on separate warehouse systems for these transactional guarantees, adding expensive data movement, duplicate copies, fragmented governance, and operational complexity. Databricks is the first lakehouse architecture to support multi-statement transactions natively across both Delta Lake and Apache Iceberg tables. Alongside stored procedures, temporary tables, and SQL scripting, multi-statement transactions bring traditional data warehousing workloads to open, governed data on the lakehouse. Problems with transactions on traditional and cloud warehouses Traditional and cloud warehouses provide transactional guarantees, but often at the cost of a fragmented data architecture: Additional copies and data movement: Moving data into and out of a separate warehouse adds latency, cost, and inconsistency risk. Fragmented governance: Teams must duplicate and synchronize access policies, auditing, and operational controls across systems. Complex consistency workarounds: Developers need custom orchestration, reconciliation, and retry logic to keep related changes consistent. Multi-statement transactions bring these guarantees directly to the open, governed lakehouse, reducing duplicate systems and the operational work required to keep them aligned. Transactions on the lakehouse deliver a unified foundation for warehousing Transactions on the lakehouse provide a foundation for running demanding warehouse workloads on open enterprise data. They combine support for open table formats with unified governance and high-concurrency performance. They also integrate with familiar warehouse development tools, data pipelines, and production observability capabilities. Open across Delta Lake and Apache Iceberg Multi-statement transactions support both Delta Lake and Apache Iceberg tables. Teams can preserve open table formats while gaining atomic guarantees across multiple statements and tables. They do not have to move governed data into a proprietary format or maintain a transactional copy in another system. Fast at high concurrency The lakehouse architecture supports row-level concurrency, allowing concurrent transactions to write to the same table as long as they do not update the same row. Unlike traditional data warehouses, which use locks to serialize writes, row-level concurrency enables more transactions to run in parallel. As a result, Databricks delivers significantly stronger performance than traditional cloud and legacy warehouses, allowing organizations to run demanding warehouse workloads directly on the lakehouse. Integrated with warehouse development and data pipelines Transactions are composable with other warehousing capabilities, enabling teams to build traditional data warehousing workloads on the lakehouse. For example, developers can stage intermediate results in a temporary table, package multi-step business logic in a stored procedure, and commit the entire workflow atomically. Transactions can also write to streaming tables, bringing atomic guarantees to workflows that continuously ingest, transform, and publish data. Now, downstream pipelines can receive consistent updates, preventing them from processing incomplete data. Governed transactions on sensitive data Transactions respect Unity Catalog attribute-based access controls, including row-level security and column masking, on both reads and writes. This is especially important for workflows that stage, validate, and publish sensitive data. Observability in in production Production transactions need to be understandable as well as correct. Developers can use commands such as DESCRIBE TABLE, SHOW COLUMNS, and GET DIAGNOSTICS to inspect the tables involved in a transaction and capture information about transactional execution. These capabilities make transactional workflows easier to develop, operate, and troubleshoot. This fully integrated experience enables Fivetran to build their connectors on top of lakehouse transactions. "Fivetran History Mode preserves each version of a record using slowly changing dimensions (SCD) Type 2. Each source change is two destination writes: closing out the prior version and inserting the new active one - both must succeed together. Previously, with transactional guarantees scoped to a single statement, we ran these as separate operations and defended against partial failure with reconciliation logic and consistency checks on subsequent syncs. With multi-statement, multi-table transactions, we now commit the close and the insert as one atomic unit so either the full change is applied or none of it is. This let Fivetran remove an entire class of consistency-repair code and gave our customers a stronger correctness guarantee on historical data."—Egidio Terra, Staff Product Manager, Fivetran Likewise, Accolade uses transactions to process thousands of claims and liabilities records every day. "At Accolade, we process roughly 1,700 transactions in production every day to handle incoming claims and liabilities. Each of those updates has to complete reliably and consistently. Multi-statement, multi-table transactions let us run these multi-step operations with full transactional guarantees, directly on our governed lakehouse. That reliability is central to our broader migration of critical workloads to the open lakehouse, without standing up a separate warehouse."—Akshay Ajgaonkar, Engineering Manager, Accolade Together, open formats, centralized governance, row-level concurrency, familiar SQL features, pipeline integration, and observability create a complete transactional experience on the lakehouse. Key use cases for multi-statement transactions Multi-statement, multi-table transactions support several common patterns for modernizing ETL, migrating warehouse workloads, and publishing trusted data. Give ETL multi-table transactional guarantees An ETL job can update related tables as one unit. In this example, the order status and inventory balance cannot fall out of sync. If either update fails, the workflow can roll back both changes rather than publishing a fulfilled order without the corresponding inventory adjustment. Migrate warehouse procedures to the lakehouse Stored procedures and temporary tables let developers preserve familiar warehouse development patterns while operating on governed lakehouse data. Unity Catalog row-level security and column masking policies continue to apply to governed reads and writes. The procedure also uses DESCRIBE TABLE to inspect the target’s schema and metadata before the next set of writes, helping developers verify its configuration and troubleshoot issues without leaving the transactional workflow. Implement Write-Audit-Publish Write-Audit-Publish prevents data from becoming visible until it passes the required quality checks. The workflow stages new data, records its quality results, and publishes the validated records in one atomic operation. If a validation or write fails, the transaction can roll back instead of exposing a partial result. The same pattern can include transactional writes to streaming tables when downstream pipelines need the published data continuously. Get started with transactions today Multi-Statement Transactions are Generally Available for UC managed Delta Lake and Apache Iceberg tables on Databricks. Get started with transactions today using Databricks’s documentation or by prompting the in-product Assistant. With transactions on the lakehouse, you can bring critical warehouse workloads to open, governed data while reducing duplicate systems, data movement, and operational complexity. Get the latest posts in your inbox Subscribe to our blog and get the latest posts delivered to your inbox. Sign up View all blogs