跳到主要內容
AI News HubLIVE
來源內容 · 翻譯待補全5 分鐘閱讀

待翻譯:What is HTAP? A plain-English guide to hybrid transactional/analytical processing

文章摘要

AI 服務暫時不可用,以下為來源摘要,待恢復後補全翻譯:Consider a payment system processing a purchase. Before approving the transaction,...

待翻譯:What is HTAP? A plain-English guide to hybrid transactional/analytical processing
報告錯誤

更正渠道尚未開通,可先複製下方文章資訊留存。

查看更正說明
直接讀正文

AI 服務暫時不可用,以下為來源正文,待恢復後補全翻譯。

What is HTAP? A plain-English guide to hybrid transactional/analytical processing | Databricks Blog Skip to main content Consider a payment system processing a purchase. Before approving the transaction, a fraud model may need to evaluate it against the account's recent activity. But if that transactional data must first move from the transactional database to a data warehouse for analysis, even a short delay can make the result less useful. Hybrid Transactional/Analytical Processing (HTAP) brings transaction processing and analytics closer together to address this problem. An HTAP database lets you run analytical queries on operational data while it is still current, without waiting for a batch data pipeline to move it into a separate analytical system. In this guide, you'll learn how HTAP works, the architectural tradeoffs involved, and where it fits in a modern data architecture. You'll also see how different HTAP systems approach transactional and analytical workloads, how HTAP compares with Lake Transactional/Analytical Processing (LTAP), and what to consider when evaluating and adopting either approach. TL;DR HTAP brings transactional and analytical processing closer together so applications can run analytics on fresh transactional data. Different HTAP architectures involve different trade-offs in performance, isolation, data consistency, and freshness. Choose the architecture based on your workload. LTAP provides greater separation between transactional and analytical compute while both operate on the same data. What is HTAP? HTAP stands for Hybrid Transactional/Analytical Processing. Gartner coined the term in 2014 to describe architectures that bring online transaction processing (OLTP) and online analytical processing (OLAP) together within an integrated database system. Gartner's original framing of hybrid transaction/analytical processing argued that combining the two would create opportunities for dramatic business innovation, because decisions could be OLTP handles high volumes of small, frequent transactional operations that keep applications running, such as creating an order, processing a payment, or updating an account. OLAP processes complex queries over large datasets to calculate metrics, identify patterns, and answer analytical questions.made on the latest transactions rather than on yesterday's extract. HTAP brings these workloads closer together, giving analytical queries access to current operational data without depending entirely on a separate batch ETL pipeline. For many operational analytics use cases, HTAP systems can reduce or remove the need for ETL processes, cut down the number of data copies teams maintain, and move closer to a single source of truth for both operational and analytical systems. In the most integrated designs, OLTP and OLAP capabilities live in a single system rather than in two products stitched together. Bringing OLTP and OLAP workloads closer together does not mean they have to run in exactly the same way. HTAP architectures therefore differ in how they combine OLTP and OLAP capabilities while preserving the characteristics each workload needs. Two common approaches are: Converged HTAP: Runs transactional and analytical workloads through a single database engine. The workloads may use different indexes, storage formats, or execution paths, but they share much of the same database. Composed HTAP: Uses separate transactional and analytical engines within an integrated architecture. The engines may share a storage layer or propagate changes through replication while remaining optimized for their respective workloads. A converged engine avoids the need to synchronize independent query engines, but it must prevent analytical operations from interfering with latency-sensitive transactions. A composed architecture provides greater freedom to optimize and isolate each workload, but it must coordinate data versions and keep the analytical path sufficiently fresh. In both cases, the central HTAP challenge is the same: provide analytical access to operational data quickly enough to be useful without compromising the performance and reliability of transactional workloads. Why OLTP and OLAP became separate systems OLTP and OLAP have different access patterns. Transactional queries typically read or update a small number of complete rows. OLTP databases therefore rely on row-based storage, selective indexes, efficient concurrency control, and predictable latency, often measured in milliseconds. Most transactional databases are relational databases tuned to keep that latency low even under heavy concurrency. Analytical queries, by contrast, often scan millions of rows across a subset of columns to calculate metrics or identify patterns. OLAP databases use columnar storage because a column store improves compression and reduces the amount of unnecessary data read during large scans and aggregations. OLAP systems can tolerate longer latency for complex queries, trading seconds of response time for the ability to analyze far more data. Optimizing OLTP and OLAP systems separately therefore made practical sense, but it introduced a new problem: the delay caused by data movement between them. Operational data typically had to be copied out of the OLTP system and into a data warehouse or other analytical database before it could be analyzed. Transactions and analytics processing ran on different schedules, on different hardware, against different copies of the data. Traditional architectures that separate OLTP and OLAP systems in this way build latency into every analytical answer. As requirements for fresher data grew, data teams started assembling their own custom systems to close the gap. Change data capture feeds, stream processors, cloud data warehouses, cloud data lakes, and real-time query layers became the building blocks of today's disaggregated data stack. Each component solved a specific problem, but every additional hop added cost, operational effort, and another place where data consistency could break. HTAP addresses that delay while preserving the performance benefits of workload-specific optimization. How HTAP architecture works An HTAP architecture provides paths for both transactional and analytical workloads while keeping operational data available for analysis with minimal delay. The exact design varies by system, but the architecture generally includes transaction processing, analytical processing, workload-specific data representations, and a mechanism for keeping those representations synchronized, as shown below. Transactional processing The transactional path handles frequent application reads and writes, such as creating orders, updating accounts, or recording payments. These transactional workloads typically operate on small numbers of rows and require low latency, efficient concurrency control, and reliable transaction guarantees. HTAP systems commonly use row-oriented storage engines, indexes, caches, and a write-ahead log designed to serve these operational access patterns efficiently and durably. Analytical processing The analytical path handles analytical queries that scan, filter, join, and aggregate larger amounts of data. Columnar representations are commonly used because analytical queries often need a subset of columns across many rows, and they are what make fast OLAP queries possible on operational data. Depending on the HTAP design, analytical processing may run in the same database engine as transactions or on separate analytical compute. Separating the compute paths allows each workload to use execution resources optimized for its access patterns, so the system can add analytical capabilities without slowing OLTP systems down. Many HTAP systems also use intelligent query optimization, where the optimizer decides whether a query should read from the row store, the column store, or both. Storage and data representation HTAP systems may maintain more than one representation of the same data. A row-oriented storage engine can serve transactional access while a columnar storage engine supports analytical scans and aggregations. These representations may exist within one database system or across separate transactional and analytical components. Some architectures also decouple compute from storage and use object storage alongside caches, logs, or other services. Synchronization and freshness When transactional and analytical processing use different storage engines or components, committed changes need a path from the transactional side to the analytical side. Systems can use mechanisms such as change data capture (CDC), replication of the write-ahead log, shared logs, or shared storage to propagate those changes. The result is an analytical view that can incorporate recently committed transactions without requiring the traditional process of periodically extracting an entire dataset into a separate analytical system. How quickly those changes become visible depends on the architecture and its consistency guarantees. Challenges of HTAP databases Bringing transactional and analytical workloads closer together introduces tradeoffs in how data is stored and processed. Where those costs appear depends on the architecture, but HTAP databases struggle with the same core problem: optimizing for both OLTP and OLAP workloads at once. Four challenges are common. Write amplification Maintaining analytical data alongside frequent transactional writes creates additional work. Because many HTAP systems store data in two formats, updates may generate deltas or require refresh and compaction, increasing compute, I/O, and storage usage even when that work stays off the transaction path. Resource contention When OLTP workloads and OLAP workloads share resources, large analytical queries can push transaction latency from milliseconds to seconds. Separate compute provides stronger isolation, but you still need to account for shared bottlenecks such as storage, networking, and coordination services. Consistency and freshness Transactional and analytical paths may not advance at the same rate. You need to decide how quickly committed changes must become available for analysis and what data consistency guarantees the workload requires. Greater freshness typically requires tighter coordination between the two paths, and every HTAP architecture manages some tension between synchronization and isolation rather than eliminating it. Operational complexity Supporting mixed workloads can mean managing multiple data representations, execution paths, resource policies, and synchronization mechanisms. Query optimizers must also choose good execution plans across two storage formats. You need to account for that complexity when monitoring, scaling, troubleshooting, and recovering the system. Evaluating HTAP solutions: Checklist When evaluating an HTAP solution, test transactional and analytical workloads together under conditions that reflect your production environment. Measure whether transaction latency, analytical performance, and data freshness remain within your requirements as both workloads run concurrently. CriterionWhat to test OLTP p99 latency under mixed loadCheck whether p99 (99th-percentile) transaction latency stays within your target as analytical load increases. Analytical performance at scaleTest scan and aggregation performance as your data volume, query complexity, and concurrency grow. Freshness and consistencyMeasure how quickly committed changes become queryable and whether the consistency guarantees meet your requirements, including when the analytical path falls behind. Cost under mixed loadMeasure compute, storage, caching, replication, network, and data-transfer costs at traffic levels representative of your workload. Failure and recoveryTest how queries behave during failures, whether correctness is maintained under degrad [truncated for AI cost control]

展開要點與分析

文章情報

研究者進階

要點

  • AI 服務暫時不可用,系統已先保留來源內容與降級元數據。
  • Consider a payment system processing a purchase. Before approving the transaction,...

要點與分析由自動化流程生成,可能有誤,請結合原始來源核實。