Tech, ExplainedData platforms and AI
Product

HTAP database

Live transactions and analytics in one engine.

BEFORE ITAFTER ITHTAP databaseSingleStoreCOVERSRows and columnsLive analyticsOne engineSub-second answersfeedslive chartsBusiness intelligence(BI) dashboards↗ BUSINESS INTELLIGENCEsendsolder dataData warehouseThe app writes eachorder, payment orstock change into itthe moment it happensCONCEPTS IT USESOLTP and OLAPColumnar databaseBEFORE ITAFTER ITHTAP databaseSingleStoreCOVERSRowsand columnsLive analyticsOne engineSub-secondanswersfeedslive chartsBusinessintelligence(BI) dashboards↗ BUSINESSINTELLIGENCEsendsolder dataData warehouseThe app writeseach order,payment orstock changeinto itthe momentit happensCONCEPTS IT USESOLTP and OLAPColumnar databaseBEFORE ITThe app writes each order, payment orstock change into it the moment it happens↓HTAP databaseSingleStoreRows and columnsLive analyticsOne engineSub-second answersAFTER IT↓ feeds live chartsBusiness intelligence (BI) dashboards↗ BUSINESS INTELLIGENCE↓ sends older dataData warehouseCONCEPTS IT USESOLTP and OLAPColumnar database
The HTAP database records each order as it happens, sends live answers to business intelligence dashboards, and sends its older data to the data warehouse.
opens a page inside this one a page in the same scene a page in another scene a concept page a label only, with no page

An HTAP database is a database that records transactions and answers analytical questions in the same engine, on the same live data. HTAP stands for hybrid transactional and analytical processing. With it, a company can ask questions of what is happening now, without waiting for a copy of the data to reach a warehouse.

Usually these two jobs run in separate systems. The app's database records each order as it happens, which is called OLTP. A pipeline copies the orders into a data warehouse, where analysts add them up, which is called OLAP. The page on OLTP and OLAP explains why. The copy is often minutes, hours or a whole night behind.

An HTAP database keeps the data in two forms inside one system. For recording, it stores each order as a row, so that a new order is written in one step. For questions, it also keeps the data column by column, so that adding up one column, such as the price of every order, reads only that column. This second form is called a columnar store. The database keeps both forms up to date with each write. It also keeps the heavy questions from slowing down the orders, for example by running them on separate processors.

Consider an online fashion store in Jaipur that runs a midnight flash sale. In the first hour, 40,000 orders arrive. The merchandising team watches the sale minute by minute. Which kurta sizes are selling out, and in which cities? Is one discount code being used far more than planned? The HTAP database answers each question in under a second, over orders written a few seconds earlier. At 12:40 a.m., the team sees that one code has been used 9,000 times, three times the plan, and ends it. With only a nightly copy in the warehouse, the team would have seen these numbers the next morning, after the sale had ended.

The answers can feed live charts on business intelligence (BI) dashboards.

An HTAP database does not replace the warehouse. It holds the data of one app, while a warehouse holds years of history from every system in the company. Keeping data in memory and in two forms also costs more per gigabyte than storing it in a warehouse. So older data is usually still copied to a warehouse.

How is it different from a real-time analytics database? That database also answers fast questions over fresh data, but it does not record the app's transactions. Its data arrives from other systems, often through event streaming. An HTAP database is the app's own database, and the answers come from the same place where each order is written.

Tech, Explained is a part of ingrid.fyi. Every page explains one piece of technology in plain words.