Databases and data warehouses

The two places every order is stored

Each record is written once by the app, then copied for analysis.

Box 1 is the database the app writes each order into. A copy then goes to box 2, where it is added up with millions of others. Box 3 companies run databases for companies that do not want to run them. Tap a title in the diagram to jump to its companies.

Databases for applications 8 companies

When a customer pays, books or signs up, the app saves that record at once in a database. These companies make those databases, each built for a different shape of data.

Document, key-value, graph and multi-model databases

Databases that store data in a shape other than tables of rows and columns.

MongoDBMongoDB sells a document database. Each record is stored as one document, much like the data an app already sends and receives, so a programmer can save a whole order in one step.Delhi NCRHiring now

AerospikeAerospike, with engineers in Bengaluru, sells a key-value store: each record is saved under one key and found by that key alone. It is built for advertising and payments, where an answer is needed in a thousandth of a second.Bengaluru

CouchbaseCouchbase sells a document database with a cache built in: the most-used data is kept in memory so it can be read very fast. It is used by apps with millions of users.BengaluruHired recently

Neo4jNeo4j sells a graph database. It stores records and the links between them, which suits questions such as which bank accounts are connected to each other.

MonkDBMonkDB, a Hyderabad start-up founded in 2023, sells one database for many kinds of data: documents, graphs, and time series, which are readings recorded over time.

Relational databases

Databases that store data in tables, and can grow across many servers.

YugabyteYugabyte was founded by engineers who built the databases inside Facebook. It sells a database that works like PostgreSQL, the popular free database, but spreads across many servers so it keeps running if one fails.BengaluruHiring now

SingleStoreSingleStore, with engineers in Pune, sells one database that both records orders and answers questions about them, so a company does not have to wait for a copy to reach a warehouse.PuneHired recently

MariaDBMariaDB is the company behind MariaDB, a free database copied from MySQL in 2009 by some of MySQL's original developers, after Oracle took over MySQL.

The tech, explainedDocument database Key-value store Graph database Multi-model database Relational database Distributed SQL database HTAP database

Data warehouses and lakehouses 7 companies

A warehouse holds a copy of every record from a company's systems, organised so that analysts can ask questions across all of them. A lakehouse also stores raw files cheaply in the same system.

Warehouses and lakehouses

One place for all of a company's data, for reports and analysis.

DatabricksDatabricks, with engineers in Bengaluru, made the lakehouse popular: one system with the low cost of storing raw files and the structure of a warehouse. It now also sells tools for building AI on that data.BengaluruHiring now

TeradataTeradata is one of the companies that built the data warehouse, the system into which a bank or a retailer copies every transaction for analysis.Hyderabad · BengaluruHiring now

SnowflakeSnowflake is one of the companies that made the warehouse a cloud service, with nothing for the customer to install. It builds part of its platform in Bengaluru.Bengaluru · PuneHired recently

ClouderaCloudera grew out of Hadoop, the free software for storing and processing data too large for one machine. It sells a data platform that runs both in the cloud and in a company's own data centre.Bengaluru

OnehouseOnehouse, founded by the creator of Apache Hudi, sells a lakehouse run as a service. Its tables are stored in an open format that any tool can read, so a company is not tied to one warehouse.

Fast answers on fresh data

Databases that add up billions of rows in under a second, including data that arrived seconds ago.

ClickHouseClickHouse sells its free column store as a cloud service. It stores data column by column, so adding up one column, such as the price of every order, takes a fraction of a second.Bengaluru

ImplyImply was founded in 2015 by the creators of Apache Druid, a free database for fast questions on data that is still streaming in, and it sells Druid as a product.

The tech, explainedData warehouse Data lakehouse Real-time analytics database

Database support and managed databases 2 companies

Some companies do not sell a database of their own. They help other companies run the free databases, or run the databases for them in the cloud.

PerconaPercona sells support and tools for companies that run the free databases MySQL, PostgreSQL and MongoDB themselves: expert help when a database breaks or slows down.

TessellTessell runs Oracle, PostgreSQL, MySQL and SQL Server databases for its customers on Amazon Web Services, Microsoft Azure and Google Cloud, so the customer does not manage them.

The tech, explainedOpen-source database support Database as a service (DBaaS)

How it all worksData engineeringOne order, recorded at once and added up later.

See how it works →

What's changing

The warehouse is becoming where AI is built.

Warehouses were built to feed reports and dashboards. Companies now also want to train AI models and run AI tools on the same data, so warehouse and lakehouse makers are adding ways to do that where the data already sits, instead of copying it somewhere else.