Data engineering is the work of copying data out of the systems where it is created into one place where it can be added up and questioned, and of keeping that copy correct and current. The people who do it are called data engineers. Together with the analysts who use the data, they make up a company's data team.
Take a single order. You buy a ₹1,299 pressure cooker on a shopping app in Indore. The moment you tap Pay, the app's backend, the part that runs on the company's servers, writes the order into a database. In the same instant, the stock count for that cooker drops by one. Later, a copy of your order reaches a much bigger store. There, the company adds it to millions of other orders to see what sold, where and when.
These are the two lives of every piece of data. Recording each event as it happens is called transactional work, or OLTP. Asking questions across all the records at once is called analytical work, or OLAP. The page on OLTP and OLAP explains why each job needs a different kind of system. The software for both lives sits in four layers.
The first layer is databases, the systems an app reads and writes while it runs. A relational database stores the order as rows in linked tables. A document database stores it as one document. Choosing a database is the decision a company finds hardest to reverse, because its code and years of data are built around it. This is called vendor lock-in.
The second layer is data pipelines, the code that copies data from one system to another. ETL and ELT tools copy data out of every source system on a schedule, often once a night. Change data capture copies each change within seconds instead. The page on batch and stream processing compares the two.
The third layer is data warehouses and lakehouses. A data warehouse is a database too, but it is kept apart from the app's database, so that heavy questions do not slow the app down. A data lakehouse keeps raw files cheaply and clean tables in the same system.
The fourth layer is data governance, the tools that tell the data team what it holds and whether to trust it. A data catalog lists every table and its owner. Data observability notices when a pipeline delivers late or short data. Master data management works out which records from different systems describe the same customer.
At the end, the answers appear on the business intelligence dashboards that managers read. Data teams either buy one suite for all four layers, or join many separate tools together, called the modern data stack. Many of the databases and pipeline tools are free, open-source software, sold through support and cloud services, as the open source business model explains.