Data observability is software that watches a company's data pipelines and tables, and raises an alert when data arrives late, arrives short or changes shape, before a wrong number reaches a dashboard. The name comes from observability in software engineering, which checks whether servers and apps are healthy. Here, the thing being checked is the data itself.
A pipeline can fail silently. The code runs and no error appears, but the table is wrong. Three problems are the most common. The first is freshness: did today's data arrive on time? The second is volume: did the expected number of rows arrive? The third is schema drift: did a source system rename a column or change its type without warning? Each of these can make a BI dashboard show a wrong figure, and the person reading it has no way to know.
The software first learns what is normal for each table. It reads the table's history: when new rows usually arrive, how many arrive on a normal Monday, and which columns the table has. Then it checks every new load against that pattern. It also uses the lineage recorded in the data catalog to work out which dashboards depend on a broken table, and who should be told.
Here is a typical case. A bus ticketing app in Indore loads its bookings into its data warehouse every hour, through an ELT pipeline. On a Tuesday, the 11 a.m. load brings 2,100 rows. On Tuesdays at that hour, the load normally brings 9,000 to 11,000 rows. The software sends an alert to the data team. The cause turns out to be a change in the booking system. It renamed a column from seat_no to seat_number, and the pipeline dropped every row with the new name. A data engineer fixes the pipeline and loads the hour again. When the founders read the revenue dashboard at noon, it shows the right figure.
Newer products also explain the warehouse bill. A cloud warehouse charges for the computing that each query uses. So the software finds the most expensive queries, such as a report that reads five years of data every fifteen minutes, and suggests a cheaper way to write them. Some products use an AI data analyst that reads the warehouse's logs and explains, in plain words, why a pipeline failed or why the bill went up.
How is it different from data quality? Data quality software checks whether the details inside each record are correct, such as a valid PIN code. Data observability checks whether a table as a whole arrived on time, in full and in the expected shape.