Open table formats are shared, public rules for storing a database table as ordinary files in a data lake, so that more than one piece of software can read and change the same table safely. The best-known are Apache Iceberg, Delta Lake and Apache Hudi. All three are open source, so anyone may use them free of charge. They are the layer that turns a data lakehouse from a pile of files into tables.
In a lake, one table is stored as many data files, usually in a column-by-column file format called Parquet, in object storage. The files alone cannot answer simple questions. Which files belong to the table today? Which were replaced yesterday? What are the column names? An open table format answers these questions in metadata files, small files kept next to the data. Each time someone changes the table, the change writes new data files and a new snapshot. A snapshot is a list of exactly which files make up the table at that moment. Finally, one pointer is switched to the new snapshot, in a single step.
That single step gives three useful properties. First, a reader never sees a half-finished change, because it reads either the old snapshot or the new one. Second, old snapshots are kept for some time, so a team can query the table exactly as it was last week. This is called time travel. Third, a column can be added or renamed without rewriting the data files.
Here is a typical case. An insurance company keeps its motor claims, 8 crore rows, as a table in its lake. The current version is snapshot 417. One night, a job must correct 2,000 claims whose amounts were entered wrongly. Those claims sit in 3 of the table's data files. The job writes 3 new files of 40 MB each, holding the corrected rows, and creates snapshot 418, which lists the new files in place of the old ones. While the job runs, an analyst's report keeps reading snapshot 417 and gets consistent figures. At 2:00 a.m., the pointer moves to 418. The next morning, the claims team reads the table through one engine, and the fraud team reads the same files through another.
Why does "open" matter? Because the format is public, any engine that supports it can read the table: a data warehouse service, a real-time analytics database or a large processing job. So the company can change its query engine later without moving its data. This is the main defence against vendor lock-in in data. Some companies sell services that keep these tables tidy, for example by merging many small files into fewer large ones.
How is a table format different from a file format such as Parquet? A file format describes how the data inside one file is laid out. A table format describes which files together make up one table, and how the table changes over time.