Object storage

Object storage is a way of storing data in which each file is kept whole, as one object, with a name and a short description, and is fetched by its name over the internet. It is the cheap storage that the big clouds sell for very large amounts of data: photos, videos, backups, logs and the raw files of data lakes. Amazon S3 is the best-known service of this kind, and every large cloud offers its own.

Here is how it works in general. The data is kept in containers called buckets. To store a file, a program sends it to the storage service with a name, called a key, such as "scans/2026/report-4471.pdf". The service stores the file together with its metadata, a few lines that describe it, such as its type, size and date. There are no real folders. The slashes in the key only make it look like a folder. The service keeps several copies of each object on different machines, and often in different data centres, so that a broken disk loses nothing. To read the file, a program asks for it by its bucket and key. It uses an ordinary web request, the same kind a browser uses to fetch a web page. An object is usually written once and then read many times. To change it, a program uploads a new version of the whole object.

Because it is built this way, object storage can grow almost without limit, and it costs little. The cloud charges for each gigabyte stored each month, for each request, and for data sent out of the cloud. Data that is rarely read can be moved to colder storage classes. These cost less each month, but take longer to fetch, from minutes to hours for the coldest ones. The big clouds run data centres in India, so an Indian company can keep its data inside the country.

Take, for example, a photo-sharing app. Each photo that a user uploads is stored as one object, and the app's database keeps only the photo's key, so the app can fetch the photo when someone opens the post.

Consider a data lakehouse. It keeps a company's tables as files in object storage, and open table formats let different engines read the same files.

Say a small business uses online backup. Each night, the files that changed on its computers are copied into object storage in the cloud.

How is object storage different from file storage and block storage? File storage is the shared drive in an office: folders inside folders, where a program can open a file and change a few bytes in the middle. Block storage is a raw disk attached to one server. A database uses it because it needs very fast, very small reads and writes. Object storage is slower for small changes, but much cheaper and much larger. The arrays described in enterprise storage systems usually offer file or block storage. A cloud file system keeps a company's files in object storage, but shows them to staff as an ordinary shared drive.