The Product

Nobody outside the company has written an independent technical history of ClickHouse, and the company’s own accounts are, understandably, product stories. This chapter is a practitioner’s version: what the architecture was built to do, which parts of that design turned into recurring operational pain, how each pain was answered over the releases, and what is still true on a server you run today. It leans on primary sources throughout: the creator’s own account of the first years, the release changelogs in the GitHub repository, the vendor’s engineering blog, a third party’s production write-up, and the system tables of a current build. Where a statement is about behaviour rather than history, it was checked on clickhouse local 26.4 (build 26.4.1.834); where it is about later releases it comes from the 26.x changelog, which at the time of writing runs through v26.9 (21 September 2026).

The people and the money come in chapter 2; this chapter is about the product.

Era one: a reporting problem, 2008 to 2016

Fifty reports and a pile of MySQL

The official history page states the starting point in one sentence: in 2009 a project began to test whether analytical reports could be generated in real time from non-aggregated data. The detail is in two documents by Alexey Milovidov: a 2017 talk and a long post written for the ten-year anniversary (15 June 2026).

Yandex.Metrica, by the 2017 talk’s account, was the largest web analytics service in Russia and the second largest in the world, processing about 25 billion events and showing reports in real time. The old system, which ran from 2008 to 2014, “was working fine” and offered about 50 pre-defined reports. The problem was everything outside those 50. The ten-year post describes the plumbing behind it: MySQL held the pre-aggregated reports, around 50 TB across 50 shards, alongside custom C++ structures stored as BLOBs. Data arrived in time order while queries asked for ranges by website, so reads were slow; the MySQL data was uncompressed; and every aggregation meant reading a BLOB out, running custom code and writing it back. Milovidov tested several column-oriented products and MySQL extensions (Infobright, InfiniDB, Vertica, MonetDB, LucidDB), and none handled the load the post gives as 100 billion records a day with 500 columns.

The argument to hold onto is that pre-aggregation is a decision made in advance about which questions will be asked. Metrica’s users wanted to decide later. Keeping raw events and aggregating at query time moves the cost from write time (and from the schema) to read time, which is only viable if reads are very cheap. Every architectural choice below follows from taking that bet seriously.

Two prototypes, then a synthesis

According to the same post there were two custom structures before ClickHouse. OLAPServer, written in December 2008 and deployed in January 2009, kept every column for every day and every website in binary files, integers only, with light compression, updated once a day. Metrage was a row-oriented structure with background merges that held arbitrary aggregates, each defined as a C++ struct with add, update and merge methods. Milovidov’s account is that both worked and that nobody demanded more; the idea of ClickHouse was to combine “a column-oriented approach for aggregation speed” with “a merge tree for realtime updates and data locality”, and to generalise it into a real query language with real data types.

The git history lets him say what was built in which order, and the order is instructive:

• The repository’s first commit is dated 29 May 2009 and is a replacement for slow libc time functions, so it predates the DBMS. The first commit of the DBMS itself implemented columns in memory (the IColumn and Field classes that still exist), then aggregate functions, then table engines, then compression.
• Compression started as QuickLZ and was replaced by LZ4 after the author read Yann Collet’s blog. Columnar plus fast compression is where the I/O saving comes from.
• Block streams, the ancestors of today’s processors, came next, with system.numbers as the first test table. Only then did a SQL parser appear, after a failed attempt with boost::spirit.
• The server was introduced on 9 March 2012 and the client on 25 March. With the Log, TinyLog, Merge, Distributed and Memory engines that was enough for production. The first production use, in the author’s words, was a persistent log queue with SQL on top.
• MergeTree came after that and, in the post’s words, let it replace both OLAPServer and Metrage. The history page dates the production launch to 2012.

Two details are worth keeping. The first real test table was the hits table that, the post notes, you can still see in ClickBench. And the post points out that Apache Arrow, RCFile, ORC and Parquet did not exist yet, which is why ClickHouse has its own in-memory column representation.

What the bet forced

Four properties follow from “raw events, aggregate on read”. The 2017 talk lists them as the answer to “why ClickHouse”: a column-oriented layout, data locality for range queries via MergeTree, vectorized execution, and specialisation, with the example that there were 17 different GROUP BY algorithms selected automatically. You can see the first three on any current build. Take three one-million-row inserts into the demo table, with merges stopped so the parts stay visible:

CREATE TABLE events (event_time DateTime, user_id UInt64, event_type LowCardinality(String),
                     url String, duration_ms UInt32, country LowCardinality(String))
ENGINE = MergeTree ORDER BY (event_type, event_time);
SYSTEM STOP MERGES events;
-- three INSERT ... SELECT ... FROM numbers(1000000), one per month
SELECT name, rows, marks, formatReadableSize(primary_key_bytes_in_memory) AS pk_mem
FROM system.parts WHERE table = 'events' AND active ORDER BY name;

all_1_1_0   1000000   124   678.00 B
all_2_2_0   1000000   124   678.00 B
all_3_3_0   1000000   124   678.00 B

Each part has one mark per 8192 rows (index_granularity, whose default system.merge_tree_settings reports as 8192), so a million rows are indexed by 124 marks, and the whole primary index of a part is under a kilobyte of memory. That is the sparse index: it does not point at rows, it bounds ranges. Asking for one event type on one day shows what it buys:

EXPLAIN indexes = 1
SELECT count() FROM events
WHERE event_type = 'buy' AND event_time >= '2026-02-01' AND event_time < '2026-02-02';

PrimaryKey
  Keys: event_type, event_time
  Parts: 2/3
  Granules: 4/369
  Search Algorithm: binary search

Four granules out of 369, two parts out of three, found by binary search over a few hundred marks. Per column, the same table stores url in 1.15 MiB compressed against 42.28 MiB uncompressed. The data is synthetic and highly repetitive, so treat the ratio as an upper bound on the idea rather than a benchmark, but it shows why compression and column layout were designed together. The mechanics are covered in parts and partitions and ORDER BY, PARTITION BY and pruning.

The same design carries a bill, and it is the origin of nearly every item in the next section. Each insert becomes a part, and a part is immutable. A hundred single-row inserts on 26.4 with merges stopped gave this:

-- 100 x: INSERT INTO t_small VALUES ('2026-01-01 00:00:00', <i>);
SELECT count() AS parts, sum(rows) AS rows FROM system.parts WHERE table = 't_small' AND active;

100   100

A hundred parts holding one row each. In normal operation merges would be combining them in the background, which is what the “Merge” in MergeTree means, but only at a finite rate. A write path that creates a part per call, and an update model that rewrites parts, are the two pressures that shaped the next decade.

Replication, and the first dependency

The 2026 post is candid about how replication arrived. Metrica ran in several data centers, and the infrastructure team deliberately switched one off for an hour each month to teach services to survive it. Double-writing with backfill was the first approach; wanting full consistency with automatic repair meant distributed consensus. Colleagues who were Java engineers “hooked us on ZooKeeper”, and Michael Kolupaev implemented ReplicatedMergeTree with ZooKeeper as the metadata layer. That, the post says, is what allowed ClickHouse to serve user-facing queries in 2014. By the end of that year it was widely used inside Yandex and, as far as Milovidov knew, outside only at CERN, for the LHCb experiment.

Keep this in mind for the next era: the replication design stores part metadata in an external consensus service and moves data between replicas by fetching parts. Much of what changed later (Keeper, SharedMergeTree) is a response to that choice.

Opening the code

Milovidov saw other companies building things “similar to OLAPServer or Metrage” because no database fit their workloads, wrote an article in 2015, and prepared a list of advantages and risks for management. It was approved, and the project was released on 15 June 2016 under the Apache 2.0 license, with a blog post, a first website and a Debian repository. The announcement’s claim was that ClickHouse “allows interactive analytical query execution on data updated in real time” and scales to tens of trillions of rows and petabytes. The 2017 talk gave a data point from that period: Metrica’s main cluster, as of November 2016, held 18.3 trillion rows on 426 servers, with a total query throughput of up to two terabytes per second. The repository on GitHub today has about 50,400 stars, and the ten-year post counts more than 2000 contributors.

Era two: the open-source years, 2016 to 2021

Who showed up

The clearest early public record of a non-Yandex user is Cloudflare’s March 2018 post on its HTTP analytics. The previous pipeline was built in 2014 on Kafka, Postgres and a Citus cluster of one main node and 11 workers with two-fold replication, and had to be replaced. A first redesign around Apache Flink was considered and set aside; ClickHouse was chosen because it was fast, scalable and fault tolerant. The new cluster had 36 nodes with three-fold replication, ingested raw (not pre-aggregated) request logs, and produced aggregates with materialized views, at an average of 6 million HTTP requests per second with peaks up to 8 million. The post is cautious about one thing: it was not confident about query performance when the cluster grew to hundreds of nodes.

The shape is the one Milovidov described for Metrica: high-volume event streams, queries that cannot be enumerated in advance, and a team that otherwise would have built its own pre-aggregation layer. Counting adopters is hard; ClickHouse’s own FAQ says as much, citing NDAs and the open-source distribution, and offers an informal “1000+ companies” estimate rather than a measured one.

The debts of the design

Adoption outside Yandex surfaced the costs. These are the ones practitioners hit hardest, with the release in which each was answered and what you get by default today. Dates are release dates from the GitHub releases page; “default today” was checked in system.settings and system.settings_changes on 26.4, which records the version in which each default changed and the maintainers’ stated reason.

Some of these deserve more than a row.

ZooKeeper to Keeper. The 22.3 release post claims Jepsen testing in CI, better latency and lower memory than ZooKeeper, checksummed data on disk, and that it had already been in production for more than half a year. Take those as the vendor’s claims; the part you can verify yourself is the migration tooling, which started with clickhouse-keeper-converter for ZooKeeper logs and snapshots. For operations, the point is that the metadata service moved from a separate JVM product to the same codebase and release cycle as the database. The trade-off is that it now shares ClickHouse’s upgrade cadence and its bug surface, and the changelog for recent releases still contains Keeper fixes, including durability and shutdown fixes in the 26.x series. See replication internals.

Small inserts. The release post for 21.11 frames asynchronous inserts as accumulating inserted data and storing it in a single batch, using less disk (IOPS), to support a high rate of INSERT queries; with wait_for_async_insert true by default the client waits until the data is flushed. The documentation still recommends client-side batches of at least 1,000 rows, ideally 10,000 to 100,000, and presents async inserts for the case where you cannot batch. The setting history shows how the feature was tuned: in 24.2 async_insert_max_data_size went from 1,000,000 to 10,485,760 bytes (“The previous value appeared to be too small”), and an adaptive busy timeout was switched on (async_insert_use_adaptive_busy_timeout 0 to 1); in 26.2 the whole feature became the default. Practically: a fleet of small writers that was a problem in 2019 is, on a current server, handled by a default, but wait_for_async_insert = 1 and deduplication interplay still matter (24.3 rejects the combination of async_insert and deduplicate_blocks_in_dependent_materialized_views unless a setting allows it). See async inserts explained and insert batching and too many parts. The guard rails on 26.4 are parts_to_delay_insert = 1000, parts_to_throw_insert = 3000 and max_parts_in_total = 100000.

Deletes and updates. The approach was to stop treating a delete as a rewrite. On 26.4 a lightweight delete is visible in the system tables for what it is:

DELETE FROM events WHERE event_type = 'scroll' AND event_time < '2026-01-02';
SELECT mutation_id, command, is_done FROM system.mutations WHERE table = 'events';

mutation_4.txt   (UPDATE _row_exists = 0 WHERE (event_type = 'scroll') AND (event_time < '2026-01-02'))   1

count() fell from 3,000,000 to 2,978,400, the parts were renamed with a mutation suffix (all_1_1_0_4) and still reported 1,000,000 rows each, and a hidden _row_exists column appeared in system.parts_columns. That is the design: write a mask, filter on read, reclaim space at the next merge. The default lightweight_delete_mode on 26.4 is alter_update. Lightweight updates extend the same idea. The 25.7 release post describes writing a compact patch part that is applied on read and merged into base parts during normal merges, instead of rewriting whole columns as a classic mutation does. A first version had been announced for the cloud alongside SharedMergeTree in August 2023. The reasoning is the same as for the whole product: you cannot make a part mutable without losing what makes it fast, so you make the mutation a new, small, immutable object. See mutations and lightweight delete.

The analyzer. The change is mostly invisible when it works, which is why it is easy to underrate. The settings history says: in 24.3 allow_experimental_analyzer went 0 to 1 with the reason “Enable analyzer and planner by default”; the analyzer documentation and the 26.9 changelog say that since 26.9 it is mandatory, the enable_analyzer setting is obsolete, and the pre-24.3 analysis is no longer supported. That is a two-and-a-half-year deprecation: if your upgrade path crosses 24.3, plan for it, and if it ends at 26.9, there is no switch to flip back. Cluster caveat from the same page: servers that disagree on whether the analyzer is used cannot run distributed queries together. See moving to the new analyzer.

Joins. Joins are a good example of a weakness being worked on from several directions. A memory-bound hash join was the only practical option for a long time; grace hash added a spilling variant in 22.12. Later releases moved to the optimizer: the default for how many tables the planner may reorder in a join (query_plan_optimize_join_order_limit) rose from 1 to 10 in 25.12 (“Allow JOIN reordering with more tables by default”), and enable_join_runtime_filters went from 0 to 1 in 26.2. See joins, memory and algorithms.

The release train

Almost everything above is visible only if you read changelogs, and the cadence is part of the product. ClickHouse ships a release roughly every month and marks some as LTS. From the GitHub releases list, LTS tags in this period are 22.8 (18 August 2022), 23.3 (31 March 2023), 23.8 (1 September 2023), 24.3 (27 March 2024), 25.3 (20 March 2025), 25.8 (29 August 2025), 26.3 (27 March 2026) and 26.8 (30 August 2026). A reasonable reading is that LTS lands in March and August or September, so a two-per-year upgrade rhythm is available if you do not want to follow the monthly train. The practical cost of the cadence is what the analyzer story shows: behaviour that was optional becomes default, and later mandatory, across versions you may not have installed.

Era three: the company and the platform, 2021 to 2026

From project to company

Yandex announced the spin-out on 20 September 2021 in partnership with Benchmark Capital and Index Ventures, saying the new company would develop commercial products such as a cloud version of the DBMS. The official history page gives an initial $50M investment that September, and the company announced a $250M Series B at a $2B valuation on 28 October 2021, led by Coatue and Altimeter, with Yandex among the existing investors. The structural point for a user is that ClickHouse, Inc. had a business model that was not the database itself, and that the database had to remain open and good for the project to matter. The answer was a managed service. The ownership, acquisitions and people are the subject of the next chapter.

ClickHouse Cloud and why SharedMergeTree exists

ClickHouse Cloud entered public beta in October 2022 and became generally available on 6 December 2022. The launch release says that the architecture decouples storage and compute, that over 100 paying customers adopted it during a two-month beta, and that the launch included SOC 2 Type II compliance and uptime SLAs. The part that matters for this history is the architecture, because it forced the first fork in the engine family.

The August 2023 SharedMergeTree post explains the pressure. ReplicatedMergeTree creates physical copies of data on a small number of replicas, while Cloud wanted many servers reading the same data in object storage. The first adaptation, zero-copy replication, replicated only metadata but still needed local disks for part metadata, coupled operations through exclusive Keeper locks and depended on three components for durability (object storage, Keeper and local storage). The post says that did not sufficiently address the problem. SharedMergeTree removes replica-to-replica communication: all coordination goes through shared storage and Keeper, there is no insert_quorum, and scaling out means new servers syncing metadata, not data.

This one you can verify as an absence. On the open-source 26.4 build, system.table_engines contains ReplicatedMergeTree but not SharedMergeTree. There are 16 engines whose names end in MergeTree, none of them the shared one. The Cloud-only status is documented on the SharedMergeTree reference page. For the practitioner this is the single largest divergence between “ClickHouse” and “ClickHouse Cloud”: the open-source engine you run yourself is still the replicated, shared-nothing one that dates to 2014, with the sharding and replica sizing decisions that implies, while the managed one uses a different replication model on object storage. Features described in cloud posts need checking against the engine you actually have. A later engineering post on stateless compute covers how far that separation went.

Data types and indexes catching up with the workloads

JSON. Semi-structured data was an early weakness. The October 2024 post (22 October 2024) describes a type built “from scratch” to replace the deprecated Object('json'): each distinct path is stored as a true column, values of different types under the same path are kept as they are via the Variant and Dynamic building blocks, and a limit on dynamic paths (default 1024) and types (default 32) prevents an avalanche of files. It was released as experimental then. The settings history shows it flipped in 25.3 with the reason “JSON data type is production-ready”. The path limit is observable. With max_dynamic_paths = 2:

CREATE TABLE logs (ts DateTime, doc JSON(max_dynamic_paths = 2)) ENGINE = MergeTree ORDER BY ts;
-- three documents with different shapes and a latency_ms that is an integer, a string and a float
SELECT doc.user.id, dynamicType(doc.latency_ms) AS t, doc.latency_ms FROM logs ORDER BY ts;

1   Int64     120
2   String    slow
3   Float64   80.5

Mixed types survive instead of being coerced, and JSONDynamicPaths versus JSONSharedDataPaths shows which paths got their own subcolumns (user.id, user.country) and which fell into shared data (latency_ms, tag, extra, region). This is the same trade-off as everywhere else in the design: make the common case a real column, put the overflow somewhere cheap. See working with JSON.

Vector search. An approximate-nearest-neighbour index existed as an experiment as early as 2022 (Annoy-based index fixes appear in the 22.10 changelog). In 25.8 the settings history records “Vector similarity indexes are GA”. The documentation says the only method available is HNSW. This is the first index in the family that is approximate by design; the sparse primary index is exact, so the results of a vector query depend on index parameters in a way that no other ClickHouse query does. See vector similarity search.

An engine for data it does not own

Reading data that ClickHouse does not store has become a strategy of its own. The changelogs give the dates: Hudi and DeltaLake read-only table engines and functions for S3 in 22.11, Iceberg in 23.2, a setting to ignore Iceberg schema evolution in 24.1 and partition pruning for Iceberg’s time transforms in 25.1. The size of the surface is easiest to see in the system tables of the 26.4 build used here: 74 table engines, 89 table functions, 108 formats, 1,795 functions and 1,555 settings. The engines include Iceberg, DeltaLake, Hudi, AzureBlobStorage, S3Queue and KeeperMap. The consequence for a practitioner is that data a table function reads from a lake has no ORDER BY chosen for your queries, so the sparse index that the first section describes cannot help unless the files were laid out well; that is a property of the source, not of ClickHouse. See ClickHouse and S3.

Defaults, drift and the cost of a living product

The 26.9 changelog shows how the project now treats compatibility. The default compression method changed from LZ4 to ZSTD(3) for network communication, with a size-aware rule for MergeTree column data: LZ4 for parts below 100 MB and ZSTD(3) at or above it. The same release removed several experimental features outright, including WINDOW VIEW, the CatBoost integration and the WasmEdge UDF engine. Whether you welcome that depends on whether you used them, but it shows the policy: experimental means experimental, and it can be removed in a release note.

The company’s products around the engine are chapter 2’s territory. What matters for the engine is the pace. Ten years after the open-source release the project is, in its creator’s words, the most popular open-source analytical database. The same post lists what it wants to be: an example of how to build a database, a place to learn C++, and a place for experiments on data structures.

What this history means on the server you run

Pulling the eras together gives a short list of operational consequences.

1. Design for the part, not the row. The sparse index, immutable parts and background merges are the 2009 to 2012 design and they have not changed. Batch inserts (or rely on async inserts) and choose ORDER BY for the queries you run. Everything else is built on that layer.
2. Treat mutation features as a ladder. Lightweight delete (on by default since 23.3) and lightweight update (beta since 25.8) are different from the classic ALTER TABLE ... UPDATE. They write masks and patch parts and defer the cost to merges. Check system.mutations and the part names to see what a statement actually did.
3. Read the settings history before and after an upgrade. system.settings_changes is the compact record of which defaults moved and why. It is how this chapter established that async inserts became default in 26.2, JSON production-ready in 25.3 and vector indexes GA in 25.8, and it is the first thing to diff when crossing several releases.
4. Plan for LTS rhythm, not monthly stress. LTS appears twice a year. Mandatory changes such as the analyzer in 26.9 are announced as defaults years earlier.
5. Know which ClickHouse you are reading about. SharedMergeTree is not in the open-source server. A lot of the platform-era writing assumes object storage and a different replication model.
6. Keeper is part of your database. If you run replicated tables, Keeper’s health, snapshot and log handling and version are part of your operational surface. It no longer comes from a separate project but is released with the server.

What this chapter does not cover

Query-engine internals that deserve a chapter of their own (the processors pipeline, the aggregation algorithms, the cache hierarchy) are only touched here. Benchmarks are deliberately absent: benchmark claims are marketing until reproduced, and the purpose here is the sequence of design decisions. The company’s structure, acquisitions and strategic moves are in chapter 2, and the AI-related developments are in chapter 3.