Philosophy

What 57 feature requests taught me about ClickHouse

From 2017 to 2026 I filed about 57 feature requests on the ClickHouse tracker. Read together, the answers draw a map of what the core team says yes to, what it refuses, and why. A table of how long the accepted ones took, and advice for filing your own.

Between 2017 and 2026 I filed roughly 57 feature requests against ClickHouse/ClickHouse. Forty-seven pages on this site cover them, some bundling several small issues, and eight are deep dives. Each issue came from a plain problem: a script that needed one step too many, an error message that said too little, a table I could not inspect. Read one at a time they are anecdotes. Read together, the answers repeat, and the repetition is the information.

The status counts are 18 implemented, 3 partially implemented, 19 open, 4 superseded and 3 declined; the ratio reflects my taste in requests more than the project’s generosity. What matters is the pattern in the reasons. Know them and you can predict whether a request will be answered in a week, in six years, or never.

For the project’s culture behind these answers, see The Brains. For the running wishlist this essay grew out of, see Feature requests worth making.

The numbers

Year is the filing year, from each page’s date; outcome is the page’s status and since. Time to ship is the days from the filing date to the merge date of the delivering pull request, as stated in the page, converted at 30.44 days per month. A dagger marks pages that give only the day the issue was closed; a double dagger marks a superseded request, where the clock stops at the merge of the design that replaced it.

Year Request Outcome Time to ship
2017 REPLACE TABLE implemented, 21.1 3 y 4 mo
2019 VALUES as a table implemented, 19.14 (as a table function) 21 days
2019 RETURNING on writes partly, in 26.11 ~7 y (INSERT only)
2019 A JSON function that infers the type superseded by Variant, Dynamic, JSON 4 y 8 mo ‡
2020 digits(n, offset, length) implemented, 26.7 6 y 5 mo
2020 Record rejected rows implemented, 22.9 2 y 7 mo
2021 Multi-column key in EmbeddedRocksDB implemented, 26.1 4 y 1 mo
2022 FROM before SELECT implemented, 22.12 88 days
2022 Specific error code in throwIf implemented, 22.9 18 days
2022 A LIMIT that applies where you write it declined none
2024 Alias table implemented, 25.10 (as an engine) 1 y 2 mo
2024 A granule virtual column implemented, 25.7 (_part_granule_offset) 9 mo †
2024 EVAL partially implemented, 26.7 (eval table function) 2 y 3 mo
2025 Native UPSERT open (duplicate of the unique-key RFC) none yet
2025 finish_time in system.mutations implemented, 26.8 10 mo
2025 XDG config for the client implemented, 25.12 8 days
2025 system.primes implemented, 26.2 (as primes()) 63 days
2026 CREATE OR REPLACE for materialized views implemented, 26.5 4 mo
2026 Echo separator in the client implemented, 26.7 5 mo
2026 Upgrade Guard declined, 8 hours none

Across the 17 implemented pages that name one clear delivering change (the 18th, a roundup of four small issues, bundles several), the median time to ship is 217 days, about seven months. Six shipped within 90 days; five took more than two years. The distribution is two clusters, not a curve. Understanding why is most of what follows.

What gets a fast yes

Four of the fastest answers, 18 to 88 days, share a profile: VALUES, throwIf with a code, FROM … SELECT and the XDG config path. None needed a new storage format or coordination protocol. Each rearranged or exposed something the engine already did.

FROM events SELECT … is the cleanest example. The planner is untouched; only the parser accepts the clauses in a different order. It took under three months, and the reasoning in the thread was about SQL, not about syntax. A sceptic preferred a separate query language; the founder replied that ClickHouse may be the one database you can hand raw SQL access to, thanks to RBAC, row-level security, quotas and metering, so the right move is to make SQL friendlier rather than hide it. That is a statement of strategy, and it explains a whole family of yeses: table functions (primes(), digits(), VALUES) and sugar that stays inside SQL. The vectorised engine does not care how a query was spelled, so the runtime cost is zero and the review burden is small.

The second family is observability columns: system.detached_tables, finish_time, _part_granule_offset, _table. The pages split it in half, usefully. The ones that shipped were derived from state that already exists. The ones that did not need something stored or decided:

  • last_insert_time was explained away: parts do not record insert times, and a trigger-style approach breaks under replication. The answer was a query over system.part_log.
  • finish_time shipped only after a first attempt was closed. A contributor asked the deciding question, whether the value must survive a restart. The second attempt was reworked to add no persistent state, so 0 means “not finished, or unknown”.
  • Query tags in system.query_log, a kafka settings table, and engine documentation in system.table_engines are open. Two of them have no maintainer comment at all.

So: yes, fast, to observability that costs nothing at runtime and persists nothing. Not no, but not fast, to observability that needs a design decision about what to store.

The shape of the yes changes

The part that surprised me most is how rarely what shipped was what I asked for. Look at the right-hand column of the table: of the 17 delivered requests, at least ten came out in a different shape.

  • I asked for VALUES syntax; the founder answered the next day that it should be a table function taking the structure as a string, like file and url. That is what shipped.
  • I asked for CREATE ALIAS; the reply was a two-line sketch: make it an engine, ENGINE = Alias(db.t). An engine inherits renames, grants, backup and replication by being a table. The first implementation was merged and then called “unnecessarily fragile” by the founder; a reimplementation shipped in 25.10.
  • I asked for a _granule column and got _part_granule_offset, per part. I asked for _table_name and found the real request, _table on every table, already open.
  • I asked for a system.primes table with a million rows and got an infinite computed table plus a primes() table function. I lost the index column and gained not storing anything.
  • I asked for a client config under ~/.config/clickhouse-client/; it shipped under ~/.config/clickhouse/.

ClickHouse has a small number of extension points (the table function, the virtual column, the setting, the system table), and a request lands fast when it maps onto one of them. The page on table functions as a first-class abstraction makes that case at length.

The same instinct shows in how things ship once accepted: narrow, opt-in, off by default. throwIf with a code needs allow_custom_error_code_in_throwif, because, as the founder put it, “we don’t want to have fake errors in normal environments”. The eval table function runs a single SELECT, only where table functions are accepted, behind allow_experimental_eval_table_function. The client’s echo separator is empty by default so existing output does not change.

What gets a no, or a “not that way”

The refusals are the most instructive, and they fall into four groups with four different reasons.

Per-row state, uniqueness, coordination

Native UPSERT was closed in a day as a duplicate of the unique-key RFC, #70589. The one-sentence reason: lightweight updates make the update cheap, but an upsert is “does this key exist?” followed by an update, and on a table of immutable parts with a sparse primary index the existence check is the expensive half. An EXPLAIN on a million-row table shows it: ten random keys touch ten of 123 granules; a thousand touch all of them. The RFC’s own design (an in-memory set of key hashes that is rebuilt from the data after a restart, with optimistic commits through Keeper on replicated tables) names the cost of the statements I wanted: they “would require running a SELECT before the INSERT” and make inserts “rather slow”.

The same wall explains RETURNING. The stated blocker, in the third of three related issues, is a parser fact: after VALUES, everything in the byte stream is treated as data, so there is no end of statement to attach a clause to. It stayed open seven years. In October 2026 a large pull request that ran a SELECT after the insert was closed, and the founder merged a much smaller one instead: in 26.11, INSERT … RETURNING (SELECT …) runs over the stream of inserted rows — the placement before the data solves the parser wall, and reading the stream instead of the table keeps it from needing per-row bookkeeping. The merge closed two of the three issues; the original request stays open, and DELETE … RETURNING is still missing.

Neither is a “no” to the need. Both refuse to bolt per-row semantics onto an engine whose speed comes from never looking at individual rows.

Version-aware magic

The Upgrade Guard was my longest request and the fastest close: eight hours, answer “We use the compatibility setting - it works well.” The answer covers less than it implies, since nothing records which version has touched your data, but it is coherent: a mechanism existed for the common case, and the proposal added a subsystem, a tag on every setting and a registry that has to stay honest forever. A request that adds permanent maintenance for a rare case loses, however good the idea.

New syntax where a setting or function already does it

The enhanced LIMIT is the case study. The maintainer liked the problem and rejected every syntax I offered: FROM events(100) looks like a table function, a second LIMIT “may be OK, but looks too surprising”. He concluded he would postpone it until there was “more motivation”, and the issue was closed years later when someone pointed out that a parameterised view covers part of it. Contrast throwIf, where I proposed a new function and the answer was a third argument to the existing one. SHOW CREATE TABLE on a table function was declined for a mechanical reason I accept: remote(...) builds a temporary table with no CREATE of its own.

Surface syntax is expensive because it is permanent: it is a promise to every future user and every external parser.

Renames of established words

PRIMARY KEY identifies nothing and ORDER BY does not order query results; I filed both on the same day. The ORDER BY twin was closed in under a day, with the observation that Vertica uses the word the same way. The PRIMARY KEY request never got a maintainer reply. Neither word has changed; a rename is a pure cost with no new capability.

The invariant

The function that would infer a JSON value’s type, issue #8327, was closed because in ClickHouse “the function return type is determined by the types of its arguments and the values of the constant arguments. It cannot depend on the values of non-constant arguments.” A column has one type, and the planner relies on it. The goal was met years later by changing what the column is: Variant, then Dynamic, then the new JSON type, announced production-ready in 25.3. That is the pattern in its purest form. If the request collides with an invariant of the engine, the project will not bend the invariant; it will add a new kind of column.

Slow, but eventually

Five delivered requests took more than two years. Three of the four longest were carried by people other than the core team.

  • REPLACE TABLE (3 y 4 mo) was blocked by a missing foundation. A statement that creates, fills, exchanges and drops needs an atomic swap, and DROP on a replicated table removes a replica, not the table. In May 2020 the founder noted that it “can be easily implemented on top of DatabaseAtomic”; the pull request was opened and merged within two days in December.
  • digits() (6 y 5 mo) and multi-column EmbeddedRocksDB keys (4 y 1 mo) carried the easy task label. A pull request for the second appeared five weeks after my request and sat unmerged for four years; the first waited until a first-time contributor asked to take it.

What changed in each case was never my argument. In one it was a foundation (Atomic databases), in three it was a person who wanted a small thing badly enough to write C++, and in every case the idea had been judged reasonable on day one. This is the most useful fact about ClickHouse feature requests: acceptance and delivery are decoupled. “That’s reasonable” on the day you file means the idea is not wrong. It does not mean anyone’s calendar has room.

The easy task label is the project saying it would take this if someone else wrote it, and it is the one route by which a small request reliably ships: finish_time, the XDG path and digits() all had it.

Silence is not a verdict

Of my 19 open pages, by my reading at least eleven record no maintainer reply at all: the whitespace-aligned input format, GRANT … TO ALL, comments on constraints, SQL SECURITY for the Alias engine, query tags, the implicit materialized view switch, _view, Kafka settings, tomorrow(), OPTIMIZE … WHERE and the primary key rename. Several are recent, so this is not neglect, but open is a poor signal. The project merged thousands of pull requests in one recent month; a request with no maintainer comment has not been refused, it has not been read by the right person. A fast no is a sign of attention, and silence a sign of queue depth.

Some open requests I do not expect to ship, and I am fine with that. A trivial function such as tomorrow() costs a review, a docs page and a permanent place in a shared namespace. The positions function was the wrong question; basename was the answer. The chained comparison is a trap as it stands: 1 <= x < 3 parses and silently returns the wrong rows. What I would want there is a rejection.

What a good request looks like here

The requests that shipped quickly have a recognisable form, and the ones that stalled lack parts of it.

  • A concrete use case, not a feature name. throwIf shipped because I could say what test I could not write. The LIMIT request had a vivid problem and a syntax the maintainer could not accept, and it died on the syntax. If the case is strong and the syntax weak, say so; the maintainers can find a better shape, but they cannot guess your problem.
  • A minimal proposal. Small, and attached to something that exists: a column in a system table, a setting that defaults to current behaviour, a third argument. My Alias request named three statements and got rewritten into one engine.
  • An argument for why the existing primitives do not compose. Three of my requests ended with someone naming a setting I had missed: skip_unavailable_shards, CHECK for the Sign type, basename for paths. Grep system.settings, system.functions and the changelog first.
  • Willingness to accept a different shape. Read the right-hand column again. When a maintainer says “better: an engine”, the winning response is “yes”, not a defence of the original syntax. Several of mine were better as shipped.
  • Brevity. My longest request, the Upgrade Guard, was closed fastest. Length is a cost you impose on the reviewer.

Advice, if you are about to file one

  1. Search first, including closed issues. One of my requests was a duplicate of a better-named one already opened by the maintainers. A duplicate is not a loss. It tells you someone inside wanted it.
  2. Ask whether it needs per-row state, a stored value or a coordination step. If yes, expect a “not that way” and ask for the mechanism in the thread, not the feature. The useful reply to my UPSERT was the link to the RFC that blocks it.
  3. Prefer an extension point to a new statement. A table function, virtual column, system table column, setting or function argument is a request the project can say yes to without a language committee.
  4. Offer a switch that defaults to current behavior. It is the cheapest way to let the maintainers ship something they are not yet sure of.
  5. If it is small and well specified, ask for the easy task label and say you are willing to test the pull request. Three of my fastest-moving small requests were carried by contributors.
  6. Return to your own issue. I found out about the error file, the detached_tables table and the _table column months or years after they had shipped, because nobody linked the pull request. Check your old issues against each release’s changelog, and close them yourself when you find the answer.
  7. Treat a decline as information. The one-line answers about compatibility, the unique-key RFC and a function’s return type were the clearest descriptions of the engine’s invariants I found anywhere.

A feature request is a cheap way to learn how the system thinks. The reasons are better than the yes.

philosophyfeature-requestsdesign

Discussion

Comments live in GitHub Discussions. Sign in with GitHub to join in.