Skip to content

Add serialized tsvector search pushdown - #352

Open
k-bx wants to merge 1 commit into
ClickHouse:mainfrom
k-bx:codex/tsvector-pushdown
Open

k-bx wants to merge 1 commit into
ClickHouse:mainfrom
k-bx:codex/tsvector-pushdown

Conversation

@k-bx

@k-bx k-bx commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • recognize compatibility views that expose ClickHouse String columns with tsvectorin(text::cstring)
  • deparse constant tsvector @@ tsquery predicates as ClickHouse RE2/array predicates
  • support exact and prefix operands, boolean operators, positional phrases, reverse @@, NULLs, and stripped vectors
  • leave weighted and non-constant queries for local PostgreSQL evaluation

Motivation

CDC and compatibility layers often serialize PostgreSQL tsvector values into ClickHouse strings. PostgreSQL currently has to retrieve every serialized value and apply @@ locally, which defeats predicate pushdown for full-text search.

Tests

  • full PostgreSQL 18 regression suite against ClickHouse 26.3.12.3 (39/39)
  • PostgreSQL 14 compile
  • upstream pre-commit suite
  • new parity coverage for Russian lexemes, numeric lexemes vs position numbers, prefixes, boolean queries, phrases/distances, reverse @@, NULLs, stripped vectors, weighted-query fallback, and custom/generic prepared plans

@theory theory added data types Improve data type support operators Improve operator pushdown pushdown Improvements to query pushdown labels Sep 14, 2026
@theory

theory commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Looks useful, but I think you need to run git submodule update --init --recursive to get the latest pg-clickhouse-c and clickhouse-c and fix a few issues.

src/binary/insert.c:38:78: error: too few arguments to function call,
      expected 4, have 3
   38 |             pgch_raise(&err, ERRCODE_FDW_ERROR, "could not prepare insert - ");
      |             ~~~~~~~~~~                                                       ^
vendor/pg-clickhouse-c/pg-clickhouse.h:54:1: note: 'pgch_raise' declared
      here
   54 | pgch_raise(const chc_err* err, int sqlstate, const char* what, const char* where);
      | ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/binary/insert.c:120:78: error: too few arguments to function call,
      expected 4, have 3
  120 |             pgch_raise(&err, ERRCODE_FDW_ERROR, "could not prepare insert - ");
      |             ~~~~~~~~~~                                                       ^
vendor/pg-clickhouse-c/pg-clickhouse.h:54:1: note: 'pgch_raise' declared
      here
   54 | pgch_raise(const chc_err* err, int sqlstate, const char* what, const char* where);
      | ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/binary/insert.c:185:74: error: too few arguments to function call,
      expected 4, have 3
  185 |         pgch_raise(&err, ERRCODE_FDW_ERROR, "could not insert columns - ");
      |         ~~~~~~~~~~                                                       ^
vendor/pg-clickhouse-c/pg-clickhouse.h:54:1: note: 'pgch_raise' declared
      here
   54 | pgch_raise(const chc_err* err, int sqlstate, const char* what, const char* where);
      | ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 errors generated.
make: *** [src/binary/insert.o] Error 1
make: *** Waiting for unfinished jobs....
src/binary/connection.c:319:84: error: too few arguments to function call,
      expected 6, have 5
  319 |         int rc = chc_client_init(&conn->client, &opts, &pgch_alloc, &conn->io, &err);
      |                  ~~~~~~~~~~~~~~~                                                   ^
vendor/pg-clickhouse-c/clickhouse-c/clickhouse-client.h:71:6: note: 
      'chc_client_init' declared here
   71 | int  chc_client_init(chc_client **out, const chc_client_opts *opts,
      |      ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   72 |                      const chc_alloc *al, chc_io *io, chc_exception **exc,
      |                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   73 |                      chc_err *err);
      |                      ~~~~~~~~~~~~
src/binary/connection.c:324:13: error: too few arguments to function call,
      expected 4, have 3
  322 |             pgch_raise(
      |             ~~~~~~~~~~
  323 |                 &err, ERRCODE_FDW_UNABLE_TO_ESTABLISH_CONNECTION, "connection error: "
  324 |             );
      |             ^
vendor/pg-clickhouse-c/pg-clickhouse.h:54:1: note: 'pgch_raise' declared
      here
   54 | pgch_raise(const chc_err* err, int sqlstate, const char* what, const char* where);
      | ^          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
make: *** [src/binary/connection.o] Error 1

@theory theory left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git submodule update --init --recursive

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data types Improve data type support operators Improve operator pushdown pushdown Improvements to query pushdown

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants