Skip to content

EQL v4: jsonb_path_match(col, …) on an eql_v4_json_search column returns false for every row, silently — document it, since a blocker cannot fix it #1439

Closed

Description

@coderdan

Revised scope (2026-10-31)

The function blockers proposed below will not be added. Review of #1444, and tests against the built v4 bundle, showed they cost more than they protect:

  • They break code that never touches an encrypted column. An overload that takes the domain as its first argument becomes a second candidate for every call whose first argument has no type. With the v4 bundle installed, jsonb_typeof('{}'), jsonb_each_text($1) and jsonb_path_exists($1, '$.a') on plain jsonb fail with function … is not unique (SQLSTATE 42725). That happens across the whole database, including when the parameter comes from a driver that sends no types (pgx, node-postgres) or from psycopg2's inlined literals.
  • They miss the usual way the mistake is written. PostgreSQL picks the overload only when every argument has a type. jsonb_path_match(col, '$.a == 1'), or jsonb_path_match(col, $1) with an untyped parameter, still runs the native function and still returns false for every row. That is the "plausible, not tested" case in the Verification section below, now confirmed.
  • They cannot be scoped to the domain. The overloads already take only the domain. The ambiguity comes from PostgreSQL's rules for an argument with no type: a domain over jsonb accepts it exactly as jsonb does.
  • The design that would close this is not available. A type of its own that jsonb functions do not accept (a base type, not a domain) makes every native call on the column fail with "function does not exist", with no ambiguity. But only a superuser can create a base type, and EQL installs without superuser (Supabase and similar).

New scope, delivered by #1444:

  1. The JSON tree reference (packages/eql/docs/reference/json-tree-leakage.md) says plainly that native jsonb functions on these columns read the stored envelope and match nothing, with no error. It lists what to query with instead: @> with an eql_v4.query_json needle, col -> selector, the subtree reads, and, from feat(eql): one-parameter jsonpath predicates on json_tree documents (json_tree_match, @@, match) #1446, eql_v4.json_tree_match.
  2. Tests on a real encrypted document pin that behaviour, so the docs cannot drift from the database.
  3. A guard test fails if EQL ever puts an overload of a native jsonb function in public, so the cost above cannot come back unnoticed.

The original text follows, kept for the record.


Original proposal (superseded)

Background

EQL is the SQL library we install into a customer's PostgreSQL database. Its v4 install adds the column type public.eql_v4_json_search, which stores an encrypted JSON document in a form the database can query without the key (containment with @>, extraction with ->, ordering at declared paths, and subtree reads). That type is a PostgreSQL domain over jsonb: a jsonb with a check constraint and its own operators. Being a domain is what gives the column free jsonb storage and tooling, but it also means every native jsonb function and operator still accepts the column, and would run over the ciphertext envelope as if it were the user's document.

EQL guards against that with blockers: operator overloads on the domain that RAISE instead of running. packages/eql/src/v3/json_tree/blockers.sql blocks ?, ?|, ?&, @?, @@, #>, #>>, -, #-, ||, positional ->, and root =/<> (the @? and @@ jsonpath blockers are at blockers.sql:104 and :124 on ffeed7adb).

Problem

The blockers cover operators only. The function forms of the same operations are not blocked, and PostgreSQL resolves them to the native pg_catalog function because the domain's base type is jsonb. So this statement runs without error on an encrypted column:

SELECT * FROM entity
WHERE jsonb_path_match(custom_field_values, '$."score" == $val'::jsonpath, '{"val": 3.14}'::jsonb);

What happens, step by step:

  1. jsonb_path_match receives the stored envelope, {"v": …, "i": …, "h": …, "sv": […], "q": {…}}.
  2. The jsonpath looks for a top-level member score. The envelope has none; the user's keys are inside keyed hashes and ciphertexts.
  3. The predicate evaluates to false (or NULL for a comparison), and the row is excluded.

Every row is excluded, for every predicate, with no error. A query that should return some rows returns none, and nothing in the result says the predicate never saw a document.

Who hits it: anyone who already uses the jsonpath function family on a plain jsonb column and migrates the column to eql_v4_json_search. A customer's filter compiler (AIP-160 filter strings to SQL) does exactly this today, and its test suite asserts on the generated SQL text, which is unchanged, so the suite stays green while production returns empty results.

Why nothing catches it today: the operator blockers were written for the operators, and the function family (jsonb_path_match, jsonb_path_exists, jsonb_path_query, jsonb_path_query_first, jsonb_path_query_array, plus jsonb_extract_path, jsonb_extract_path_text, jsonb_array_elements, jsonb_array_elements_text, jsonb_each, jsonb_object_keys) is not covered. blockers.sql contains no jsonb_path reference at all (verified: git grep -c jsonb_path ffeed7adb -- packages/eql/src/v3/json_tree/blockers.sql is 0).

Verification

Run on PostgreSQL 17.10 with a scratch domain over jsonb:

native|f                      -- jsonb_path_match(doc, '$."score" == $val'::jsonpath, '{"val":3.14}'::jsonb) with no blocker
ERROR:  jsonb_path_match is not supported on an encrypted document   -- same call, after creating the overload below
blocked-untyped|f             -- same call with bare untyped literals: the native function still wins

The overload that produced the second line:

CREATE FUNCTION public.jsonb_path_match(public.scratch_json_search, jsonpath, jsonb) RETURNS boolean
  LANGUAGE plpgsql IMMUTABLE AS $$
  BEGIN RAISE 'jsonb_path_match is not supported on an encrypted document' USING ERRCODE = 'feature_not_supported'; END $$;

So an overload on the domain, in public (which EQL already owns for the domain types), wins function resolution over pg_catalog when the other arguments carry a type. PostgreSQL's resolution checks for an exact match on the input types before it flattens a domain to its base type, which is the same rule the existing operator blockers rely on.

Verified: the silent false, and the typed-argument blocker winning. Plausible, not tested: that the untyped-literal case cannot be closed by a function overload at all. A bare '…' literal is of type unknown, and for unknown arguments resolution prefers the pg_catalog candidate. Parameterised callers (the realistic ones, since a driver sends $1::jsonpath) are covered.

Proposal

  1. Add a function-form blocker section to packages/eql/src/v3/json_tree/blockers.sql: one overload per function in the jsonpath family and the extraction family listed above, with the domain as the first argument and the native signature's remaining types, each raising feature_not_supported with a message that names the function and points at the EQL operation to use instead (jsonb_path_match → "compile the predicate client-side; see the json_tree query reference").
  2. Add the same for public.eql_v4_json_entry where the native function would accept it (an extracted entry is also a domain over jsonb).
  3. Add a test in the SQLx suite that calls each blocked function on a stored document and asserts the feature_not_supported SQLSTATE, so a function added to the family later is caught when the test is extended, and a blocker dropped by accident fails loudly.
  4. Document in the json_tree reference that the untyped-literal form (jsonb_path_match(col, '$.a == 1', '{}') with no casts) is not blocked, and why.

The failure we want: a loud feature_not_supported error at the first call, not an empty result set.

Relationship to other work

Activity

  1. added
    bugSomething isn't working
    needs-triageMaintainer needs to evaluate the issue
    on Oct 29, 2026
  2. coderdan commented on Oct 29, 2026

    @coderdan
    ContributorAuthor

    Follow-ons that give callers something to use instead of jsonb_path_match: #1441 (compiled predicate operand and eql_v4.json_tree_match) and its Go call #1443. The element-scoped case is #1440 / #1442.

  3. added a commit that references this issue on Oct 30, 2026
    0883fb1
  4. coderdan commented on Oct 30, 2026

    @coderdan
    ContributorAuthor

    Two deviations from the proposal in #1444, recorded here so they are not lost:

    1. jsonb_array_elements, jsonb_array_elements_text and jsonb_array_length are not blocked. The hazard this issue targets is a silent empty result. These three cannot produce one: on both public.eql_v4_json_search and public.eql_v4_json_entry the native function already raises (cannot extract elements from an object, cannot get array length of a non-array), because the stored envelope and an extracted entry are JSON objects. An overload would only change the error message, and it would make an untyped first argument on plain jsonb ambiguous for those three as well (jsonb_array_elements($1) is a common bulk-load pattern). A test pins that the three still raise natively on both domains, so the reasoning fails loudly if it stops holding. Each is one overload to add if the uniform message is preferred.
    2. The tests are in the encryption test crate, not the SQLx suite. EQL tests must run over real encrypted data, and the SQLx suite's fixtures come from cipherstash-client, which writes EQL v3 only. The tests (packages/eql/tests/encryption/tests/json_tree/blockers.rs) run over a document Stack Encrypt seals under the fake key source, against PostgreSQL 14 to 17 in CI (mise run test:encryption:postgres).

    Also beyond the list here: the _tz jsonpath forms, jsonb_each_text, jsonb_typeof, jsonb_strip_nulls, jsonb_set, jsonb_insert and jsonb_pretty are blocked, 20 functions on each domain. Their cost on plain jsonb (an untyped first argument becomes ambiguous) is in upgrade note U-004 of the 3.1 guide.

  5. changed the title [-]EQL v4: `jsonb_path_match(col, …)` on an `eql_v4_json_search` column returns false for every row, silently — add function-form blockers[/-] [+]EQL v4: `jsonb_path_match(col, …)` on an `eql_v4_json_search` column returns false for every row, silently — document it, since a blocker cannot fix it[/+] on Oct 31, 2026
  6. self-assigned this
    on Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

SDKbugSomething isn't workingneeds-triageMaintainer needs to evaluate the issue

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions