CI probe: same commit as #1444 (temporary, will be closed)

.changeset/eql-json-tree-function-blockers.md

@@ -0,0 +1,9 @@
+---
+"@cipherstash/eql": patch
+---
+
+EQL v4 `json_tree`: calling a native jsonpath or extraction function on a `public.eql_v4_json_search` column, or on an entry extracted from one, now raises `feature_not_supported` (SQLSTATE `0A000`) instead of returning nothing (#1439).
+
+The column is a domain over `jsonb`, so `jsonb_path_match(col, $1::jsonpath, $2::jsonb)` used to resolve to PostgreSQL's own function, which ran over the stored ciphertext envelope rather than the document and returned false for every row, with no error. EQL now installs overloads in `public` for `jsonb_path_exists`, `jsonb_path_match`, `jsonb_path_query`, `jsonb_path_query_array` and `jsonb_path_query_first` (and their `_tz` forms), `jsonb_extract_path`, `jsonb_extract_path_text`, `jsonb_each`, `jsonb_each_text`, `jsonb_object_keys`, `jsonb_typeof`, `jsonb_strip_nulls`, `jsonb_set`, `jsonb_insert` and `jsonb_pretty` on both `public.eql_v4_json_search` and `public.eql_v4_json_entry`. Each raises with a message naming the function and what to do instead. On a document, the jsonpath and extraction blockers name the EQL query to use (`@>` with an `eql_v4.query_json` needle, `col -> selector`, or a subtree walk), and `jsonb_typeof`, `jsonb_set`, `jsonb_insert`, `jsonb_strip_nulls` and `jsonb_pretty` direct the caller to decrypt the document in the client. On an entry, `jsonb_path_exists` and `jsonb_path_match` name the comparison with an `eql_v4.query_json_leaf`, and the others direct the caller to decrypt in the client.
+
+Two limits, documented in `docs/reference/json-tree-leakage.md`. A call is caught only when every argument has a type: a bare literal (`jsonb_path_match(col, '$.a == 1')`) or a parameter sent without a type still reaches the native function, so cast every argument. And with the `eql_v4` bundle installed, a call to any of these 20 functions whose first argument is an untyped literal or an untyped bound parameter now fails with `function … is not unique` (SQLSTATE `42725`), on plain `jsonb` too: `jsonb_typeof('{}')`, `jsonb_typeof($1)`, `jsonb_path_exists($1, '$.a')`. Cast the first argument to `jsonb`; a `jsonb` column or a parameter typed `jsonb` is unaffected. See [U-004](../../docs/upgrading/v3.1.md#u-004-an-untyped-first-argument-to-20-jsonb-functions-is-ambiguous).

packages/eql/docs/reference/json-tree-leakage.md

@@ -6,6 +6,7 @@
 
 - [What a stored row shows](#what-a-stored-row-shows)
 - [What a query shows](#what-a-query-shows)
+- [Native jsonb operators and functions are blocked](#native-jsonb-operators-and-functions-are-blocked)
 - [What is authenticated](#what-is-authenticated)
 - [Compared with EQL v3 (`public.eql_v3_json_search`)](#compared-with-eql-v3-publiceql_v3_json_search)
 - [The two options that change the picture](#the-two-options-that-change-the-picture)
@@ -51,6 +52,24 @@ A query hands the database a term derived from the plaintext it asks about, and
   - **The root.** A read of `$` sends the root's token, and a read of anything below `$` does not. With the sealed root on (the default) that token covers no document's structure: it finds the root entry, whose ciphertext is the document, and nothing beneath it, since the top-level fields chain from nothing. With it off, the root's token covers every row's whole document.
   - **`json_tree_children` and `json_tree_child_count`** take one token, of any node a path names, and send nothing else. Under the sealed root the root's token enumerates nothing (its count is NULL), for the same reason.
 
+## Native jsonb operators and functions are blocked
+
+`public.eql_v4_json_search` and `public.eql_v4_json_entry` are PostgreSQL domains over `jsonb`, so PostgreSQL's own `jsonb` operators and functions accept them. Run on a column, they would see the stored envelope (`{v, i, h, sv, q}`), not the document: `jsonb_path_match(col, '$.score == 3'::jsonpath)` finds no `score` key and returns false for every row, with no error. EQL blocks them so that they raise instead:
+
+- **Operators.** `?`, `?|`, `?&`, `@?`, `@@`, `#>`, `#>>`, `-`, `#-`, `||`, `->` and `->>` by position, root-document comparisons, and containment against a plain `jsonb` operand raise `operator … is not supported for public.eql_v4_json_search`.
+- **Functions.** `jsonb_path_exists`, `jsonb_path_match`, `jsonb_path_query`, `jsonb_path_query_array`, `jsonb_path_query_first` (and their `_tz` forms, with or without `vars` and `silent`), `jsonb_extract_path`, `jsonb_extract_path_text`, `jsonb_each`, `jsonb_each_text`, `jsonb_object_keys`, `jsonb_typeof`, `jsonb_strip_nulls`, `jsonb_set`, `jsonb_insert` and `jsonb_pretty` raise SQLSTATE `0A000` (`feature_not_supported`) on a document or an entry, naming the function and what to use instead: `@>` with an `eql_v4.query_json` needle for equality, `col -> selector` compared with an `eql_v4.query_json_leaf` for ranges, `col -> selector`, `eql_v4.json_tree_path_query` or `eql_v4.json_tree_subtree` to extract, then decryption in the client. `jsonb_array_elements`, `jsonb_array_elements_text` and `jsonb_array_length` are not blocked: they already raise on both domains, because the envelope and an entry are objects.
+
+The function blockers are overloads in `public` that take the domain as their first argument. PostgreSQL picks one only when **every argument has a type**, so two forms still reach the native function, which reads the stored envelope: a key of the document returns nothing there, but a key of the envelope, such as `v`, returns the envelope's own value (`jsonb_extract_path(col, 'v')` is `4`):
+
+- a bare literal anywhere in the call, as in `jsonb_path_match(col, '$.a == 1', '{}')`;
+- a parameter the driver sends without a type, as in `jsonb_path_match(col, $1)`.
+
+Cast every argument, `jsonb_path_match(col, $1::jsonpath, $2::jsonb)`, and the call raises. No overload can close the untyped forms.
+
+The overloads have one cost outside EQL's columns. A call on plain `jsonb` whose first argument is untyped, such as `jsonb_typeof($1)` or `jsonb_each('{"a": 1}')`, now has two candidates, and PostgreSQL refuses it as ambiguous (`function … is not unique`). Cast that argument (`$1::jsonb`). The jsonb operators behave the same way once EQL is installed. [U-004 in the 3.1 upgrade guide](../upgrading/v3.1.md#u-004-an-untyped-first-argument-to-20-jsonb-functions-is-ambiguous) lists the 20 functions.
+
+To read the stored envelope on purpose, cast the column: `jsonb_pretty(col::jsonb)`.
+
 ## What is authenticated
 
 Only the ciphertext `c`. Opening an entry verifies that it was sealed under this keyset, for this column, under the data key its row's header names (a fresh one for every row), with the selector `s` and the slot `j` it is opened with, and over the AEAD tags of exactly the children it is opened with: each entry's AAD binds a digest of its children's tags, in order. A ciphertext moved into another row or column, or opened under another selector, slot or set of children, fails. Nothing in it names a row, so an entry that travels with its own row's header still opens; see below. The selectors, the projection, the parent tags, the self tags, the ordering terms, the header's sealed-root flag, and which entries a row holds are not authenticated, by the same rule every EQL stored value follows (ADR 0010 in `packages/stack-encrypt/docs/adr`).

packages/eql/docs/upgrading/v3.1.md

@@ -13,6 +13,11 @@ terms this release's `eql_v4` expects. The notes below are for anyone who
 builds `eql_v4` payloads by hand, or who stored `eql_v4` block ORE terms
 before this release.
 
+One more change reaches SQL that never touches an EQL column: installing
+the `eql_v4` bundle, which carries the new JSON tree family, makes a call to
+20 jsonb functions ambiguous when its first argument has no type
+([U-004](#u-004-an-untyped-first-argument-to-20-jsonb-functions-is-ambiguous)).
+
 ## TL;DR
 
 1. An `eql_v4` text search query carries only the term its operator reads,
@@ -30,6 +35,12 @@ before this release.
    `eql_v4` bundle must drop its stored order domains before installing this
    one —
    [U-003](#u-003-an-order-term-is-one-hex-string-checked-when-it-is-written).
+4. With the `eql_v4` bundle installed, a call to one of 20 jsonb functions
+   (`jsonb_typeof`, `jsonb_path_exists`, `jsonb_extract_path`, …) whose
+   **first argument is an untyped literal or an untyped bound parameter
+   fails** with `function … is not unique` (SQLSTATE `42725`), on plain
+   `jsonb` too. Cast that argument to `jsonb` —
+   [U-004](#u-004-an-untyped-first-argument-to-20-jsonb-functions-is-ambiguous).
 
 ## Compatibility
 
@@ -44,6 +55,7 @@ before this release.
 | `ob` in every `eql_v4` block ORE domain and range bound | **Wire shape changed.** One hex string, `"ob": "0202…"`, where it was a one-element array. The CHECK requires a string starting `0202` or `0203`. See U-003. |
 | `op` in every `eql_v4` OPE domain and range bound | **Tightened CHECK.** A string starting `00` or `01`, the term's carry byte. The term itself is unchanged. See U-003. |
 | `eql_v4` equality (`hm`) terms, ciphertexts (`c`) | **Unchanged** by this release. |
+| 20 jsonb functions (`jsonb_typeof`, the `jsonb_path_*` family, `jsonb_extract_path`, …) called with an untyped first argument | **Now ambiguous** (SQLSTATE `42725`) once the `eql_v4` bundle is installed, on plain `jsonb` too. A `jsonb` column, a `::jsonb` cast, or a parameter typed `jsonb` is unaffected. See U-004. |
 | Postgres 14, 15, 16, 17 | Supported. |
 | Supabase / managed Postgres (no superuser) | As before: the block ORE domains need the operator class the installer creates, so a non-superuser install disables them ([U-003 in the 3.0 guide](./v3.0.md#u-003-non-superuser-installs-disable-the-ore-backed-domains)). U-002 only concerns installs where they are enabled. |
 
@@ -238,13 +250,97 @@ SELECT '{"v": 4, "i": {"t": "t", "c": "c"}, "c": "x", "ob": ["0202"]}'::jsonb
 -- ERROR:  value for domain eql_v4_integer_ord_ore violates check constraint
 ```
 
+### U-004: An untyped first argument to 20 jsonb functions is ambiguous
+
+**What changed.** The `eql_v4` bundle carries the JSON tree family, whose
+stored document `public.eql_v4_json_search` and extracted entry
+`public.eql_v4_json_entry` are domains over `jsonb`. So that a native jsonb
+function called on them raises instead of reading the stored envelope, the
+install adds an overload in `public` of each of these 20 functions on each
+of the two domains, 40 in all:
+
+| Functions | |
+| --- | --- |
+| `jsonb_path_exists`, `jsonb_path_exists_tz` | `jsonb_path_match`, `jsonb_path_match_tz` |
+| `jsonb_path_query`, `jsonb_path_query_tz` | `jsonb_path_query_array`, `jsonb_path_query_array_tz` |
+| `jsonb_path_query_first`, `jsonb_path_query_first_tz` | `jsonb_extract_path`, `jsonb_extract_path_text` |
+| `jsonb_each`, `jsonb_each_text` | `jsonb_object_keys`, `jsonb_typeof` |
+| `jsonb_strip_nulls`, `jsonb_set` | `jsonb_insert`, `jsonb_pretty` |
+
+With `public` on the search path, a call to any of them whose **first
+argument has no type** now has more than one candidate, PostgreSQL's own
+function and the overloads, and PostgreSQL refuses it. That holds when the
+call has nothing to do with an EQL column:
+
+```text
+SELECT jsonb_typeof('{}');
+ERROR:  function jsonb_typeof(unknown) is not unique
+SQLSTATE: 42725 (ambiguous_function)
+```
+
+An untyped first argument is a bare string literal, as in
+`jsonb_typeof('{}')` or `jsonb_each('{"a": 1}')`, or a bound parameter the
+driver sends without a type and the SQL does not cast, as in
+`jsonb_typeof($1)` or `jsonb_path_exists($1, '$.a')`. The statement fails
+when it is parsed or prepared, before it reads a row.
+
+These are **unaffected**: a first argument that is a `jsonb` column or
+expression, one cast to `jsonb` (`'{}'::jsonb`, `$1::jsonb`), and a
+parameter the driver declares as `jsonb`. Only the first argument decides
+this; the others may stay untyped. `jsonb_array_elements`,
+`jsonb_array_elements_text` and `jsonb_array_length` have no overload and
+are unaffected.
+
+**Why.** On the two domains, the native functions run over the stored
+envelope rather than the document, so `jsonb_path_match(col, $1::jsonpath,
+$2::jsonb)` returned false for every row, with no error. The overloads make
+those calls raise `feature_not_supported` (`0A000`) instead. A literal of
+unknown type can be read as `jsonb` or as either domain, so PostgreSQL has
+no single best candidate. The jsonb operators already behave this way with
+EQL installed (`'{"a": 1}' -> 'a'` is `operator is not unique`).
+
+**Who is affected.** Any SQL in a database where the `eql_v4` bundle is
+installed that calls one of the 20 functions with an untyped first
+argument, whatever table it reads. Drivers commonly prepare parameters
+without a type, so look for calls that pass a parameter straight in as the
+first argument. A database with only the `eql_v3` bundle is not affected:
+the overloads are in the `eql_v4` bundle alone.
+
+**What to do.** Cast the first argument to `jsonb`, once:
+
+```sql
+SELECT jsonb_typeof($1::jsonb);
+SELECT jsonb_path_exists($1::jsonb, '$.a');
+SELECT jsonb_each('{"a": 1}'::jsonb);
+```
+
+To find the calls, search your SQL for each of the 20 names above followed
+by `(` and a parameter or a quoted literal.
+
+**Verification.**
+
+```sql
+-- 40 overloads, 20 on each domain:
+SELECT count(*)
+FROM pg_proc p
+WHERE p.pronamespace = 'public'::regnamespace
+  AND p.pronargs >= 1
+  AND p.proargtypes[0] IN ('public.eql_v4_json_search'::regtype,
+                           'public.eql_v4_json_entry'::regtype)
+  AND starts_with(p.proname::text, 'jsonb_');
+
+SELECT jsonb_typeof('{}');         -- ERROR: 42725, not unique
+SELECT jsonb_typeof('{}'::jsonb);  -- object
+```
+
 ## Verification checklist
 
 - [ ] The `eql_v4` install reports `3.1.0` (`SELECT obj_description('eql_v4'::regnamespace, 'pg_namespace')`).
 - [ ] A one-term `eql_v4.query_text_search` operand casts (U-001).
 - [ ] `eql_v4_internal` lists `ore_block_v2` objects and no `ore_block_256` (U-002).
 - [ ] Every `_ord_ore` / `text_search_ore` column passes the self-comparison scan (U-002).
 - [ ] The stored `eql_v4` order domains require a hex string with a scheme prefix (U-003).
+- [ ] No call to one of the 20 jsonb functions passes an untyped first argument (U-004).
 - [ ] Grants and functional indexes have been re-applied after the install
       ([U-002 in the 3.0.5 guide](./v3.0.5.md#u-002-grants-and-dependent-objects-do-not-survive-an-install)).
 
@@ -264,6 +360,32 @@ reversal misreads data:
   block ORE domains and order through the `_ord` / `_ord_ope` domains, whose
   terms this release does not change.
 
+U-004 is not reversed by installing an earlier bundle, nor by the `eql_v4`
+uninstaller. The overloads live in `public` beside the JSON tree domains,
+which survive both by design, and they keep raising on those domains. To
+remove them, drop them; nothing else depends on them:
+
+```sql
+DO $$
+DECLARE r record;
+BEGIN
+  FOR r IN
+    SELECT p.oid::regprocedure AS f
+    FROM pg_proc p
+    WHERE p.pronamespace = 'public'::regnamespace
+      AND p.pronargs >= 1
+      AND p.proargtypes[0] IN ('public.eql_v4_json_search'::regtype,
+                               'public.eql_v4_json_entry'::regtype)
+      AND starts_with(p.proname::text, 'jsonb_')
+  LOOP
+    EXECUTE 'DROP FUNCTION ' || r.f;
+  END LOOP;
+END $$;
+```
+
+A native jsonb function called on a JSON tree column then reads the stored
+envelope again, without an error.
+
 Any install, in either direction, re-runs `DROP SCHEMA … CASCADE`, so grants
 and dependent objects need re-applying afterwards.