Clarify that test/install only prepares the test environment

CLAUDE.md

@@ -207,7 +207,7 @@ When tests fail, examine the diff output carefully. The actual test output in `t
 **Exceptions to the above** -- `test-build` and `test/install` (both optional, see `README.asc`) don't follow the `test/results` vs `test/expected` model:
 
 - **test-build** runs first, in its own separate `pg_regress` pass over `test/build/*.sql`, and gates the main suite: if it fails, `test/install`/`test/sql` never run at all. It does compare actual vs expected normally (`test/build/results/` vs `test/build/expected/`) -- use `make results-build` to refresh its expected output, not `make results`.
-- **test/install** does NOT get a real diff at all: its actual output is written to the exact same file as its expected output, so a content difference can never fail the build, no matter what changed. The only thing that still fails the build is a hard SQL error, and only if the file has `ON_ERROR_STOP` set (directly, or via `\i test/pgxntool/psql.sql`) -- pgxntool checks for this by default, passing any file that includes `psql.sql` or has its own `\set`/`\unset ON_ERROR_STOP`, whatever the value. If a `test/install/*.sql` file is misbehaving, don't go looking for a diff; check whether it errored, and don't assume a stale-looking `.out` for it means anything.
+- **test/install** only sets up the environment for the real tests; its output is not a test result (not compared against expected output, not validated as TAP), but a failure in it must abort the run. It does NOT get a real diff at all: its actual output is written to the exact same file as its expected output, so a content difference can never fail the build, no matter what changed. The only thing that still fails the build is a hard SQL error, and only if the file has `ON_ERROR_STOP` set (directly, or via `\i test/pgxntool/psql.sql`) -- pgxntool checks for this by default, passing any file that includes `psql.sql` or has its own `\set`/`\unset ON_ERROR_STOP`, whatever the value. If a `test/install/*.sql` file is misbehaving, don't go looking for a diff; check whether it errored, and don't assume a stale-looking `.out` for it means anything.
 
 ## Key Implementation Details
 

README.asc

@@ -166,7 +166,11 @@ cp test/build/results/<name>.out test/build/expected/<name>.out
 ----
 
 === test/install
-Runs setup files before the main test suite within the same `pg_regress` invocation. This allows expensive one-time operations (like extension installation) to set up state that persists into the regular test files.
+`test/install` only prepares the environment for your real tests; its output is not a test result. If a file in it fails, the run aborts, but short of a failure nothing about its output is checked: it is not compared against expected output and not validated as TAP, so don't write tests there (put them in `test/sql`).
+
+Abort-on-failure relies on `ON_ERROR_STOP`: pgxntool's static check requires every `test/install/*.sql` file to include `test/pgxntool/psql.sql` (which sets it) or to `\set` or `\unset ON_ERROR_STOP` itself. An explicit setting means the author has chosen deliberately.
+
+It runs setup files before the main test suite within the same `pg_regress` invocation. This allows expensive one-time operations (like extension installation) to set up state that persists into the regular test files.
 
 **How it works:**
 
@@ -202,11 +206,11 @@ PGXNTOOL_ENABLE_TEST_INSTALL = yes  # or no
 
 **Why this is useful:**
 
-Without `test/install`, each test file typically needs to run `CREATE EXTENSION` in its setup, which adds overhead and doesn't allow validating the installation step separately. With `test/install`, setup runs once before all tests, and any state it creates (tables, extensions, etc.) is available to every subsequent test file.
+Without `test/install`, each test file typically needs to run `CREATE EXTENSION` in its setup, which adds overhead and repeats the installation step in every test. With `test/install`, setup runs once before all tests, and any state it creates (tables, extensions, etc.) is available to every subsequent test file.
 
 **Key detail:** Install files and regular tests run in a single `pg_regress` invocation. This means the database is NOT dropped between install and test phases — state created by install files persists into the main test suite.
 
-WARNING: **`+test/install/*.out+` is never actually compared against anything.** Unlike every other test type pgxntool supports, the `test/install` directory's actual output is written to the exact same file as its expected output, so there is no diff — a wrong or changed result will never fail the build, no matter how much it changes. The *only* thing that still fails the build is a hard SQL error, and only if the file sets `ON_ERROR_STOP` (directly, or via `\i test/pgxntool/psql.sql`) — without it, psql prints the error, keeps going, and the file "passes" regardless. pgxntool enforces this by default (`_check-test-install-error-stop`, see `PGXNTOOL_ENABLE_TEST_INSTALL_ERROR_STOP_CHECK` to disable), but that only guarantees errors are caught — it does not give you real output validation. If you need actual output comparison, put that logic in `test/sql` instead (or use pgTap assertions from within the install file itself).
+WARNING: **`+test/install/*.out+` is not a test result and is never compared against anything.** Unlike every other test type pgxntool supports, the `test/install` directory's actual output is written to the exact same file as its expected output, so there is no diff — a wrong or changed result will never fail the build, no matter how much it changes. The *only* thing that still fails the build is a hard SQL error, and only if the file sets `ON_ERROR_STOP` (directly, or via `\i test/pgxntool/psql.sql`) — without it, psql prints the error, keeps going, and the file "passes" regardless. pgxntool enforces this by default (`_check-test-install-error-stop`, see `PGXNTOOL_ENABLE_TEST_INSTALL_ERROR_STOP_CHECK` to disable), but that only guarantees errors are caught — it does not give you real output validation. If you need actual output comparison, put that logic in `test/sql` instead (or use pgTap assertions from within the install file itself).
 
 ==== Update & Upgrade (U&U) Testing
 

README.html

@@ -850,7 +850,13 @@ <h4 id="_results_build"><a class="anchor" href="#_results_build"></a><a class="l
 <div class="sect2">
 <h3 id="_testinstall"><a class="anchor" href="#_testinstall"></a><a class="link" href="#_testinstall">4.5. test/install</a></h3>
 <div class="paragraph">
-<p>Runs setup files before the main test suite within the same <code>pg_regress</code> invocation. This allows expensive one-time operations (like extension installation) to set up state that persists into the regular test files.</p>
+<p><code>test/install</code> only prepares the environment for your real tests; its output is not a test result. If a file in it fails, the run aborts, but short of a failure nothing about its output is checked: it is not compared against expected output and not validated as TAP, so don&#8217;t write tests there (put them in <code>test/sql</code>).</p>
+</div>
+<div class="paragraph">
+<p>Abort-on-failure relies on <code>ON_ERROR_STOP</code>: pgxntool&#8217;s static check requires every <code>test/install/*.sql</code> file to include <code>test/pgxntool/psql.sql</code> (which sets it) or to <code>\set</code> or <code>\unset ON_ERROR_STOP</code> itself. An explicit setting means the author has chosen deliberately.</p>
+</div>
+<div class="paragraph">
+<p>It runs setup files before the main test suite within the same <code>pg_regress</code> invocation. This allows expensive one-time operations (like extension installation) to set up state that persists into the regular test files.</p>
 </div>
 <div class="paragraph">
 <p><strong>How it works:</strong></p>
@@ -914,7 +920,7 @@ <h3 id="_testinstall"><a class="anchor" href="#_testinstall"></a><a class="link"
 <p><strong>Why this is useful:</strong></p>
 </div>
 <div class="paragraph">
-<p>Without <code>test/install</code>, each test file typically needs to run <code>CREATE EXTENSION</code> in its setup, which adds overhead and doesn&#8217;t allow validating the installation step separately. With <code>test/install</code>, setup runs once before all tests, and any state it creates (tables, extensions, etc.) is available to every subsequent test file.</p>
+<p>Without <code>test/install</code>, each test file typically needs to run <code>CREATE EXTENSION</code> in its setup, which adds overhead and repeats the installation step in every test. With <code>test/install</code>, setup runs once before all tests, and any state it creates (tables, extensions, etc.) is available to every subsequent test file.</p>
 </div>
 <div class="paragraph">
 <p><strong>Key detail:</strong> Install files and regular tests run in a single <code>pg_regress</code> invocation. This means the database is NOT dropped between install and test phases — state created by install files persists into the main test suite.</p>
@@ -926,7 +932,7 @@ <h3 id="_testinstall"><a class="anchor" href="#_testinstall"></a><a class="link"
 <div class="title">Warning</div>
 </td>
 <td class="content">
-<strong><code>test/install/*.out</code> is never actually compared against anything.</strong> Unlike every other test type pgxntool supports, the <code>test/install</code> directory&#8217;s actual output is written to the exact same file as its expected output, so there is no diff — a wrong or changed result will never fail the build, no matter how much it changes. The <strong>only</strong> thing that still fails the build is a hard SQL error, and only if the file sets <code>ON_ERROR_STOP</code> (directly, or via <code>\i test/pgxntool/psql.sql</code>) — without it, psql prints the error, keeps going, and the file "passes" regardless. pgxntool enforces this by default (<code>_check-test-install-error-stop</code>, see <code>PGXNTOOL_ENABLE_TEST_INSTALL_ERROR_STOP_CHECK</code> to disable), but that only guarantees errors are caught — it does not give you real output validation. If you need actual output comparison, put that logic in <code>test/sql</code> instead (or use pgTap assertions from within the install file itself).
+<strong><code>test/install/*.out</code> is not a test result and is never compared against anything.</strong> Unlike every other test type pgxntool supports, the <code>test/install</code> directory&#8217;s actual output is written to the exact same file as its expected output, so there is no diff — a wrong or changed result will never fail the build, no matter how much it changes. The <strong>only</strong> thing that still fails the build is a hard SQL error, and only if the file sets <code>ON_ERROR_STOP</code> (directly, or via <code>\i test/pgxntool/psql.sql</code>) — without it, psql prints the error, keeps going, and the file "passes" regardless. pgxntool enforces this by default (<code>_check-test-install-error-stop</code>, see <code>PGXNTOOL_ENABLE_TEST_INSTALL_ERROR_STOP_CHECK</code> to disable), but that only guarantees errors are caught — it does not give you real output validation. If you need actual output comparison, put that logic in <code>test/sql</code> instead (or use pgTap assertions from within the install file itself).
 </td>
 </tr>
 </table>
@@ -2217,7 +2223,7 @@ <h2 id="_copyright"><a class="anchor" href="#_copyright"></a><a class="link" hre
 </div>
 <div id="footer">
 <div id="footer-text">
-Last updated 2026-10-07 17:56:47 -0500
+Last updated 2026-10-15 17:03:08 -0500
 </div>
 </div>
 </body>