pgxntool/verify-tap.sh [--skip-non-tap] [FILE|-]...+
PGXNTOOL_ENABLE_FS_INSTALL=no) and runs it through PGXS’s installcheck, first pulling in anything you’ve hooked into testdeps and, if enabled, sanity-checking your test SQL via test-build.
make test also fails on two safeguard checks:
make test also fails on these safeguard checks:
In pgtap mode (the default; see verify-results safeguard), the TAP check: fails if verify-tap.sh rejects any test/results/*.out file holding TAP output, even when that output matches its expected file. Files with no TAP output are skipped, and test/install output is never checked. This runs alongside the regression.diffs check, and both are always reported.
The stale-expected-file check, run after pg_regress finishes: fails if test/expected/ (or test/build/expected/) has a .out file with no matching .sql, or a file that isn’t .out. See PGXNTOOL_ENABLE_CHECK_STALE_EXPECTED *.
test exits non-zero (after printing regression.diffs) if any test fails, so CI and other automation can rely on its exit code.
+test exits non-zero (after printing regression.diffs and any TAP check failures) if any test fails, so CI and other automation can rely on its exit code.
By default, make results will refuse to run if your tests are failing, so you can’t accidentally promote incorrect output into the new expected results. Which failures it looks for is controlled by PGXNTOOL_VERIFY_RESULTS_MODE, which must be one of these two values (case-insensitive; anything else is a hard make error):
By default, make results will refuse to run if your tests are failing, so you can’t accidentally promote incorrect output into the new expected results. Which failures it looks for is controlled by PGXNTOOL_VERIFY_RESULTS_MODE, which also decides whether make test runs its TAP check (see test). It must be one of these two values (case-insensitive; anything else is a hard make error):
pgtap (the default)Scans test/results/*.out for pgTAP not ok lines (ignoring # TODO items) and plan-count mismatches, then also checks test/regression.diffs for any output mismatch pgTAP itself wouldn’t catch. A test that has no expected output file yet is the one exception: it always shows up in regression.diffs (pgxntool creates an empty expected file for it, because pg_regress aborts on a missing one), so make results is allowed to create its first expected output — provided its own pgTAP output is clean and free of SQL errors. If a new test legitimately needs output containing a SQL error blessed, use PGXNTOOL_ENABLE_VERIFY_RESULTS=no for that one run.
Runs verify-tap.sh --skip-non-tap on every test/results/*.out file, so a not ok (ignoring # TODO items), a Bail out!, or a missing, duplicate or misplaced plan line or a test count that differs from the plan all block; then also checks test/regression.diffs for any output mismatch pgTAP itself wouldn’t catch. A test that has no expected output file yet is the one exception: it always shows up in regression.diffs (pgxntool creates an empty expected file for it, because pg_regress aborts on a missing one), so make results is allowed to create its first expected output — provided its own pgTAP output is clean and free of SQL errors. If a new test legitimately needs output containing a SQL error blessed, use PGXNTOOL_ENABLE_VERIFY_RESULTS=no for that one run.
+make test runs the same verify-tap.sh check, so a TAP failure fails make test even after it has been blessed into an expected file.
diffsOnly checks whether test/regression.diffs exists, so it can’t create a new test’s first expected output; use pgtap mode, or PGXNTOOL_ENABLE_VERIFY_RESULTS=no for that one run.
Only checks whether test/regression.diffs exists, and make test skips its TAP check. It can’t create a new test’s first expected output; use pgtap mode, or PGXNTOOL_ENABLE_VERIFY_RESULTS=no for that one run.
pgxntool/verify-tap.sh checks TAP output (such as pgTAP’s) and fails if it shows a failed test run. It doesn’t need make or a database, so you can run it on any TAP output, including outside of pgxntool; make test and make results use it in pgtap mode (see verify-results safeguard).
pgxntool/verify-tap.sh [--skip-non-tap] [FILE|-]...+
With no arguments, or for a - argument, it reads stdin; otherwise each FILE is judged as its own TAP stream:
psql -XtA -f my_test.sql | pgxntool/verify-tap.sh +pgxntool/verify-tap.sh test/results/*.out+
Only unindented lines count, so subtest output is ignored. A stream passes only if:
+no not ok line lacks # TODO
there is no Bail out! line
there is no pgTAP # Looks like you planned line
there is exactly one plan line (1..N), before every test line (plan(N)) or after every one (no_plan() plus finish())
the number of test lines (ok N/not ok N) equals N; 1..0 with no tests passes
Input with no plan line and no test lines fails, since it means nothing ran. With --skip-non-tap, such input (plain SQL output, say) is skipped silently instead, which lets you pass a mix of TAP and non-TAP files. Each problem is reported on stderr, naming its source (stdin or the file’s path).
Exit status is 0 if every input passes, 1 if any input fails, and 2 on a usage error or an unreadable file (in which case nothing is checked).
+PGXNtool automatically generates version-specific SQL files from your base SQL file. These files follow the pattern sql/{extension}--{version}.sql and are used by PostgreSQL’s extension system to install specific versions of your extension.
PGXN distinguishes two things it’s easy to conflate: a distribution and an extension. Per the PGXN meta spec, a distribution is "a collection of extensions, source code, utilities, tests, and/or documents that are distributed together" — the thing you release and upload to PGXN, identified by the top-level name/version in META.json (generated from META.in.json — you edit the latter, never META.json directly). An extension is the individual thing installed via CREATE EXTENSION, described by its own .control file. A single distribution routinely provides more than one extension — META.in.json’s `provides map lists each one; the pgTAP distribution, for example, provides both the pgtap and schematap extensions from one release.
When you run make (or make all), PGXNtool:
The version number comes from each extension’s own .control file → default_version, not from META.json. PostgreSQL itself uses default_version to pick which versioned SQL file to load, so control.mk.sh parses the .control file(s) directly to keep the generated filename in sync with what PostgreSQL will actually use. META.in.json has its own, separate provides.{extension}.version for that same extension (see PGXN Distributions vs. Extensions above) — pgxntool never reads it when generating SQL files, and nothing keeps it in sync with the .control file’s default_version.
Version-specific SQL files are treated as permanent files that should be committed to your repository by default. This makes it much easier to test updates to extensions, as you can see exactly what SQL was included in each version.
The primary value of committing a version’s install script is update testing: with the file in place, you can install that exact version (CREATE EXTENSION ext VERSION 'x.y.z'), run ALTER EXTENSION ext UPDATE, and verify the update path actually works. In principle you would only ever need the very first committed version’s install script — every later version is reachable by chaining upgrade scripts from it. See test/install for how to wire this into your test suite.
For a minor version change that doesn’t meaningfully alter the extension (e.g. a small bug fix), it’s unlikely to straddle a PostgreSQL supported-version boundary, so there’s much less test-coverage value in committing that specific version’s generated install script. For large extensions, deliberately not committing some of these individual version files is worth doing to keep repository size down — the base sql/{extension}.sql still fully captures the change in git history, and the install script itself is trivially regenerated from that base file at build time.
.gitignore a Skipped Version — rm It Once.gitignore a Skipped Version — rm It OnceWhen you decide not to track a particular version’s generated install script, do not add a .gitignore entry for it.
control.mk.sh generates a Make rule along these lines for the current version’s file: