Repository navigation
Test make test's pgTap output check and pgtap-only mode - #107
Open
jnasbyupgrade wants to merge 7 commits into
Open
jnasbyupgrade wants to merge 7 commits into
jnasbyupgrade wants to merge 7 commits into
Conversation
Add tests for Postgres-Extensions/pgxntool#141 (commit d62a875, part of Postgres-Extensions/pgxntool#106): - `make test` fails on pgTap failures, missing or duplicate plans and plan/count mismatches in `test/results` and `test/install` output - `verify-results-pgtap.sh` uses the same check, with the `test/install` outputs as extra arguments - New `check-tap-output-script.bats` invokes the script directly and covers each failure rule, non-TAP pass-through, `1..0`, subtests and nonexistent arguments. - `verify-results-pgtap-script.bats` covers the plan checks and the extra file arguments. - `make-test.bats` covers wiring only: the arguments (dry run), the `PGXNTOOL_ENABLE_CHECK_TAP_OUTPUT=no` skip and exit-status propagation (stub), and one end-to-end case where a blessed `not ok` fails `make test`. - Add the new variables to `pgxntool-variables.txt` and the new script to `dist-expected-files.txt`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…heck Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Add tests for Postgres-Extensions/pgxntool@3e8be3f (part of Postgres-Extensions/pgxntool#141): with `TESTOUT` different from `TESTDIR`, `_test-install-output-link` makes `$(TESTOUT)/install` a symlink to `$(TESTDIR)/install`, and fails rather than replace a pre-existing directory there. Both checks run without Postgres, folded into the existing schedule test in `test-test-install.bats`.
It is always `$(TESTDIR)/install/`, per Postgres-Extensions/pgxntool@3e8be3f.
Add tests for Postgres-Extensions/pgxntool@15e558b (part of Postgres-Extensions/pgxntool#141, for Postgres-Extensions/pgxntool#106): - `verify-results-pgtap-script.bats`: under `--pgtap-only` (with and without `--for-test`), a TAP test whose only difference is its description passes, and fails under plain pgtap; a non-TAP diff still fails; a TAP test with a SQL error line fails. - `check-tap-output-script.bats`: `--list` names only TAP files. - `make-test.bats`: the mode validation tests cover `pgtap-only` and its parse-time error with `PGXNTOOL_ENABLE_CHECK_TAP_OUTPUT=no`, and a dry run shows `make test` and `verify-results` using the new logic.
jnasbyupgrade
changed the title
make test's pgTap output checkmake test's pgTap output check and pgtap-only mode
Update tests for Postgres-Extensions/pgxntool@7eec927 (part of Postgres-Extensions/pgxntool#141, for Postgres-Extensions/pgxntool#106): - `make-test.bats`: the dry-run and stub tests prove `PGXNTOOL_VERIFY_RESULTS_MODE=diffs` skips the TAP check, instead of the removed variable; drop its invalid-value test and the `pgtap-only` conflict assertion. - `pgxntool-variables.txt`: drop the variable from the inventory.
A bare Postgres-Extensions#106 in pgxntool-test resolves to this repo's PR, so the reference now names Postgres-Extensions/pgxntool#106.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds test coverage for Postgres-Extensions/pgxntool#141, for Postgres-Extensions/pgxntool#106. That PR makes
make testfail on pgTap failures in actual output (anot okwithout# TODO, a missing or duplicate plan, or a plan/count mismatch, intest/resultsandtest/installoutput), addsPGXNTOOL_VERIFY_RESULTS_MODE=pgtap-only, and fixestest/installunder a separateTESTOUT.Merge order: merge Postgres-Extensions/pgxntool#141 first. These tests exercise behavior that exists only on that branch, so merging this PR first would break CI for every other pgxntool-test PR until #141 lands.
check-tap-output-script.batsruns the script directly. It covers every failure rule, non-TAP pass-through,1..0, subtests, nonexistent arguments and--list.verify-results-pgtap-script.batscovers the plan checks, the extra-file arguments (thetest/installoutputs), and--pgtap-onlywith and without--for-test: a description-only TAP diff passes there and fails under plain pgtap, a non-TAP diff still fails, and a TAP test with a SQL error line fails.make-test.batscovers wiring only:pgtap-only's use ofverify-results-pgtap.sh --pgtap-only --for-test;PGXNTOOL_VERIFY_RESULTS_MODE=diffsskip and exit-status propagation, through a stub;PGXNTOOL_VERIFY_RESULTS_MODEvalidation for the new value;not okfailsmake test.test-test-install.batschecks, without Postgres, that_test-install-output-linkcreates the$(TESTOUT)/installsymlink for a separateTESTOUTand fails on a pre-existing directory there.pgxntool-variables.txtanddist-expected-files.txtare updated for the new variables and script.