feat(standard-tests): build sources that require catalog, config and state

[Anatolii Yatsuk (tolik0)]

No connector changes behaviour until it pins the release. Once one does, the in-process suite builds Python sources that require catalog, config and state by itself, and the 7 sources of suite group 1 in airbytehq/airbyte#88940 can delete their integration_tests/standard_tests.py.

What

The in-process Standard Tests (SourceTestSuiteBase, used by airbyte-cdk connector test and by create_connector_test_suite()) now build Python sources the way their run.py does:

• create_connector() inspects the constructor (or factory) signature.
  • When it requires catalog, config or state, the suite passes the scenario's config (an empty dict when there is none) and None for catalog and state, as run.py passes them when it launches without a catalog or state.
  • An optional config also gets the scenario's config, when the scenario has one, since run.py always passes --config.
  • Any other required argument raises a TypeError that names it and says how to work around it.
• When the class accepts catalog, config or state (required or optional), its read is done by a source built from the read command's catalog, config and state, again as run.py builds it.

Why

The suite called the connector class with no arguments. Python sources built on YamlDeclarativeSource, FileBasedSource or ConcurrentSourceAdapter require catalog, config and state, so every in-process test of those connectors failed with a TypeError before reaching the connector. This blocks running the suite in connector CI (airbytehq/airbyte-internal-issues#17133) for source-asana, source-freshcaller, source-google-ads, source-google-drive, source-microsoft-sharepoint, source-salesforce and source-sftp-bulk.

Two kinds of source also need to be built with the command's own arguments:

• A file-based source takes its stream schemas from the catalog it is built with. Built without one, every read fails with a missing json_schema error.
• A declarative Python source with an optional config (source-facebook-pages) evaluates the config in its components when it is built. Built without one, check and read send requests with empty config values.

Verification

• poetry run pytest unit_tests/test/ -p no:cacheprovider: 795 passed, 9 skipped. The new unit tests cover:
  • a source built for the read command;
  • a source with an optional config, with and without a scenario config;
  • a source with no arguments, which keeps its own read.
• ruff format --check, ruff check and mypy airbyte_cdk/test are clean.
• Connector check: this branch was installed into the Poetry environments of source-facebook-pages, source-google-drive and source-asana, then the generated suite ran with no connector-side suite file.
  • source-facebook-pages passes (6 passed).
  • source-google-drive passes (9 passed).
  • source-asana now reaches the connector and fails only check, on its test credentials, which are known to be invalid (it already fails airbyte-cdk image test).

Related PRs

This PR is part of airbytehq/airbyte-internal-issues#17133. It waits for no monorepo PR, and none of them waits for it: the new connector CI step runs each connector's pinned CDK, and the monorepo suite files build these sources themselves until a connector pins a release with this change.

• ci(connectors): run the in-process Standard Tests for Python sources in connector CI airbyte#88945: the in-process Standard Tests step for Python sources in connector CI.
• test(connectors): add in-process Standard Test suites for Python sources the generated suite cannot run airbyte#88940: integration_tests/standard_tests.py suites for 14 sources. The 7 sources of its group 1 and source-facebook-pages (group 3) can delete theirs once they pin a release with this PR; the file-based sources of group 2 keep theirs, since FileBasedSource also requires a stream reader and a spec class.
• test(connectors): fix acceptance-test-config.yml entries for the in-process Standard Tests airbyte#88941, fix(source-tplcentral): allow additional properties in stream schemas airbyte#88942, fix(source-gcs): update airbyte-cdk to 7.36.0 airbyte#88943 and fix(source-zoho-crm, source-google-ads): fix two Docker read reds of the standard tests airbyte#88944: connector fixes for the same step and for the Docker read.

Migration

cdk-migrations.md gets a new section under "Upgrading to the next minor release (unreleased)". No action is needed for most connectors. Connectors that override create_connector() only to pass catalog, config and state can drop the override once they pin this release. The airbyte monorepo adds such overrides as integration_tests/standard_tests.py (airbytehq/airbyte#88940).

🤖 Generated with Claude Code

[github-actions]

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@tolik0/cdk/python-connector-test-support#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch tolik0/cdk/python-connector-test-support

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

• /autofix - Fixes most formatting and linting issues
• /poetry-lock - Updates poetry.lock file
• /test - Runs connector tests with the updated CDK
• /prerelease - Triggers a prerelease publish with default arguments
• /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
• /poe <command> - Runs any poe command in the CDK environment

📚 Show Repo Guidance

Helpful Resources

• CDK API Reference

📝 Edit this welcome message.

[github-actions]

PyTest Results (Fast)

5 759 tests  +11   5 728 ✅ +10   12m 54s ⏱️ +44s
    1 suites ± 0      31 💤 + 1 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 88a1d44. ± Comparison against base commit 2082c2d.

This pull request skips 1 test.

unit_tests.resources.source_pokeapi_w_components_py.integration_tests.test_airbyte_standards.TestSuiteSourcePokeAPI ‑ test_docker_image_spec_backward_compatibility

[github-actions]

PyTest Results (Full)

5 762 tests  +11   5 731 ✅ +10   13m 29s ⏱️ - 3m 31s
    1 suites ± 0      31 💤 + 1 
    1 files   ± 0       0 ❌ ± 0 

Results for commit 88a1d44. ± Comparison against base commit 2082c2d.

This pull request skips 1 test.

unit_tests.resources.source_pokeapi_w_components_py.integration_tests.test_airbyte_standards.TestSuiteSourcePokeAPI ‑ test_docker_image_spec_backward_compatibility