Skip to content

test(ffe): add direct dependent flags coverage - #8190

Open
typotter wants to merge 7 commits into
mainfrom
typo/dependent-flags-direct-system-tests
Open

typotter wants to merge 7 commits into
mainfrom
typo/dependent-flags-direct-system-tests

Conversation

@typotter

@typotter typotter commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Add direct dependent-flags evaluation coverage to the existing parametric FFE suite while keeping SDK activation and tracer branch targeting in a separate stacked integration PR.

Changes

  • add an explicitly named test_ffe_dependent_flags method beside the existing dynamic-evaluation test
  • keep its UFC and cases isolated under tests/parametric/test_ffe/dependent-flags/
  • assert the expected value, reason, and errorCode fields provided by each case, while rejecting unexpected errors from successful evaluations
  • mark the method as missing_feature (FFL-1694) for every server SDK
  • leave existing root fixture discovery and test_ffe_flag_evaluation unchanged

The fixtures are an exact copy of evaluation-suites/dependent-flags from DataDog/ffe-system-test-data commit f20ab80 (the merge of PR #45).

SDK activation, Node tracer artifact selection, and target-branch CI execution are intentionally outside this harness PR and belong in the stacked activation PR.

Telemetry remains outside this system-test assertion. Exposure and flag-evaluation telemetry are covered in the SDK repository.

Validation

  • ./format.sh
  • verified every server SDK manifest declares test_ffe_dependent_flags as missing_feature (FFL-1694)

Run isolated dependent-flag fixtures through the existing parametric FFE evaluation path while keeping every SDK disabled until its supporting release is known.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
@github-actions

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

tests/parametric/test_ffe/dependent-flags/cases/dependent-flags.json    @DataDog/feature-flagging-and-experimentation-sdk
tests/parametric/test_ffe/dependent-flags/ufc-config.json               @DataDog/feature-flagging-and-experimentation-sdk
manifests/dotnet.yml                                                    @DataDog/system-tests-reviewers
manifests/golang.yml                                                    @DataDog/system-tests-reviewers
manifests/java.yml                                                      @DataDog/system-tests-reviewers
manifests/nodejs.yml                                                    @DataDog/system-tests-reviewers
manifests/php.yml                                                       @DataDog/system-tests-reviewers
manifests/python.yml                                                    @DataDog/system-tests-reviewers
manifests/ruby.yml                                                      @DataDog/system-tests-reviewers
manifests/rust.yml                                                      @DataDog/system-tests-reviewers
tests/parametric/test_ffe/test_dynamic_evaluation.py                    @DataDog/feature-flagging-and-experimentation-sdk

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Pipelines  Tests

⏳ PR checks are still running. This summary will update when all results arrive.

🎉 All green!

🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 246c918 | Docs | View more details | Give us feedback!

Enable the direct dependent-flags method for the 7.0.0-pre SDK integration branch carrying the pinned WIP evaluator bundle.

Environment: Datadog workspace

Co-Authored-By: Codex GPT-5 <noreply@localhost>
@typotter typotter changed the title test(ffe): add direct dependent flags coverage [nodejs@dependent-flags-variant-key-node-integration] test(ffe): add direct dependent flags coverage [nodejs@typo/dependent-flags-evaluator-system-tests] Oct 27, 2026
@typotter typotter changed the title test(ffe): add direct dependent flags coverage [nodejs@typo/dependent-flags-evaluator-system-tests] test(ffe): add direct dependent flags coverage Oct 27, 2026
…direct-system-tests

# Conflicts:
#	manifests/golang.yml
#	manifests/java.yml
#	manifests/nodejs.yml
#	manifests/php.yml
#	manifests/python.yml
#	manifests/ruby.yml
@typotter
typotter marked this pull request as ready for review October 27, 2026 21:02
@typotter
typotter requested review from a team as code owners October 27, 2026 21:02
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T21:05:05.713756Z ebbcce6 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@typotter
typotter requested a review from hhan2024 October 27, 2026 21:03
@typotter typotter changed the title test(ffe): add direct dependent flags coverage test(ffe): add direct dependent flags coverage [nodejs@typo/dependent-flags-evaluator-system-tests] Oct 27, 2026
Run against the Node tracer artifact branch containing openfeature-js-client PR #437 at df51d65f0d89dc192c9c96cf7ac000dcf9ac1f43.
Keep the direct dependent-flags harness disabled for every SDK so activation and tracer branch targeting remain in the stacked integration PR.
@typotter typotter changed the title test(ffe): add direct dependent flags coverage [nodejs@typo/dependent-flags-evaluator-system-tests] test(ffe): add direct dependent flags coverage Oct 28, 2026
Trigger system-tests CI after removing the Node target-branch selector from the pull request title.
@typotter
typotter enabled auto-merge (squash) October 28, 2026 16:37

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants