FFLSDK-258: Add experimentation opt-out modes by leoromanovsky · Pull Request #4024 · DataDog/dd-sdk-android

leoromanovsky · 2026-10-21T16:16:50Z

What does this PR do?

Add immediate and future-experiments-only opt-out to FlagsClient.Builder. The OpenFeature provider uses the same configuration.

// Initialize Datadog and enable Flags first.
val provider = FlagsClient.Builder()
    .experimentationOptOutMode(ExperimentationOptOutMode.FUTURE_EXPERIMENTS_ONLY)
    .build()
    .asOpenFeatureProvider()
OpenFeatureAPI.setEvaluationContext(ImmutableContext(
    targetingKey = "subject-id",
    attributes = mapOf("dd_experimentation_opt_out_at" to Value.String("2026-10-21T12:00:00Z"))
))
OpenFeatureAPI.setProviderAndWait(provider)
val enabled = OpenFeatureAPI.getClient().getBooleanValue("new-feature", false)

Mode	Behavior with a valid timestamp
`DISABLED`, the default	Ignore the attribute.
`IMMEDIATE`	Exclude all experiment allocations.
`FUTURE_EXPERIMENTS_ONLY`	Retain allocations activated before the cutoff; exclude equal, later, or unknown activation.

Omit the timestamp for eligible subjects. Invalid enabled input returns INVALID_CONTEXT without an assignment request. Future timestamps take effect immediately.

Require acknowledgement of the effective policy and, for future mode, the exact cutoff. Bind caches to the mode and context. Enabled modes withhold older contexts during refresh, including when the timestamp is absent. Flag resolution waits for persistence once before acquiring the policy lock. Request cleanup uses a nonblocking snapshot.

A client-owned queue delivers callbacks on background workers. Each worker shuts down after its drain task is submitted. A blocked fetch cannot delay initialization timeout callbacks. Callback exceptions cannot prevent later delivery. Recoverable errors remain recoverable through OpenFeature.

Motivation

Applications need to exclude subjects from experimentation while retaining ordinary feature delivery. Experimentation and evaluation telemetry controls remain independent. Future mode uses allocation activation, not prior subject enrollment. Retained allocations still perform normal targeting and exposure logging.

The three granular tracking bridge methods reject snapshots from superseded contexts. Timestamp documentation describes the supported RFC3339 subset. Tests cover both modes, mode-only acknowledgement rejection, invalid-context mapping, and worker-start recovery.

Additional Notes

Both mobile implementations pass the 187-case shared precompute profile. The catalog still marks D006 as unresolved and fleet reuse as proposed. Backend acknowledgement, ordinary fallback, and activation delivery remain release prerequisites. The deployment-verification thread remains open.

Validation

Tested commit	Validation	Result
`DataDog/dd-sdk-android@d95597bc31e78c280547cb33e6c10f242df21e06`	Native integration: both modes, telemetry control, local HTTP intake, callback saturation, shutdown and startup recovery; API 36 arm64	3 passed; 8 values, 4 evaluation events, 2 retained exposures; 9/9 saturated callbacks and 4/4 recovery scenarios, including thread-start OutOfMemoryError, 0 workers after stop. Supplied assignments and explicit flush.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Make sure you discussed the feature or bugfix with the maintaining team in an Issue
Make sure each commit and the PR mention the Issue number (cf the CONTRIBUTING doc)

FFE reviewer selection

Reviewers: @typotter, @sameerank

Why: android — Adds Android Flags SDK experimentation opt-out modes (DISABLED/IMMEDIATE/FUTURE_EXPERIMENTS_ONLY) driven by a dd_experimentation_opt_out_at context attribute. Reviewers come from the matched pod.

_{🤖 Automated selection.}

Selection details

Initiative: fallback / android

Why this match: Adds Android Flags SDK experimentation opt-out modes (DISABLED/IMMEDIATE/FUTURE_EXPERIMENTS_ONLY) driven by a dd_experimentation_opt_out_at context attribute. The modes exclude experiment allocations from assignment and bind caches to the mode and context, with OpenFeature provider support and callback dispatch changes. This removes subjects from experiment assignment. It does not suppress evaluation/exposure event egress (the PR keeps telemetry controls independent, so ufc-compliance does not fit), and it does not implement layers, holdouts, bandits or any other registered project. The Android fallback for Kotlin flags/OpenFeature code applies.

@typotter — primary (client-android): Matched pod member. Existing review coverage preserved.
@sameerank — core (client-android): Matched pod member. Existing review coverage preserved.

The PR author was excluded. Existing reviewers and teams were preserved.

Debug or fix a problem

Workflow run · Bits Code session

Wrong project or pod? Clarify this PR’s purpose, or correct the reviewer registry and publish the change. Sheet edits take effect after publication.
Need another reviewer now? Request them in GitHub. This workflow only adds requests and keeps existing reviewers.
Stuck beyond five minutes or failed? Inspect the run’s failed step and available Bits session. After fixing the cause, rerun the workflow with this PR URL and dry_run=false. Unchanged completed runs are skipped. If the receipt says needs_reconciliation or a write outcome is uncertain, ask the workflow owner to reconcile GitHub and the receipt before retrying; do not delete the lock blindly.
Report a problem: Comment on the reviewer-routing skill PR with this PR URL, the workflow run link, and the expected initiative or reviewer correction.

Evidence and run details

✅ Complete — reviewer coverage verified. The workflow checked pending requests or submitted reviews on the evaluated head.

Generated asynchronously by the FFE reviewer-routing workflow.

Changed paths: features/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/ExperimentationOptOutMode.kt, features/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/evaluation/ExperimentationOptOut.kt, features/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/FlagsClient.kt, features/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/DatadogFlagsClient.kt, features/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/repository/net/PrecomputeMapper.kt, features/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/evaluation/EvaluationsManager.kt, features/dd-sdk-android-flags-openfeature/src/main/kotlin/com/datadog/android/flags/openfeature/DatadogFlagsProvider.kt

Evaluated head: 0a4f91eed3bdd0f1b2f8800e30dd72776e6d6e71

Registry revision: revision:1946aa2a-530e-487a-84a7-83aff44a1089

Pinned skill source: d214cfc65ae154b97d39af5cf32caf48aaec9bf0

Instructions: workflow-injected snapshot (no plugin loaded)

Instruction snapshot SHA-256: f23b2a88c13c0da90e2dd58246a14f257af91e2195c555039bd555b52e6f3516

Add native and OpenFeature opt-out configuration with timestamp validation. Require server acknowledgment and isolate cached assignments and pending contexts.

datadog-datadog-prod-us1-2 · 2026-10-21T16:33:02Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog retried 1 test - 1 passed on retry

🎯 Code Coverage (details)
• Patch Coverage: 86.21%
• Overall Coverage: 72.00% (+0.19%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 0a4f91e | Docs | View more details | Give us feedback!}

Bind assignment acknowledgements and persisted caches to the selected mode and exact subject cutoff. Cover request, restart, and stale-response behavior.

Deliver committed state changes and callbacks outside the policy lock. Read one assignment snapshot and retain the attempted subject for error telemetry.

chatgpt-codex-connector · 2026-10-21T19:38:31Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-27T12:33:09.957887Z	`0a4f91e`	New commits
🔒 Security Review	✅ Completed 2026-10-27T12:25:20.467661Z	`0a4f91e`	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f335b48a3d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-datadog-prod-us1-2

The most critical concern is that EvaluationContextCallback completion callbacks can be delivered on the UI thread — both via drain() executing on whichever thread calls it and via the direct onFailure call in the invalid-context path — violating the background-thread contract. A separate issue in DatadogFlagsClient.kt causes the React Native snapshot path to block for up to 200 ms (two sequential 100 ms waits) when persistence is loading and the opt-out policy is disabled.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit f335b48}

buraizu · 2026-10-21T23:16:29Z

Created DOCS-16087 for the editorial review.

Use Gregorian dates and explicit UTC bounds for opt-out timestamps. Derive request options from the effective policy and validate acknowledgements.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b7ab71778

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

leoromanovsky · 2026-10-23T02:20:07Z

+Invalid timestamps return application defaults with `INVALID_CONTEXT` and do not send an assignment request.
+
+The backend skips experiment allocations and continues through ordinary feature allocations.
+The SDK requires the backend to acknowledge exclusion before using the response.


🤖 Comment from Codex working with Leo:

Recorded the precompute acknowledgement and activation-metadata prerequisites in 9c18ede. Local checks confirm that Android rejects missing or mismatched acknowledgements and recovers after a valid response. These checks use supplied responses and do not confirm backend deployment. Release enablement still requires verified site coverage for the acknowledgement, fallback, and activation changes linked here.

This release gate remains open until site deployment is verified. No backend deployment was performed in this SDK work.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9c18edeca8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

linear-code · 2026-10-23T02:25:59Z

FFLSDK-258

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cda7955b63

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 347884bca8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e5fe6cf62

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

aarsilv

Thanks for iterating! 🚀

Recreate the develop merge without the unrelated master ancestry. Preserve the complete file tree from d95597b. Ref: FFLSDK-258

Add immediate experimentation opt-out

f9b4aa3

Add native and OpenFeature opt-out configuration with timestamp validation. Require server acknowledgment and isolate cached assignments and pending contexts.

leoromanovsky added 2 commits October 21, 2026 13:58

Merge develop into experimentation opt-out

087b9b7

Add future-experiments-only opt-out mode

29ea98c

Bind assignment acknowledgements and persisted caches to the selected mode and exact subject cutoff. Cover request, restart, and stale-response behavior.

leoromanovsky changed the title ~~Add immediate experimentation opt-out~~ Add experimentation opt-out modes Oct 21, 2026

Fix opt-out state delivery and evaluation snapshots

f335b48

Deliver committed state changes and callbacks outside the policy lock. Read one assignment snapshot and retain the attempted subject for error telemetry.

leoromanovsky marked this pull request as ready for review October 21, 2026 19:38

leoromanovsky requested review from a team as code owners October 21, 2026 19:38

chatgpt-codex-connector Bot reviewed Oct 21, 2026

View reviewed changes

Comment thread ...id-flags/src/main/kotlin/com/datadog/android/flags/internal/evaluation/EvaluationsManager.kt Outdated

datadog-datadog-prod-us1-2 Bot reviewed Oct 21, 2026

View reviewed changes

Comment thread ...oid-flags/src/main/kotlin/com/datadog/android/flags/internal/evaluation/EvaluationEffects.kt

Comment thread ...d-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/DatadogFlagsClient.kt Outdated

buraizu self-assigned this Oct 21, 2026

buraizu added the editorial review label Oct 21, 2026 — with ddtool CLI

aarsilv reviewed Oct 22, 2026

View reviewed changes

aarsilv mentioned this pull request Oct 22, 2026

[FFLSDK-258] Add experimentation opt-out modes for Flags clients DataDog/dd-sdk-ios#3338

Open

3 of 5 tasks

maciejburda requested a review from aleksandr-gringauz October 22, 2026 07:06

leoromanovsky added 2 commits October 22, 2026 10:43

Align experimentation opt-out timestamps and request policy

b82d984

Use Gregorian dates and explicit UTC bounds for opt-out timestamps. Derive request options from the effective policy and validate acknowledgements.

Check effective opt-out policy in native integration test

7b7ab71

chatgpt-codex-connector Bot reviewed Oct 22, 2026

View reviewed changes

Comment thread ...d-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/DatadogFlagsClient.kt Outdated

FFLSDK-258: Align experimentation opt-out with shared decisions

9c18ede

leoromanovsky commented Oct 23, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Oct 23, 2026

View reviewed changes

leoromanovsky changed the title ~~Add experimentation opt-out modes~~ FFLSDK-258: Add experimentation opt-out modes Oct 23, 2026

leoromanovsky added 2 commits October 22, 2026 22:27

FFLSDK-258: Match the callback listener parameter name

f15ab92

FFLSDK-258: Preserve callback delivery and shut down workers

cda7955

chatgpt-codex-connector Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread ...ures/dd-sdk-android-flags/src/main/kotlin/com/datadog/android/flags/internal/FlagsFeature.kt

FFLSDK-258: Warn on concurrent client mode mismatches

347884b

chatgpt-codex-connector Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread ...rc/main/kotlin/com/datadog/android/flags/internal/evaluation/EvaluationCallbackDispatcher.kt Outdated

Comment thread features/dd-sdk-android-flags/README.md Outdated

FFLSDK-258: Retry callback worker startup and await context in example

6e5fe6c

chatgpt-codex-connector Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread ...-flags/src/main/kotlin/com/datadog/android/flags/internal/repository/net/PrecomputeMapper.kt

leoromanovsky requested a review from aarsilv October 23, 2026 12:29

aarsilv reviewed Oct 25, 2026

View reviewed changes

leoromanovsky requested a review from aarsilv October 26, 2026 19:59

aarsilv approved these changes Oct 27, 2026

View reviewed changes

aarsilv mentioned this pull request Oct 27, 2026

Support per-subject experimentation opt-out DataDog/dd-trace-java#12950

Open

leoromanovsky requested review from greghuels, sameerank and typotter October 27, 2026 03:11

greghuels removed their request for review October 27, 2026 09:32

FFLSDK-258: Merge develop and preserve opt-out review fixes

0a4f91e

Recreate the develop merge without the unrelated master ancestry. Preserve the complete file tree from d95597b. Ref: FFLSDK-258

leoromanovsky force-pushed the leoromanovsky/experimentation-opt-out-immediate branch from d95597b to 0a4f91e Compare October 27, 2026 12:18

typotter approved these changes Oct 27, 2026

View reviewed changes

Conversation

leoromanovsky commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Additional Notes

Validation

Review checklist (to be filled by reviewers)

FFE reviewer selection

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Oct 21, 2026 • edited by datadog-prod-us1-5 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

buraizu commented Oct 21, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

leoromanovsky Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

linear-code Bot commented Oct 23, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aarsilv left a comment

Choose a reason for hiding this comment

Uh oh!

leoromanovsky commented Oct 21, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Oct 21, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

chatgpt-codex-connector Bot commented Oct 21, 2026 •

edited

Loading