Skip to content

feat: [FME-17261]: add experiment noun (CRUD) - #258

Open
apetruccelli wants to merge 6 commits into
harness:mainfrom
apetruccelli:FME-17261-experiment-v2
Open

apetruccelli wants to merge 6 commits into
harness:mainfrom
apetruccelli:FME-17261-experiment-v2

Conversation

@apetruccelli

@apetruccelli apetruccelli commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Description

Note: stacked on top of #257 (still open) — this branch depends on the fme:owners field type introduced there, so the diff below includes #257's commits until it merges.

  • Ticket FME-17261 asks for the experiment noun in the FME CLI module with full CRUD support.
  • experiment noun, full CRUD: list, get, create, update, delete against v4 (/fme/api/v4/experiments).
    • Experiments are scoped by parent — --parent-type FEATURE_FLAG|AI_CONFIG plus --parent-name/--parent-id — and by --env.
    • create builds the request from --start-at/--end-at/--baseline-treatment/--comparison-treatment/--key-metric/--supporting-metric/--description/--hypothesis, or accepts -f experiment.json as a full alternative.
    • update narrows the get-then-patch body to UpdateExperimentRequest's accepted fields, keeping read-only fields (id, type, parent, environment, timestamps) out of the PATCH — v4 rejects unknown properties.
  • fme:metric_refs field type (modules/fme/metric_refs.go): keyMetrics/supportingMetrics read as [{id,name}] but write as plain metric id strings — an asymmetric read/write shape core's built-in set type can't handle since it assumes both shapes match. Normalize strips the display name down to a plain id string; Mutate adds/deletes/dedups on the normalized id list.
  • owners is fully mutable, reusing the fme:owners field type already wired onto feature_flag/segment in feat: [FME-17257]: add owners/tags field types, flag_sets, and file (-f) support for feature_flag and segment #257 — no new Go code needed. --add/--del owners.user:<email|id> and owners.group:<identifier> work at both create and update, since flags_builtin: set: true already enables add/del.
  • Not supported by v4 (confirmed at the DTO level, not a CLI gap): tags and a client-settable targeting rule are absent from CreateExperimentRequest/UpdateExperimentRequest. The FME web UI's experiment tagging goes through a separate legacy webconsole-bff path, not the public v4 API.

Commands

  • harness create experiment <name> --parent-type feature_flag --parent-name my-flag --env <env-id> --start-at 2026-01-01T00:00:00Z --end-at 2026-02-01T00:00:00Z --baseline-treatment off --comparison-treatment on --add owners.user:<email>
  • harness create experiment <name> -f experiment.json
  • harness list experiment --parent-type FEATURE_FLAG --parent-name my-flag [--env <env-id>] [--status ACTIVE]
  • harness get experiment <id>
  • harness update experiment <id> --set description="new desc" --add key_metrics.<metric-id> --add owners.user:<email> --del owners.user:<id>
  • harness delete experiment <id> --confirm

Testing

  • go build ./... and go test ./... pass.
  • Unit tests in pkg/specloader/fme_spec_test.go covering all 5 operations: TestFMESpec_ListExperiment, GetExperiment, CreateExperiment, UpdateExperiment, DeleteExperiment, plus TestFMESpec_UpdateExperiment_AddDelOwners.
  • Live-verified against qa0: create/list/get/update/delete round trip, owners add/del, key-metric add on update, read-only field exclusion from PATCH.
  • Live-verified that --org/--project scope flags and --format json|table (via the shared --format/--json flags) work for list/get experiment, per the ticket's acceptance criteria.

Acceptance Criteria

  • All 5 operations implemented (list, get, create, update, delete)
  • --org/--project scope flags work
  • --format json|table output works for list/get
  • Unit tests for each operation

🤖 Generated with Claude Code

apetruccelli and others added 6 commits September 29, 2026 10:35
…e_flag

Supersedes the object_set-based approach from cli#244/harness#245, which the team
decided against (see docs/mutation.md, merged via harness#247-harness#256). This wires
feature_flag's owners/tags collections into the new per-module field_type
handler framework instead.

- New modules/fme package, registered in main-harness.go: fme:owners and
  fme:tags handlers (Normalize/Mutate/Encode), following the Normalize ->
  Mutate (per --add/--del op, in CLI order) -> Encode lifecycle.
- feature_flag's owners/tags fields get mutable_path + field_type; the
  update command's update_body_pick now carries both collections so a
  mutation can preserve the other members (merge-PATCH replaces the whole
  array).
- Owners support add/del by "user:<email>", "user:<id>", or
  "group:<identifier>", matching v4's OwnerReferenceInput write shape and
  the email support added in Main #12882. Existing GROUP owners picked up
  from a GET cannot be re-encoded (v4's read side returns a group's name,
  never its identifier), so encodeOwners errors clearly instead of
  dropping the group or sending a broken request -- a known limitation
  until the server returns it (documented in docs/mutation.md).
- Tags support add/del by name; Encode strips the read-only id before
  writing, since v4's TagReferenceInput only accepts {name}.
- segment is intentionally untouched in this change; same treatment
  follows in a separate pass.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
v4's read shape carries a GROUP owner's identifier under "id" (the
same key USER owners use), not a separate "identifier" field —
confirmed live against qa0. encodeOwners required "identifier" and
had no fallback, so any flag with a GROUP owner became permanently
stuck on the next owners edit. Fall back to "id" when "identifier"
is absent, and match on it too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Follow-up to harness#257 — wires the same fme:owners/fme:tags field types
(no new Go code needed, they're already generic) into the segment
noun: mutable_path + field_type on tags/owners, and update_body_pick
widened to carry both collections so add/del preserves the other's
members under merge-PATCH.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…y for feature_flag and segment

- create/update feature_flag and create/update segment now accept file_body (-f) as an
  alternative to building the request from flags
- dropped required: true from --traffic-type (feature_flag, segment create) and
  --segment-type (segment create), matching metric's existing required-flag fix, so a
  file body alone can satisfy them; update segment keeps --segment-type required since it
  is a query-param used for routing, not a body field a file could supply
- added fme:flag_sets field type (modules/fme/flagsets.go) so
  --add/--del flag_sets.<id> can manage a feature flag definition's flag set
  associations, keyed by id per FlagSetReference

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- create/list/get/update/delete experiment against v4 (/fme/api/v4/experiments),
  scoped by parent-type (FEATURE_FLAG or AI_CONFIG) and parent-name/parent-id
- fme:metric_refs field type (modules/fme/metric_refs.go): keyMetrics/supportingMetrics
  read as [{id,name}] but write as plain metric id strings, which core's built-in
  "set" type can't handle since it assumes read/write shapes match
- owners mutable via --add/--del owners.user:<email|id> and owners.group:<identifier>,
  reusing the fme:owners field type already wired onto feature_flag/segment
- update narrows the get-then-patch body to UpdateExperimentRequest's accepted fields,
  keeping read-only fields (id/type/parent/environment/timestamps) out of the PATCH
- unit tests for all 5 operations plus owners add/del, in pkg/specloader/fme_spec_test.go

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…gs_builtin

BuiltinFlags has no Add field — set: true already enables --add/--del together.
The extra add: true broke YAML unmarshaling in CI's check:specs task.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant