Skip to content

fix(config/db): seed-group rollbacks, partition-safe 000027 dedupe, fail-closed plan providers, revocation_in_flight clear - #1268

Merged
IAmJSD merged 8 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-316
Oct 26, 2026
Merged

IAmJSD merged 8 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-316

Conversation

@IAmJSD

@IAmJSD IAmJSD commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR addresses the four 2026-09-02 audit findings bundled in the issue (A04-010 … A04-013). One part of A04-012 is deferred (normalizing the plan-services key delimiter, see below), so this is a partial fix. I re-checked each finding against current main (a14dc8f2), and all four still reproduced. git log --grep also turns up commit da731285, which mentions this issue number, but it is an old federation-test commit from before the renumbering and has nothing to do with these findings.

Refs #316

Per-finding status

Finding Status on main a14dc8f2 This PR
A04-010 000059 down deletes Standard Users Reproduces Fixed. Only removes ...0005 when it really is Purchaser
A04-010 (same hazard) 000096 down Partly: users_min_one_group already aborts the DO block atomically, so nothing is destroyed, but the error is a bare CHECK violation Hardened. Refuses up front with an error naming the migration and the group
A04-010 (same hazard) 000024 down Not a defect. 000024's down can only run after 000057's down, which drops users_min_one_group and restores users.role. An empty group_ids is then the legitimate pre-000024 state, so failing loud would block a valid rollback No change
A04-011 000027 dedupes by ctid across partitions Reproduces Fixed. Joins on (tableoid, ctid)
A04-012 DerivePlanProviders fails open Reproduces Fixed. Accepts provider:service keys, identifies legacy-keyed entries by their Provider field, and errors instead of returning an empty set
A04-012 key-delimiter normalization migration Reproduces: 000040 only rewrote colon keys, so API-written aws/savings-plans keys were never split Deferred, see "Out of scope"
A04-013 MarkPurchaseRevoked leaves revocation_in_flight set Reproduces Fixed. SET list now clears it, and backfill migration 000142 fixes rows already revoked

Root cause and fix

A04-010: seed-group down migrations

000057 seeds Standard Users at 00000000-0000-5000-8000-000000000005. 000059 tried to seed Purchaser at the same UUID, and its ON CONFLICT DO NOTHING made that a no-op (#942). 000064 then put Purchaser at ...0007 and owns that row from then on. Its own down is a no-op. Even so, 000059's down removed ...0005 from every user and deleted the group. That destroyed Standard Users, or aborted on users_min_one_group if a user had no other group.

  • 000059_seed_purchaser_group.down.sql: the membership strip and the delete now run only if the row at ...0005 is named Purchaser. If any user would be left with no group, the down raises a clear error instead. system_managed is still dropped. One thing stays as-is: 000059's admin backfill added ...0005 (Standard Users) to every admin, and those memberships can't be told apart from deliberate ones (documented in the file).
  • 000096_seed_ri_exchanger_group.down.sql: the same up-front refusal for RI-Exchanger-only users, inside the existing DO block.

A04-011: 000027 dedupe

savings_snapshots is partitioned by month, and ctid is a position inside a single partition's heap. Matching on ctid alone meant a duplicate at (0,1) in one partition also deleted the row at (0,1) in every other partition. The CTE now selects tableoid and the DELETE matches on (tableoid, ctid). Duplicates of one (id, timestamp) always sit in the same partition, so the tableoid tie-break added to the ORDER BY never changes which row is kept. This edits an already-applied up migration. It only affects fresh databases and the forced re-run that the file's header describes.

A04-012: fail-open provider derivation

DerivePlanProviders cut keys on / and silently skipped anything else. That included the aws:savings-plans-<type> keys 000040 wrote, which findServiceKey in internal/api already accepts as valid. An empty result means "no restriction" to both guards (Handler.validatePlanAccountProviders and PostgresStore.validatePlanAccountProvidersTx), so a colon-keyed AWS plan accepted an Azure account. Now each entry's provider is resolved like this:

  • Key prefix first: the text before the first / or : in the key.
  • Then the entry's own Provider field, when the key has no prefix. This covers legacy aliases kept under their original key by fix(api): preserve legacy service aliases on plan update #1123's singleton-alias edit path (e.g. "original-key"), and plans keyed "ec2" with Provider: "aws". That is the same field the legacy edit path (mergeRequestedServices) and plan step resolution go by. Main skipped such entries, so a legacy-keyed plan accepted an account of any provider.
  • An entry identified neither way is skipped. Skipping can only shrink the set, so the check never gets looser. TestPlanWriteBodiesAssociationCompatibility relies on this: it stores an unknown "future" services entry and expects account assignment to keep working.
  • Fail closed: the function returns ([]string, error). If the plan has services but none of them identifies a provider, it returns an error. The handler maps that to a logged 500 (no plan UUID in the response), and the store returns it as-is.

Behavior change from main for mixed plans: a plan with aws/ec2 plus a legacy-keyed Azure entry now derives {aws, azure} instead of {aws}. That matches what the plan actually buys. A key prefix still wins over a conflicting Provider field, as before.

The first commit on this branch failed closed whenever no key had a provider prefix. That broke the supported legacy-key plans (TestLegacyServiceB1S1RoutedLedger, …RoutedRefusals, …Migration127Replay). The second commit (acd60dd4) adds the Provider-field fallback.

A04-013: revocation_in_flight never cleared

Migration 000072, PurchaseHistoryRecord.RevocationInFlight and the interface doc all say a successful MarkPurchaseRevoked resets the flag, but the UPDATE didn't. MarkPurchaseRevoked now sets revocation_in_flight = false in the same guarded UPDATE (WHERE revoked_at IS NULL). The new 000142_purchase_history_clear_revoked_in_flight clears the flag on rows that are already revoked. It leaves unrevoked flagged rows alone, because those are the finalize sweep's queue. Its down is a documented no-op.

Concurrency: concurrent MarkPurchaseRevoked calls on one row (the handler's retry loop plus the finalize sweep) serialize on the row lock. The losers re-check revoked_at IS NULL, match nothing and take the idempotent path. The integration test runs 8 concurrent marks on one in-flight row and checks that exactly one stamp lands and the flag ends cleared. A late FlipPurchaseRevocationInFlight from a stale concurrent request can still re-set the flag on a row that is already revoked. That was already possible before this PR (see "Out of scope").

Migration number

Migration number: 000142 (renumbered from 000146 on 2026-10-26). main is at 000139. The reviewed, ready PRs take 000140-000145 and merge in this order: #1240 (000140) → #1302 (000141) → #1268 (000142) → #1287 (000143) → #1289 (000144) → #1116 (000145). golang-migrate skips a version below the database's current one, so merge this after #1302 and before #1287. #1239 (was 000140-000141), #1241 (was 000143) and #1253 (was 000144) are held on owner decisions and will be renumbered to start at 000146 when decided. The test uses previousMigrationVersion(t, 142).

Verification

All evidence below is local. "Integration" means real PostgreSQL in Docker, using the shared container (testhelpers.RequireMigratedDatabase). The handler tests are mock-store unit tests. Fail-before runs swapped in origin/main's production files (and an empty 000142 body), kept the new tests, and then restored the files.

export PATH=$HOME/.local/go/bin:$HOME/go/bin:$HOME/.local/bin:$PATH AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true
go build ./... ; go vet ./... ; go vet -tags integration ./...                      # all OK
go test -count=1 ./internal/config/ ./internal/api/ ./internal/database/postgres/migrations/ ./internal/mocks/   # one package at a time: all ok
golangci-lint run --allow-parallel-runners [--build-tags integration] ./internal/config/... ./internal/api/... ./internal/database/postgres/migrations/...   # 0 issues (both)
gocyclo -over 10 -ignore "_test\.go" <changed .go files>                          # none (validatePlanAccountProviders is at 10)
go test -tags integration -count=1 -timeout 20m -run 'TestMigration_SavingsSnapshotsPK_DedupeStaysInPartition|TestMigration_SeedPurchaserGroup_Down|TestMigration_SeedRIExchangerGroup_Down|TestMigration_000142_ClearRevokedInFlight|TestMigration_SeedRIExchangerGroup$' ./internal/database/postgres/migrations/
go test -tags integration -count=1 -timeout 20m -run 'TestMarkPurchaseRevoked_ClearsRevocationInFlight|TestSetPlanAccounts_ProviderCheckFailsClosed|TestPlanWriteBodiesAssociationCompatibility|TestIntegrationTestsShareOnePostgresContainer' ./internal/config/
go test -count=1 -run 'TestDerivePlanProviders' ./internal/config/ ; go test -count=1 -run 'TestSetPlanAccounts|TestValidatePlanAccountProviders|TestGetPlanForAccountProviderValidation' ./internal/api/
Test Before (main code) After
TestMigration_SeedPurchaserGroup_Down/Standard_Users_and_its_members_survive_the_rollback (integration) FAIL: users_min_one_group violation PASS
…/Standard_Users_is_not_silently_destroyed (integration) FAIL: Standard Users deleted PASS
…/a_Purchaser-only_user_stops_the_rollback_with_a_clear_error (integration) FAIL: bare CHECK error PASS
…/a_real_Purchaser_at_...0005_is_still_removed (integration, preservation) PASS PASS
TestMigration_SeedRIExchangerGroup_Down/an_RI-Exchanger-only_user_stops… (integration) FAIL: bare CHECK error PASS
TestMigration_SavingsSnapshotsPK_DedupeStaysInPartition (integration) FAIL: other partition's row deleted PASS
TestMigration_000142_ClearRevokedInFlight (integration) FAIL with a no-op body: stale flag kept PASS
TestMarkPurchaseRevoked_ClearsRevocationInFlight (integration, incl. 8-way concurrent marks) FAIL: flag still true, row still in partial index PASS
TestSetPlanAccounts_ProviderCheckFailsClosed (integration, store path: colon key, legacy key rejected and accepted, unidentifiable) FAIL: Azure account accepted PASS
TestSetPlanAccounts_ColonKeyedPlanStillChecksProvider, …_UnparseableServiceKeyFailsClosed, …_LegacyKeyedPlanUsesProviderField (unit, mock store) FAIL: no error PASS
TestDerivePlanProviders* (unit) does not compile (signature change) PASS
TestDerivePlanProviders_LegacyKeyUsesProviderField, TestSetPlanAccounts_LegacyKeyedPlanUsesProviderField, the legacy subtests of TestSetPlanAccounts_ProviderCheckFailsClosed FAIL on the first commit's parser (aaeab564, the regression) PASS

Regression check for the legacy-key plans, plus every test file that reaches SetPlanAccounts or createPlan (integration, run by name):

go test -tags integration -count=1 -timeout 60m -run 'TestLegacyServiceB1S1|PlanAccount|PlanProvider' ./internal/api/    # ok (39 top-level tests pass, incl. the three B1S1 tests)
go test -tags integration -count=1 -timeout 60m -run '^(<top-level tests of store_postgres_{delete_account,plan_writes,ramp_step}_integration_test.go>)$' ./internal/config/    # ok
go test -tags integration -count=1 -timeout 60m -run '^(<top-level tests of manual_plan_step, plan_step_cumulative, plan_step_retention, ramp_step_progress integration tests, fan_out_shape_test.go>)$' ./internal/purchase/    # ok
go test -tags integration -count=1 -timeout 60m -run '^(<top-level tests of handler_plans_{concurrency,legacy_version,managed_add_accounts,legacy_service_alias}_integration_test.go>)$' ./internal/api/    # ok
go test -count=1 ./internal/purchase/    # ok

I grepped internal/ for services fixtures whose keys have no provider prefix ("ec2", "original-key", "original-unusual-key", "one", "service"). Every one that reaches the provider check sets Provider and is covered by the runs above. The rest (mergeRequestedServices unit tests, plan_display_test.go, and the "aws|rds" key in recommendation_overrides_test.go, which is a different map type) never call DerivePlanProviders.

Combined-tree verification. This branch was tested together with the other queued branches on a local, never-pushed tree: queue/platform @ 936996ad = origin/main a14dc8f2 plus these heads, merged in this order:

Branch Tested head
fix/issue-337 ff3420c
fix/issue-169 7d75eb5
fix/issue-342 66b221f
fix/issue-310 5e11c21
fix/issue-326 5f0ea59
fix/issue-243 365e959
fix/issue-343 6c94036
fix/issue-312 d2848a8
fix/issue-316 acd60dd
fix/issue-339 75ea9b9
fix/issue-339-commitment-type 73bdb19
fix/issue-353 5d25fcd
fix/issue-313 3156bc4
fix/issue-357 53d9e58
fix/issue-368 0ba727d
fix/issue-419 61648c9
fix/issue-111 b04c372
fix/issue-686 0ad90b7
fix/issue-674 793095d
fix/issue-1055 4ec96f1
fix/issue-559 dfa9525
fix/issue-558 034eed4
fix/issue-874 1720539
fix/issue-666 1e120da
fix/issue-668 fb26345
fix/issue-568 ce04173
fix/issue-311 e54f250
fix/issue-177 0cbb88a
fix/issue-151 c21377c
fix/issue-166 41fb77b
fix/issue-27 2d67622
fix/issue-323 69288ae
fix/issue-59 36463e8
fix/issue-43 35e4069
fix/issue-158 6f76f8a
fix/issue-104 bb00491
fix/issue-55 971ef1b
fix/issue-361 89ac220
fix/issue-120 b458719
fix/issue-36 42bac6e
fix/issue-525 57c97dd
fix/issue-712 e5bcda0
fix/issue-72 6cc63ed

fix/issue-72 and fix/issue-312 conflict on one line in internal/api/handler_dashboard.go (both change the calculateCommitmentMetrics call). The tree used #72's side with #312's validated provider variable; whichever PR merges second takes that same resolution.

Linux, Go 1.26.9, AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true; Docker Postgres via testcontainers:

  • Passed at 936996ad: go build ./..., go vet ./..., go vet -tags integration ./... and the full go test -race -short -count=1 ./....
  • Static checks at 936996ad: golangci-lint v2.10.1 found 0 issues on every changed package, with and without --build-tags integration. tests/e2e is a separate module, so it was vetted under its own e2e tag. gocyclo -over 10 is silent on the changed non-test files.
  • Integration tests, every changed package, one package at a time (-tags integration -count=1, 60-minute local timeout because the VM was shared):
    • At 936996ad, these pass: internal/server, internal/credentials, internal/scheduler, internal/database/postgres/migrations, internal/purchase and internal/api.
    • internal/config passes except TestPostgresStoreDB_ResolveStaleRIExchange_ValidatesCloudCheck, which fails the same way on unmodified main (fixed by fix(config): compare resolve checked_at at the database's microsecond precision #1252).
    • These were run on the first build of this tree (26795376, same branches at earlier heads): internal/analytics, internal/auth, internal/commitmentopts, internal/database and cmd/lambda/clear-rate-limit all pass. The later fixes don't touch those packages.
  • What the combined run caught, each fixed on its own branch before this PR opened:
    • a compile break between fix/issue-104's test and fix/issue-326's httpToLambdaRequest signature change;
    • fix/issue-316 rejecting legacy-keyed plans;
    • fix/issue-120 versus main's global-filter test.
  • Secrets: git-secrets, using the repo's scripts/setup-git-secrets.sh patterns in a fresh repo with an isolated HOME, found nothing in the changed files, and a planted fake key was flagged. scripts/test-git-secrets-allowlist.sh: 137 PASS, 0 FAIL.
  • Nothing was applied to any cloud.

Out of scope / deferred

  • Normalizing the plan-services key delimiter (A04-012, second half). The audit suggests one data migration that rewrites every key to a single delimiter, including splitting the API-written aws/savings-plans umbrella keys that 000040 never touched. I left it out for three reasons. Open fix(plans): migrate stored pre-#608 Azure/GCP service ids to canonical slugs #1140 (000136_canonical_legacy_plan_service_ids) rewrites purchase_plans.services too and deliberately keeps keys unchanged because fix(api): preserve legacy service aliases on plan update #1123's edit path relies on them. What to do when a plan has both aws/x and aws:x is an owner call. And the savings-plans identifier cleanup belongs to umbrella Normalize 'savings-plans' vs 'savingsplans' identifier across frontend + backend #7, which the maintainer asked to keep open. The fail-closed fix above removes the security-relevant part without it.
  • FlipPurchaseRevocationInFlight has no revoked_at IS NULL guard, so a stale concurrent revoke request can re-flag a revoked row. Readers check revoked_at first, and that request's failed Return clears the flag again. Guarding it would change its not-found error semantics, so I left it unchanged.
  • 000059's admin backfill put every admin in Standard Users (...0005). Rolling back can't tell those memberships from deliberate ones, so they stay.

No frontend change needed.

Maintainer: please mirror the issue's triage labels on this PR: impact/internal, triaged, priority/p3, severity/low, urgency/eventually, effort/m, type/bug.

🤖 Generated with Claude Code

IAmJSD and others added 2 commits October 23, 2026 05:14
…config store

- 000059 down: only remove ...0005 when it really is Purchaser. 000057 gave
  that UUID to Standard Users, so the old down deleted Standard Users and
  stripped it from every member. Refuse with a clear error rather than leave
  a Purchaser-only user with no group. 000096 down gets the same up-front
  refusal instead of a bare users_min_one_group violation.
- 000027: dedupe savings_snapshots on (tableoid, ctid). ctid is per
  partition, so matching on it alone deleted unrelated rows of other
  partitions on a forced re-run.
- DerivePlanProviders: accept the "provider:service" keys 000040 wrote, and
  return an error instead of an empty provider set when no key names a
  provider, so both plan-account guards fail closed.
- MarkPurchaseRevoked clears revocation_in_flight, as 000072 documents;
  migration 000146 clears it on rows already revoked.

Real-Postgres tests for each, using the shared container. Normalizing the
plan-services key delimiter is deferred (see the PR).

Refs LeanerCloud#316

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first LeanerCloud#316 commit made DerivePlanProviders fail when no service key
had a provider prefix. That refused supported plans: a legacy alias kept
under its original key (LeanerCloud#1123's singleton-alias edit path, exercised by
TestLegacyServiceB1S1*) has a key like "original-key", and so do fixtures
keyed "ec2" with Provider "aws".

Each entry's provider now comes from its key prefix ("provider/service"
or "provider:service"), falling back to the entry's own Provider field,
which is what mergeRequestedServices and plan step resolution use. The
check fails closed only when no entry identifies a provider either way.

Refs LeanerCloud#316

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD IAmJSD added impact/internal Team-internal only triaged Item has been triaged priority/p3 Polish / idea / may never ship severity/low Minor harm urgency/eventually No deadline effort/m Days type/bug Defect labels Oct 23, 2026
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

IAmJSD and others added 4 commits October 25, 2026 11:52
Brings in main's CI fixes (LeanerCloud#1236, LeanerCloud#1252) and CI changes since, so this PR gets a fresh CI run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main's LeanerCloud#1400 added main_integration_test.go with a TestMain that stops
the shared PostgreSQL container after the run. This branch added an
identical TestMain to helpers_test.go, so after merging main the
integration build of the package declared TestMain twice and failed to
compile. Keep main's TestMain, which does the same teardown, and
restore helpers_test.go to main's version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

CI fix pushed. New head: 26270404.

Root cause. #1400 on main added internal/database/postgres/migrations/main_integration_test.go, which declares a TestMain that stops the shared PostgreSQL container after the run. This branch had added an identical TestMain to helpers_test.go. After merging main, the integration build of the package declared TestMain twice, which failed Lint and the migrations integration job with TestMain redeclared in this block.

Fix. Commit 873cc10c drops the branch's copy and keeps main's TestMain. Main's copy runs m.Run() and then testhelpers.TerminateSharedPostgres(), which is the same teardown the branch's version did, so the package now has one TestMain and the behaviour is unchanged. helpers_test.go is back to main's version. No tests or assertions changed. The afterTests hook pattern exists only in internal/api and internal/purchase, and nothing else needed to hook in here.

Other changes: I merged origin/main twice as plain merges with no conflicts.

Checks run locally on the new head:

  • go build ./..., go vet ./..., go vet -tags integration ./... and the tests/e2e vet all pass.
  • golangci-lint v2.10.1 reports 0 issues on internal/api, internal/config and internal/database/postgres/migrations, both with and without --build-tags integration.
  • gocyclo -over 10 on the changed non-test files reports nothing.
  • go test -race -short passes on all three packages.
  • go test -race -tags integration passes on internal/api and internal/config.
  • go test -race -tags integration on internal/database/postgres/migrations compiles again with a single TestMain, and the package passes, including this branch's TestMigration_SeedPurchaserGroup_Down, TestMigration_SeedRIExchangerGroup*, TestMigration_SavingsSnapshotsPK* and TestMigration_000146_ClearRevokedInFlight. On this heavily loaded host, some of main's 000136, 000137 and 000139 tests first failed only because their Postgres container did not start in time. I reran those tests until they passed.
  • The secrets scan of the changed files prints "clean; planted key flagged".
  • bash scripts/test-git-secrets-allowlist.sh: 137 PASS, 0 FAIL.

@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 26270404620960bac117412da2302a4e89038051

Blocking:

  • none

Non-blocking:

Attacked:

  • A04-012: checked every plan_accounts write path: setPlanAccountsTx is the only INSERT, and the handler callers are setPlanAccounts and createPlan. Probed DerivePlanProviders with empty, :x//x, colon, legacy and mixed keys. A skipped entry can only shrink the set, and an all-unidentified plan now errors instead of returning an empty "no restriction" set.
  • A04-013: checked that GetPurchaseHistoryInFlight and the revoke handler (which checks RevokedAt before RevocationInFlight) are unaffected by clearing the flag, and that the 000146 backfill leaves unrevoked flagged rows (the sweep's queue) alone. For 000059/000096, checked that users_min_one_group is in force when those downs run, and that the 000024 "not a defect" reasoning holds.
  • Ran: go build ./..., go vet ./..., go vet -tags integration ./..., and go test -race -short on config, api and migrations (ok). Also go test -race -tags integration on the whole internal/database/postgres/migrations package (ok, 407s, includes the up/down round trips) and on the targeted config tests (TestMarkPurchaseRevoked|TestSetPlanAccounts|TestPlanWriteBodiesAssociationCompatibility|InFlight|PlanProvider|PlanAccount, ok). golangci-lint v2.10.1 found 0 issues with and without --build-tags integration; gocyclo is silent.
  • Fail-before, each restored afterwards:
    • Dropped revocation_in_flight = false from MarkPurchaseRevoked: TestMarkPurchaseRevoked_ClearsRevocationInFlight fails in 2 subtests.
    • Restored main's fail-open parser body, keeping the new signature: 3 TestDerivePlanProviders_* and 3 TestSetPlanAccounts_* handler tests fail.
    • Checked out main's 000027.up and 000059.down: TestMigration_SavingsSnapshotsPK_DedupeStaysInPartition and 3 TestMigration_SeedPurchaserGroup_Down subtests fail.

Body-stale:

Reviewer: rB14. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 25, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 26270404620960bac117412da2302a4e89038051

Blocking:

  • none

Non-blocking:

Attacked:

  • A04-012 fail-closed (security): DerivePlanProviders has two callers, the handler validatePlanAccountProviders and the store validatePlanAccountProvidersTx. Both now fail on the error. Neither can see an empty set while the plan has services, so the old "empty = no restriction" fail-open is gone. Unidentifiable entries can only drop providers, never add one. A key prefix still wins over the Provider field. API-written keys and the Provider field both come from r.Provider (buildServiceConfig), so they can't disagree for new plans.
  • Migrations: 000059 down only acts when ...0005 is really named Purchaser, and it refuses before stranding a user. 000096 down refuses up front. 000027 now joins on (tableoid, ctid). golang-migrate has no checksums, so editing an applied up only affects fresh or forced re-runs, as the header says. 000146 is a narrow UPDATE behind the existing partial index, with a documented no-op down. During a rolling deploy, old pods still leave the flag set on rows they revoke, which is harmless because readers check revoked_at first.
  • Fail-before: I restored main's 000059 down, 000096 down and 000027 up, plus main's MarkPurchaseRevoked SET list. That fails TestMigration_SeedPurchaserGroup_Down (3 subtests), TestMigration_SeedRIExchangerGroup_Down/an_RI-Exchanger-only_user…, TestMigration_SavingsSnapshotsPK_DedupeStaysInPartition and TestMarkPurchaseRevoked_ClearsRevocationInFlight (sequential and 8-way concurrent). Files restored.
  • Ran go build ./..., go vet (plain and -tags integration) on api, config and migrations, go test -race -short -count=1 on all three, and golangci-lint with and without --build-tags integration (0 issues). Integration (testcontainers): the full ./internal/database/postgres/migrations/ package, plus -run 'Plan|Revok|Revoc' on internal/config and -run 'Plan' / 'TestLegacyServiceB1S1|PlanAccount|PlanProvider' on internal/api. All ok.

Body-stale:

  • "Main ends at 000134. Open PRs use 000135–000143, … fix/issue-47 takes 000144, … fix/issue-339 already has 000145": main is now at 000139, and 000140-000148 are held by open PRs (000145 by fix(analytics): label snapshot commitment types from provider and service #1302). 000146 is still this PR's reserved number, so no renumber is needed, but the paragraph is out of date.
  • "I re-checked each finding against current main (a14dc8f2)": main has moved since. The branch has main merged in and passes as above.

Reviewer: rB15. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@cristim

cristim commented Oct 26, 2026

Copy link
Copy Markdown
Member

Independent review 1 (cold, fresh context):

VERDICT: APPROVE 2627040

Findings (none blocking):

Claims checked against code and issue #316:

  • A04-010: 000059 down now only strips/deletes ...0005 when its name is Purchaser; 000064 up relocates any Purchaser at ...0005 to ...0007, so on real DBs the down is effectively a no-op beyond dropping system_managed. Matches the issue's suggested fix. 000096 down pre-check is correct (NULL group_ids excluded by COALESCE/ANY). 000024 left alone: PR's reasoning (users_min_one_group dropped by 000057 down first) is plausible; not refuted.
  • A04-011: (tableoid, ctid) join is correct on the partitioned parent; duplicates of (id,timestamp) share a partition, so the ORDER BY tie-break is inert. Fixed.
  • A04-012: DerivePlanProviders handles "/" and ":" prefixes, falls back to entry Provider, errors when services exist but none identify a provider. Both callers (handler 500 without plan UUID; store wrapped error via setPlanAccountsTx) fail closed. Key-normalization migration deferred; PR body says "Refs chore(config/db): low correctness findings from the 2026-09-02 audit #316", closingIssuesReferences is [] (correct for a partial fix). Labels mirror the issue.
  • A04-013: MarkPurchaseRevoked clears the flag in the same guarded UPDATE; the only Postgres implementation. 000146 clears only revoked rows, leaves the finalize sweep's queue (revoked_at IS NULL) intact; idempotent on re-run; down is a documented no-op (acceptable: data-only).

Attacked:

  • Ran at head (integration, testcontainers): TestMigration_SavingsSnapshotsPK*, SeedPurchaserGroup_Down (4 subtests), SeedRIExchangerGroup(_Down), 000146_ClearRevokedInFlight, TestMigrations_DownAllRoundTrip, TestMigrations_FullStackIdempotent; config: TestMarkPurchaseRevoked*, TestSetPlanAccounts*, TestPlanWriteBodiesAssociationCompatibility, TestLegacyService*; unit: TestDerivePlanProviders*, api TestSetPlanAccounts|TestValidatePlanAccountProviders|TestGetPlanForAccount*. All pass.
  • Migration numbering: listed all open PRs' migration dirs; no 000146 collision.
  • Mutation probes (scratch copy, each restored and verified byte-identical to HEAD):
    P1 000027 join back to ctid-only -> DedupeStaysInPartition FAILS (unrelated row deleted). Killed.
    P2 drop revocation_in_flight = false from MarkPurchaseRevoked -> ClearsRevocationInFlight (both subtests) FAIL. Killed.
    P3 000059 down without the name='Purchaser' guard -> 2 SeedPurchaserGroup_Down subtests FAIL. Killed.
    P4 000146 without revoked_at IS NOT NULL -> 000146 test FAILS. Killed.
    P5 drop Provider-field fallback -> TestDerivePlanProviders_LegacyKeyUsesProviderField FAILS. Killed.
    P6 colon prefix not recognised -> TestDerivePlanProviders_ColonKeys FAILS. Killed.
    P7 remove fail-closed error -> TestDerivePlanProviders_UnidentifiableEntries FAILS. Killed.
    P8 disable 000096 stranded-user pre-check -> test PASSES. Survived (see nit 1).
  • Security: no auth/tenant regressions found; change strictly narrows the plan/account provider check. No private findings.

@IAmJSD on the merge-order nit: once 000146 is on main, any of #1239/#1240/#1241/#1253/#1302 that still carries 000140-000145 will be skipped by golang-migrate on every DB already at 146, so whichever lands later needs renumbering above the highest on main. Happy to merge this one last among them, or have it renumbered, whichever you prefer.

@cristim

cristim commented Oct 26, 2026

Copy link
Copy Markdown
Member

Independent review 2 (claude-fleet, cold): APPROVE at 2627040

Blocking items: none.

Non-blocking items:

  1. Merge-order gate. internal/database/postgres/migrations/000146_purchase_history_clear_revoked_in_flight.up.sql:1. origin/main (9cdeb78) ends at 000139. Clone refs show open PRs holding 000140 and 000141 (fix(recommendations): key stored recs, rec ID and purchase identity on RDS az_config (#408) #1239), 000142 (perf(api): page recommendations from an ID-ordered keyset scan instead of the full list #1240) and 000143 (feat(plans): settle a failed-with-submission row so a managed plan can be deleted #1241). The PR body (line 50) says 000144 and 000145 are taken by unopened branches. golang-migrate never applies a lower version below the current one, so if 146 lands first, deployed DBs silently skip 140-145. Either merge this after them or renumber at merge time. The body's "main ends at 000134" is also stale.
  2. Test gap. 000059_seed_purchaser_group_test.go:131 and 000096_seed_ri_exchanger_group_test.go:166 assert that err contains "migration 0000NN down". golang-migrate echoes the whole migration SQL in its error, and that SQL includes the RAISE literal. Replacing IF v_stranded > 0 with IF false in either down still passes, because the bare users_min_one_group CHECK violation is raised instead (confirmed: the ERRTEXT log shows the CHECK error and the test PASSes). The "clear error" claim is therefore untested. The atomicity claim is tested. Fix: assert on "user(s) belong only to", or check that "users_min_one_group" is absent.
  3. Pre-existing, not introduced here. A full down to version 1 followed by up fails at 000059_seed_purchaser_group.up.sql:37. The cause is that 000064's down is a no-op (000064 down.sql:13), so its Purchaser at ...0007 survives and trips 000059's name guard. Head and base behave the same.
  4. Pre-existing. FlipPurchaseRevocationInFlight (store_postgres.go:2977-2982) is an unconditional UPDATE, not a CAS. Two concurrent revokes that both read revoked_at NULL can both call Azure Return. This PR does not widen that window (see the analysis below). The PR body lists it as out of scope.

Per-claim table (claim / implemented? / test fails without fix? / evidence):

  • A04-010 seed-group downs / yes (000059 down.sql:17-41, guarded DO; 000096 down.sql:14-23) / yes: on base, 059_Down and 096_Down fail with the users_min_one_group violation / probe P1 (drop the name guard) kills 2 subtests. The message half survives (item 2).
  • A04-011 000027 dedupe / yes (000027 up.sql:62,65,71, joins on tableoid+ctid) / yes: DedupeStaysInPartition and my multi-partition test fail on base / P4 (ASC) and P4b (ctid-only) both killed.
  • A04-012 fail-closed providers / yes (purchase_plan_providers.go:24-51; handler_accounts.go:1281-1287 maps to 500; store_postgres.go:4414-4417) / yes: 3 api unit tests and the store integration test fail on base / P2, P2b, P2c (fail open), P5 (no colon), P6 (field-first), P7 (400 instead of 500) and P8 (no TrimSpace) all killed.
  • A04-013 revocation_in_flight clear / yes (store_postgres.go:2950, guarded by :2952 revoked_at IS NULL; 000146 backfill) / yes: ClearsRevocationInFlight and 000146 tests fail on base / P3a (unconditional backfill), P3b (no clear) and P3c (drop the revoked_at guard) all killed.

Migration analysis:

  • 000146 up is a single UPDATE limited to revocation_in_flight AND revoked_at IS NOT NULL. It uses the partial index from 000072, needs no DDL, takes row locks only and is idempotent. The down is a documented no-op, which is acceptable because the flag means nothing once revoked_at is set (GetPurchaseHistoryInFlight at store_postgres.go:3025 filters revoked_at IS NULL). No CONCURRENTLY, and TestMigrationsAreTransactional passes.
  • 000027: this edits an already-shipped up migration. Deployed DBs never re-run it and already have the PK, so the fix only reaches fresh DBs and CUDLY_FORCE_MIGRATION_VERSION re-runs. The header says the prod audit found zero duplicates at first apply, so the old bug most likely deleted nothing in prod. That cannot be verified from here, and any loss would be unrecoverable either way. There is no checksum mechanism in migrate.go.
  • The kept row is the newest created_at, with ctid DESC breaking ties. Since timestamp is the partition key, duplicates of one (id, timestamp) share a partition, so the tableoid term does not change the choice. The PK (id, timestamp) includes the partition key. My test (dups in 2 of 3 partitions, a 3-way and a tied pair, unrelated rows at colliding ctids, same id at a different timestamp) keeps every unrelated row on head and loses rows on base.
  • 000059 and 000096 edit only the down files, so the fix reaches deployed DBs at rollback time. Each down is one DO block inside the per-migration transaction, so a refusal leaves no partial state; the tests verify the group and memberships are intact.

Caller enumeration for DerivePlanProviders (head):

  • api/handler_accounts.go:1281 validatePlanAccountProviders. It is called from handler_accounts.go:1388 (PUT /plans/:id/accounts) and handler_plans.go:207 (createPlan, which rolls back the plan row on error).
  • config/store_postgres.go:4414 validatePlanAccountProvidersTx, reached via store_postgres_plan_writes.go:66 setPlanAccountsTx, from SetPlanAccounts at store_postgres.go:4396. SetPlanAccounts is called from the two handlers above.
  • Nothing else calls it. No caller relied on fail-open for valid data: ServiceConfig.validateProvider (config/validation.go:476-483) requires a lowercase, non-empty provider, so the error path is reachable only for corrupt or legacy rows.
  • The refusal surfaces as 500 "failed to validate plan accounts" with no plan UUID in the response. The store path wraps it as a plain 500.
  • Behaviour widens for one case, as the body states: aws/ec2 plus a legacy-keyed Azure entry now accepts Azure accounts. That matches planstep/resolve.go:202, which buys by the Provider field.

Revocation race analysis:

  • The handler checks RevokedAt before RevocationInFlight (handler_purchases_revoke.go:193 and 206), so clearing the flag on a revoked row cannot reopen a revoke.
  • The clear sits in the same statement that sets revoked_at, behind revoked_at IS NULL, so it is CAS-guarded.
  • Concurrent marks serialize on the row lock (the PR's 8-worker test passes).
  • No purchase path reads the flag.
  • A stale Flip after a Mark behaves the same before and after this PR.

Commands and results (GOTOOLCHAIN=go1.26.10, credentials nulled, heavy runs under the lock):

  • go build ./... passes on head and on the merged tree.
  • go vet, with and without -tags integration, on api, config and migrations: clean on head and merged.
  • golangci-lint run --allow-parallel-runners on the 3 packages: 0 issues. gofmt -l: clean.
  • Unit tests for the 3 packages pass on head and merged.
  • Integration tests on head: migrations (full package) pass, apart from my first full-chain variant, which hit item 3. The config targeted set and api targeted set pass.
  • Integration tests on merged (git merge-tree --write-tree origin/main gives a clean tree, ccf2b4cf; main is 92 commits ahead of the merge-base): migrations full package, config targeted plus the full config integration suite, and api targeted all pass.
  • Fail-before (head tests copied into base): all 11 new or changed tests fail.
  • Full chain up from empty to 146, down to 1, fresh up, down to 64 and back up passes on head. On base, the same chain hits the identical 000059 error.
  • Git history: the merges d138c9e, b362714 and 2627040 have empty remerge-diffs. 873cc10 drops a duplicate TestMain.
  • Probes ran in the head export only, restored with cp and checked with cmp. Every PR file in the export matches the PR head blob, and my scratch test files have been removed.

Reviewed by an independent agent with no author context (second reviewer). Review only: nothing was pushed, and the GitHub review state is unchanged.

IAmJSD and others added 2 commits October 26, 2026 08:46
…n 000146 -> 000142

golang-migrate only applies versions above the database's current one,
so PRs must merge in migration-number order. 000140-000144 had been
taken by PRs that are now held on owner decisions (LeanerCloud#1239, LeanerCloud#1241,
LeanerCloud#1253). The ready PRs move to 000140-000145 so they can merge ahead of
them, and the held ones will take the next free numbers when decided.
Rename only: the SQL statements are unchanged, and the test version
constants and the comments naming the migration follow the new number.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Renumbered migration 000146 -> 000142 so it can merge ahead of PRs held on owner decisions (#1239, #1241, #1253, which will take the next free numbers when decided). Rename only; SQL unchanged.

New head: 7a662259. The branch first merged origin/main (2baf23d5); then one commit renames the up/down/test files and updates the test's version constant and the comments that name the number (including the self-referencing header comments in the .sql files; no statement changed).

The migration test's name follows the number too (TestMigration_000146_ClearRevokedInFlight is now TestMigration_000142_ClearRevokedInFlight), as do its fixture purchase IDs (ri-146-* is now ri-142-*). A first, parallel integration run hit testcontainers start failures caused by machine load. A serial re-run of api, config and migrations passed.

Merge order for the ready PRs: #1240 (000140) -> #1302 (000141) -> #1268 (000142) -> #1287 (000143) -> #1289 (000144) -> #1116 (000145). The PR body's migration-number paragraph is updated to match.

Checks on this head (local): build, vet (plain, integration, e2e), golangci-lint v2.10.1 with and without the integration tag and gocyclo -over 10 on the changed packages, go test -race -short and go test -race -tags integration on the changed packages (including the migrations package's up/down round trip and number guards), secrets scan and git-secrets allowlist test. The six renumbered branches were also merged onto origin/main in that order in one scratch tree, and go test -race -tags integration ./internal/database/postgres/migrations/ passed there, so the chain applies up and down cleanly together.

@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 7a6622594bd378d032b4ea98650e5f1a0f206786

Blocking:

  • none

Non-blocking:

Attacked: I checked each finding against issue #316 and traced both DerivePlanProviders callers (the handler's validatePlanAccountProviders and the store's validatePlanAccountProvidersTx). Both now fail closed on an error, and an unidentifiable entry only narrows the set. The key prefix still wins over the Provider field. I also checked that clearing revocation_in_flight in MarkPurchaseRevoked cannot hide a pending reconcile: the revoke handler checks revoked_at before RevocationInFlight, and the guarded WHERE revoked_at IS NULL serializes concurrent marks. 000142 touches only already-revoked rows, and its no-op down is justified. Fail-before probes, each reverted afterwards:

  • The old /-only parser without the empty-set error fails TestDerivePlanProviders_{ColonKeys,UnidentifiableEntries,LegacyKeyUsesProviderField}, TestSetPlanAccounts_{ColonKeyedPlanStillChecksProvider,LegacyKeyedPlanUsesProviderField,UnparseableServiceKeyFailsClosed} and the integration TestSetPlanAccounts_ProviderCheckFailsClosed (3 subtests).
  • Dropping the revocation_in_flight = false line fails TestMarkPurchaseRevoked_ClearsRevocationInFlight.
  • The merge-base 000027 up fails TestMigration_SavingsSnapshotsPK_DedupeStaysInPartition.
  • The merge-base 000059 down fails 3 subtests of TestMigration_SeedPurchaserGroup_Down.

Ran go build ./..., go vet (plain and integration) on api, config and database, go test -race -short -count=1 on config, migrations and api (ok), and go test -race -tags integration -count=1 on the full config, migrations and api packages (all ok).

Body-stale:

Reviewer: rE5. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 7a6622594bd378d032b4ea98650e5f1a0f206786

Blocking:

  • none

Non-blocking:

  • internal/config/purchase_plan_providers.go:47-52: the Provider-field fallback only trims whitespace, while the key prefix is used verbatim. A legacy entry stored with Provider: "AWS" would derive AWS and then reject every aws account. That fails closed, so it's not a security risk, and I found no writer that stores mixed case. Still, strings.ToLower would make the two sources agree.
  • internal/api/handler_accounts.go:1314-1319: an unidentifiable plan now returns a generic 500 on account assignment. That's the right fail-closed call, but the operator only sees "failed to validate plan accounts", and the log line has no hint about which plan to fix (on purpose, since it omits the plan UUID). A follow-up could log the plan's key count or the keys themselves (keys aren't identifiers) to make triage easier.
  • FlipPurchaseRevocationInFlight can still re-flag an already-revoked row. The body says so and leaves it out of scope. Readers check revoked_at first, so that's acceptable.

Attacked:

  • Provider derivation: traced both guards (Handler.validatePlanAccountProviders via setPlanAccounts and createPlan, and PostgresStore.validatePlanAccountProvidersTx under FOR SHARE). An empty set is still only reachable for a plan with no services. Skipping unknown entries can only shrink the set. A key prefix still wins over the field.
  • Revocation: checked the flag readers (revoke handler 207 path, finalize sweep revoked_at IS NULL filter). Clearing the flag on revoked rows can't hide pending reconciliation. 000142 touches only revoked_at IS NOT NULL rows.
  • Migrations: 000059 down is gated on name = 'Purchaser'. 000096 down refuses stranded users up front. 000027 now joins on (tableoid, ctid). 000142 > main's 000141. The no-op down is fine.
  • Fail-before (scratch, then restored): removed revocation_in_flight = false from MarkPurchaseRevoked, which made TestMarkPurchaseRevoked_ClearsRevocationInFlight FAIL. Restored main's 000059 down and 000027 up, which made 3 TestMigration_SeedPurchaserGroup_Down subtests and TestMigration_SavingsSnapshotsPK_DedupeStaysInPartition FAIL.
  • Ran at the head: go build ./..., go vet (with and without -tags integration) on config/api/migrations, go test -race -short -count=1 on those 3 packages (ok), and the targeted -race -tags integration tests (ok). On a local merge with origin/main (2449fba3, never pushed): build, vet, golangci-lint v2.10.1 with and without integration tags (0 issues), gocyclo -over 10 (silent), the full -race -tags integration ./internal/database/postgres/migrations/ (ok), the config revoke/plan-account integration tests (ok), and 119 api integration tests matching TestLegacyServiceB1S1|PlanAccount|PlanProvider|Revoke|LegacyServiceAlias|ManagedAddAccounts (ok).

Body-stale:

Reviewer: rE6. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Local verification at 7a6622594bd378d032b4ea98650e5f1a0f206786 merged onto main (dfad3e98), together with #1268, #1287 in that order (combined tree db750f37, Linux, Go 1.26.10): go build ./..., go vet with and without -tags integration, the full go test -race -short -count=1 ./..., and go test -race -tags integration -count=1 on every changed package, against Docker Postgres. All pass. CI Success and pre-commit are green on this head, with 2 approving independent review(s) on it. Nothing was run against any cloud.

@IAmJSD
IAmJSD marked this pull request as ready for review October 26, 2026 13:55
@IAmJSD
IAmJSD requested a review from cristim as a code owner October 26, 2026 13:55
@IAmJSD
IAmJSD merged commit 7595cef into LeanerCloud:main Oct 26, 2026
41 checks passed
@IAmJSD

IAmJSD commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

Merged by IAmJSD's agent at IAmJSD's request (IAmJSD is acting owner while the maintainers are away), under the bar IAmJSD set: 2 approving independent review(s) on 7a662259 (two required: money, auth or security path), green CI, and the combined-tree verification above. Squash-merged at exactly 7a6622594bd378d032b4ea98650e5f1a0f206786.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/internal Team-internal only priority/p3 Polish / idea / may never ship severity/low Minor harm triaged Item has been triaged type/bug Defect urgency/eventually No deadline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants