[IAmJSD]

Summary

The ladder compared an on-demand-equivalent (OD) target with existing Savings Plan coverage measured in commitment dollars. This PR converts existing and expiring coverage to OD dollars before Allocate. Following the maintainer's note on the issue, it also makes legacy NULL-basis and on_demand_equivalent tranches impossible to claim.

Closes #946

Units

• Commitment $/hr: what is bought and stored. The AWS provider's LayerState.ExistingUSDPerHour and ExpiringUSDPerHour are in this unit: SavingsPlan.Commitment for the SP layers and amortized RI cost for the RI layer.
• OD $/hr: on-demand-equivalent spend covered. Baseline, target, gap and the netting totals are in this unit.
• Rate r is in (0, 1] and means commitment dollars per OD dollar. od = commitment / r and commitment = od * r. This is the same rule go's allocate_units.go uses for tranches.

Root cause

executeLadderRun passed GetLayerStates straight to Allocate and assembleLadderPlan. go's LayerState doc says callers must convert these values to OD first. Existing coverage was therefore understated by a factor of r and the gap overstated. Example: a 2.5 commitment $/hr EC2 Instance SP at r = 0.5 really covers 5 OD $/hr. It was netted as 2.5, so against a target of 10 the run placed 7.5 OD (3.75 commitment) instead of 5 OD (2.5 commitment).

Fix

1. internal/server/ladder_existing_units.go (new): ladderLayerStatesOnDemand(states, rates) returns a converted copy of the states and never mutates the provider's map.
   • It converts each layer with the rate that executeLadderRun already fetched from GetCommitmentRates for the config's term and payment option. This is option 2 on the issue. DescribeSavingsPlans does not expose each plan's own rate, so plans bought at a different discount are approximated by today's rate.
   • The math is exact (big.Rat). When the quotient is not a float64, existing rounds up and expiring rounds down. Net coverage (existing - expiring) is therefore never understated, which errs toward under-buying.
   • A layer with no commitment (nil or 0) needs no rate. Nil stays nil, and Allocate still rejects it.
   • A layer with non-zero existing or expiring commitment and no rate fails the run (Errored, nothing persisted). Netting it 1:1 would reproduce this bug, and no value is a safe upper bound. Invalid rates (outside (0, 1], NaN, Inf) and negative, NaN or overflowing amounts also fail.
2. executeLadderRun calls it right after the rate fetch. The converted states feed Allocate and the plan, so ladder_runs.existing_usd_hr and plan JSON existing are now OD. That matches go's LadderPlan.ExistingUSDPerHour doc and the amend ceiling (LEAST(target, baseline*coverage) - existing), which already assumed OD. The ceiling therefore gets stricter, never looser.
3. Maintainer comment / known issue: ClaimDueLadderTranches (selectDueLadderTranchesTx) now selects only amount_basis = 'commitment' rows. The validated ladder_tranche_amount_provenance constraint guarantees those rows have a commitment_rate.
   • Legacy NULL-basis and on_demand_equivalent rows are never claimed. They stay netted while within LadderFireGrace. After that, the existing overdue sweep cancels them and the next run re-plans their gap in commitment dollars ("cancelled or re-planned", with no migration).
   • The filter is in SQL, before LIMIT ... FOR UPDATE SKIP LOCKED. Older legacy rows therefore cannot use up a claim's limit and starve a due commitment row.
   • Basis is never updated after insert, and the claim still runs under the per-config ladder lock, so it adds no new race.
   • The known_issues entry is marked resolved.

Visible behavior changes

• AWS accounts with active Compute SPs or Convertible RIs now get an Errored ladder run (logged as existing coverage: layer compute-sp: no commitment rate ...). Today the AWS provider supplies a rate only for EC2 Instance SP (fix(ladder): bump cloud-commitments-go and plan tranches in commitment dollars #976 already holds those layers for new purchases). Before this PR, such runs silently produced over-sized plans; now they produce no plan. Once go's provider returns Compute SP and RI rates, or option 3 (Cost Explorer covered OD cost) lands, these runs plan again. A softer alternative would be a persisted Hold-only run with this reason. I kept fail-loud to match fix(ladder): bump cloud-commitments-go and plan tranches in commitment dollars #976's "no capability → Errored" precedent, and I'm happy to switch if you prefer.
• existing_usd_hr on new runs is OD. Historical ladder_runs rows keep their commitment-dollar snapshot because no rate was stored to backfill from. If the latest run predates this change, the amend ceiling uses that older, smaller existing value until the next run writes an OD one.
• No money moves. The ladder executor still has no production caller.

Verification

All evidence comes from unit, fake-capability and local-PostgreSQL tests. Nothing ran against a real AWS account. Tests ran with AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null because this machine's default AWS profile is invalid.

Fail-before (on origin/main c8bdd46, with the new tests and no fix):

• go test ./internal/server/ -run 'LadderLayerStatesOnDemand|TestExecuteLadderRun_(Existing|Expiring)':
  • TestExecuteLadderRun_ExistingSavingsPlanCoverageIsOnDemandEquivalent: existing 2.5 (want 5), gap 7.5 OD (want 5), total_hourly_commit 3.75 (want 2.5), plan JSON existing 2.5 (want 5).
  • TestExecuteLadderRun_ExpiringCoverageIsOnDemandEquivalent: gap 8 (want 6), commit 4 (want 3).
  • TestExecuteLadderRun_ExistingCoverageWithoutRateIsErrored: no error (the run was netted 1:1).
• go test -tags integration ./internal/config/ -run TestClaimDueLadderTranches_NeverClaimsOnDemandOrUnknownBasisRows: the limit-1 claim took the older NULL row, 3 execution rows were created (want 1), and the NULL and OD rows were fired (want scheduled, then cancelled after grace).

Pass-after (commit 6af6413):

• Tests that pin the conversion: TestLadderLayerStatesOnDemand_PinsCommitmentToOnDemandConversion covers 5 @ 0.5 → 10, 3 @ 0.25 → 12, 7 @ 1 → 7, and expiring by the same rate. ..._RoundsTowardUnderBuying checks that at 1/0.3, existing is the nearest float at or above the exact value and expiring the nearest at or below. ..._DoesNotMutateInput, ..._EmptyLayersNeedNoRate and ..._RejectsUnconvertibleCoverage (8 cases) cover the rest, plus the three executeLadderRun tests above.
• Mutation probes, all killed by the tests above:
  • expiring rounded up;
  • existing rounded down;
  • a missing rate treated as 1:1;
  • multiply instead of divide;
  • expiring left unconverted;
  • dropping the claim filter (the fail-before run above).
• go build ./..., go vet ./..., go vet -tags integration on config/purchase/api/server, and make tidy-check are clean.
• go test -count=1 passes for ./internal/server/ ./internal/config/ ./internal/purchase/ ./internal/api/ ./internal/mocks/, and go test -race -run Ladder ./internal/server/ passes.
• go test -tags integration -count=1 ./internal/config/ (whole package) passes, as do -run Ladder ./internal/purchase/ and -run Ladder ./internal/api/. These ran on the repo's testcontainers PostgreSQL via Docker, with no skips.
• golangci-lint v2.10.1 (--allow-parallel-runners, with and without --build-tags integration) reports 0 issues.
• gocyclo -over 10 -ignore "_test\.go" on the changed non-test files is silent.
• scripts/gosec-hook.sh on the changed packages and markdownlint-cli 0.47.0 on the known_issues file are clean.
• git-secrets ran in a fresh temp repo with an isolated HOME and every changed file tracked. A planted AKIA key was flagged (rc 1); the clean scan returned rc 0. scripts/test-git-secrets-allowlist.sh reports 137 PASS, 0 FAIL.

Test fixture changes: the existing claim tests now seed commitment-basis rows at rate 1 through a new claimableTranche helper (or explicit basis fields in the purchase and api rigs), so their netting values are unchanged. TestLadderTrancheProvenance_InsertClaimAndReconcile now asserts that only the commitment row is claimed and fires the NULL and OD rows directly, so the fired-row reader still covers every basis. TestLadderAmendCeiling_CancelsOverdueBeforeCounting marks its overdue row as commitment, so the cancel, not the basis filter, is what keeps it unclaimed.

Out of scope / follow-ups

• Per-plan rates (option 1) or covered-OD from Cost Explorer (option 3) would remove the today's-rate approximation. Both need provider work in cloud-commitments-go.
• Compute SP and Convertible RI rates from the AWS provider: without them, accounts holding those commitments error (see above).
• go's sumSPHourlyCost sums EC2 Instance SPs from every region, while the EC2 Instance SP rate is regional. This is pre-existing and noted here only.
• Historical ladder_runs.existing_usd_hr rows are not backfilled.

Frontend follow-up for the UI owner

No change is required. The timeline chart already treats existing_usd_hr as the OD base. New runs now match that label, so only runs from before this PR show a commitment-dollar base. Optionally, the run snapshot text ("existing $X") could say "on-demand-equivalent".

Maintainer: please mirror the issue's triage labels on this PR: priority/p2, severity/medium, urgency/this-quarter, impact/few, effort/m, type/bug, triaged.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[cristim]

Independent review 2 (claude-fleet, cold): APPROVE 6af6413

Findings (none blocking on their own):

• [cross-PR, blocking for whichever merges second] internal/config/store_postgres_ladder_lifecycle.go:160 conflicts in meaning with open fix(ladder): floor tranche amounts to the stored scale instead of rounding them up #1199 (head f1bdefa).
  • git merge-tree of the two heads is clean.
  • On the stacked tree, fix(ladder): floor tranche amounts to the stored scale instead of rounding them up #1199's TestLadderTrancheAmount_RoundTripsProviderPrecisionContract fails with "[]" should have 19 item(s), but has 0. Its rows come from lifecycleTranche, which has a NULL basis, and this PR stops claiming those.
  • The fix(ladder): floor tranche amounts to the stored scale instead of rounding them up #1199 body also says it "pins" the legacy pass-through that this PR removes.
  • Whichever of the two merges second must seed commitment-basis rows (claimableTranche).
• [non-blocking, wording] store_postgres_ladder_lifecycle.go:104-109, known_issues/05_config_store_postgres.md:139 and the PR body all say NULL-basis rows "stay netted while within LadderFireGrace". That is wrong.
  • Netting fails loudly on a NULL basis (internal/config/ladder_provenance.go:60, "cannot net an unproven amount basis").
  • So a NULL-basis row that is scheduled within grace, or fired and unresolved, makes the run error.
  • The behaviour predates this PR and fails closed. Only the text needs fixing.
• [non-blocking, track before enabling the executor] internal/server/ladder_existing_units.go:24-28 converts existing coverage at today's rate for the config's term and payment (option 2). That is a known over-buy path.
  • Probe: 36 commitment/hr of 3yr All-Upfront SPs at a true rate of 0.45 covers 80 OD/hr.
  • With a 1yr No-Upfront config at r=0.72, it nets as 50 OD/hr. Against a target of 80, the run buys 21.6 commitment/hr when nothing is needed.
  • The PR's "errs toward under-buying" covers only ULP-level float rounding, not this.
  • The issue accepted this approximation, and it is strictly better than base, which nets 36 here.
• [non-blocking, disclosed] The AWS provider never returns a Compute SP or RI rate (go providers/aws ladder/commitment_rates.go:18-19, 45).
  • So every AWS account that holds a Compute SP or Convertible RI gets an Errored ladder run.
  • That fails closed, but the maintainer has to choose it over a Hold-only run.

Attacked:

• Formula, against go at the pinned versions (pkg 541e05e5fd29, providers/aws aeb8a6f7ff51):
  • pkg/ladder/allocate_units.go:9-13 defines od = commitment / r.
  • The AWS rate is spCost/coveredOD (recommendations/sp_commitment_rate.go:74). That puts it in (0, 1], so the direction is right.
  • The SP layer's Existing is SavingsPlan.Commitment (layer_states.go:122, interfaces.go:57), so the input is commitment $/hr.
  • There is no double conversion:
    • Allocate takes LayerStates as given (allocate.go:421-431).
    • It converts only in-flight rows, by their stored rate (allocate_units.go:84).
    • It converts the gap OD to commitment once (:114).
  • The only GetLayerStates/Allocate caller is handler_ladder.go:317/386.
  • The plan's existing and the amend ceiling (store_postgres_ladder_amend_ceiling.go:116) are both OD now.
• Table probe: run through executeLadderRun, asserting bought commitment ≤ (target - true net OD)·r, measured with big.Rat. All 6 cases pass at equality:

Case	Bought
40 @0.72	17.6
30/10 @0.6	28
13.37/2.11 @0.3	9.74
over target	0 tranches
r=1	30
0.05 @0.0001	0.03

• Rounding:
  • Existing rounds up and expiring rounds down. So net coverage is never understated, and expiring cannot exceed existing.
  • This is consistent with fix(ladder): floor tranche amounts to the stored scale instead of rounding them up #1199's floor at the purchase boundary: both only ever shrink what is bought.
• Legacy classification:
  • amount_basis = 'commitment' is a positive match, so NULL fails closed.
  • The provenance CHECK is validated in 000122.
  • The basis is never updated after insert. The only UPDATEs are at store_postgres_ladder.go:341, amend.go:105 and lifecycle.go:77/135.
  • A legacy row that has already fired keeps its execution and is untouched.
  • A scheduled OD row nets 1:1, is never claimed, and is cancelled by the sweep (store_postgres_ladder.go:341-347).
• Mutation probes on a scratch copy, each restored and confirmed identical with cmp. All were caught:
  • existing rounds down;
  • expiring rounds up;
  • Quo becomes Mul;
  • a missing rate treated as 1:1;
  • the zero/nil guard's && changed to ||;
  • the conversion result discarded in the handler;
  • a rate > 1 allowed;
  • the first overflow check removed;
  • the claim filter changed to IS DISTINCT FROM 'on_demand_equivalent', or deleted. Both fail NeverClaims, and the deletion also fails ProvenanceInsertClaim.

Verified locally, with GOTOOLCHAIN=go1.26.10 and no cloud credentials, under the heavy lock:

• Head:
  • go build ./... and go vet, with and without -tags integration: clean. gofmt: clean.
  • go test -count=1 on server, config, purchase and api: pass. -race -run Ladder on server: pass.
  • Integration on testcontainers Postgres: the whole config package passes, and -run Ladder passes on purchase, api and server.
• Fail-before, on base c8bdd46:
  • The new unit file does not compile.
  • The trimmed executeLadderRun tests FAIL: 2.5 vs 5, 7.5 vs 5, 3.75 vs 2.5, 8 vs 6, and no error where one is expected.
  • NeverClaims FAILS, because the NULL and OD rows get claimed.
• Merged with origin/main (merge-tree clean):
  • Build and vet: clean.
  • Unit tests: pass.
  • Integration -run 'Ladder|Claim' on config, purchase, api and server: pass. This includes main's new immediate and lock ladder tests.
• No migration.

Reviewed by an independent agent with no author context (second reviewer). Review only: nothing was pushed, and the GitHub review state is unchanged.

[cristim]

VERDICT: APPROVE 6af6413

Merge check: merge-tree vs origin/main 2ded648 is clean (tree 9f4c82b0). go build ./... and go vet -tags integration on config/server/purchase/api pass on the merged tree.

Findings:

• medium (disclosure, non-blocking) internal/server/ladder_existing_units.go:68-71 with providers/aws@aeb8a6f7 ladder/commitment_rates.go:39-41. AWS returns an EMPTY rate map whenever Cost Explorer has no EC2 Instance SP recommendation (ErrNoSPCommitmentRate), which is common for accounts already well covered by SPs. Any non-zero existing coverage then makes the run Errored, where main persisted a Hold-only run. The body discloses this only for Compute SP and RI. Failing the whole run is still correct, since an unconverted layer understates the total and over-buys other layers, and no money moves. The body should say that EC2-SP-only accounts can also error, and the owner should decide whether Errored or a persisted Hold is wanted.
• nit process: the single commit 6af6413 has 461 changed lines, over the 400 cap.
• nit handler_ladder.go:284 has an overlong rewrapped comment line.

Attacked:

• Units checked against go pkg@541e05e5. ladder/types.go:473-478 says callers must normalize to OD. allocate.go:38-40 and allocate_units.go:9-23 give od = commitment / rate with rate in (0,1]. The PR divides, validates (0,1] the same way, and uses exact big.Rat.
• Rounding: existing rounds up and expiring rounds down, so net coverage is not understated and expiring <= existing still holds for Allocate's consistency check (allocate.go:431).
• Double count and drops:
  • Fired, unresolved and completed-after-now tranches still net at their own stored rate. Unchanged.
  • Legacy NULL and OD rows still net within grace. cancelOverdueLadderTranchesTx (store_postgres_ladder.go:336,404) then cancels them.
  • scheduled->fired is only possible via ClaimDueLadderTranches (lifecycle.go:71), and the basis filter sits before LIMIT/SKIP LOCKED.
• Unit: ./internal/server and ./internal/config passed 1492 tests.
• Integration on the merged tree (-run Ladder, config/purchase/api/server): all ok. 83 config ladder tests passed, and 1 pre-existing env skip (LadderArming).
• Mutations, all killed:
  • M1, drop the conversion call: 3 executeLadderRun tests FAIL.
  • M2, expiring rounds up: RoundsTowardUnderBuying FAILS.
  • M3, skip the rate check when only existing is zero: RejectsUnconvertible FAILS.
  • M4, claim filter IS DISTINCT FROM 'on_demand_equivalent' (lets NULL through): NeverClaims integration FAILS.
• CI at 6af6413: Security Scanning, Build Docker Image and CI Success are red. govulncheck GO-2026-6809 is a stdlib advisory fixed in Go 1.26.10. That is the OLD base c8bdd46, and main is now on 1.26.10, so a rebase or re-run should clear it. Everything else is green.

@IAmJSD CI on this head is red only from the old merge base (govulncheck on go1.26.9, fixed on main by #1236). Please merge or rebase onto current main so CI runs on a fresh merge ref.

[cristim]

Merge-order note from review: when this PR and #1199 are both on one tree, TestLadderTrancheAmount_RoundTripsProviderPrecisionContract claims 0 of 19 rows, because its fixture uses lifecycleTranche (NULL basis). #1199 is queued to merge first, so @IAmJSD, when you next refresh this branch, please switch that fixture to claimableTranche. Separately, LadderAmountFloat has no production caller once both land, so it can go.