Skip to content

feat(api): report the next scheduled collection in recommendations freshness - #1567

Merged
IAmJSD merged 2 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-756-next-collection
Oct 28, 2026
Merged

IAmJSD merged 2 commits into
LeanerCloud:mainfrom
IAmJSD:fix/issue-756-next-collection

Conversation

@IAmJSD

@IAmJSD IAmJSD commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator

#1389 added the first-run empty state, which shows the last collection time. The issue also asks for the next one, but the API had no field for it, and the server did not know its own schedule. Terraform gave the schedule to EventBridge, Cloud Scheduler, CronJobs or the Logic App, never to the app. This PR adds next_collection_at to GET /api/recommendations/freshness and wires the schedule through to the server.

Refs #756

Partial: this is the backend half. Showing the time on the empty-state card is a frontend change for the UI owner (see below).

Root cause

config.RecommendationsFreshness only had last_collected_at, last_collection_error, last_collection_started_at and last_collection_partial. The collection schedule lived only in Terraform (var.recommendation_schedule), so nothing at runtime could work out when the next run was due.

What changed

  • Terraform: every compute module that schedules collect_recommendations now sets RECOMMENDATION_SCHEDULE on the API container. The value is var.enable_scheduled_tasks ? var.recommendation_schedule : "", so it is empty whenever the scheduler resource is not created:
    • modules/compute/aws/lambda/main.tf and aws/fargate/main.tf (EventBridge rate()/cron(), UTC);
    • gcp/cloud-run/main.tf and gcp/gke/main.tf (Cloud Scheduler 5-field cron, time_zone = "UTC");
    • azure/aks/main.tf (local.app_env, shared with the CronJobs, 5-field cron, UTC).
    • azure/container-apps/main.tf is different. Its Logic App reads only the hour field and runs daily at <hour>:00 UTC (scheduled-tasks.tf), so the env var is "0 ${local.schedule_hour} * * *", the schedule that actually runs, not the raw cron.
  • New internal/schedexpr package (schedexpr.go and cron.go). Parse accepts rate(N unit), EventBridge cron(min hour dom month dow year) and 5-field cron. It supports *, ?, lists, ranges, steps and month/day names, and maps EventBridge day-of-week 1-7 (Sunday is 1) and classic 0-7 (Sunday is 0 or 7). Next(t) returns the first fire time strictly after t, in UTC, and searches up to 10 years ahead. The parser is strict. Anything it does not fully support is an error: L, W, #, @daily, backwards ranges, signed numbers, a wrong field count, or an EventBridge cron where both or neither day field is ?.
  • rate() always gives null. EventBridge counts a rate schedule from the moment the rule was created, and the server cannot see that. Using last_collected_at as the starting point would also be wrong, because manual refreshes and the 6h on-read background refresh (non-Lambda) move it. So the field reports "unknown" instead of guessing.
  • API: HandlerConfig.RecommendationSchedule is read from RECOMMENDATION_SCHEDULE in LoadApplicationConfig. It is passed to both NewHandler call sites (pre-connect and reinitializeAfterConnect) and parsed once per handler. An unparsable value logs one warning and is treated as unset. getRecommendationsFreshness sets NextCollectionAt from the handler clock. The value is computed, not stored, so there is no migration and no DB write. The store returns a new struct on every call, so setting the field does not race with other requests.
  • internal/api/openapi.yaml: documents next_collection_at (nullable date-time, always present, so it is listed in required) and when it is null. .env.example has a commented example.

Verification

Unit, mock and fixture evidence only. Nothing was deployed and no Terraform plan or apply was run.

Commands (with AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true, Go 1.26.9):

  • go build ./..., go vet ./..., go vet -tags integration ./...: clean.
  • go test -count=1 run one package at a time: ./internal/schedexpr/, ./internal/api/, ./internal/server/, ./internal/config/, ./terraform/modules/compute/, ./internal/mocks/. All ok.
  • golangci-lint run --allow-parallel-runners (v2.10.1) on internal/schedexpr, internal/api, internal/server, internal/config and terraform/modules/compute: 0 issues.
  • gocyclo -over 10 -ignore "_test\.go" on the changed non-test files: nothing over 10.
  • terraform fmt -check -recursive terraform/modules/compute: clean. terraform init -backend=false plus terraform validate on temp copies of aws/lambda, aws/fargate, gcp/cloud-run, gcp/gke, azure/aks and azure/container-apps: "Success! The configuration is valid." for all six.

New tests:

  • internal/schedexpr/schedexpr_test.go:
    • rate with minute(s), hour(s) and day(s): the period is parsed and Next reports unknown.
    • Invalid rate: zero, negative, +2, singular/plural mismatch, weeks, non-integer, malformed.
    • EventBridge cron with ? and *, 0/15, lists, names, numeric day-of-week (2 is Monday), the year field, 29 February (gives 2028), "strictly after", and a non-UTC input evaluated as UTC.
    • 5-field cron with */30, day-of-week 0 and 7 as Sunday, and OR matching when both day fields are restricted.
    • No fire time within the horizon (a past year, 31 February).
    • 29 invalid cron expressions.
  • internal/api/recommendations_next_collection_test.go: the freshness JSON gives "2026-10-28T02:00:00Z" for cron(0 2 * * ? *) and "2026-10-27T18:00:00Z" for 0 */6 * * * at a fixed clock. It gives null, with the key present, for unset, rate(1 day), unparsable and no-run-in-horizon schedules. NewHandler parses HandlerConfig.RecommendationSchedule.
  • internal/server/app_test.go: LoadApplicationConfig reads RECOMMENDATION_SCHEDULE, and NewApplicationFromDeps passes it to the API handler.
  • terraform/modules/compute/recommendation_schedule_env_test.go: each of the six modules has exactly one RECOMMENDATION_SCHEDULE entry with the expected gated expression. A shape test covers the map, ECS and AKS-list forms of the matcher.

Fail-before / pass-after:

  • origin/main (7b86e386) with a probe test calling getRecommendationsFreshness: body {"last_collected_at":null,...,"last_collection_partial":null} gives FAIL: response must carry "next_collection_at". On this branch the same probe passes with "next_collection_at":null.
  • origin/main with the new tests copied in: the api and server tests do not compile (undefined: parseCollectionSchedule, ApplicationConfig has no field ... RecommendationSchedule). TestRecommendationScheduleEnvWiring fails for all six modules with "want exactly one RECOMMENDATION_SCHEDULE env entry, got 0".
  • Mutation probes on this branch. Each was applied, the target test failed, and the code was restored:
    • dropping the handler assignment: TestFreshnessNextCollectionAt fails;
    • dropping the NewApplicationFromDeps wiring: TestNewApplicationFromDeps fails;
    • dropping the env read: TestLoadApplicationConfig fails;
    • EventBridge day-of-week off by one: TestNextEventBridgeCron fails;
    • forcing AND day matching: TestNextUnixCron fails;
    • making Next inclusive of t: TestNextEventBridgeCron fails;
    • loosening the "exactly one ?" rule: TestParseCronInvalid fails;
    • making rate return a time: TestParseRate fails;
    • removing the Lambda enable gate in Terraform: TestRecommendationScheduleEnvWiring fails.

Not covered by a test: the reinitializeAfterConnect call site (it needs a DB). It reads the same appConfig field the tested pre-connect path uses.

Combined-tree verification. This branch was tested together with the other queued branches on a local, never-pushed tree: queue/platform @ c00ef619 = origin/main 15dc8feb plus these heads, merged in this order:

Branch Tested head
fix/issue-712-fargate 2df9723
fix/issue-41-ri-exchange-wiring bb548fe
fix/issue-349-ci-hygiene 360ae6c
fix/issue-756-next-collection 2cb1b2f
fix/issue-36-go-clock aaea757
fix/issue-36-fe-waits 7feb01d
fix/issue-35-purchase-asserts fbafeb0
fix/issue-134-mfa-token 33da813

Linux, Go 1.26.9, AWS_CONFIG_FILE=/dev/null AWS_SHARED_CREDENTIALS_FILE=/dev/null AWS_EC2_METADATA_DISABLED=true; Docker Postgres via testcontainers. The full suite ran on 644bfcaf, which differs from this tree only by a test-only rename on fix/issue-134-mfa-token (a password-like constant that git-secrets flagged):

  • Passed: go build ./..., go vet ./..., go vet -tags integration ./... and the full go test -race -short -count=1 ./....
  • Static checks: golangci-lint v2.10.1 found 0 issues on every changed package, with and without --build-tags integration. gocyclo -over 10 is silent on the changed non-test files.
  • Integration tests, every changed package that has them, one package at a time (-tags integration -count=1): internal/api, internal/auth, internal/config, internal/database/postgres/migrations, internal/purchase and internal/server all pass. Migration 000152 applies on top of main's 000151.
  • Frontend: the full jest suite passes on this tree (175 suites, 4595 tests, 1 skipped).
  • Secrets: git-secrets, using the repo's scripts/setup-git-secrets.sh patterns in a fresh repo with an isolated HOME, found nothing in the changed files, and a planted fake key was flagged. scripts/test-git-secrets-allowlist.sh: 183 PASS, 0 FAIL.
  • Terraform .tftest.hcl files were not run on this tree (CI runs them). Nothing was applied to any cloud.

Left out of scope

  • Frontend: the empty-state card does not show the time yet (see below).
  • The AWS default schedule is rate(1 day), so a default AWS deployment reports next_collection_at: null. Fixing that means switching the AWS default to a cron such as cron(0 2 * * ? *) (02:00 UTC, the same as GCP and Azure). That moves the collection time of existing deployments, so it is an owner decision and is not changed here.
  • CloudFormation (cloudformation/stacks/CUDly/template.yaml) also schedules recommendation collection and does not set RECOMMENDATION_SCHEDULE yet. The brief covered Terraform only. A CFN follow-up would add the env var from the same schedule parameter.
  • next_collection_at is the next scheduled run only. Manual refreshes and the non-Lambda on-read background refresh (CUDLY_RECOMMENDATION_CACHE_TTL, default 6h) are not predicted.
  • Existing deployments report null until Terraform is applied again and the new env var appears.

Frontend follow-up for the UI owner

  • frontend/src/api/recommendations.ts: add next_collection_at?: string | null to RecommendationsFreshness.
  • frontend/src/first-run-empty-state.ts: next to the last-collection line (around line 52, formatDateTime(f.last_collected_at)), show "Next collection: <formatDateTime(next_collection_at)>" when the field is non-null. Show nothing when it is null, which is what every rate() schedule and disabled scheduling return. Hide it while a collection is running (last_collection_started_at set).
  • Optional: the same line in the freshness indicator on Opportunities.

Maintainer: please mirror the issue's triage labels onto this PR: priority/p2, severity/medium, urgency/this-quarter, impact/many, effort/m, type/bug, triaged.

🤖 Generated with Claude Code

IAmJSD and others added 2 commits October 27, 2026 23:42
…eshness

GET /api/recommendations/freshness now carries next_collection_at so the
first-run empty state can say when recommendations will next be
collected. The runtime did not know its own schedule, so every compute
module that schedules collect_recommendations (Lambda, Fargate, Cloud
Run, GKE, AKS, Container Apps) now passes it as RECOMMENDATION_SCHEDULE,
empty when the scheduler resource is not created.

A new internal/schedexpr package parses EventBridge rate()/cron() and
five-field cron and computes the next UTC fire time. rate() schedules
report null: EventBridge counts them from rule creation, which the
server cannot see. Unset, disabled or unparsable schedules also report
null. Container Apps passes the daily recurrence its Logic App really
runs (hour field only), not the raw cron.

Refs LeanerCloud#756

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…llection

Conflict in internal/server/app_test.go (TestLoadApplicationConfig): kept
both RECOMMENDATION_SCHEDULE (this branch) and main's
SCHEDULED_PURCHASE_MAX_OVERDUE (LeanerCloud#1549) in the env list, setters and
assertions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@IAmJSD IAmJSD added priority/p2 Backlog-worthy severity/medium Moderate harm triaged Item has been triaged urgency/this-quarter Within the quarter impact/many Affects most users effort/m Days type/bug Defect labels Oct 28, 2026
@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 15dc8fe and 2cb1b2f.

Files selected for processing: 20
  • .env.example
  • internal/api/handler.go
  • internal/api/handler_recommendations.go
  • internal/api/openapi.yaml
  • internal/api/recommendations_next_collection.go
  • internal/api/recommendations_next_collection_test.go
  • internal/api/types.go
  • internal/config/types.go
  • internal/schedexpr/cron.go
  • internal/schedexpr/schedexpr.go
  • internal/schedexpr/schedexpr_test.go
  • internal/server/app.go
  • internal/server/app_test.go
  • terraform/modules/compute/aws/fargate/main.tf
  • terraform/modules/compute/aws/lambda/main.tf
  • terraform/modules/compute/azure/aks/main.tf
  • terraform/modules/compute/azure/container-apps/main.tf
  • terraform/modules/compute/gcp/cloud-run/main.tf
  • terraform/modules/compute/gcp/gke/main.tf
  • terraform/modules/compute/recommendation_schedule_env_test.go
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@IAmJSD

IAmJSD commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Independent review (IAmJSD's agent, fresh context, given only the diff, the issue and the PR text): VERDICT: APPROVE 2cb1b2f7a2b3f59c9fb3565067b9f365779c1045

Blocking:

  • none

Non-blocking:

  • Default AWS deployments (Lambda, Fargate, terraform/environments/aws) use recommendation_schedule = "rate(1 day)". For them next_collection_at is always null until an operator switches to cron(...). The PR explains why, and that is the right call: the server cannot see the rule's creation phase. The upshot is that on the primary platform the empty-state card's "next collection" line will be blank by default. Changing the AWS default to e.g. cron(0 2 * * ? *) would fix that, but it is a product decision because it moves the run time for existing stacks (see the note to the owner).
  • CloudFormation (cloudformation/stacks/CUDly, ScheduledRule uses ExecutionFrequency) doesn't pass RECOMMENDATION_SCHEDULE, so CFN stacks also report null. The PR doesn't claim CFN, so this is a follow-up, not a bug.
  • azure/container-apps/main.tf: "0 ${local.schedule_hour} * * *" matches the Logic App's daily recurrence for a numeric hour. If someone sets a non-numeric hour field (e.g. 0 */6 * * *), the env var says "every 6 hours" while the Logic App's start_time is already malformed. That is a pre-existing limitation documented on the variable, so I'm only noting it.
  • Issue fix(frontend): first-run empty state says no rows match filters with no next step #756 is a frontend issue. This is the backend half, so Refs is correct. The card change still has to land on the UI branch.

Attacked:

  • internal/schedexpr adversarially, with a throwaway probe test (deleted afterwards). Checked:

    • a non-UTC input instant
    • EventBridge DOW 4 = Wednesday at 18:59:30Z giving the next week's Wednesday (strictly after)
    • MON-FRI, */5, 0 0 29 2 * giving 2028-02-29
    • classic OR semantics for 0 2 1 * MON
    • ? in five-field cron
    • L and cron(0 2 * * * *) being rejected

    All correct. Next is bounded: at most about 3,650 day steps, then hour and minute steps.

  • Terraform: every env var is gated on the same var.enable_scheduled_tasks as its scheduler resource (Lambda scheduled_recommendations, Fargate recommendations, GCP google_cloud_scheduler_job.recommendations with time_zone = "UTC", AKS sweep_tasks.collect_recommendations, Container Apps daily recurrence in UTC). terraform fmt -check is clean. terraform test with the mock provider passes on aws/fargate (1/1) and aws/lambda (15/15).

  • Race claim: PostgresStore.GetRecommendationsFreshness returns a fresh &RecommendationsFreshness{} per call, so setting the field is not shared state.

  • Ran go build ./... and go vet ./... with and without -tags integration (clean). Ran go test -race -short -count=1 on internal/schedexpr, internal/api, internal/server, internal/config and terraform/modules/compute (all ok). Ran -tags integration -race on internal/server (ok) and internal/config (ok). golangci-lint with and without --build-tags integration found 0 issues, and gocyclo -over 10 is silent.

  • Fail-before:

    • Removing the freshness.NextCollectionAt = ... assignment: TestFreshnessNextCollectionAt fails (both cron subtests).
    • Dropping the RECOMMENDATION_SCHEDULE env read: TestLoadApplicationConfig/custom_values fails.
    • Restored afterwards.

Body-stale:

  • none

Reviewer: rI1. Bar while the maintainers are away (set by IAmJSD as acting owner): one approving independent review on the head, two for money, auth and security paths.

@IAmJSD

IAmJSD commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Local verification at 2cb1b2f7a2b3f59c9fb3565067b9f365779c1045 merged onto main (15dc8feb), together with #1565, #1566, #1567, #1569 in that order (combined tree 189311ee, Linux, Go 1.26.10): go build ./..., go vet with and without -tags integration, the full go test -race -short -count=1 ./..., and go test -race -tags integration -count=1 on every changed package, against Docker Postgres. All pass. CI Success and pre-commit are green on this head, with 1 approving independent review(s) on it. Nothing was run against any cloud.

@IAmJSD
IAmJSD marked this pull request as ready for review October 28, 2026 04:46
@IAmJSD
IAmJSD requested a review from cristim as a code owner October 28, 2026 04:46
@IAmJSD
IAmJSD merged commit cd70634 into LeanerCloud:main Oct 28, 2026
41 checks passed
@IAmJSD

IAmJSD commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Merged by IAmJSD's agent at IAmJSD's request (IAmJSD is acting owner while the maintainers are away), under the bar IAmJSD set: 1 approving independent review(s) on 2cb1b2f7, green CI, and the combined-tree verification above. Squash-merged at exactly 2cb1b2f7a2b3f59c9fb3565067b9f365779c1045.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/m Days impact/many Affects most users priority/p2 Backlog-worthy severity/medium Moderate harm triaged Item has been triaged type/bug Defect urgency/this-quarter Within the quarter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant