Skip to content

ci(cache): config-server cache-backend integration-tests job - #498

Open
steiler wants to merge 6 commits into
cache-write-backendfrom
cache-ci-integration
Open

ci(cache): config-server cache-backend integration-tests job#498
steiler wants to merge 6 commits into
cache-write-backendfrom
cache-ci-integration

Conversation

@steiler

@steiler steiler commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Ticket 08 of the #460 + #471 PR-split reorg
(see graph.md/index.md in the tracking gist for the full stack). Base:
cache-write-backend (#497). Top of the stack.

Scope

Wires a real Cache.Type: config-server deployment into cicd.yml's integration-tests
job, exercising the read+write ConfigServerCache path built in tickets 05-07 end-to-end.

Cross-repo dependency

Pairs-with: sdcio/config-server#476
Pairs-with: sdcio/integration-tests#115

What shipped

  • reflection.Register(s.srv) on the gRPC server — enables grpcurl-/tooling-based
    introspection the integration suite relies on.
  • New integration-tests-cache-backend job in cicd.yml: a second single.yml
    invocation with cache_type: config-server, running 00-setup,01-crs,02-crud, 03-deviations,04-sensitive,05-cache-backend against the paired integration-tests
    branch (falling back to config-server-cache-backend instead of main when
    unresolved, so 05-cache-backend doesn't run against a checkout that lacks the suite).
  • New run-manifest job that records the resolved pairing/version inputs for both
    integration jobs (data-server/config-server/integration-tests refs+SHAs, schema/cache/
    cert-manager versions, suites_to_run) as a build artifact, for triage without
    re-deriving Pairs-with resolution by hand.

Topology decision

Kept the existing cache_type: local integration-tests job running unchanged,
rather than dropping it (source branch's 01b93e2). The two backends are independent
deployments launched by separate single.yml invocations/runners here, not one shared
cluster, so there's no forced-coexistence problem in this repo's CI — dropping the job
would only cost regression coverage on cache_type: local, which tickets 05-07 leave
untouched and still need to keep passing.

Adapted from the source branch

  • Cherry-picked/adapted e07bb92, b2c4ed7, 92a7d85, 96eee5b (job/manifest
    structure) and the server-side half of 71fab9d from origin/config-server-cache-backend.
  • Folded 71fab9d/5cb4100/00e5224's sensitive-suite back-and-forth directly into the
    final 04-sensitive suite entry on the config-server job, rather than replaying the
    intermediate integration-tests-sensitive-local job that 00e5224 immediately removed
    again — net effect only.
  • 01b93e2 (drop local-cache job) intentionally not applied — see topology decision
    above.
  • 69aab566/6c1bf7d8 (empty CI re-trigger commits) dropped, consistent with prior tickets.

Known issue found while re-verifying cross-repo pairing (2026-09-23)

Both paired PRs resolve MERGEABLE against their own bases, but neither base is main:

  • sdcio/integration-tests#115 bases on feat/sensitiveData (Bump google.golang.org/grpc from 1.61.0 to 1.62.0 #108), which is itself
    OPEN/MERGEABLE against main — this chain is fine.
  • sdcio/config-server#476 bases on the sensitive branch, which only has
    sdcio/config-server#441 open against it, and #441 is CONFLICTING against main.
    This is a config-server-repo problem (unrelated author/PR), not something this ticket's
    scope covers, but it means integration-tests-cache-backend's config-server side won't
    be reliably green until someone rebases config-server#441 (or #476 gets re-based
    directly onto main). Flagging for whoever owns that PR/repo before expecting this job
    to go green.

Verification

go build ./..., go vet ./..., go test ./... (full repo), and golangci-lint run ./... all pass. .github/workflows/cicd.yml validated with yaml.safe_load. Integration
CI itself won't be green until the config-server-side issue above is resolved upstream.

@steiler
steiler requested a review from a team as a code owner September 23, 2026 12:21
@steiler
steiler added this pull request to stack #494 September 23, 2026 12:32
This was referenced Sep 23, 2026
Open
Open
@steiler
steiler force-pushed the cache-ci-integration branch from 7505b87 to 4a40df6 Compare October 5, 2026 09:58
@steiler
steiler force-pushed the cache-ci-integration branch from 1d357f7 to 0e42e33 Compare October 14, 2026 08:18
@steiler
steiler removed this pull request from stack #494 October 14, 2026 08:20
@steiler
steiler added this pull request to stack #494 October 14, 2026 08:20
@steiler
steiler removed this pull request from stack #494 October 14, 2026 08:21
@steiler
steiler added this pull request to stack #528 October 14, 2026 08:21
@steiler
steiler force-pushed the cache-ci-integration branch from 0e42e33 to 4d2709c Compare October 14, 2026 09:06
steiler and others added 6 commits October 14, 2026 11:09
@steiler
netconf: record ADRs and CONTEXT.md for transport-error decisions
14fe42f 
- ADR 0001: normalize driver-library errors (scrapligo's
  util.ErrConnectionError) to io.EOF at the driver/scrapligo adapter
  boundary, not in nc.go, so isTransportError stays driver-agnostic.
- ADR 0002: deliberately exclude scrapligo's util.ErrTimeoutError from
  transport-error classification, to avoid config-server retrying
  transactions against slow-but-alive devices.
- CONTEXT.md: document the netconf target and note that ErrNotConnected
  is intentionally reused for both the pre-flight (never connected) and
  mid-flight (transport dropped during an in-progress RPC) scenarios.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
chore: retrigger CI to pick up Pairs-with tag
94caea3 
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
Delete pkg/datastore/target/netconf/docs/adr directory
735b9c4 
cleanup

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
feat(server): register gRPC reflection service
424aef4 
Enables grpcurl- and tooling-based introspection used by the
config-server cache-backend integration suite added in this ticket.

Adapted from 71fab9d on origin/config-server-cache-backend (split out
from that commit's cicd.yml change, which is folded into the next
commit instead).

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
ci: add config-server cache-backend integration-tests job
5b71320 
Runs a second single.yml invocation (cache_type: config-server)
alongside the existing cache_type: local job, exercising 00-setup,
01-crs, 02-crud, 03-deviations, 04-sensitive and the new
05-cache-backend suite against the ConfigServerCache read+write path
built in tickets 05-07. Pinned to the paired integration-tests branch
as a fallback ref so a plain 'main' resolution doesn't run the new
05-cache-backend suite against a checkout that doesn't have it yet.

Also adds a run-manifest job that records the resolved pairing/version
inputs (data-server, config-server, integration-tests, schema/cache/
cert-manager versions, suites_to_run) as a build artifact, so a
failing run can be triaged without re-deriving Pairs-with resolution
by hand.

Topology decision: kept the existing cache_type: local job running
unchanged rather than dropping it (see ticket 08's note on
01b93e2/'ci: drop local-cache integration-tests job from cicd.yml').
The two backends are independent deployments launched by separate
single.yml invocations/runners in this repo's CI, not a single shared
cluster, so there's no forced coexistence problem here -- dropping the
job would only lose regression coverage on cache_type: local, which
tickets 05-07 leave untouched and still need to keep working.

Adapted from origin/config-server-cache-backend's e07bb92, b2c4ed7,
92a7d85, 96eee5b (job/manifest structure) and the net effect of
71fab9d/5cb4100/00e5224's back-and-forth (folded directly into the
04-sensitive suite entry the config-server job ends with, rather than
replaying the intermediate sensitive-local job that immediately got
removed again). 01b93e2 (drop local-cache job) intentionally not
applied, per the topology decision above.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
ci: drop local-cache integration-tests job
e4bcefa 
Keep only the config-server cache backend integration job. Local cache is a separate snapshot-handling mode and is not part of the CI regression gate.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
steiler force-pushed the cache-ci-integration branch from 4d2709c to e4bcefa Compare October 14, 2026 09:09
@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/server/server.go 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alexandernorth alexandernorth alexandernorth approved these changes

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@steiler @alexandernorth