Skip to content

feat(cache): config-server cache read backend - #496

Open
steiler wants to merge 13 commits into
cache-interface-prepfrom
cache-read-backend
Open

steiler wants to merge 13 commits into
cache-interface-prepfrom
cache-read-backend

Conversation

@steiler

@steiler steiler commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Ticket 06 of the #460 + #471 PR-split reorg
(see graph.md/index.md in the tracking gist for the full stack). Base:
cache-interface-prep (#495).

Scope

Read-only ConfigServerCache (Cache.Type: config-server): local-read seam,
fake for tests, real gRPC client, and wiring into Server.createCacheClient.
No writes — IntentWriter stays a no-op (noopIntentWriter, composed via
NewConfigServerClient) until the next ticket in the stack.

Cross-repo dependency

Depends on sdcio/sdc-protos#134, a fresh rebase of the dangling
config-server-cache-backend branch's config_read.proto/ConfigReadService
commit onto the tip of sdcio/sdc-protos#123 (the sensitive-flag rebase
sdcio/data-server#493 already pinned to). go.mod is pinned to that
commit; swap for a tagged sdc-protos release once #123/#134 merge.

What shipped

  • configserver.LocalConfigReader seam (Get/List) + FakeLocalConfigReader
    for tests, Document/Target/ConfigBlob types per the ADR's field-mapping
    table.
  • ConfigServerCache (pkg/cache/configserver.go): real-intent reads go
    through the seam and wrap into importer.IntentAdapter; writes are
    unconditional no-ops; running is served from its own independent
    in-memory, per-instance store, entirely separate from the seam.
  • GRPCConfigReader: real seam implementation over the generated
    config_read.ConfigReadServiceClient, dialed by
    Server.createConfigServerCacheClient.
  • Cache.Type: config-server now requires only Address (Namespace config
    field deleted) — ConfigServerCache derives the per-call target
    namespace/name from the datastore name itself (splitDatastoreName),
    correctly serving datastores across multiple Kubernetes namespaces from one
    process.
  • Intent names are emitted GVKNSN-style (<namespace>.<name>), matching how
    config-server keys TransactionSet owners, with a lookup-side strip
    (lookupConfigName) so Get/Exists still work off either form.
  • mergeConfigBlobs now handles config-server's root-path ("/") blob shape
    used by LoadAllButRunningIntents/revert-after-sync.

Adapted from the source branch

  • Re-derived against this ticket's actual, narrower IntentAdapter/
    ImportConfigAdapter split and noopIntentWriter composition (both from
    the prior cache-interface-prep ticket), rather than the single
    ImportConfigAdapter shape / concrete-ConfigServerCache-implements-
    `IntentWriter shape some of the original commits were written against.
  • Per the prior ticket's follow-up note: InstanceCreate seeds running
    with an empty &tree_persist.Intent{} (not nil) from the start, and
    InstanceRunningGet has no ErrRunningNotFound/nil-guard — built in
    correctly rather than reintroduced-then-fixed.
  • Dropped two late "rename"/"colocate" commits from the source branch
    (706ecb6, 2e8ffc6): both rename ConfigReadService → ConfigSnapshotService
    and touch a pkg/cache/configsnapshot/ package that belongs to the next
    ticket's write-backend work (ConfigSnapshotService gains Modify/Delete
    there) — premature here, where the service is still read-only
    ConfigReadService.
  • Dropped 8bebc69 (GetIntent redaction-scope doc comment): stale relative to
    the already-shipped always-union hardening from the sensitive-redaction
    ticket; the current comments already describe that behavior correctly.
  • Dropped .scratch/ planning artifacts and doc-only "mark ticket done"
    commits, consistent with prior tickets in this stack.

Verification

go build ./..., go vet ./..., go test ./... (full repo), and
golangci-lint run ./... all pass.

@steiler
steiler requested a review from a team as a code owner September 23, 2026 10:45
@steiler
steiler added this pull request to stack #494 September 23, 2026 10:53
@steiler
steiler force-pushed the cache-read-backend branch from 128faa7 to 4258422 Compare October 5, 2026 09:58

@alexandernorth alexandernorth left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, one minor suggestion

Comment thread pkg/cache/configserver/fake.go Outdated
@steiler
steiler force-pushed the cache-read-backend branch from 1ef4e22 to 518f4d1 Compare October 13, 2026 07:32
@steiler
steiler force-pushed the cache-read-backend branch from 25487e3 to 3846890 Compare October 13, 2026 09:44
@steiler
steiler force-pushed the cache-read-backend branch 2 times, most recently from d12d108 to 533ae01 Compare October 14, 2026 07:59
@steiler
steiler force-pushed the cache-read-backend branch from 533ae01 to dc979ab Compare October 14, 2026 08:18
@steiler
steiler removed this pull request from stack #494 October 14, 2026 08:20
@steiler
steiler added this pull request to stack #494 October 14, 2026 08:20
@steiler
steiler removed this pull request from stack #494 October 14, 2026 08:21
@steiler
steiler added this pull request to stack #528 October 14, 2026 08:21
steiler and others added 12 commits October 14, 2026 11:06
Pins to sdcio/sdc-protos@f894efb (feat(config-read): define local
ConfigReadService for config-server Config reads), the sdc-protos#134
rebase of the dangling config-server-cache-backend branch's a8f3da0
onto the now-merged-ready #123 tip (fd8bd02).

Swap for a tagged sdc-protos release once #123 and #134 merge upstream.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Defines LocalConfigReader (Get-by-name / List-by-target), the Target,
Document, and ConfigBlob types per the ADR's field-mapping table, and
an in-memory FakeLocalConfigReader seedable with fixtures. Lets the
config-server-backed cache.Client (ticket 06) be built and unit-tested
now, decoupled from the real cross-repo gRPC transport (ticket 01/07/08).

Ticket 05 of docs/adr/0001-config-server-backed-cache-client.md; no
existing file touched.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
…fake

Builds ConfigServerCache (pkg/cache/configserver.go), the new cache.Client
for Cache.Type: config-server, behind ticket 05's LocalConfigReader seam:
real-intent reads map onto the seam's Get/List and wrap results as
importer.IntentAdapter via a new configserver.NewImportAdapter
(pkg/cache/configserver/importer.go, merge.go) per the ADR's field-mapping
table; writes on real intents are unconditional no-ops; "running" is served
from its own independent in-memory, per-instance store, entirely separate
from the seam.

Re-derived against ticket 05's actual (post-34728c2) IntentAdapter/
ImportConfigAdapter split rather than the single ImportConfigAdapter shape
this was originally written against.

Per ticket 05's follow-up note, InstanceCreate seeds "running" with an
empty &tree_persist.Intent{} instead of nil, and InstanceRunningGet has no
ErrRunningNotFound/nil-guard: a freshly created instance's running store
simply reads back as an empty adapter, mirroring LocalCache's on-disk
running store, which also starts out with no data written to it.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Implements ticket 08: GRPCConfigReader (pkg/cache/configserver/grpc.go)
implements the local-read seam by calling the generated
config_read.ConfigReadServiceClient (sdc-protos ticket 01), replacing
ticket 05's fake behind ticket 06's backend. Connection settings are
caller-supplied (grpc.ClientConnInterface / Dial(address)), never
hardcoded. Unit-tested against a new generated mock
(mocks/mockconfigread) of the real client.

Pins sdc-protos to the sibling local checkout (untagged commit a8f3da0,
which adds config_read) until that branch is pushed/tagged upstream.

No wiring into Server.createCacheClient/CacheConfig — that's ticket 09.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Add the case "config-server" branch to createCacheClient, dialing the
colocated config-server controller and constructing ConfigServerCache
against it. Extend CacheConfig with Namespace and require both Address
and Namespace for Cache.Type: config-server; local/remote behavior is
unchanged.

Closes ticket 09.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
…rver backend

Adds splitDatastoreName + ErrMalformedDatastoreName so ConfigServerCache can
later derive per-call target namespace/name from the datastore name instead
of a fixed, deployment-wide namespace (ticket 01).

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
target() now decodes each call's cacheInstanceName via splitDatastoreName
instead of trusting a single, deployment-wide namespace field, so one
data-server process correctly serves datastores across multiple Kubernetes
namespaces and config-server's ConfigReadService receives the bare target
name. Every target() caller, plus InstanceCreate, propagates/surfaces
ErrMalformedDatastoreName on a malformed datastore name. Constructors drop
their namespace parameter.

Also re-derived against ticket 05's noopIntentWriter capability split:
ConfigServerCache no longer implements InstanceIntentModify/
InstanceIntentDelete itself — NewConfigServerClient composes it with the
generic noopIntentWriter into a full Client instead, per
noop_intent_writer.go's own doc comment. Server.createConfigServerCacheClient
and the corresponding tests now go through NewConfigServerClient rather than
asserting a concrete *ConfigServerCache.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Namespace was never meaningful once ConfigServerCache derives it per-call
from the datastore name (ticket 02); config-server cache type now requires
only Address (ticket 03).

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Config-server stores intents as a single JSON object at the YANG root path,
which mergeConfigBlobs rejected during LoadAllButRunningIntents and broke
performRevert after sync.

Re-derived against this ticket's package layout: the branch this was
written against had since moved this code to
pkg/tree/importer/configserver; here it's still pkg/cache/configserver
(ticket 06's original location), so the new tests reference the local
ConfigBlob type directly instead of an imported csreader.ConfigBlob alias.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Config-server keys TransactionSet intents as namespace.name, but the
config-server cache was returning bare Config names. That mismatch made
DeviationWatcher split intent owners incorrectly and panic the controller.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
s.cacheClient is already statically typed cache.Client (Server.cacheClient's
field type), so the explicit var _ cache.Client = s.cacheClient assertion
added alongside 7b1a78a is redundant — staticcheck (QF1011) flags it.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Use make(map[string]*Document, len(docs)) when seeding a new target map;
map composite literals are fine elsewhere, but the prior review fix used
make with a composite-literal type suffix, which fails typecheck.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
Run 05-cache-backend against cache_type=config-server and fail the workflow when GitHub release lookups return empty values.

Signed-off-by: Markus Vahlenkamp <markus.vahlenkamp@nokia.com>
@steiler
steiler force-pushed the cache-read-backend branch from dc979ab to e2add12 Compare October 14, 2026 09:06
@codecov

codecov Bot commented Oct 14, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.31615% with 34 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pkg/cache/configserver.go 88.18% 9 Missing and 4 partials ⚠️
pkg/server/cache.go 50.00% 7 Missing and 1 partial ⚠️
pkg/cache/configserver/merge.go 90.16% 3 Missing and 3 partials ⚠️
pkg/cache/configserver/grpc.go 88.37% 5 Missing ⚠️
pkg/cache/configserver/fake.go 94.28% 1 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

if len(rest) == 0 {
// A keyed element with nothing after it has only its own key leaves
// to contribute — already seeded by findOrCreateListEntry.
return nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible that here the values can be lost in case a path in the config ends with a list entry?

Example config:
`apiVersion: config.sdcio.dev/v1alpha1
kind: Config
metadata:
name: intf-uplink
namespace: default
labels:
config.sdcio.dev/targetName: dev1
config.sdcio.dev/targetNamespace: default
spec:
priority: 10
config:

  • path: /interface[name=ethernet-1/1]
    value:
    admin-state: enable
    description: uplink`

And what data server builds from it:

{"interface":[{"name":"ethernet-1/1"}]}

Config-server itself still sends the full intent with TransactionSet, so the first apply to the device is correct. The lost leaves only affect intents that data-server reads back through this backend:

Next change to the intent: the old version data-server reads back is missing those leaves, so leaves that were removed in the new spec are never deleted from the device.
Rollback: the restored intent is missing those leaves.
BlameConfig / GetIntent: the leaves have no owner.
Deviations: the leaves look like running values that no intent manages ("unhandled").
Changes to other intents: a transaction for another intent on the same device rebuilds the tree from the cache. Leaves only this intent set lose their owner, and may be treated as deleted or no longer managed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understood config-server will never send that type of config, so the path will only be / so up to you if you see this as an issue for the future or not

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants