Skip to content

Refactor configuration handling - #1381

Open
mozesl-nokia wants to merge 39 commits into
kptdev:mainfrom
nokia:config-refact
Open

mozesl-nokia wants to merge 39 commits into
kptdev:mainfrom
nokia:config-refact

Conversation

@mozesl-nokia

Copy link
Copy Markdown
Contributor

Refactor configuration handling


Description

  • What changed: Most of the configuration for all components now lives in the config folder. You can now configure Porch using a mounted YAML. There is also a json schema and a schema generator which can be used to validate said YAML config.
  • Why it’s needed: To simplify how configuration is handled and centralize it.
  • How it works: A new loader handles flag registration, env var and config file reading based on the names and tags in the structs in config/types.

Related Issue(s)


Type of Change

  • New feature
  • Enhancement
  • Refactor
  • Documentation

Checklist

  • Code follows project style guidelines
  • Self-reviewed changes
  • Tests added/updated
  • Documentation added/updated
  • All tests and gating checks pass

Additional Notes (Optional)

  • Known Issues: Had to wrap the config YAML in a pseudo-manifest in order to allow kpt to run apply-setters on it, which is then extracted to a real ConfigMap with starlark with the deploy scripts. Thus, the final config is never actually validating against the schema.
  • Further improvements: We likely need to break BC in order to unify the configs even further, added many TODO comments in the appropriate places.
  • Review notes: The final version of the YAML differs from the original issue description, please see deployments/config/porch-config.yaml for the final version.

AI Disclosure

  • I have used AI in the creation of this PR.

If so, please describe how:

  • Multiple models were used (Grok 4.6, 4.7, Composer 2.5, Claude Haiku and Sonnet 5.5) through Cursor to author big chunks of the PR, like deduplication and extraction of types, writing most of the loader and schema generator and reviewing.

Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
…flag gen omittable

Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.6
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
- fix contradictory comments
- don't make loader users import x/tools
- dedupe grpcPort
- rename db-related flags

Assisted-by: Cursor:composer-2.5
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
…d later

Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
@mozesl-nokia
mozesl-nokia requested review from a team November 18, 2026 10:55
@mozesl-nokia mozesl-nokia added enhancement New feature or request go Pull requests that update go code labels Nov 18, 2026
@mozesl-nokia mozesl-nokia added the refactoring #b33d8f label Nov 18, 2026
Copilot AI balanced review requested due to automatic review settings November 18, 2026 10:55
@netlify

netlify Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for kpt-porch canceled.

Built without sensitive environment variables

Name Link
🔨 Latest commit 7531034
🔍 Latest deploy log https://app.netlify.com/projects/kpt-porch/deploys/6afda1c075b66c14e3ae7b9b

Assisted-by: Cursor:claude-sonnet-5.5
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical defects can expose database credentials in logs, prevent deployment startup, discard database pool limits, and leave unselected controllers or explicitly disabled profiling enabled.

6 open issues
What changed in this PR

This PR centralizes Porch’s component configuration under config, adding mounted YAML support for the ConfigMap configuration workflow requested in #1185.

Changes:

  • Adds shared configuration types, a defaults/YAML/environment/flag loader, and JSON schema generation.
  • Integrates configuration loading across the server, controllers, function runner, and wrapper server.
  • Updates deployment rendering, container builds, documentation, and tests; moves relative binary-path resolution into the function runner.
File Description
test/​e2e/​crd/​podevaluator_test.go Removes the environment-specific deployment assertion.
scripts/​README.md Documents configuration rendering.
scripts/​lib/​common.sh Adds the apply-setters image reference.
scripts/​deploy/​render-porch-config.sh Renders configuration into the ConfigMap.
pkg/​util/​secret/​secret.go Adds connection-string password redaction.
pkg/​util/​datasize/​datasize.go Removes the former data-size utility.
pkg/​registry/​porch/​packagerevision_approval_test.go Uses shared lifecycle configuration.
pkg/​registry/​porch/​lifecycle_policy.go Adopts the shared lifecycle type.
pkg/​lifecycle/​transitions.go Accepts shared lifecycle options.
pkg/​lifecycle/​transitions_test.go Updates lifecycle option fixtures.
pkg/​externalrepo/​types/​externalrepotypes.go Embeds shared Git cache options.
pkg/​externalrepo/​git/​gogit.go Uses the new data-size type.
pkg/​engine/​podevaluator/​podevaluator.go Consumes component and global configuration.
pkg/​engine/​podevaluator/​podevaluator_unit_test.go Updates evaluator configuration fixtures.
pkg/​engine/​podevaluator/​podevaluator_podcachemanager_test.go Updates FunctionConfig store construction.
pkg/​engine/​podevaluator/​podcachemanager_eventloop_test.go Updates FunctionConfig store construction.
pkg/​engine/​podevaluator/​grpcserver.go Uses shared options and message sizes.
pkg/​engine/​podevaluator/​grpcserver_test.go Updates the message-size helper type.
pkg/​engine/​options.go Passes shared configuration into runtimes.
pkg/​engine/​grpcruntime.go Adopts global runtime settings.
pkg/​engine/​grpcruntime_test.go Updates runtime configuration fixtures.
pkg/​engine/​engine.go Stores shared lifecycle options.
pkg/​engine/​engine_test.go Updates lifecycle configuration fixtures.
pkg/​engine/​builtinruntime_test.go Updates FunctionConfig store construction.
pkg/​cache/​types/​cachetypes.go References centralized cache options.
pkg/​cache/​dbcache/​dbhandler.go Unwraps configured connection lifetimes.
pkg/​cache/​dbcache/​dbcache_test.go Uses shared database options.
pkg/​cache/​crcache/​cache.go Unwraps configured list timeouts.
pkg/​apiserver/​new_test.go Updates server configuration fixtures.
pkg/​apiserver/​config.go Wires shared configuration into server dependencies.
pkg/​apiserver/​config_test.go Tests preservation of relative binary paths.
make/​build.mk Adds configuration schema generation.
internal/​telemetry/​metrics.go Uses the relocated binary-size constant.
internal/​telemetry/​config_env.go Exports effective telemetry settings to environment.
go.sum Updates dependency checksums.
go.mod Updates direct and indirect dependencies.
func/​wrapper-server/​main.go Loads wrapper and global configuration.
func/​server/​server.go Loads and logs function-runner configuration.
func/​internal/​executableevaluator.go Resolves runner-relative executable paths.
func/​internal/​executableevaluator_test.go Covers relative executable-path resolution.
func/​Dockerfile-wrapperserver Includes configuration sources in builds.
func/​Dockerfile Updates copied build sources.
docs/​content/​en/​docs/​6_configuration_and_deployments/​configurations/​components/​porch-server-config/​_index.md Removes the server-side functions-directory argument.
docs/​content/​en/​docs/​6_configuration_and_deployments/​configurations/​components/​porch-controllers-config/​_index.md Removes the controller binary-directory setting.
docs/​content/​en/​docs/​6_configuration_and_deployments/​configurations/​components/​function-runner-config/​function-configuration.md Updates the default functions directory.
docs/​content/​en/​docs/​6_configuration_and_deployments/​configurations/​components/​function-runner-config/​_index.md Updates the functions-directory example.
docs/​content/​en/​docs/​5_architecture_and_components/​function-runner/​functionality/​function-evaluation.md Clarifies runner-owned binary-path resolution.
docs/​content/​en/​docs/​5_architecture_and_components/​engine/​functionality/​function-evaluation.md Documents unchanged path forwarding and runner checks.
docs/​content/​en/​docs/​5_architecture_and_components/​controllers/​packagerevision-controller/​interactions.md Removes the controller binary-cache setting.
deployments/​porch/​README.md Documents the mounted configuration workflow.
deployments/​porch/​porch-config.yaml Removes the previous local configuration resource.
deployments/​porch/​9-controllers.yaml Mounts configuration for controllers.
deployments/​porch/​3-porch-server.yaml Replaces server arguments with mounted configuration.
deployments/​porch/​2-function-runner.yaml Mounts configuration for the function runner.
deployments/​porch/​1-porch-config.yaml Adds the mounted ConfigMap placeholder.
deployments/​config/​porch-config.yaml Adds the setter-enabled configuration source.
controllers/​repositories/​pkg/​controllers/​repository/​sync.go Uses shared settings and fixed sync polling.
controllers/​repositories/​pkg/​controllers/​repository/​repository_controller.go Stores centralized repository-controller configuration.
controllers/​repositories/​pkg/​controllers/​repository/​repository_controller_test.go Updates configuration fixtures and polling assertions.
controllers/​repositories/​pkg/​controllers/​repository/​config.go Validates and logs shared configuration.
controllers/​repositories/​pkg/​controllers/​repository/​cache.go Builds caches from loaded configuration.
controllers/​repositories/​pkg/​controllers/​repository/​cache_test.go Replaces environment fixtures with configuration structs.
controllers/​repositories/​integration/​suite_test.go Updates repository-controller configuration fixtures.
controllers/​repositories/​integration/​repository_integration_test.go Updates sync-capacity configuration fixtures.
controllers/​packagevariantsets/​pkg/​controllers/​packagevariantset/​packagevariantset_controller.go Removes empty options and flag methods.
controllers/​packagevariants/​pkg/​controllers/​packagevariant/​packagevariant_controller.go Removes empty options and flag methods.
controllers/​packagerevisions/​pkg/​webhooks/​packagerevision_webhook.go Uses shared lifecycle policy options.
controllers/​packagerevisions/​pkg/​controllers/​packagerevision/​status.go Accepts the shared duration type.
controllers/​packagerevisions/​pkg/​controllers/​packagerevision/​render.go Uses shared settings and explicit retry delays.
controllers/​packagerevisions/​pkg/​controllers/​packagerevision/​metadata.go Uses an explicit render retry delay.
controllers/​packagerevisions/​pkg/​controllers/​packagerevision/​metadata_test.go Updates retry-delay assertions.
controllers/​packagerevisions/​pkg/​controllers/​packagerevision/​config.go Wires loaded runtime and lifecycle settings.
controllers/​packagerevisions/​pkg/​controllers/​packagerevision/​config_test.go Initializes fixtures through shared defaults.
controllers/​packagerevisions/​integration/​suite_test.go Applies shared defaults to integration fixtures.
controllers/​packagerevisions/​integration/​packagerevision_integration_test.go Updates reconciler configuration fixtures.
controllers/​main.go Loads controller/global settings and enablement.
controllers/​main_test.go Tests file-based controller configuration.
controllers/​functionconfigs/​functionconfigreconciler.go Stores binary paths without local resolution.
controllers/​Dockerfile Includes configuration sources in builds.
config/​types/​wrapper_server_types.go Defines wrapper-server settings.
config/​types/​server_types.go Defines server, evaluator, and serving settings.
config/​types/​master_types.go Defines master and global configuration.
config/​types/​global_types.go Defines telemetry and lifecycle settings.
config/​types/​function_runner_types.go Defines function-runner settings.
config/​types/​duration.go Adds serializable duration values.
config/​types/​duration_test.go Tests duration serialization.
config/​types/​datasize/​datasize.go Adds serializable decimal/binary data sizes.
config/​types/​datasize/​datasize_test.go Tests data-size parsing and serialization.
config/​types/​database_types.go Defines database connection settings.
config/​types/​controller_types.go Defines controller configuration and enablement.
config/​types/​cache_types.go Defines shared cache configuration.
config/​loader/​server_load_test.go Tests server defaults and configuration overrides.
config/​loader/​schemafields.go Exposes component-aware fields for schema generation.
config/​loader/​schemafields_test.go Tests component-level schema flag names.
config/​loader/​overlay_test.go Tests section loading and override behavior.
config/​loader/​names.go Derives file, flag, and environment names.
config/​loader/​names_test.go Tests naming and tag rules.
config/​loader/​load.go Provides configuration loading entry points.
config/​loader/​load_test.go Tests precedence and invalid inputs.
config/​loader/​flags.go Registers flags and applies defaults.
config/​loader/​flags_test.go Tests flag types, defaults, and validation.
config/​loader/​fields_test.go Tests field naming and configuration mappings.
config/​loader/​embedded_test.go Tests embedded configuration fields.
config/​loader/​defaults.go Exposes default application for tests.
config/​loader/​controllers_load_test.go Tests controller defaults and overrides.
config/​cmd/​genschema/​main.go Adds the schema-generation command.
cmd/​porch/​main.go Removes early telemetry initialization.
build/​Dockerfile Includes configuration sources in server builds.
.gitignore Ignores compiled test binaries.

Review effort: Balanced

if strings.ToUpper(r.cacheType) == string(cachetypes.DBCacheType) {
var dbOptions configtypes.DBCacheOptions
if strings.ToUpper(r.Cfg.CacheType) == string(cachetypes.DBCacheType) {
dbOptions, err = r.setupDBCacheOptionsFromEnv()
Comment on lines +66 to +67
enablePackageVariants: true # kpt-set: ${enablePackageVariants}
enablePackageVariantSets: true # kpt-set: ${enablePackageVariantSets}
Comment on lines +15 to +25
# The porch-config ConfigMap mounted by the Deployments. porch.yaml is empty
# here, so when this folder is applied on its own the components run on their
# built-in defaults. `make deployment-config` fills porch.yaml in from
# deployments/config/porch-config.yaml.
apiVersion: v1
kind: ConfigMap
metadata:
namespace: porch-system
name: porch-config
data:
porch.yaml: ""
Comment on lines +37 to +42
if cfg.PprofPort != 0 {
values[PProfPortEnvVar] = strconv.Itoa(cfg.PprofPort)
}
if cfg.PackageSizeDetailed {
values[packageSizeDetailedEnv] = "true"
}
Comment thread pkg/util/secret/secret.go
Comment on lines +17 to +45
import (
"net/url"
"strings"
)

// RedactDataSourcePassword returns a copy of the given database connection
// string with the password replaced by "REDACTED", so it is safe to log.
func RedactDataSourcePassword(dataSource *string) {
if dataSource == nil || *dataSource == "" {
return
}

// URL-style connection strings (e.g. postgres://user:pass@host/db).
if u, err := url.Parse(*dataSource); err == nil && u.User != nil {
if _, hasPassword := u.User.Password(); hasPassword {
u.User = url.UserPassword(u.User.Username(), "REDACTED")
*dataSource = u.String()
}
return
}

// mysql-style DSN: user:pass@tcp(host:port)/db.
if idx := strings.LastIndex(*dataSource, "@"); idx > 0 {
creds := (*dataSource)[:idx]
if before, _, ok := strings.Cut(creds, ":"); ok {
*dataSource = before + ":REDACTED" + (*dataSource)[idx:]
}
}
}
// maximum HTTP request body. 6MiB is 6 * datasize.MiB.
MaxGrpcMessageSize datasize.DataSize `flag:"max-request-body-size" default:"6MiB" usage:"Maximum size of the request body and gRPC messages, for example 6MB or a number of bytes"`

LifecyclePolicy LifecycleOptions `json:",omitzero"`
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
57.7% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code refactoring #b33d8f

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Porch components should support reading configuration from a ConfigMap

2 participants