Repository navigation
Refactor configuration handling - #1381
Open
mozesl-nokia wants to merge 39 commits into
Open
mozesl-nokia wants to merge 39 commits into
mozesl-nokia wants to merge 39 commits into
Conversation
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
…flag gen omittable Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.7 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:grok-4.6 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
- fix contradictory comments - don't make loader users import x/tools - dedupe grpcPort - rename db-related flags Assisted-by: Cursor:composer-2.5 Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
…d later Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
✅ Deploy Preview for kpt-porch canceled.Built without sensitive environment variables
|
Assisted-by: Cursor:claude-sonnet-5.5 Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical defects can expose database credentials in logs, prevent deployment startup, discard database pool limits, and leave unselected controllers or explicitly disabled profiling enabled.
6 open issues
Preserve configured database pool settings when building cache options · New Default unselected variant controllers to disabled · New Add a deployment-compatible baseline for the Porch manifest · New Apply zero profiling ports and false detailed-metrics overrides · New Redact credentials from all supported connection string formats · New Preserve existing lifecycle-policy flag names · New
What changed in this PR
This PR centralizes Porch’s component configuration under config, adding mounted YAML support for the ConfigMap configuration workflow requested in #1185.
Changes:
- Adds shared configuration types, a defaults/YAML/environment/flag loader, and JSON schema generation.
- Integrates configuration loading across the server, controllers, function runner, and wrapper server.
- Updates deployment rendering, container builds, documentation, and tests; moves relative binary-path resolution into the function runner.
| File | Description |
|---|---|
| test/e2e/crd/podevaluator_test.go | Removes the environment-specific deployment assertion. |
| scripts/README.md | Documents configuration rendering. |
| scripts/lib/common.sh | Adds the apply-setters image reference. |
| scripts/deploy/render-porch-config.sh | Renders configuration into the ConfigMap. |
| pkg/util/secret/secret.go | Adds connection-string password redaction. |
| pkg/util/datasize/datasize.go | Removes the former data-size utility. |
| pkg/registry/porch/packagerevision_approval_test.go | Uses shared lifecycle configuration. |
| pkg/registry/porch/lifecycle_policy.go | Adopts the shared lifecycle type. |
| pkg/lifecycle/transitions.go | Accepts shared lifecycle options. |
| pkg/lifecycle/transitions_test.go | Updates lifecycle option fixtures. |
| pkg/externalrepo/types/externalrepotypes.go | Embeds shared Git cache options. |
| pkg/externalrepo/git/gogit.go | Uses the new data-size type. |
| pkg/engine/podevaluator/podevaluator.go | Consumes component and global configuration. |
| pkg/engine/podevaluator/podevaluator_unit_test.go | Updates evaluator configuration fixtures. |
| pkg/engine/podevaluator/podevaluator_podcachemanager_test.go | Updates FunctionConfig store construction. |
| pkg/engine/podevaluator/podcachemanager_eventloop_test.go | Updates FunctionConfig store construction. |
| pkg/engine/podevaluator/grpcserver.go | Uses shared options and message sizes. |
| pkg/engine/podevaluator/grpcserver_test.go | Updates the message-size helper type. |
| pkg/engine/options.go | Passes shared configuration into runtimes. |
| pkg/engine/grpcruntime.go | Adopts global runtime settings. |
| pkg/engine/grpcruntime_test.go | Updates runtime configuration fixtures. |
| pkg/engine/engine.go | Stores shared lifecycle options. |
| pkg/engine/engine_test.go | Updates lifecycle configuration fixtures. |
| pkg/engine/builtinruntime_test.go | Updates FunctionConfig store construction. |
| pkg/cache/types/cachetypes.go | References centralized cache options. |
| pkg/cache/dbcache/dbhandler.go | Unwraps configured connection lifetimes. |
| pkg/cache/dbcache/dbcache_test.go | Uses shared database options. |
| pkg/cache/crcache/cache.go | Unwraps configured list timeouts. |
| pkg/apiserver/new_test.go | Updates server configuration fixtures. |
| pkg/apiserver/config.go | Wires shared configuration into server dependencies. |
| pkg/apiserver/config_test.go | Tests preservation of relative binary paths. |
| make/build.mk | Adds configuration schema generation. |
| internal/telemetry/metrics.go | Uses the relocated binary-size constant. |
| internal/telemetry/config_env.go | Exports effective telemetry settings to environment. |
| go.sum | Updates dependency checksums. |
| go.mod | Updates direct and indirect dependencies. |
| func/wrapper-server/main.go | Loads wrapper and global configuration. |
| func/server/server.go | Loads and logs function-runner configuration. |
| func/internal/executableevaluator.go | Resolves runner-relative executable paths. |
| func/internal/executableevaluator_test.go | Covers relative executable-path resolution. |
| func/Dockerfile-wrapperserver | Includes configuration sources in builds. |
| func/Dockerfile | Updates copied build sources. |
| docs/content/en/docs/6_configuration_and_deployments/configurations/components/porch-server-config/_index.md | Removes the server-side functions-directory argument. |
| docs/content/en/docs/6_configuration_and_deployments/configurations/components/porch-controllers-config/_index.md | Removes the controller binary-directory setting. |
| docs/content/en/docs/6_configuration_and_deployments/configurations/components/function-runner-config/function-configuration.md | Updates the default functions directory. |
| docs/content/en/docs/6_configuration_and_deployments/configurations/components/function-runner-config/_index.md | Updates the functions-directory example. |
| docs/content/en/docs/5_architecture_and_components/function-runner/functionality/function-evaluation.md | Clarifies runner-owned binary-path resolution. |
| docs/content/en/docs/5_architecture_and_components/engine/functionality/function-evaluation.md | Documents unchanged path forwarding and runner checks. |
| docs/content/en/docs/5_architecture_and_components/controllers/packagerevision-controller/interactions.md | Removes the controller binary-cache setting. |
| deployments/porch/README.md | Documents the mounted configuration workflow. |
| deployments/porch/porch-config.yaml | Removes the previous local configuration resource. |
| deployments/porch/9-controllers.yaml | Mounts configuration for controllers. |
| deployments/porch/3-porch-server.yaml | Replaces server arguments with mounted configuration. |
| deployments/porch/2-function-runner.yaml | Mounts configuration for the function runner. |
| deployments/porch/1-porch-config.yaml | Adds the mounted ConfigMap placeholder. |
| deployments/config/porch-config.yaml | Adds the setter-enabled configuration source. |
| controllers/repositories/pkg/controllers/repository/sync.go | Uses shared settings and fixed sync polling. |
| controllers/repositories/pkg/controllers/repository/repository_controller.go | Stores centralized repository-controller configuration. |
| controllers/repositories/pkg/controllers/repository/repository_controller_test.go | Updates configuration fixtures and polling assertions. |
| controllers/repositories/pkg/controllers/repository/config.go | Validates and logs shared configuration. |
| controllers/repositories/pkg/controllers/repository/cache.go | Builds caches from loaded configuration. |
| controllers/repositories/pkg/controllers/repository/cache_test.go | Replaces environment fixtures with configuration structs. |
| controllers/repositories/integration/suite_test.go | Updates repository-controller configuration fixtures. |
| controllers/repositories/integration/repository_integration_test.go | Updates sync-capacity configuration fixtures. |
| controllers/packagevariantsets/pkg/controllers/packagevariantset/packagevariantset_controller.go | Removes empty options and flag methods. |
| controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller.go | Removes empty options and flag methods. |
| controllers/packagerevisions/pkg/webhooks/packagerevision_webhook.go | Uses shared lifecycle policy options. |
| controllers/packagerevisions/pkg/controllers/packagerevision/status.go | Accepts the shared duration type. |
| controllers/packagerevisions/pkg/controllers/packagerevision/render.go | Uses shared settings and explicit retry delays. |
| controllers/packagerevisions/pkg/controllers/packagerevision/metadata.go | Uses an explicit render retry delay. |
| controllers/packagerevisions/pkg/controllers/packagerevision/metadata_test.go | Updates retry-delay assertions. |
| controllers/packagerevisions/pkg/controllers/packagerevision/config.go | Wires loaded runtime and lifecycle settings. |
| controllers/packagerevisions/pkg/controllers/packagerevision/config_test.go | Initializes fixtures through shared defaults. |
| controllers/packagerevisions/integration/suite_test.go | Applies shared defaults to integration fixtures. |
| controllers/packagerevisions/integration/packagerevision_integration_test.go | Updates reconciler configuration fixtures. |
| controllers/main.go | Loads controller/global settings and enablement. |
| controllers/main_test.go | Tests file-based controller configuration. |
| controllers/functionconfigs/functionconfigreconciler.go | Stores binary paths without local resolution. |
| controllers/Dockerfile | Includes configuration sources in builds. |
| config/types/wrapper_server_types.go | Defines wrapper-server settings. |
| config/types/server_types.go | Defines server, evaluator, and serving settings. |
| config/types/master_types.go | Defines master and global configuration. |
| config/types/global_types.go | Defines telemetry and lifecycle settings. |
| config/types/function_runner_types.go | Defines function-runner settings. |
| config/types/duration.go | Adds serializable duration values. |
| config/types/duration_test.go | Tests duration serialization. |
| config/types/datasize/datasize.go | Adds serializable decimal/binary data sizes. |
| config/types/datasize/datasize_test.go | Tests data-size parsing and serialization. |
| config/types/database_types.go | Defines database connection settings. |
| config/types/controller_types.go | Defines controller configuration and enablement. |
| config/types/cache_types.go | Defines shared cache configuration. |
| config/loader/server_load_test.go | Tests server defaults and configuration overrides. |
| config/loader/schemafields.go | Exposes component-aware fields for schema generation. |
| config/loader/schemafields_test.go | Tests component-level schema flag names. |
| config/loader/overlay_test.go | Tests section loading and override behavior. |
| config/loader/names.go | Derives file, flag, and environment names. |
| config/loader/names_test.go | Tests naming and tag rules. |
| config/loader/load.go | Provides configuration loading entry points. |
| config/loader/load_test.go | Tests precedence and invalid inputs. |
| config/loader/flags.go | Registers flags and applies defaults. |
| config/loader/flags_test.go | Tests flag types, defaults, and validation. |
| config/loader/fields_test.go | Tests field naming and configuration mappings. |
| config/loader/embedded_test.go | Tests embedded configuration fields. |
| config/loader/defaults.go | Exposes default application for tests. |
| config/loader/controllers_load_test.go | Tests controller defaults and overrides. |
| config/cmd/genschema/main.go | Adds the schema-generation command. |
| cmd/porch/main.go | Removes early telemetry initialization. |
| build/Dockerfile | Includes configuration sources in server builds. |
| .gitignore | Ignores compiled test binaries. |
Review effort: Balanced
| if strings.ToUpper(r.cacheType) == string(cachetypes.DBCacheType) { | ||
| var dbOptions configtypes.DBCacheOptions | ||
| if strings.ToUpper(r.Cfg.CacheType) == string(cachetypes.DBCacheType) { | ||
| dbOptions, err = r.setupDBCacheOptionsFromEnv() |
Comment on lines
+66
to
+67
| enablePackageVariants: true # kpt-set: ${enablePackageVariants} | ||
| enablePackageVariantSets: true # kpt-set: ${enablePackageVariantSets} |
Comment on lines
+15
to
+25
| # The porch-config ConfigMap mounted by the Deployments. porch.yaml is empty | ||
| # here, so when this folder is applied on its own the components run on their | ||
| # built-in defaults. `make deployment-config` fills porch.yaml in from | ||
| # deployments/config/porch-config.yaml. | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| namespace: porch-system | ||
| name: porch-config | ||
| data: | ||
| porch.yaml: "" |
Comment on lines
+37
to
+42
| if cfg.PprofPort != 0 { | ||
| values[PProfPortEnvVar] = strconv.Itoa(cfg.PprofPort) | ||
| } | ||
| if cfg.PackageSizeDetailed { | ||
| values[packageSizeDetailedEnv] = "true" | ||
| } |
Comment on lines
+17
to
+45
| import ( | ||
| "net/url" | ||
| "strings" | ||
| ) | ||
|
|
||
| // RedactDataSourcePassword returns a copy of the given database connection | ||
| // string with the password replaced by "REDACTED", so it is safe to log. | ||
| func RedactDataSourcePassword(dataSource *string) { | ||
| if dataSource == nil || *dataSource == "" { | ||
| return | ||
| } | ||
|
|
||
| // URL-style connection strings (e.g. postgres://user:pass@host/db). | ||
| if u, err := url.Parse(*dataSource); err == nil && u.User != nil { | ||
| if _, hasPassword := u.User.Password(); hasPassword { | ||
| u.User = url.UserPassword(u.User.Username(), "REDACTED") | ||
| *dataSource = u.String() | ||
| } | ||
| return | ||
| } | ||
|
|
||
| // mysql-style DSN: user:pass@tcp(host:port)/db. | ||
| if idx := strings.LastIndex(*dataSource, "@"); idx > 0 { | ||
| creds := (*dataSource)[:idx] | ||
| if before, _, ok := strings.Cut(creds, ":"); ok { | ||
| *dataSource = before + ":REDACTED" + (*dataSource)[idx:] | ||
| } | ||
| } | ||
| } |
| // maximum HTTP request body. 6MiB is 6 * datasize.MiB. | ||
| MaxGrpcMessageSize datasize.DataSize `flag:"max-request-body-size" default:"6MiB" usage:"Maximum size of the request body and gRPC messages, for example 6MB or a number of bytes"` | ||
|
|
||
| LifecyclePolicy LifecycleOptions `json:",omitzero"` |
Signed-off-by: Mózes László Máté <laszlo.mozes@nokia.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Refactor configuration handling
Description
configfolder. You can now configure Porch using a mounted YAML. There is also a json schema and a schema generator which can be used to validate said YAML config.config/types.Related Issue(s)
Type of Change
Checklist
Additional Notes (Optional)
deployments/config/porch-config.yamlfor the final version.AI Disclosure
If so, please describe how: