Repository navigation
Conversation
…cripts Replace the Makefile and the scripts around it with a justfile and a small set of scripts in hack/ that work on the per-minor-version layout (providers/<provider>/<stack>/1-XX/). - justfile: build, publish, dev, install-cso, matrix, update, add-version, generate-resources, generate-clusterstack, generate-cluster, generate-image-manifests, generate-docs, migrate-cluster, clean and the tools container. flake.nix and Containerfile provide just, helm, kubectl, yq and oras. - build.sh: builds the release of a Kubernetes minor version (cluster class and cluster addon charts, clusteraddon.yaml, metadata). Development builds get the version v0-ttl.<timestamp>; the release directory and the OCI tag use the dash notation (v0-ttl-<timestamp>), which is what the Cluster Stack Operator requests. Stable releases are numbered highest published version + 1, a build without changes is not published again, --release-version selects a version. --validate checks the addon bundle, --install-cso installs the operator with the matching OCI configuration (CSO_CHART, CSO_IMAGE_REPOSITORY/TAG). - update.sh: Kubernetes patch versions (dl.k8s.io), node images in image-manager.yaml (--check-image-repo) and addon chart versions. Minor versions with a FROZEN file are skipped. - add-version.sh: scaffold a new Kubernetes minor version from the latest one. - generate-resources.sh: ClusterStack (channel stable, or custom for development builds) and Cluster. The Cluster uses the API version of the ClusterClass of the stack (classRef for v1beta2, class for v1beta1). - generate-image-manifests.sh (ORC images), show-matrix.sh (version and addon matrix). - docugen.py: configuration documentation from the ClusterClass variables. Only the sections between BEGIN/END GENERATED markers are replaced (--update), hand written text is kept. - update-gateway-api-crds.sh: vendors the Gateway API CRDs for the gateway-api addon of the scs stacks. Removes the scripts that are replaced (ensure-*, generate_version.py, generate_openstack_image_manager_yaml.sh, kind-dev.sh, the release notes tool). Co-authored-by: Nils Arnold <arnold@aov.de> Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
Co-authored-by: Nils Arnold <arnold@aov.de> Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
The scs stack for Kubernetes 1.33 to 1.36 (1.33.11, 1.34.12, 1.35.9, 1.36.5), every minor version in its own self-contained directory. The ClusterClass uses cluster.x-k8s.io/v1beta2 and needs Cluster API >= 1.11 and a Cluster Stack Operator that supports it. Kubernetes 1.36 additionally needs Cluster API >= 1.12.8 or >= 1.13.1. - Unified variables: flavor, rootDisk, serverGroupID, ... apply to all nodes and are overridden per control plane or machine deployment via variables.overrides, instead of separate controlPlane*/worker* variables. Node images are managed in image-manager.yaml. - Cilium with native routing and kube-proxy replacement (1.35, 1.36), Cilium 1.19.4 (1.33 to 1.35) and 1.20.2 (1.36). - serviceLoadBalancer (#241): Octavia OVN (default) or Amphora for Services of type LoadBalancer. With OVN the cloud controller manager manages the security groups, otherwise clients outside of the cluster do not reach the node ports. - diskPinAZ: create root and additional volumes in the AZ of the machine. - gateway-api addon: the Gateway API CRDs (v1.4.1 for Cilium 1.19, v1.6.3 for Cilium 1.20) are applied before Cilium and the cilium GatewayClass is created, Cilium installs neither. - 1.35 and 1.36: machine health checks and rollout strategy, security groups with allowAllInClusterTraffic, containerd limits and registry mirror CA, API server audit logging, componentOverrides for the resources of components and addons, ciliumHostLegacyRouting, OCCM on the control plane with hostNetwork (1.36). Co-authored-by: Nils Arnold <arnold@aov.de> Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
scs 1.32 is the only scs version that has been released before. It is kept as it was (v1beta1 ClusterClass, variables like cloud_name and ssh_key), moved into 1-32/, so that existing clusters can be upgraded within the minor version. The unified variables and the new features start with 1.33. The directory is marked with a FROZEN file, hack/update.sh skips it. Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
… the stack scs2 is only maintained from now on and will be phased out, scs is the stack to use. - 1-33 to 1-36, every minor version in its own directory. The ClusterClass templates and the variables (controlPlane*/worker*) are unchanged and still v1beta1, only the chart metadata differs, so existing clusters keep working. - Node images are managed in image-manager.yaml, the per-version versions.yaml, kubernetes.yaml and image.yaml are gone. - DEPRECATED marker (add-version skips the stack) and a README with the deprecation note. - hack/migrate-cluster.sh (just migrate-cluster) converts the variables of a Cluster manifest from scs2 to the unified variables of scs. Changing the ClusterStack of an existing cluster is not possible with the Cluster Stack Operator, the script only converts the variables. Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
Cluster stack for Kubernetes 1.33 to 1.36 that runs the control plane as pods in the management cluster with the teutonet hosted control plane provider. Only the workers are OpenStack servers. The API server is exposed through the Gateway API of the management cluster, Cilium runs on the workers. The ClusterClass is v1beta1. The HostedControlPlaneTemplate passes allocate-node-cidrs to the hosted controller manager. Cilium uses ipam.mode kubernetes and waits for the pod CIDR of its node. Without the flag the nodes never got a podCIDR, the Cilium agents did not become ready, the nodes stayed tainted and the cloud controller manager was never scheduled. Co-authored-by: Nils Arnold <arnold@aov.de> Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
…rovider docker/scs for Kubernetes 1.32 to 1.36 with the Cluster API Docker provider, no cloud credentials needed. The ClusterClass uses v1beta2, Cilium 1.19.4 and the metrics-server are applied as addons. Stacks from 1.33 on also ship the Gateway API CRDs (gateway-api addon). Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
… stacks - README and docs/overview.md: the stacks, the just commands and the quickstart. - docs/versioning.md: layout, requirements (Cluster API >= 1.11, Kubernetes 1.36 needs >= 1.12.8 or >= 1.13.1, because older versions set the removed kubeadm feature gate ControlPlaneKubeletLocalMode), breaking changes only with a new minor version, status of the stacks. - docs/build-system.md: layout, commands, releases and versions, updating, the Gateway API CRDs and the migration from scs2 to scs. - docs/providers: configuration of scs, scs2 and docker/scs (generated sections with the variables and the version matrix), hcp with its prerequisites (Gateway API and provider version, cert-manager, etcd storage, networkMTU). Co-authored-by: Nils Arnold <arnold@aov.de> Signed-off-by: Jan Schoone <6106846+jschoone@users.noreply.github.com>
image-manager.yaml requires 2048 MiB of RAM per image, but update.sh and generate-image-manifests.sh still wrote 1024, which would revert the value on the next update. Signed-off-by: Nils Arnold <arnold@aov.de>
Otherwise a temporary download error removes the checksum of an unchanged version from image-manager.yaml. Signed-off-by: Nils Arnold <arnold@aov.de>
Cilium 1.20 requires it. Without the CRD the operator reports "Required GatewayAPI resources are not found" and does not enable the Gateway API. Signed-off-by: Nils Arnold <arnold@aov.de>
The vendored CRDs have to match the Cilium version of the cni addon, otherwise a Cilium update breaks the Gateway API. update.sh reads the version from the Cilium release and calls update-gateway-api-crds.sh when it differs. Signed-off-by: Nils Arnold <arnold@aov.de>
This also moves 1-35 to metrics-server 3.14.0 and to Gateway API v1.6.3, which Cilium 1.20 needs. Signed-off-by: Nils Arnold <arnold@aov.de>
…h version Signed-off-by: Nils Arnold <arnold@aov.de>
The ccm-cloud-config secret lacked settings the old cloud-config secret had. Add them for both octavia-ovn and octavia-amphora: - floating-network-id from the networkExternalID variable, needed in projects with more than one external network. - manage-security-groups, now also for amphora, so the amphora can reach the NodePorts and the health check node port. - create-monitor, required for externalTrafficPolicy Local. The CCM uses TCP monitors for ovn and HTTP monitors for amphora. enable-ingress-hostname is left out: it only works with the PROXY protocol (amphora only), and Cilium honors ipMode Proxy, which the CCM sets without it. Signed-off-by: Nils Arnold <arnold@aov.de>
Same version as in 1.35. Signed-off-by: Nils Arnold <arnold@aov.de>
Nils98Ar
approved these changes
Nov 10, 2026
Nils98Ar
left a comment
Member
There was a problem hiding this comment.
Looks good and works for our dev clusters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi @Nils98Ar and @garloff. I decided to put #270, #259, #255, #252 and #248 together again in this PR since we lost the overview and other PRs were already that old again that they needed to be reworked completely. This PR will solve all of them together.
Mostly generated code but tested, including HCP.
This PR will deprecate the
scs2since the number is no longer needed when using subdirs and continues withscsagain.What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when PR gets merged):Fixes #
Special notes for your reviewer:
Please confirm that if this PR changes any image versions, then that's the sole change this PR makes.
TODOs: