Repository navigation
Select the Ceph backend with CEPH_STACK - #2960
Merged
Merged
Conversation
This was referenced Sep 30, 2026
The testbed has carried a deploy script for each Ceph backend for a while, scripts/deploy/100-ceph-with-ansible.sh and 100-ceph-with-cephadm.sh, and deploy-manager.sh links both into /usr/local/bin. Nothing chose between them: deploy-services.sh and deploy-ceph.sh named the ceph-ansible script outright, so a cephadm deployment could only be had by running the script by hand after the fact. The variable to choose with already exists too. terraform's ceph_stack is written into /opt/manager-vars.sh as CEPH_STACK, which every script here sources, and terraform/Makefile passes it through from a make variable. It was inert because nothing read it. Dispatch on it. deploy_ceph in include.sh picks the script for the configured backend and fails on an unknown value rather than silently deploying the default, and the two deploy wrappers call it. deploy.yml threads a ceph_stack job variable to make, so a Zuul job can select the backend the same way it selects ceph_version, defaulting to ceph-ansible where it says nothing. The upgrade path deliberately does not follow. Upgrading Ceph under cephadm runs through "ceph orch upgrade" and there is no script for it, so upgrade-services.sh and upgrade-ceph.sh refuse on a cephadm cluster instead of running the ceph-ansible upgrade against one. In upgrade-services.sh the refusal is skipped when SKIP_CEPH_UPGRADE is already set, so the rest of an upgrade can still be exercised. Accepting the value also needs osism/terraform-base, which owns the ceph_stack variable and validates it. Assisted-by: Claude:claude-opus-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
ideaship
force-pushed
the
ceph-stack-selectable
branch
2 times, most recently
from
October 1, 2026 09:37
25cf7b5 to
8ba1984
Compare
scripts/check/100-ceph-with-ansible.sh has nothing ceph-ansible specific left in it. Every command it runs is the plain Ceph CLI through the cephclient container, which both backends deploy, and the three validators it calls -- osism validate ceph-mons, ceph-mgrs and ceph-osds -- were taught to match cephadm's daemon container names and to reach Ceph through cephclient. Only the name still said ceph-ansible, and the two wrappers called it unconditionally, so a cephadm testbed ran a check that claimed to be for the other backend. Rename it to 100-ceph.sh rather than adding a second, near-identical script and a dispatch in front of it. The deploy and upgrade steps keep their per-backend names because those really are different procedures. Nothing else referred to the old check name: deploy-manager.sh links the deploy and upgrade scripts into /usr/local/bin, not this one. Append the one thing that is backend specific. Under cephadm the orchestrator holds state with no ceph-ansible equivalent -- the services it has been asked to run and the daemons it placed for them. A service running fewer daemons than its placement asks for, down to none at all, is exactly the failure a healthy-looking "ceph -s" hides, so print orch ls, orch ps and orch host ls, and fail when a service is short. Read the backend from the cluster (ceph orch status) rather than from CEPH_STACK: a nutshell testbed selects cephadm in python-osism and leaves CEPH_STACK unset, so keying on it would skip the check exactly on the lanes that deploy cephadm that way. Assisted-by: Claude:claude-opus-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
On the latest track the manager's ceph-ansible image tag follows ceph_version. A cephadm testbed on Tentacle therefore asks for ceph-ansible:tentacle, an image that does not exist, because ceph-ansible has no branch past squid. Nothing on the cephadm path needs that container any more. The Ceph version pins it supplied come from osism-ansible for numbered releases and from osism/defaults on the latest track, and the OSD LVM plays run in osism-ansible. So set ceph_ansible_enable: false in the manager configuration when CEPH_STACK is cephadm, as cfg-cookiecutter does for Ceph releases ceph-ansible cannot deploy. The manager waits for the ceph-ansible container only when the manager configuration enables it, rather than keying on CEPH_STACK, so a container disabled for any reason is not waited for. A nutshell testbed selects its Ceph backend in python-osism and leaves CEPH_STACK unset, so CEPH_STACK alone cannot tell whether the container runs. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
set-ceph-version.sh wrote ceph_version into the manager environment only. The plays of the other environments never load that file; they got their Ceph pins from the ceph-ansible container's versions.yml, which names the release its image tag selects. A cephadm testbed runs no ceph-ansible container, so cephclient_version and ceph_image_version fall back to ceph_version, which those plays do not see, and `osism apply cephclient` fails on it. Write ceph_version into environments/configuration.yml as well, which every environment loads, next to ceph_cluster_fsid. The manager environment keeps its copy, which the testbed's own manager deployment reads. cfg-cookiecutter makes the same move for generated configurations. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Roger Luethi <luethi@osism.tech>
ideaship
force-pushed
the
ceph-stack-selectable
branch
from
October 1, 2026 13:32
8ba1984 to
f6dd01c
Compare
ideaship
marked this pull request as ready for review
October 2, 2026 10:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The testbed has carried a deploy script for each Ceph backend for a while,
scripts/deploy/100-ceph-with-ansible.shand100-ceph-with-cephadm.sh, anddeploy-manager.shlinks both into/usr/local/bin. Nothing chose between them:deploy-services.shanddeploy-ceph.shnamed the ceph-ansible script outright, so a cephadm deployment could only be had by running the script by hand after the fact.The variable to choose with already exists too. terraform's
ceph_stackis written into/opt/manager-vars.shasCEPH_STACK, which every script here sources, andterraform/Makefilepasses it through from a make variable. It was inert because nothing read it.Dispatch on it.
deploy_cephininclude.shpicks the script for the configured backend and fails on an unknown value rather than silently deploying the default, and the two deploy wrappers call it.deploy.ymlthreads aceph_stackjob variable tomake, so a Zuul job can select the backend the same way it selectsceph_version, defaulting toceph-ansiblewhere it says nothing.The upgrade path deliberately does not follow. Upgrading Ceph under cephadm runs through
ceph orch upgradeand there is no script for it, soupgrade-services.shandupgrade-ceph.shrefuse on a cephadm cluster instead of running the ceph-ansible upgrade against one. Inupgrade-services.shthe refusal is skipped whenSKIP_CEPH_UPGRADEis already set, so the rest of an upgrade can still be exercised.The Ceph check keeps no backend in its name.
check/100-ceph-with-ansible.shhad nothing ceph-ansible specific left in it: every command is the plain Ceph CLI through cephclient, which both backends deploy, and theceph-mons,ceph-mgrsandceph-osdsvalidators match either backend's daemon container names. It is renamed tocheck/100-ceph.shrather than duplicated behind a dispatch, and gains an orchestrator section under cephadm: it fails when a service runs fewer daemons than its placement asks for — down to none, the failure a healthy-lookingceph -shides. The backend is read from the cluster (ceph orch status), not fromCEPH_STACK, which a nutshell testbed leaves unset. The dedicated-key scope check (101-ceph-ssh-key-scope.sh) runs after it as before.A cephadm testbed runs no ceph-ansible container. On the latest track the manager's ceph-ansible image tag follows
ceph_version, and ceph-ansible has no branch past squid, so a Tentacle testbed would ask for an image that does not exist.000-manager.shsetsceph_ansible_enable: falsewhenCEPH_STACKiscephadm, the same way cfg-cookiecutter disables the container for Ceph releases ceph-ansible cannot deploy, and the manager waits for the container only when the manager configuration enables it, so a container disabled for any reason is not waited for. A nutshell testbed leavesCEPH_STACKunset because python-osism selects the backend there; the Ceph-release rule for that case is in #2961. Nothing on the cephadm path needs it: the Ceph version pins come from osism-ansible for numbered releases and from osism/defaults on the latest track, and the OSD LVM plays run in osism-ansible (osism/ansible-playbooks#576, used by #2959).Every environment sees the selected Ceph release.
set-ceph-version.shalso writesceph_versionintoenvironments/configuration.yml. Without a ceph-ansible container, whoseversions.ymlused to carry the Ceph pins,cephclient_versionandceph_image_versionfall back toceph_version, which only the manager environment had. cfg-cookiecutter makes the same move for generated configurations (osism/cfg-cookiecutter#882).Accepting the value also needs osism/terraform-base, which owns the
ceph_stackvariable and validates it.Part of a cross-repo effort:
🤖 Generated with Claude Code