Skip to content

Initial Support for Slurm 26.05 - #742

Merged
hanwen-cluster merged 1 commit into
aws:developfrom
hanwen-cluster:developslurm265
Oct 14, 2026
Merged

hanwen-cluster merged 1 commit into
aws:developfrom
hanwen-cluster:developslurm265

Conversation

@hanwen-cluster

@hanwen-cluster hanwen-cluster commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Description of changes

Slurm 26.05 adds two keys to each job in the SLURM_RESUME_FILE:

Changelog: 26.05.0rc1 ("Add job priority value to SLURM_RESUME_FILE"), L465 ("Add exclusive to the SLURM_RESUME_FILE"). Code: power_save.c.

SlurmResumeJob(**job) raised TypeError: unexpected keyword argument 'exclusive', which failed the whole resume. As a result, ParallelCluster could not launch dynamic nodes with Slurm 26.05.

This commit:

Backward compatible: Slurm 25.11 doesn't emit either key, so both default to None and parsing is unchanged.

Note: this is an initial support for Slurm 26.05 to allow ParallelCluster scale correctly. This commit doesn't upgrade consider incorporating new Slurm features.

Tests

Build image is successful. The following tests have passed:

{%- import 'common.jinja2' as common with context -%}
{{- common.OSS_COMMERCIAL_X86.append("rocky8") or "" -}}
{{- common.OSS_COMMERCIAL_X86.append("rocky9") or "" -}}
---
test-suites:
  basic:
    test_essential_features.py::test_essential_features:
...
  schedulers:
    test_slurm.py::test_slurm:
...
    test_slurm.py::test_slurm_from_login_nodes_in_private_network:
...
    test_slurm.py::test_slurm_scaling:
...
    test_slurm.py::test_error_handling:
...
    test_slurm.py::test_clustermgtd_instance_id_matching:
...
    test_slurm.py::test_slurm_maintenance_reservation:
...
    test_slurm.py::test_slurm_protected_mode:
...
    test_slurm.py::test_slurm_protected_mode_on_cluster_create:
...
    test_slurm.py::test_fast_capacity_failover:
...
    test_slurm.py::test_expedited_requeue:
...
    test_slurm.py::test_slurm_config_update:
...
    test_slurm.py::test_slurm_memory_based_scheduling:
...
    test_slurm.py::test_scontrol_reboot:
...
    test_slurm.py::test_scontrol_reboot_ec2_health_checks:
...
    test_slurm.py::test_slurm_overrides:
...
    test_slurm_accounting.py::test_slurm_accounting:
...
    test_slurm_accounting.py::test_slurm_accounting_external_dbd:
...
    test_slurm.py::test_slurm_custom_config_parameters:
...
    test_slurm.py::test_slurm_ticket_17399:
...
    test_slurm.py::test_slurm_custom_partitions:
...
    test_custom_munge_key.py::test_custom_munge_key:
...
    test_slurm_rest_api.py::test_slurm_rest_api:
...
  pyxis:
    test_pyxis.py::test_pyxis:
...

Checklist

  • Make sure you are pointing to the right branch.
  • If you're creating a patch for a branch other than develop add the branch name as prefix in the PR title (e.g. [release-3.6]).
  • Check all commits' messages are clear, describing what and why vs how.
  • Make sure to have added unit tests or integration tests to cover the new/modified code.
  • Check if documentation is impacted by this change.

Please review the guidelines for contributing and Pull Request Instructions.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@hanwen-cluster
hanwen-cluster requested review from a team as code owners October 14, 2026 12:16
Slurm 26.05 adds two keys to each job in the `SLURM_RESUME_FILE`:
- [exclusive](SchedMD/slurm@7f63a8c)
- [priority](SchedMD/slurm@219d033)

Changelog: [26.05.0rc1](https://github.com/SchedMD/slurm/blob/476de143d0fd4137fd4606882a68443965529ba3/CHANGELOG/slurm-26.05.md?plain=1#L332) ("Add job priority value to SLURM_RESUME_FILE"), [L465](https://github.com/SchedMD/slurm/blob/476de143d0fd4137fd4606882a68443965529ba3/CHANGELOG/slurm-26.05.md?plain=1#L465) ("Add exclusive to the SLURM_RESUME_FILE"). Code: [power_save.c](https://github.com/SchedMD/slurm/blob/slurm-26-05-5-1/src/slurmctld/power_save.c#L650-L671).

`SlurmResumeJob(**job)` raised `TypeError: unexpected keyword argument 'exclusive'`, which failed the whole resume. As a result, ParallelCluster could not launch dynamic nodes with Slurm 26.05.

This commit:
- [Must have] Accepts `exclusive` and `priority`, and ignores any key a future Slurm adds, so new Resume File fields no longer break scaling.
- [Cleanup BTW] Removes `SlurmResumeJob.is_exclusive()`. It had no caller since aws#574 (job-level scaling for all jobs).

Backward compatible: Slurm 25.11 doesn't emit either key, so both default to `None` and parsing is unchanged.

Note: this is an initial support for Slurm 26.05 to allow ParallelCluster scale correctly. This commit doesn't consider incorporating new Slurm features.
@hanwen-cluster hanwen-cluster changed the title Adapt ParallelCluster node code for Slurm 26.05 Initial Support for Slurm 26.05 Oct 14, 2026
Comment on lines -143 to -144
def is_exclusive(self):
return self.oversubscribe == JobOversubscribe.NO

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is dangling code, so is removed.
See details in the PR description

@hanwen-cluster
hanwen-cluster merged commit b625438 into aws:develop Oct 14, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants