CI: Reduce workflow overhead - #3962
Open
kevinjqliu wants to merge 4 commits into
Open
kevinjqliu wants to merge 4 commits into
kevinjqliu wants to merge 4 commits into
Conversation
Overlap validation while retaining one-at-a-time ALLGREEN squash merges. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Apply the existing workflow/ref concurrency policy to policy and documentation checks, cancelling active predecessors only for pull requests. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use ubuntu-slim for eleven validation, reporting, artifact-merging, documentation, and maintenance jobs that do not require Docker or a full VM. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Build and serve documentation with the docs group without installing the unrelated default test dependency stack. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
kevinjqliu
requested review from
Fokko and
geruh
and
a lite review from Copilot
September 14, 2026 16:37
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The dependency-heavy integration report may exceed the ubuntu-slim 15-minute job limit and block the required gate.
Pull request overview
Reduces CI and merge-queue overhead through lighter documentation dependencies, slimmer runners, concurrency cancellation, and parallel queue builds.
Changes:
- Restricts documentation commands to the
docsdependency group. - Moves lightweight jobs to
ubuntu-slimand cancels redundant PR runs. - Increases merge-queue build parallelism from 1 to 2.
File summaries
| File | Summary |
|---|---|
Makefile |
Limits documentation commands to docs dependencies. |
.github/workflows/zizmor.yml |
Adds workflow concurrency controls. |
.github/workflows/svn-build-artifacts.yml |
Moves artifact merging to a slim runner. |
.github/workflows/stale.yml |
Moves stale processing to a slim runner. |
.github/workflows/python-release.yml |
Moves validation jobs to slim runners. |
.github/workflows/python-release-docs.yml |
Moves documentation release work to a slim runner. |
.github/workflows/python-integration.yml |
Moves selected integration jobs to slim runners; the dependency-heavy report job may exceed the 15-minute limit. |
.github/workflows/python-ci.yml |
Moves the aggregate gate to a slim runner. |
.github/workflows/pypi-build-artifacts.yml |
Moves artifact merging to a slim runner. |
.github/workflows/nightly-pypi-build.yml |
Moves version setup to a slim runner. |
.github/workflows/license_check.yml |
Adds workflow concurrency controls. |
.github/workflows/codeql.yml |
Adds concurrency controls and a slim runner. |
.github/workflows/check-md-link.yml |
Adds workflow concurrency controls. |
.github/workflows/asf-allowlist-check.yml |
Adds concurrency controls and a slim runner. |
.asf.yaml |
Allows two merge-queue entries to build concurrently. |
Review details
Suppressed comments (1)
.github/workflows/python-integration.yml:155
ubuntu-slimhas a hard 15-minute job limit, but this job performs a fulluv sync --group dev(including heavyweight development dependencies such as PySpark) before generating the report. A cold-cache run can hit that limit and fail the requiredpython-integration-requiredgate; keep this dependency-heavy report on a full runner, or reduce the install to only the report's dependencies before moving it to slim.
runs-on: ubuntu-slim
- Files reviewed: 15/15 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
rambleraptor
approved these changes
Sep 14, 2026
rambleraptor
left a comment
rambleraptor
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for doing this! Excited to see better CI performance, especially on the merge queue.
jayceslesar
reviewed
Sep 17, 2026
Comment on lines
38
to
+39
| asf-allowlist-check: | ||
| runs-on: ubuntu-24.04 | ||
| runs-on: ubuntu-slim |
Contributor
There was a problem hiding this comment.
is it possible to still be explicit about this versioning?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Grouping a few changes together in this PR. Overall, this helps improve CI and merge queue.
max_entries_to_buildfrom1to2. This allows merge queue to run in paralleldocs-serveanddocs-buildubuntu-slim. Hopefully this will reduce contention waiting for runnerconcurrencysetting to cancel redundant runs in the same PR