Repository navigation
Try running the backend and webapp image builds at the same time #567
Description
Part of #554. Change set E. Work on main. Rulings: #554 rulings 1, 2, 4, 5, 7, 11, 14, 15, 16.
Summary
- Today.
🏗️ Build Docker Imagesruns twodocker compose … buildcalls in series. The backend call must finish before the front call starts. - Measured cost. On six deploys, the backend call took 69–80 s and the front call took 414–487 s. The gap between them was 0 s. M3 equals their sum.
- After. Both calls start at once with
&. The step checks both exit codes withwait, so a failed call still fails the step. - Expected saving. 45–70 s off M3. The ceiling is the backend call time. CPU contention on the 4-vCPU host lowers the saving. Nobody has measured that contention.
- This is a trial over two normal deploys. The maintainer keeps or reverts it from M3 and the host numbers the step records.
| Run | Build job | M3 (s) | Backend call (s) | Front call (s) |
|---|---|---|---|---|
| 39398067267 | 118549257854 | 513 | 76 | 436 |
| 39297468969 | 118217497479 | 483 | 69 | 414 |
| 39275929831 | 118142985559 | 550 | 74 | 475 |
| 38392614215 | 115281929767 | 542 | 70 | 471 |
| 38239392350 | 114780192678 | 559 | 71 | 487 |
| 38068638114 | 114262648443 | 557 | 80 | 477 |
Each call runs from its first Image docsplus-<name>:<sha> Building log line to its Image docsplus-<name>:<sha> Built line. The backend uses docsplus-hocuspocus; the front uses docsplus-webapp. The Step 0.5 script reads the same lines.
The admin image finished 96–112 s after the front call started. So the backend call overlaps the most CPU-heavy part of the front call: the admin build and both webapp installs.
These six deploys are history only. Sets B and D change M3 before this issue lands. Step 0 reads a fresh baseline.
Depends on / Blocks
- Depends on Build the deploy images while the quality gates run #565. It moves
🏗️ Build Docker Imagesinto the newimagesjob. This issue edits the step in that job. - Lands after the Measure whether the backend image build still needs --no-cache #566 A/B is finished. Both read the backend call time, so they must not share deploys.
- Shares the step with Skip the duplicate type check inside the webapp Docker build #560. It adds a
--build-argto the front call. Keep whatever arguments the calls have onmain. - Reads the step name that Write a timing table to every CI/CD run summary #556 reads. Do not rename the step.
- Changes the build log that Measure the CI/CD after benchmark for v2.3 #569 and Measure and trim the Next.js build-trace step in the webapp build #568 parse. Every build line now carries a
[backend]or[front]prefix, after the GitHub timestamp. The two calls also reuse BuildKit ids such as#43.- Measure the CI/CD after benchmark for v2.3 #569
bench.tsalready matches the optional[front]or[backend]prefix (Measure the CI/CD after benchmark for v2.3 #569 Step 1). - Measure and trim the Next.js build-trace step in the webapp build #568 Step 5, item 4, already filters
[front]lines. - Before merge, confirm both still hold. If one does not, comment there.
- Measure the CI/CD after benchmark for v2.3 #569
- Blocks: nothing.
Current behaviour
Links are pinned to d0a53756f8676556358b6adc6c2397d2ed7c64c8. After #565, these lines sit in the images job at new line numbers. Step 0 finds them by name.
- The step runs the two calls in series (
prod.docs.plus.yml:612-642). The backend call is at:633-634:The front call is atdocker compose -f "${COMPOSE_FILE}" --env-file "${ENV_FILE}" \ build --no-cache rest-api hocuspocus-server
:639-640:docker compose -f "${COMPOSE_FILE}" --env-file "${ENV_FILE}" \ build webapp admin-dashboard
- Inside one call, the named services already build at once (
:636-638). The series is between the two calls. - The calls stay separate because of
--no-cache. Only the backend call passes it (:628-632). One call cannot apply the flag to some services only. - The compose file has no
no_cachekey, and other callers read it.- Build sections:
rest-apidocker-compose.prod.yml:170-176,hocuspocus-server:288-294,webapp:490-497,admin-dashboard:589-596. - Stage sets
COMPOSE_FILE: docker-compose.prod.yml(stage.docs.plus.yml:48). Prod sets it once (prod.docs.plus.yml:76). - The
buildandbuild-prod-ciMake targets read it too (Makefile:68-79). So ano_cache: truekey would change stage and the Makefile as well.
- Build sections:
- Host ([Tracking] Faster, change-aware CI/CD pipeline #554 §Host facts): 4 vCPUs.
next buildruns several workers, so two builds at once compete for CPU. The calls write the same layers as today, so disk use does not change. Prune Docker build cache toward a free-disk target and keep rollback images #564 owns disk. - Settled Docker rules: [Tracking] Faster, change-aware CI/CD pipeline #554 §Shared rules.
--parallelis not adocker compose buildflag in Compose v2 (AGENTS.md:300). Docker build timings do not transfer from a laptop (AGENTS.md:293). No build-once-then-docker tag(AGENTS.md:294). Never measure a cache fix under--no-cache(AGENTS.md:301). Prod deploys read only the pushed tip commit subject (AGENTS.md:416).
Target behaviour
- The step keeps its name,
🏗️ Build Docker Images, and its job. - The backend call and the front call start at the same time, in that one step. Each keeps its exact arguments from
main. - Each call streams live into the job log. Every line starts with
[backend]or[front], so a hung call still shows its log. - The step waits for both calls. When one call fails, the other runs to its end. A failed backend call then reports only when the front call ends, up to about 487 s later.
- The step fails when either call exits non-zero, and the error names both exit codes.
- The step summary gets an "Image build calls" block:
- the wall time of the step;
- each call's exit code and seconds;
load1_max,mem_avail_min_mb,swap_used_start_mb,swap_used_max_mband the sample count, sampled every 5 s;restarts_during_build,unhealthy_beforeandunhealthy_afterfor the live containers.
- The load sampler and the restart check stay after the trial.
- Nothing else changes: the compose file, the Dockerfiles, the other steps, stage and the Makefile.
Implementation steps
Read every file in Read first before Step 0. "Comment here" means a comment on this GitHub issue, never a code comment.
Step 0. Read and check the base
- Confirm Build the deploy images while the quality gates run #565 is merged on
main.At the pinned commit it printsgit show main:.github/workflows/prod.docs.plus.yml \ | awk '/^ [a-z][a-z0-9-]*:$/ {job=$1} /name: 🏗️ Build Docker Images/ {print job, NR}'
deploy: 612. It must now printimages:with some line number. If it printsdeploy:, stop and comment here. - Confirm the Measure whether the backend image build still needs --no-cache #566 A/B is finished. Its
backend_no_cacheinput must be gone fromworkflow_dispatch. After a drop, Measure whether the backend image build still needs --no-cache #566 must also have its second "Measured, " comment for the next two(build):deploys. If either check fails, stop and comment here. - Print the step and copy both calls.
git show main:.github/workflows/prod.docs.plus.yml | grep -n -A 60 "name: 🏗️ Build Docker Images"
- There must be exactly two
docker compose … buildcalls. The first namesrest-api hocuspocus-server. The second nameswebapp admin-dashboard. - Copy every line of the step's
env:map. For example, Skip the duplicate type check inside the webapp Docker build #560 addsSKIP_QUALITY_GATES. - Copy every argument of each call exactly. For example, Skip the duplicate type check inside the webapp Docker build #560 adds
--build-arg "NEXT_SKIP_TYPECHECK=${NEXT_SKIP_TYPECHECK}"to the front call, and the line that sets it. Measure whether the backend image build still needs --no-cache #566 may have changed--no-cache. - Copy any shell line that sets a variable a call uses.
- If the step is missing, has more or fewer than two calls, or a call names other services, stop and comment here.
- There must be exactly two
- Confirm the compose file has no
no_cachekey. Rungit grep -n no_cache main -- docker-compose.prod.yml. Expect no output. - Read the baseline, before you edit anything.
- List recent deploy runs:
gh run list --repo docs-plus/docs.plus --workflow prod.docs.plus.yml --event push --branch main \ --status success --limit 100 --json databaseId,createdAt,attempt,headSha
- Keep a run only when its attempt is 1 and its subject starts with
(build):. Read the subject withgh api repos/docs-plus/docs.plus/actions/runs/<id> -q .head_commit.message. - Take the last 3 such runs on
mainwhose build step ran in theimagesjob. The script below prints that job asbuild_job_name. It must not be🚀 Deploy Production. - Run
git fetch origin mainfirst, so every SHA exists locally. The last baseline SHA is theheadShaof the latest of the three runs. Each baseline run must build the same way asmaintoday:git show <headSha>:.github/workflows/prod.docs.plus.yml | grep -n -A 60 'name: 🏗️ Build Docker Images'must show the same two calls, with the same arguments, as Step 0.3.git diff --quiet <headSha> <last baseline SHA> -- apps/webapp/docker/Dockerfile.bun apps/admin-dashboard/docker/Dockerfile.bun apps/hocuspocus.server/docker/Dockerfile.bun docker-compose.prod.yml apps/webapp/next.config.js apps/admin-dashboard/next.config.jsmust exit 0.- If a run fails either check, skip it and walk further back. The
imagesjob proves only that Build the deploy images while the quality gates run #565 landed, not set B or the Measure whether the backend image build still needs --no-cache #566 result.
- If fewer than 3 such runs exist, stop and wait for more normal deploys. Do not fall back to runs that fail the checks.
- Note each run's cache class:
- Warm: the previous build started 24 h or less before it, and
git diff --quiet <previous SHA> <SHA> -- bun.lock extensions/exits 0. - The previous build is the latest earlier
prod.docs.plus.ymlrun, of any event or conclusion, wherebun x2-read.ts <id>prints a non-nullm3. ItsheadShais the previous SHA. Compare the two runs'createdAtfor the 24 h rule. - List the candidates with
gh run list --repo docs-plus/docs.plus --workflow prod.docs.plus.yml --branch main --limit 100 --json databaseId,createdAt,event,conclusion,headSha. Walk back from the baseline run, one run at a time. - Cold: every other run.
- Warm: the previous build started 24 h or less before it, and
- Save this read-only script outside the repo as
x2-read.ts. Runbun x2-read.ts <runId> <runId> <runId>.// bun x2-read.ts <runId> ... → one JSON row per run (read-only gh calls) const R = "docs-plus/docs.plus"; const gh = (...a: string[]) => Bun.spawnSync(["gh", ...a]).stdout.toString(); for (const id of Bun.argv.slice(2)) { const d = JSON.parse(gh("run", "view", id, "--repo", R, "--json", "jobs,createdAt,attempt,headSha")); const t0 = Date.parse(d.createdAt); const off = (s?: string) => (s ? Math.round((Date.parse(s) - t0) / 1000) : null); const dur = (s: any) => (s ? Math.round((Date.parse(s.completedAt) - Date.parse(s.startedAt)) / 1000) : null); const steps = d.jobs.flatMap((j: any) => (j.steps ?? []).map((s: any) => ({ ...s, job: j }))); const step = (re: RegExp) => steps.find((s: any) => re.test(s.name)); const build = steps.find((s: any) => /Build Docker Images/.test(s.name) && s.conclusion !== "skipped"); const log = build ? gh("run", "view", "--repo", R, "--job", String(build.job.databaseId), "--log") : ""; const at = (re: RegExp) => { const m = log.split("\n").find((l) => re.test(l))?.match(/\d{4}-\d\d-\d\dT[\d:.]+Z/); return m ? Date.parse(m[0]) : NaN; }; const sec = (a: number, b: number) => (Number.isNaN(a) || Number.isNaN(b) ? null : Math.round((b - a) / 1000)); const tag = d.headSha; const bs = at(new RegExp(`Image docsplus-hocuspocus:${tag} Building`)); const be = at(new RegExp(`Image docsplus-hocuspocus:${tag} Built`)); const fs = at(new RegExp(`Image docsplus-webapp:${tag} Building`)); const fe = at(new RegExp(`Image docsplus-webapp:${tag} Built`)); console.log(JSON.stringify({ id, attempt: d.attempt, sha: tag.slice(0, 9), build_job: build?.job.databaseId ?? null, build_job_name: build?.job.name ?? null, m1: off(step(/Verify Deployment/)?.completedAt), m2g: off(d.jobs.find((j: any) => /Build Verification/.test(j.name))?.completedAt), m2i: off(build?.completedAt), m3: dur(build), back_s: sec(bs, be), front_s: sec(fs, fe), overlap_s: Math.max(0, sec(Math.max(bs, fs), Math.min(be, fe)) ?? 0), })); }
- Check the script on run 39398067267 first. It prints
build_job_name🚀 Deploy Production,m3513,back_s76,front_s436 andoverlap_s0. A run whose build step was skipped or never started printsm3null. - Post the three rows and their cache classes as a comment here, headed "Baseline, ". Name the full last baseline SHA.
- List recent deploy runs:
- Check that the image path is the long one. After Build the deploy images while the quality gates run #565, M1 follows the later of M2g and M2i. If
m2iis belowm2gon all three baseline runs, this issue cannot lower M1. Stop, and comment here with the numbers and a proposal to close this issue.
Step 1. Run the two calls at once
File: .github/workflows/prod.docs.plus.yml, step 🏗️ Build Docker Images, in the images job.
- Keep the step's
nameandenvlines. Keep the comment block above the backend call, which starts "rest-api, hocuspocus-server and hocuspocus-worker share one build definition". - Replace the rest of the
run:block with the block below. Put the two calls from Step 0.3 in the twobuild_calllines. Write only the arguments that followbuild, not thedocker composeprefix. - Delete the old 3-line comment that starts "No --parallel: it is not a
docker compose buildflag". The new comment replaces it.
- name: 🏗️ Build Docker Images
env:
DOCKER_BUILDKIT: '1'
COMPOSE_DOCKER_CLI_BUILD: '1'
# Keep every other env line from main here, such as SKIP_QUALITY_GATES.
run: |
echo "🔨 Building images with tag: ${DEPLOY_TAG}"
# (keep the existing comment block about the shared backend build and --no-cache here)
BUILD_DIR="${RUNNER_TEMP}/image-build"
mkdir -p "${BUILD_DIR}"
: > "${BUILD_DIR}/times.txt"
# Sum of restart counts over running containers; a rise means a live container died during the build.
restarts() { { docker ps -q | xargs -r docker inspect -f '{{.RestartCount}}' 2>/dev/null || true; } | awk '{s += $1} END {print s + 0}'; }
unhealthy() { { docker ps -q --filter health=unhealthy || true; } | awk 'END {print NR}'; }
RESTARTS_BEFORE=$(restarts)
UNHEALTHY_BEFORE=$(unhealthy)
# Host load, available memory and used swap every 5 s, for the keep-or-revert call (#567).
( while sleep 5; do
printf '%s %s %s\n' "$(date +%s)" "$(cut -d' ' -f1 /proc/loadavg)" \
"$(awk '/^MemAvailable:/ {a=$2} /^SwapTotal:/ {t=$2} /^SwapFree:/ {f=$2} END {print int(a/1024), int((t-f)/1024)}' /proc/meminfo)"
done ) > "${BUILD_DIR}/host.txt" 2>/dev/null &
SAMPLER_PID=$!
# Stop the sampler and both build calls on every exit, also on a cancel.
stop_all() {
kill "${SAMPLER_PID}" 2>/dev/null || true
for p in "${BACK_PID:-}" "${FRONT_PID:-}"; do
[ -z "$p" ] && continue
pkill -TERM -P "$p" 2>/dev/null || true
kill "$p" 2>/dev/null || true
done
}
trap stop_all EXIT
trap 'exit 143' INT TERM
# Streams one compose build call with a name prefix, so a hung call still shows its log.
build_call() {
local name=$1 start=$SECONDS rc=0
shift
docker compose -f "${COMPOSE_FILE}" --env-file "${ENV_FILE}" build "$@" 2>&1 \
| sed -u "s/^/[${name}] /" || rc=$?
echo "${name} exit=${rc} seconds=$((SECONDS - start))" >> "${BUILD_DIR}/times.txt"
return "${rc}"
}
# The calls stay separate because only the backend passes --no-cache. They run
# at once, and Compose v2 has no --parallel flag (AGENTS.md §Docker).
STEP_START=$SECONDS
build_call backend --no-cache rest-api hocuspocus-server &
BACK_PID=$!
build_call front webapp admin-dashboard &
FRONT_PID=$!
BACK_RC=0
wait "${BACK_PID}" || BACK_RC=$?
FRONT_RC=0
wait "${FRONT_PID}" || FRONT_RC=$?
kill "${SAMPLER_PID}" 2>/dev/null || true
RESTARTS_AFTER=$(restarts)
UNHEALTHY_AFTER=$(unhealthy)
HOST_LINE=$(awk 'NR == 1 {l = $2; m = $3; s0 = $4; s = $4} {if ($2 > l) l = $2; if ($3 < m) m = $3; if ($4 > s) s = $4} END {printf "load1_max %s, mem_avail_min_mb %s, swap_used_start_mb %s, swap_used_max_mb %s, samples %d", l, m, s0, s, NR}' "${BUILD_DIR}/host.txt")
{
echo "### 🏗️ Image build calls"
echo "- wall: $((SECONDS - STEP_START)) s"
sed 's/^/- /' "${BUILD_DIR}/times.txt"
echo "- host: ${HOST_LINE}"
echo "- live containers: restarts_during_build $((RESTARTS_AFTER - RESTARTS_BEFORE)), unhealthy_before ${UNHEALTHY_BEFORE}, unhealthy_after ${UNHEALTHY_AFTER}"
} | tee -a "${GITHUB_STEP_SUMMARY}"
if [ "${BACK_RC}" -ne 0 ] || [ "${FRONT_RC}" -ne 0 ]; then
echo "::error::Image build failed: backend exit ${BACK_RC}, front exit ${FRONT_RC}. Read the [backend] and [front] lines above."
exit 1
fi
echo "✅ Images built"Notes for the builder:
|| BACK_RC=$?.prod.docs.plus.yml:64-67setsshell: bash, so the step runsbash --noprofile --norc -eo pipefail. A barewaiton a failed call exits before the second exit code is read.|| rc=$?inbuild_call. Underpipefail, the pipe status is thedocker composeexit code. Without the guard,set -eends the call before it writestimes.txt.stop_all. A&job in non-interactive bash ignores SIGINT. So the step stops both calls and theirdocker composechildren itself.|| trueinsidestop_all,restartsandunhealthy.set -ealso applies inside a trap, and a container can stop betweendocker psanddocker inspect. Either would change the step's exit code.- The front call with Skip the duplicate type check inside the webapp Docker build #560. Keep its two lines that set
NEXT_SKIP_TYPECHECKaboveSTEP_START. The call then readsbuild_call front --build-arg "NEXT_SKIP_TYPECHECK=${NEXT_SKIP_TYPECHECK}" webapp admin-dashboard &. - The backend call after Measure whether the backend image build still needs --no-cache #566. If
--no-cacheis gone, the call readsbuild_call backend rest-api hocuspocus-server &. Then replace "because only the backend passes --no-cache" in the new comment with: "The calls stay separate so each keeps its own flags." {{.RestartCount}}has no$, so GitHub does not read it as an expression. Line 740 of the pinned file uses{{.Names}}the same way.- The
[backend]and[front]lines keep GitHub's timestamps, sox2-read.tsstill reads each call's seconds.
Check:
git diff --stat
git diff .github/workflows/prod.docs.plus.yml
git diff -U0 .github/workflows/prod.docs.plus.yml | grep -E '^[-+] +[A-Z_]+: 'Expect one file changed, only inside 🏗️ Build Docker Images. The third command must print nothing, so the env: lines are unchanged.
Step 2. Lint the workflow
Install shellcheck first, so actionlint also lints the run: block.
actionlint -config-file .github/actionlint.yaml .github/workflows/prod.docs.plus.ymlExpect exit code 0 and no output.
Step 3. Prove the exit-code logic with a fake docker
A one-off check outside the repo, not a committed test.
- Copy the
run:block from Step 1 intostep.shin a scratch folder. Remove the 10-space YAML indent. - Create
bin/dockerand runchmod +x bin/docker:#!/bin/bash case "$1" in ps|inspect) exit 0 ;; esac echo "fake docker $*" case "$*" in *rest-api*) sleep 2; [ "$FAIL_SVC" = back ] && exit 3 ;; *webapp*) sleep 3; [ "$FAIL_SVC" = front ] && exit 4 ;; esac exit 0
- Run the three cases from that folder:
for f in none back front; do mkdir -p "t-$f"; : > "t-$f/sum.md" PATH="$PWD/bin:$PATH" FAIL_SVC=$f RUNNER_TEMP="$PWD/t-$f" GITHUB_STEP_SUMMARY="$PWD/t-$f/sum.md" \ DEPLOY_TAG=abc COMPOSE_FILE=c.yml ENV_FILE=e.env bash --noprofile --norc -eo pipefail step.sh echo "FAIL_SVC=$f step exit=$?" done
- Pass rules:
noneends with✅ Images builtandstep exit=0. The wall time is about 3 s, not 5 s.backprintsbackend exit 3, front exit 0andstep exit=1.frontprintsbackend exit 0, front exit 4andstep exit=1.- Every fake line starts with
[backend]or[front]. grep -c -e '- backend exit=' -e '- front exit=' t-<case>/sum.mdprints2in every case.- On macOS,
/procdoes not exist, so the host line shows empty values. On Linux it shows numbers.
- Prove the final check by sabotage. Restore each line after its leg.
- Change
if [ "${BACK_RC}" -ne 0 ] || [ "${FRONT_RC}" -ne 0 ]; thentoif [ "${BACK_RC}" -ne 0 ]; then. Thefrontcase must print✅ Images builtandstep exit=0. - Change
wait "${BACK_PID}" || BACK_RC=$?towait "${BACK_PID}". Thebackcase must printstep exit=3with no summary block and no::error::line. - In
build_call, delete|| rc=$?from thesed -uline. Thebackcase printsstep exit=1, butt-back/sum.mdholds no- backend exit=line. - Run all three cases again. Expect the pass rules in item 4.
- Change
- Wait 6 s, then run
ps -ax -o pid,command | grep step.sh | grep -v grep. Expect no output. - Cancel check.
- In
bin/docker, changesleep 3tosleep 30. - Run
PATH="$PWD/bin:$PATH" FAIL_SVC=none RUNNER_TEMP="$PWD/t-none" GITHUB_STEP_SUMMARY="$PWD/t-none/sum.md" DEPLOY_TAG=abc COMPOSE_FILE=c.yml ENV_FILE=e.env bash --noprofile --norc -eo pipefail step.sh & S=$!; sleep 1; kill -TERM "$S"; sleep 3; ps -ax -o pid,command | grep '[b]in/docker'. - Expect no output. Use TERM, not INT, because a test shell starts
&jobs with SIGINT ignored. - Restore
sleep 3.
- In
Review checkpoint
Show git diff, the actionlint output and the Step 3 output. Confirm the log-prefix handling in #569 and #568. Then stop. Do not run git add, git commit, git push or git stash. The maintainer reviews the diff, then commits and pushes per SP2 (#554), as its own commit.
Acceptance criteria
- Step 0 checks passed, and the "Baseline, " comment holds three runs with their cache classes.
- The step is still named
🏗️ Build Docker Imagesand still sits in theimagesjob. - Both calls start with
&in that one step. Each keeps its exact arguments frommain. - The step's
env:lines are unchanged.git diff -U0 .github/workflows/prod.docs.plus.yml | grep -E '^[-+] +[A-Z_]+: 'prints nothing. - Before merge, Measure the CI/CD after benchmark for v2.3 #569 Step 1 and Measure and trim the Next.js build-trace step in the webapp build #568 Step 5, item 4, still handle the prefix, or a comment there names it.
-
docker-compose.prod.yml,stage.docs.plus.yml,Makefileand every Dockerfile are unchanged. - No
--parallelwas added anywhere. - actionlint exits 0 with the repo config.
- The Step 3 fake-
dockercases pass, and the sabotage case failed as described. - The Step 3 cancel check leaves no fake
dockerprocess. - On the first real deploy, the step summary shows the "Image build calls" block with numbers in every field.
- The job log shows
[backend]and[front]lines that overlap in time. - Two "Measured, " comments exist here, one per deploy.
- A maintainer ruling comment says keep or revert, and the
AGENTS.md§Docker bullet records the result.
Verification
Before merge:
actionlint -config-file .github/actionlint.yaml .github/workflows/prod.docs.plus.yml
git diff --exit-code docker-compose.prod.yml .github/workflows/stage.docs.plus.yml Makefile
grep -n -- "--parallel" .github/workflows/prod.docs.plus.yml | grep -v "#"- The second command must exit 0. The third must print nothing; it drops comment lines, because the new comment names
--parallel. bash .github/scripts/parse-build-trigger.test.shand adry_run_subjectdispatch do not apply. This issue changes no triage, jobif,needsor gating.
After merge, read each of the first two deploys with bun x2-read.ts <runId> (Step 0.5), and copy the "Image build calls" block from the step summary. Expected values are in Accountability. overlap_s must be close to back_s, which shows the calls ran at once. front_s minus the baseline front_s of the same class is the contention cost.
Accountability
- Who pushes. The maintainer pushes the two measured deploys, at an hour the maintainer judges quiet. They are the first two
(build):push runs whose tree contains this change, and the push that carries it can be the first. They are normal deploys with real changes; noforce_deploydispatch counts. - A measured deploy counts only when its two calls carry the Step 0.3 arguments and
git diff --quiet <last baseline SHA> <measured SHA> -- apps/webapp/docker/Dockerfile.bun apps/admin-dashboard/docker/Dockerfile.bun apps/hocuspocus.server/docker/Dockerfile.bun docker-compose.prod.yml apps/webapp/next.config.js apps/admin-dashboard/next.config.jsexits 0. Otherwise comment here, take a new Step 0.5 baseline, and measure the next deploys. - Who reads. The builder of this issue, or the agent the maintainer names, reads each run after
🩺 Verify Deploymentends. - Record per SP3 ([Tracking] Faster, change-aware CI/CD pipeline #554), one comment per deploy. Metrics as defined in Measure the CI/CD after benchmark for v2.3 #569.
- Metric row: M3, read from each of the two measured runs, against the class baseline, expected 45–70 s lower.
- M2i falls by the same amount. M1 falls by the M3 saving only when M2i is later than M2g. M2g, M4, M5, W and M6 stay unchanged.
- Also add the build job id, the cache class and its baseline runs,
back_s,front_s,overlap_s, and the "Image build calls" block.
- The class baseline is the median M3 of the Step 0.5 runs in the deploy's cache class. With two runs, it is their mean. With one run, it is that run's M3. With none, it is the median M3 of all three baseline runs, and the comment says so.
- Who rules. The maintainer, in one comment after the second deploy.
- Keep when both deploys pass every stop condition, and M3 is at least 30 s below the class baseline on both.
- Stop condition: revert when any of these holds on either deploy:
- either call fails with exit code 137,
signal: killedorcannot allocate memory; mem_avail_min_mbis below 2048;swap_used_max_mbminusswap_used_start_mbis above 512;restarts_during_buildis above 0, orunhealthy_afteris aboveunhealthy_before;🩺 Verify Deploymentfails on that run;- M3 is not below the class baseline.
- either call fails with exit code 137,
- A stop condition on the first measured deploy. Do not wait for a second deploy. Comment here the same day with the numbers. The maintainer reverts before the next
(build):push. - A mixed result (both pass every stop condition, but one saves less than 30 s) needs a third deploy. Comment here with the numbers. The maintainer rules after the third one.
Rollback
Revert per SP4 (#554) on a stop condition in Accountability. The commit touches only .github/workflows/prod.docs.plus.yml. No host step is needed: the step writes only under ${RUNNER_TEMP}, which the runner clears after each job. A failed build leaves production unchanged, because the build ends before 🔧 Ensure Infrastructure.
Docs to update
- In this change: only the new comment inside the step (Step 1). It replaces the old
--parallelcomment. - After the maintainer's ruling, in the revert commit, or in its own commit for a keep: add one bullet to
AGENTS.md§Docker after the--parallelbullet (line 300 at the pinned commit). Name the two run ids, M3 before and after, the cache class,mem_avail_min_mb, and "kept" or "reverted". .github/WORKFLOWS.md: no change. It does not describe the image build calls.
Out of scope
- Anything the settled Docker rules ([Tracking] Faster, change-aware CI/CD pipeline #554 §Shared rules) forbid.
- Do not touch
stage.docs.plus.yml:260orMakefile:78. Both still passbuild --paralleltoday. The maintainer may file that as its own issue. - No
no_cache: trueindocker-compose.prod.yml. - Do not merge the two calls into one, and do not change
--no-cache. Measure whether the backend image build still needs --no-cache #566 owns that flag. - No Dockerfile change and no BuildKit cache mount. Do not change the Next build worker count. Do not add a CPU or memory limit to the build.
- No change to the
imagesjob'sif,needs,concurrencyortimeout-minutes. Build the deploy images while the quality gates run #565 owns them. - No disk guard or prune change. Prune Docker build cache toward a free-disk target and keep rollback images #564 owns it.
- No change to the runner setup. Restrict the production self-hosted runner to main deploys #552 owns runner hardening.
- No change to the Bun version, the metascraper pin, or the
oven/bun:1-slimbase tag. The Bun floor stays 1.4.0 (AGENTS.md:66).
Read first
AGENTS.md, mainly §Docker, §Git And Commits, §Test Policy and §Simplified English Mandate..github/workflows/prod.docs.plus.yml, the wholeimagesjob onmain.docker-compose.prod.yml, thebuild:sections ofrest-api,hocuspocus-server,webappandadmin-dashboard..github/WORKFLOWS.md.- [Tracking] Faster, change-aware CI/CD pipeline #554, then Build the deploy images while the quality gates run #565, Measure whether the backend image build still needs --no-cache #566, Skip the duplicate type check inside the webapp Docker build #560 and Write a timing table to every CI/CD run summary #556.
Activity