Skip to content

Raise bug-fix workflow credit cap and run pytest from PATH - #4508

Open
gyanu2507 wants to merge 2 commits into
github:mainfrom
gyanu2507:fix/4472-bug-fix-credits-pytest
Open

Raise bug-fix workflow credit cap and run pytest from PATH#4508
gyanu2507 wants to merge 2 commits into
github:mainfrom
gyanu2507:fix/4472-bug-fix-credits-pytest

Conversation

@gyanu2507

@gyanu2507 gyanu2507 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Description

The agentic bug-fix workflow hit the 1K default credit cap, then failed trying ./.venv/bin/python -m pytest with Permission denied.

This sets max-ai-credits: 2000, allows python as well as python3 on the bash allowlist, and tells the agent to use python3 -m pytest / pytest from PATH instead of a project venv interpreter.

Testing

  • Ran existing tests with uv sync && uv run pytest
  • uv run pytest tests/test_bug_fix_workflow.py

AI Disclosure

  • I did use AI assistance (describe below)

LLM-assisted implementation. I chose the issue, wrote the tests, and ran them locally.

Fixes #4472

The agentic bug-fix run hit the 1K default and then failed trying
./.venv/bin/python (Permission denied). Give the workflow 2K credits,
allow `python` as well as python3, and tell the agent to use PATH.

Fixes github#4472
@gyanu2507
gyanu2507 requested a review from mnriem as a code owner September 10, 2026 11:50
@mnriem

mnriem commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Thanks @gyanu2507 — a process heads-up. This one turns out to touch security-sensitive agentic-workflow infra: the change only takes effect once the compiled bug-fix.lock.yml is regenerated, and that has to be done with the exact pinned gh-aw compiler (v0.79.8), plus a careful diff to confirm nothing else in the lock (container digests, action SHAs, firewall allow-domains) shifts. Rather than ask you to chase our pinned toolchain, I'm going to pick up the recompile and finish this directly on your branch — you'll stay the author and I'll keep your commits.

Two small design tweaks I'm weighing as I do, with rationale to follow in the finishing commit: keeping the credit cap as-is if the pytest-from-PATH fix removes the retry loop that caused the ~8-credit overrun, and keeping the bash allowlist at python3 only unless bare python is genuinely needed. Appreciate you surfacing #4472 and getting it most of the way there.

@mnriem mnriem added the triage-must-have Verdict: high-value, important work for Spec Kit — do first label Sep 10, 2026
…fect

PR github#4508 raised max-ai-credits to 2000 and added `python` to the bash
allowlist in bug-fix.md, but never regenerated the compiled
bug-fix.lock.yml — the artifact GitHub Actions actually runs. Both
frontmatter-derived settings were therefore inert: the lock still pinned
maxAiCredits to the 1000 default and allowed shell(python3) but not
shell(python). (The prompt guidance was already live via {{#runtime-import}}.)

Recompile with gh-aw v0.79.8 (the pinned compiler) to apply exactly:
- agent + summary jobs' effective maxAiCredits: 1000 -> 2000
- harness allowlist: + shell(python)

Dependabot's action SHA pins in the committed lock (actions/checkout
v7.0.1, actions/setup-node v7.0.0) are preserved rather than reverted to
the compiler's older baked-in defaults. No container digest, firewall
domain, permission, or detection-job (400) change.

Rework tests/test_bug_fix_workflow.py to assert the compiled lock (the real
running artifact) for the cap and allowlist, and the runtime-imported
Markdown for the pytest-PATH guidance, replacing the prior .md-only string
checks that passed even while the running workflow was unchanged.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@gyanu2507

Copy link
Copy Markdown
Contributor Author

Sounds good. I'll leave the lockfile recompile and the credit / python allowlist tweaks to you.

@mnriem

mnriem commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Maintainer note: I (@mnriem) pushed one agent-assisted commit (5c52420) to complete this PR. Disclosure: it was generated autonomously by GitHub Copilot (model: Claude Opus 4.8) and reviewed by me before pushing.

The core gap was that .github/workflows/bug-fix.lock.yml — the compiled artifact GitHub Actions actually runs — was never regenerated, so two of the three changes were inert as merged: the effective maxAiCredits stayed at the compiled 1000 default, and the harness still allowed shell(python3) but not shell(python). (The pytest-PATH guidance was already live, since the lock imports the Markdown body at runtime via {{#runtime-import}}.)

I recompiled with the pinned gh-aw v0.79.8 and verified the lock diff is only the intended deltas: agent+summary maxAiCredits 1000 -> 2000 and + shell(python). Container image digests, action SHAs, firewall allow-domains, permissions, min-integrity, and the detection job's 400 cap are all byte-identical to main. One wrinkle worth flagging: a plain recompile wanted to revert Dependabot's committed action bumps (actions/checkout v7.0.1->v6.0.3, actions/setup-node v7.0.0->v6.4.0), so I preserved the v7 pins to avoid a downgrade.

I also reworked tests/test_bug_fix_workflow.py, which previously only asserted strings in bug-fix.md and stayed green even though the running workflow was unchanged; it now asserts the compiled lock for the cap and allowlist, plus the runtime-imported guidance.

One judgment call for reviewers: keeping the cap at 2000 is generous given the original overrun was ~0.84% and the real root cause was the permission-denied .venv interpreter (addressed by the prose). It stays bounded by the unchanged daily 5000 and detection 400 caps, and is a one-line revert if you prefer 1000.

CI has not run on this push and will need a maintainer trigger.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The source, generated workflow, and focused regression coverage consistently address both reported failures.

Pull request overview

Fixes #4472 by raising the bug-fix workflow’s credit budget and avoiding disallowed project-local Python executables.

Changes:

  • Raises AI credit limit to 2000.
  • Allows PATH-based python, python3, and pytest.
  • Adds regression tests covering source and compiled workflow consistency.
File summaries
File Description
.github/workflows/bug-fix.md Updates limits, allowlist, and test guidance.
.github/workflows/bug-fix.lock.yml Regenerates the executable workflow.
tests/test_bug_fix_workflow.py Guards the workflow configuration.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-must-have Verdict: high-value, important work for Spec Kit — do first

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Fix Bug from Labeled Issue exceeded max AI credits

3 participants