Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@ jobs:
matrix:
python-version: ["3.10"]
activitysim: [locked, main]
os: [ubuntu-latest, windows-latest]
defaults:
run:
shell: bash -l {0}
name: linux-64-py${{ matrix.python-version }}-${{ matrix.activitysim }}
runs-on: ubuntu-latest
name: ${{ matrix.os }}-py${{ matrix.python-version }}-${{ matrix.activitysim }}
runs-on: ${{ matrix.os }}
steps:
# checkout the code in this repository
- uses: actions/checkout@v4
Expand Down Expand Up @@ -67,4 +68,15 @@ jobs:
ACTIVITYSIM_TEST_SOURCE: ${{ matrix.activitysim }}
UV_NO_SYNC: ${{ matrix.activitysim == 'main' && '1' || '0' }}
run: |
uv run --no-sync pytest ./test
# Linux runs the full regression suite; Windows additionally checks
# script startup, including its platform-specific launcher.
if [ "${{ runner.os }}" = "Windows" ]; then
uv run --no-sync pytest ./test/test_run_scripts.py
else
uv run --no-sync pytest ./test
fi

- name: Test scripts from an external environment
run: |
# Match ActivitySim's CI: no opt-in to this example's lockfile checks.
uv run --no-sync pytest ./test/test_run_scripts.py
17 changes: 13 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ uv run --locked scripts/run-small-sharrow.py
uv run --locked scripts/run-large-sharrow.py
```

The executable scripts can also be called directly from the repository root.
On POSIX systems, the executable scripts can also be called directly from the repository root.
They use `pyproject.toml` and `uv.lock`; no separate script environments or
script lockfiles are needed. Output goes into `scripts/run-small-sharrow-output`
or `scripts/run-large-sharrow-output`, respectively.
Expand All @@ -68,15 +68,24 @@ or `scripts/run-large-sharrow-output`, respectively.
Run the regression suite and script smoke tests against the locked release:

```sh
uv run --locked pytest test
ACTIVITYSIM_TEST_SOURCE=locked uv run --locked pytest test
```

CI runs this suite against both the locked ActivitySim release and current
In PowerShell, first set `$env:ACTIVITYSIM_TEST_SOURCE = "locked"`, then run
`uv run --locked pytest test`.

CI runs the full suite on Linux and the script smoke tests on Windows, against
both the locked ActivitySim release and current
ActivitySim `main`. The main job replaces only ActivitySim, keeping the other
locked dependencies, and uses `--no-sync` to preserve that installation.
The script smoke tests validate startup, configuration, and filesystem behavior;
they do not download the full dataset or simulate 500,000 households.

When running the tests from an external ActivitySim development environment,
leave `ACTIVITYSIM_TEST_SOURCE` unset. This skips the project launcher and locked
version checks, which apply only to this example's environment. Model regression
and script configuration tests still run against the installed ActivitySim.

# Benchmarking

The `prototype_mtc` example model is run using the `activitysim` command line tool.
Expand All @@ -85,4 +94,4 @@ A quick and easy way to run the model for benchmarking is to use the following c
```shell
cd activitysim-prototype-mtc-extended
activitysim workflow performance-benchmarking
```
```
22 changes: 18 additions & 4 deletions test/test_run_scripts.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,19 @@
SCRIPTS = ["run-small-sharrow.py", "run-large-sharrow.py"]


@pytest.fixture
def project_environment():
# ActivitySim also runs this suite using its own environment and lockfile.
# Only opt into checks of this project's environment when it was installed
# explicitly; a launcher could otherwise sync away the version under test.
source = os.environ.get("ACTIVITYSIM_TEST_SOURCE")
if source not in ("locked", "main"):
pytest.skip("requires the example environment (ACTIVITYSIM_TEST_SOURCE=locked or main)")
return source


@pytest.mark.parametrize("script_name", SCRIPTS)
def test_script_launcher_uses_project_environment(tmp_path, script_name):
def test_script_launcher_uses_project_environment(tmp_path, script_name, project_environment):
# Execute the actual launcher and dependency metadata, replacing only the
# model body with a version probe so this test never launches a large run.
source = (ROOT / "scripts" / script_name).read_text()
Expand All @@ -28,8 +39,11 @@ def test_script_launcher_uses_project_environment(tmp_path, script_name):
'for p in ["activitysim", "sharrow", "numpy"]}))\n'
)
probe.chmod(0o755)
# Windows cannot execute a Python shebang. Use the documented uv command
# there, while continuing to exercise direct execution on POSIX systems.
command = ["uv", "run", "--locked", str(probe)] if os.name == "nt" else [str(probe)]
result = subprocess.run(
[str(probe)], cwd=ROOT, env=os.environ.copy(),
command, cwd=ROOT, env=os.environ.copy(),
text=True, capture_output=True, check=True, timeout=120,
)
versions = json.loads(result.stdout)
Expand Down Expand Up @@ -90,13 +104,13 @@ def configure(**kwargs):
download.assert_not_called()


def test_locked_dependencies_are_installed():
def test_locked_dependencies_are_installed(project_environment):
import tomli

with (ROOT / "uv.lock").open("rb") as stream:
packages = tomli.load(stream)["package"]
for name in ("activitysim", "sharrow", "numpy"):
if name == "activitysim" and os.environ.get("ACTIVITYSIM_TEST_SOURCE") == "main":
if name == "activitysim" and project_environment == "main":
continue # Only ActivitySim is replaced in the compatibility job.
locked = next(p["version"] for p in packages if p["name"] == name)
assert importlib.metadata.version(name) == locked
Loading