feat(hosting): ship the /setup wizard with the framework (#351)

CHANGELOG.md

@@ -12,6 +12,21 @@ All notable changes to this project are documented in this file. The format is b
 ## [Unreleased]
 
 ### Added
+- **The `/setup` wizard ships with the framework** (#351). Since 0.0.33
+  `SetupMiddleware` redirected a fresh install to `/setup`, but the route and
+  page lived only in this repository's unpublished host, so any other host got
+  `/` → `/setup` → 404. `create_app` now mounts the wizard
+  (`simple_module_hosting.setup_wizard`) and `gen-pages` registers its page
+  (`Setup/Wizard`) from the wheel; a host needs no setup code and should delete
+  any copy it carries. Steps complete from the browser through a new
+  `SetupStep.action` (`SetupAction` / `SetupField` in `simple_module_core`),
+  POSTed to `/setup/steps/<id>` with the session CSRF token. An action runs only
+  while its own step is pending (409 otherwise); `users` ships the
+  first-administrator action, which re-checks under a database lock inside the
+  inserting transaction so concurrent requests create one superuser. Required
+  steps without an action are logged at boot. The old host-only
+  `/setup/administrator`, `/setup/migrations` and the UI-less
+  `/setup/site-basics` endpoints are gone.
 - **Postgres test runs** (#343) — `SM_TEST_DATABASE_URL` points the
   `simple_module_test` fixtures at Postgres, and `make test-py-pg` runs the
   whole Python suite there. The schema is reset once per test, so `app` and

CLAUDE.md

@@ -76,7 +76,7 @@ cascade layer is inert, while unlayered CSS beats every Tailwind utility —
 hence `SM022`/`SM023`. See `docs/module-authoring.md` § Styling.
 
 **Lifecycle hooks** (in `framework/core/simple_module_core/module.py`) — all no-op by default; subclasses override as needed:
-`register_settings` → `register_menu_items` / `register_permissions` / `register_feature_flags` / `register_event_handlers` / `register_invalidations` / `register_health_checks` / `register_public_routes` / `register_csp_sources` / `register_setup_steps` → `register_exception_handlers` → `register_middleware` → `register_routes(api_router, view_router)` / `register_admin_routes(admin_router)` → async `on_startup` / `on_shutdown` (reverse order). `register_admin_routes` is only for modules that serve **both** public and admin pages: a module gets exactly one router per `view_prefix`, which `users` cannot express (sign-in at `/users/login`, management at `/admin/users`). Setting `ModuleMeta.admin_view_prefix` mounts a second view router there. A module whose views are *all* administrative just points `view_prefix` at `/admin/<name>` and keeps using `register_routes`. The prefix is a URL convention, not a permission — guard these routes exactly as you would any other. `register_csp_sources(registry)` lets a module whitelist external asset origins (`registry.add("style-src", "https://rsms.me")`) — fetch directives only, validated at boot. `register_public_routes(registry)` lets a module exempt anonymous/read-only routes (STAC/OGC, webhooks) from `AuthMiddleware`; rules are method-aware (`registry.add_regex(r"…/tilejson$", methods={"GET"})`), so a GET read route can be public while sibling POST/PATCH mutations under the same prefix stay gated. See [docs/framework/public-routes.md](docs/framework/public-routes.md). `register_setup_steps(registry)` lets a module declare what a usable install still needs; while any required step is incomplete `SetupMiddleware` serves the first-run wizard at `/setup` instead of the app. A module that registers nothing never gates — that is how `keycloak` opts out, since its local users table is legitimately empty forever and a host-level superuser count would lock those installs out permanently. `register_invalidations(bus, app)` subscribes a module's **per-process caches** to `InvalidationBus`, so another worker's write drops this worker's entry instead of leaving it stale for its whole TTL; handlers may only *forget*, since there is no delivery guarantee. Publishing takes no hook — `await request.app.state.sm.invalidation.publish(channel, key=...)` from a `db.on_commit` callback. Cross-process delivery needs a transport, which `background_tasks` installs on its Redis connection (`SM_BG_TASKS_BROADCAST_INVALIDATIONS`); with none the bus is in-process and every cache still needs its TTL as a floor. See [docs/framework/invalidation.md](docs/framework/invalidation.md).
+`register_settings` → `register_menu_items` / `register_permissions` / `register_feature_flags` / `register_event_handlers` / `register_invalidations` / `register_health_checks` / `register_public_routes` / `register_csp_sources` / `register_setup_steps` → `register_exception_handlers` → `register_middleware` → `register_routes(api_router, view_router)` / `register_admin_routes(admin_router)` → async `on_startup` / `on_shutdown` (reverse order). `register_admin_routes` is only for modules that serve **both** public and admin pages: a module gets exactly one router per `view_prefix`, which `users` cannot express (sign-in at `/users/login`, management at `/admin/users`). Setting `ModuleMeta.admin_view_prefix` mounts a second view router there. A module whose views are *all* administrative just points `view_prefix` at `/admin/<name>` and keeps using `register_routes`. The prefix is a URL convention, not a permission — guard these routes exactly as you would any other. `register_csp_sources(registry)` lets a module whitelist external asset origins (`registry.add("style-src", "https://rsms.me")`) — fetch directives only, validated at boot. `register_public_routes(registry)` lets a module exempt anonymous/read-only routes (STAC/OGC, webhooks) from `AuthMiddleware`; rules are method-aware (`registry.add_regex(r"…/tilejson$", methods={"GET"})`), so a GET read route can be public while sibling POST/PATCH mutations under the same prefix stay gated. See [docs/framework/public-routes.md](docs/framework/public-routes.md). `register_setup_steps(registry)` lets a module declare what a usable install still needs; while any required step is incomplete `SetupMiddleware` serves the first-run wizard at `/setup` instead of the app. The wizard ships with `simple_module_hosting` (`setup_wizard/` — routes mounted by `create_app`, page `Setup/Wizard` registered by `gen-pages`), so hosts carry no setup code; a step completes from the browser through its `SetupAction`, which runs only while *that* step is pending. A module that registers nothing never gates — that is how `keycloak` opts out, since its local users table is legitimately empty forever and a host-level superuser count would lock those installs out permanently. `register_invalidations(bus, app)` subscribes a module's **per-process caches** to `InvalidationBus`, so another worker's write drops this worker's entry instead of leaving it stale for its whole TTL; handlers may only *forget*, since there is no delivery guarantee. Publishing takes no hook — `await request.app.state.sm.invalidation.publish(channel, key=...)` from a `db.on_commit` callback. Cross-process delivery needs a transport, which `background_tasks` installs on its Redis connection (`SM_BG_TASKS_BROADCAST_INVALIDATIONS`); with none the bus is in-process and every cache still needs its TTL as a floor. See [docs/framework/invalidation.md](docs/framework/invalidation.md).
 
 `MenuRegistry.add_provider(fn)` (from `register_menu_items`) contributes per-request menu items evaluated in `InertiaLayoutDataMiddleware` after auth/tenant resolution, and `PermissionRegistry.add_source(name, provider)` (from `register_permissions`) contributes runtime-defined permissions from a sync in-memory cache, refreshed with `invalidate_source(name)`; see [docs/framework/permissions.md](docs/framework/permissions.md).
 

Makefile

@@ -135,7 +135,7 @@ ci-js-typecheck:
 			exit 1; \
 		fi; \
 	done
-	@for cfg in modules/*/tsconfig.json packages/*/tsconfig.json; do \
+	@for cfg in modules/*/tsconfig.json packages/*/tsconfig.json framework/*/tsconfig.json; do \
 		[ -f "$$cfg" ] || continue; \
 		echo "tsc -p $$cfg"; \
 		npx tsc --noEmit -p "$$cfg" || exit 1; \

biome.json

@@ -7,6 +7,8 @@
       "modules/*/*/pages/**",
       "modules/*/*/**/components/**",
       "modules/*/tests-js/**",
+      "framework/hosting/simple_module_hosting/setup_wizard/**",
+      "framework/hosting/tsconfig.json",
       "!.claude",
       "!host/client_app/modules.generated.ts",
       "!host/client_app/modules.manifest.json",

docs/module-authoring.md

@@ -574,39 +574,79 @@ origin/scheme token, validated at boot. See
 ## First-run setup steps
 
 A module can declare what an install still needs before it is usable. While
-any required step reports incomplete, `SetupMiddleware` serves the wizard at
-`/setup` instead of the app:
+any required step reports incomplete, `SetupMiddleware` redirects every
+request to the wizard at `/setup`. The wizard ships with
+`simple_module_hosting`: `create_app` mounts its routes and `smpy gen-pages`
+registers its page (`Setup/Wizard`), so a host needs no code of its own for it.
 
 ```python
-from simple_module_core import SetupRegistry, SetupStep
+from simple_module_core import SetupAction, SetupField, SetupRegistry, SetupStep
 
 
 async def has_administrator(app) -> bool:
     async with app.state.sm.db.session_factory() as session:
         ...  # return True once satisfied
 
 
+async def create_administrator(request, data: dict) -> dict:
+    ...  # validate `data`, lock, re-check, create; raise HTTPException to refuse
+    return {"created": True}
+
+
 class MyModule(ModuleBase):
     def register_setup_steps(self, registry: SetupRegistry) -> None:
         registry.add(
             SetupStep(
                 id="mymodule.administrator",
                 title="Create an administrator",
+                title_key="mymodule.setup.administrator.title",
                 description="An account that can sign in and manage this install.",
                 is_complete=has_administrator,
                 order=30,
+                action=SetupAction(
+                    handler=create_administrator,
+                    fields=[
+                        SetupField(name="email", label="Email", type="email"),
+                        SetupField(name="password", label="Password", type="password"),
+                    ],
+                    submit_label="Create administrator",
+                ),
             )
         )
 ```
 
-Three things are worth knowing before you add one.
+The wizard lists every registered step and, for each pending step with an
+`action`, renders `fields` as a form. Submitting it POSTs the values as JSON to
+`/setup/steps/<step id>`, which calls `handler(request, data)` and returns its
+dict. Titles, descriptions, field labels and the submit label reach the page as
+backend data, so give each a `*_key` into your module's catalog; an unresolved
+key falls back to the literal. A step with no `action` can only be completed
+out of band (a CLI command, an environment variable); the host logs each such
+required step at boot so an operator facing a form-less wizard can find out why.
+
+A few things are worth knowing before you add one.
 
 **Registering nothing is a valid answer, and it is how a module opts out.** The
 `users` module contributes the "an administrator exists" step; `keycloak`
 deliberately does not, because an install using an external identity provider
 has a legitimately empty local users table and a host-level superuser count
 would hold it behind the wizard forever.
 
+**An action runs only while its own step is pending.** The wizard answers 404
+once setup is complete, and 409 for a step that is already done even while
+other steps keep the wizard open. "Setup mode" alone is not a safe gate: the
+host always registers `host.migrations`, so an install whose schema falls
+behind head re-enters setup mode with its administrators intact. Every
+`/setup` mutation also carries the session's CSRF token
+(`simple_module_hosting.csrf`); the wizard page sends it for you.
+
+**An action that creates something unique must re-check under a lock.** The
+step check runs before your handler and outside its transaction, so two
+concurrent requests can both pass it. `users.setup_action` shows the pattern:
+take a database lock (`pg_advisory_xact_lock` on Postgres, a no-op `UPDATE`
+that claims SQLite's write lock), re-check the predicate, then insert, all in
+one transaction.
+
 **A step whose predicate raises counts as complete.** Failing closed on a
 transient database error would open an anonymous admin-creation form on a
 live install — that is failing *open* on security, so the framework fails the

framework/cli/simple_module_cli/templates/host/migrations/env.py

@@ -27,7 +27,9 @@
 config = context.config
 
 if config.config_file_name is not None:
-    fileConfig(config.config_file_name)
+    # Not the default disable_existing_loggers=True: the setup wizard runs this
+    # in-process, and that default would silence every app logger until restart.
+    fileConfig(config.config_file_name, disable_existing_loggers=False)
 
 target_metadata = build_module_metadata()
 include_object = make_include_object(target_metadata)

framework/cli/simple_module_cli/templates/host/migrations/script.py.mako

@@ -8,6 +8,7 @@ Create Date: ${create_date}
 from collections.abc import Sequence
 
 import sqlalchemy as sa
+import sqlmodel  # noqa: F401  (autogenerate may emit sqlmodel types)
 from alembic import op
 ${imports if imports else ""}
 

framework/cli/tests/test_scaffolded_host_setup_wizard.py

@@ -0,0 +1,57 @@
+"""A freshly scaffolded host gets a working /setup with no files of its own.
+
+GH #351: ``SetupMiddleware`` redirected every request to ``/setup`` while the
+route and page lived only in the framework repo's unpublished host, so a host
+made by ``smpy create-host`` answered ``/`` → ``/setup`` → 404. The wizard now
+ships with ``simple_module_hosting``: ``create_app`` mounts the route and
+``gen-pages`` registers the page, so the scaffold must stay free of both.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+
+import pytest
+from simple_module_hosting.manifest import write_module_pages_manifest
+from simple_module_hosting.setup_wizard import PAGES_NAME, pages_dir
+
+pytestmark = pytest.mark.anyio
+
+
+async def test_scaffold_carries_no_setup_code_and_resolves_the_wizard(tmp_path) -> None:
+    from simple_module_cli.scaffolding import create_host
+
+    dest = tmp_path / "demo"
+    create_host(dest, name="demo-host", modules=[])
+    client_app = dest / "client_app"
+
+    # Nothing host-side: no route module, no page.
+    assert not (dest / "routes_setup.py").exists()
+    assert not list((client_app / "pages").rglob("Setup*"))
+    assert "setup" not in (dest / "main.py").read_text(encoding="utf-8").replace(
+        "setup_logging", ""
+    )
+
+    # What `smpy gen-pages` writes for this host, with no modules installed.
+    write_module_pages_manifest([], client_app, repo_root=dest)
+
+    manifest = json.loads((client_app / "modules.manifest.json").read_text(encoding="utf-8"))
+    assert manifest[PAGES_NAME] == pages_dir().as_posix()
+    assert (pages_dir() / "Wizard.tsx").is_file()
+
+    generated = (client_app / "modules.generated.ts").read_text(encoding="utf-8")
+    assert f'"{PAGES_NAME}": import.meta.glob' in generated
+
+    # The scaffold's resolver keys a module glob entry as `<name>/<path under pages/>`,
+    # which is how `inertia.render("Setup/Wizard")` finds the page.
+    pages_ts = (client_app / "pages.ts").read_text(encoding="utf-8")
+    assert "pages[`${moduleName}/${match[1]}`]" in pages_ts
+    match = re.search(r"/pages/(.+)\.tsx$", (pages_dir() / "Wizard.tsx").as_posix())
+    assert match and f"{PAGES_NAME}/{match.group(1)}" == "Setup/Wizard"
+
+    # Tailwind must scan the wheel's wizard, and Vite must be allowed to serve it.
+    css = (client_app / "modules.generated.css").read_text(encoding="utf-8")
+    assert f'@source "{pages_dir().as_posix()}/**/*.{{ts,tsx}}";' in css
+    assets = json.loads((client_app / "modules.assets.json").read_text(encoding="utf-8"))
+    assert assets[PAGES_NAME]["pages"] == pages_dir().as_posix()

framework/core/simple_module_core/__init__.py

@@ -47,7 +47,7 @@
 from simple_module_core.permissions import PermissionRegistry
 from simple_module_core.public_routes import PublicRoute, PublicRouteRegistry
 from simple_module_core.services import Services
-from simple_module_core.setup_steps import SetupRegistry, SetupStep
+from simple_module_core.setup_steps import SetupAction, SetupField, SetupRegistry, SetupStep
 from simple_module_core.tenancy import TENANT_ROLE_PREFIX, TenantRole, is_tenant_role, tenant_role
 from simple_module_core.versioning import FRAMEWORK_API_VERSION, check_framework_compatibility
 
@@ -91,6 +91,8 @@
     "PublicRoute",
     "PublicRouteRegistry",
     "Services",
+    "SetupAction",
+    "SetupField",
     "SetupRegistry",
     "SetupStep",
     "TenantRole",

framework/core/simple_module_core/dotenv.py

@@ -38,7 +38,7 @@ def find_env_file() -> Path:
     settings layer (``BootstrapSettings``) and every out-of-process tool
     (diagnostics CLI, worker entrypoints, users bootstrap) resolve through
     here, so they can never disagree about which file is in effect. Compare
-    ``app_builder._resolve_project_root`` in the hosting package — a
+    ``_project_root.resolve_project_root`` in the hosting package — a
     separate walk that anchors the static/i18n root instead; the two are
     kept distinct on purpose (see that function's docstring).
     """