Repository navigation
feat(dev): add @data-fair/lib-dev and the data-fair-service skill - #63
Merged
Merged
Conversation
- dev/dev.json lists the port variables, the panes, the pages to open, setup steps and the config overrides of optional services; setup builds the ui workspace - df-dev setup / init-env: idempotent .env with free ports and a valid DEV_HOST, build-types, SPA build, repo-specific setup commands - df-dev zellij: attaches, recreates exited sessions or creates (zellij >= 0.44) - df-dev status / logs / restart (readiness wait) / up / down / wait-deps for agents - df-dev absent-config: NODE_CONFIG overrides for optional services not running - df-status: the status pane in bash (~4 MB instead of a ~55 MB node process) - df-mem-cap, df-worktree and df-worktree-delete with branch name validation - messages and docs suggest npx --no df-dev, scripts call the local binary
…onventions - migrating a service to lib-dev, its CI and NODE_CONFIG for optional services - tooling reference: eslint, commitlint, tsconfig, .gitignore template, node, Docker - settled conventions: test-env only in development, BUILD.json, /dev pages, vite - commit layout of a service branch - dev docker-compose reference: canonical service blocks, mongo/ES versions (the only place they change), shared dev secrets, maildev only where a mail flow exists - dev nginx reference: shared head mirroring the prod ingress settings, the 3 route shapes
Inside a capped cgroup node sizes its heap from the cap, about half of it: 2.2 GB under the default 4G. vite died of a V8 heap OOM at ~2.2 GB on the portals pilot while the cgroup still had room. df-mem-cap now exports --max-old-space-size at 3/4 of DEV_MEMORY_MAX (3072 MB for 4G), keeps the rest for native memory (rolldown), keeps the other NODE_OPTIONS and never overrides a heap size chosen by the caller.
The stash stack is shared by every worktree and session: the stash@{n} index computed at push
time could point to someone else's entry by the time it was popped.
…ose and CI rules - migrate-a-service: no env key in dev.json, data-fair's NGINX_PORT / NGINX_PORT2, no test profile, service blocks from docker-compose.md only, NODE_CONFIG on the catalogs worker and its /dev footer URL, the lib-node and lib-types-builder versions the recipe relies on - docker-compose.md: the maildev blocks back inside the yaml block - tooling.md: light pre-push, nightly only on data-fair, portals, simple-directory, split CI test jobs only over ~10 min - data-fair-security: SSRF_PUBLIC_IPS is written by df-dev init-env - skill description and skills/README cover tooling, compose and nginx; lib-dev README setup line
Every service has one page to open; the label map was never used.
…ting, then every minute A mongosh probe costs ~1.3 s of CPU: every 2 s it held ~65 % of a core per running env. start_period 3m matches the df-dev wait-deps timeout, so readiness is detected as fast as before.
… folder and session length - a failure after stashing the branch's local changes (main checked out elsewhere, worktree add, npm ci) left them in the shared stash stack: the exit trap re-applies them in place, or prints the stash sha to apply - df-worktree and df-worktree-delete refuse a subfolder or a linked worktree: the worktree path comes from the current folder - the zellij session <repo>_<branch> must fit in 60 characters: its socket path is capped at 107 bytes, zellij 0.45 refused a 67-character name after the whole setup
…er worktrees - values that parseEnv would cut or change (#, spaces, quotes, new lines) are written quoted: a hand-added SECRET='abc#def' became SECRET=abc after init-env --force - ports written in the .env of the other worktrees of the repository are never drawn: they are free while their env is stopped, then both envs collide
- a container that exited with 0 shows as done only while the stack runs: a stack stopped by docker stop showed every container as done - wait-deps fails after 3 s on a dependency that stays exited with an error code instead of waiting 3 minutes (a short exit between two restarts still waits) - df-dev status survives a ps failure like its other parts - the status pane parses the compose file once instead of every 3 s - one jsonLines helper for the compose output, one config import in the cli
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
@data-fair/lib-dev(packages/dev), the dev environment shared by the data-fair services, replacing the dev scripts each repo carried:df-dev:setup/init-env(.envwith free ports drawn in 20000-29999,DEV_HOST,SSRF_PUBLIC_IPS;--forcekeeps hand-added variables),zellij(one session per worktree,--servicesas compose profiles),status,logs,restart(waits for the pane's readiness URL),up/downof optional compose services,wait-deps, andabsent-config(config overrides of the stopped optional services, exported asNODE_CONFIG).df-status: the bash status pane; it starts the containers and shows them coming up.df-mem-cap: runs vite / nuxt under a cgroup memory cap, V8 heap at 3/4 of it.df-worktree/df-worktree-delete: a worktree with full setup, and its removal with containers, volumes and zellij session.A repo only declares
dev/dev.json(port variables, pane readiness URLs, URLs, setup commands, optional services) and its.zellij.kdl.Add the
data-fair-serviceskill: how agents operate a running env (npx --no df-dev …), the rules that keep it light and crash-free, the migration recipe, and the shared service conventions (tooling, dev docker-compose, dev nginx, architecture).Why: one shared, tested implementation instead of a copy of the dev scripts in every service, and one documented way for agents to check status, read logs and restart a process without touching the session. Validated live on the portals, data-fair, catalogs and processings pilots.
Heads-up: published as lib-dev 0.1.0 after merge (the version is already set:
npm publish -w packages/dev, no release bump). The pilot PRs follow and replace theirfile:dependency. The skill reaches every global install at merge, before the services adopt lib-dev; on a repo withoutdev/dev.json,df-devstops with an explicit message.df-worktreeswitches the main checkout's branch and moves its local changes through a stash (restored by sha);df-worktree-deleteremoves containers, volumes and the worktree.