Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
395d3f6
fix(sea): files inside symlinks are not resolved correctly (#295)
mpotthoff Aug 20, 2026
0d3b743
Address review comments
mpotthoff Aug 25, 2026
86e86b0
refactor(prelude): bound symlink memo and shorten the resolution walk
robertsLando Sep 4, 2026
6d8f560
fix(prelude): keep exact symlink entries ahead of their symlinked parent
robertsLando Sep 4, 2026
e807a4f
fix(sea): resolve symlinks in realpath, which also unbreaks readlink
robertsLando Sep 4, 2026
28bfa42
fix(prelude): match the deepest symlink key and charge cached hops
robertsLando Sep 7, 2026
13355f2
fix(prelude): report symlinks as links in readdir withFileTypes
robertsLando Sep 7, 2026
13de24a
test(#295): build the symlink fixture at test time so Windows runs it
robertsLando Sep 7, 2026
aecc3f8
docs: sync the two bootstrap-shared.js line counts
robertsLando Sep 7, 2026
888d2db
fix(prelude): service symlinks in classic-mode readlink and lstat
robertsLando Sep 7, 2026
7686b94
fix: address review round 1 on symlink resolution
robertsLando Sep 18, 2026
a668fed
fix(sea): report symlinks in lstat, readlink and readdir
robertsLando Sep 18, 2026
bc930d6
fix: route ELOOP through callbacks, share the readlink contract
robertsLando Sep 18, 2026
7fb8467
chore: require @roberts_lando/vfs ^0.3.4 for the lstat/readlink routing
robertsLando Sep 18, 2026
5179fc8
fix: close the remaining ELOOP call sites and share the parent walk
robertsLando Sep 18, 2026
eee1abc
refactor(prelude): one error shape, one link-target policy, one ELOOP…
robertsLando Sep 18, 2026
82aa589
chore: install @roberts_lando/vfs from the PR commit until it releases
robertsLando Sep 18, 2026
14f9872
fix(prelude): give the identity resolver a .parent
robertsLando Sep 18, 2026
d42be23
refactor(prelude): narrow #296 to symlink resolution
robertsLando Oct 15, 2026
5333e16
feat(prelude)!: report snapshot symlinks in lstat, readlink and readdir
robertsLando Oct 15, 2026
5426852
chore(deps): use @roberts_lando/vfs 0.4.0
robertsLando Oct 23, 2026
7a5e83d
fix: address review round 1 on symlink reporting
robertsLando Oct 30, 2026
f795506
fix: address review round 2 on symlink reporting
robertsLando Oct 30, 2026
66385bb
fix(walker): key a link inside a linked directory under its real path
robertsLando Oct 30, 2026
22036a2
fix: address review round 3 on symlink reporting
robertsLando Oct 30, 2026
d460af6
Merge branch 'main' into feat/symlink-reporting
robertsLando Oct 30, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,6 @@ prelude/sea-bootstrap.bundle.js
# Built by test-99-#295/main.js at test time, removed again afterwards
test/test-99-#295/lib
test/test-99-#295/reallib/inner.js
test/test-99-#295/reallib/other.js
test/test-99-#295/linkinfo.json
test/test-99-#295/esm/node_modules
18 changes: 18 additions & 0 deletions docs-site/guide/snapshot-fs.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,24 @@ const configPath = path.join(path.dirname(process.execPath), 'config.json');
const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
```

## Symlinks

Symlinks collected into the snapshot (pnpm and workspace `node_modules`, `node_modules/.bin`, links among your assets) behave as they do outside the binary:

```js
fs.lstatSync(link).isSymbolicLink(); // true
fs.readdirSync(dir, { withFileTypes: true }); // links report isSymbolicLink()
fs.readlinkSync(link); // '/snapshot/app/node_modules/.pnpm/x/node_modules/x'
```

`readlink` returns the absolute target inside the snapshot, not the relative link body on disk.

::: warning Changed in 7.0.0
Before, links looked like the file or directory they point at. Code that finds entries with `d.isDirectory()` or `d.isFile()` now skips linked ones, the same as plain Node. Check `d.isSymbolicLink()` and `fs.statSync()` the entry, or pass `follow: true` to `glob`. fast-glob and readdirp follow links by default.
:::

See [symlink semantics](https://github.com/yao-pkg/pkg/blob/main/docs/ARCHITECTURE.md#symlink-semantics-in-packaged-binaries) for the details.

## Detecting that you're packaged

Check for `process.pkg`:
Expand Down
50 changes: 36 additions & 14 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ Each file is stored with one or more store types:

### Runtime Bootstrap

`prelude/bootstrap.js` (1964 lines) executes before user code. It:
`prelude/bootstrap.js` (~2049 lines) executes before user code. It:

1. **Sets up entrypoint** — Reads `DEFAULT_ENTRYPOINT` from injected parameters, sets `process.argv[1]`
2. **Initializes VFS** — Builds in-memory lookup from `VIRTUAL_FILESYSTEM` dictionary with optional path compression via `DICT`
Expand Down Expand Up @@ -394,15 +394,16 @@ ASCII version:

The `SEAProvider` (in `prelude/sea-vfs-setup.js`) implements lazy loading from a single archive blob:

| Method | Behavior |
| -------------------------- | ----------------------------------------------------------------------------- |
| `readFileSync(path)` | Resolve symlinks, `subarray()` from archive via `offsets` map, cache in `Map` |
| `statSync(path)` | Return metadata from manifest `stats` |
| `internalModuleStat(path)` | Fast path for module resolution: 0 (file), 1 (dir), or a negative errno |
| `readdirSync(path)` | Return directory entries from manifest `directories` |
| `existsSync(path)` | O(1) check against manifest `stats` |
| `readlinkSync(path)` | Unused by `fs.readlinkSync`, which vfs answers via `realpathSync` (#310) |
| `realpathSync(path)` | Follow symlinks, return the path if the manifest has it |
| Method | Behavior |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `readFileSync(path)` | Resolve symlinks, `subarray()` from archive via `offsets` map, cache in `Map` |
| `statSync(path)` | Return metadata from manifest `stats` |
| `internalModuleStat(path)` | Fast path for module resolution: returns 0 (file), 1 (dir), or a negative errno (not found) |
| `readdirSync(path)` | Return manifest directory entries directly; with `withFileTypes` builds shared `Dirent`s, marking links from `manifest.symlinks` |
| `existsSync(path)` | O(1) check against manifest `stats` |
| `readlinkSync(path)` | Return the symlink target from `manifest.symlinks`, resolving a symlinked parent first; `EINVAL` for a path that is present but not a link, `ENOENT` otherwise. Reached through `fs.readlinkSync` by the VFS's own patch (robertsLando/vfs#4) |
| `lstatSync(path)` | Describe the link itself: the target's stat with link semantics and `S_IFLNK` type bits. Reached through `fs.lstatSync` the same way as `readlinkSync` above |
| `realpathSync(path)` | Follow the symlink chain, then return the path if the manifest has it. Load-bearing: without it every archive path raises `ENOENT`. The VFS mounts under a POSIX `/snapshot` prefix, so on Windows the result is converted back to `C:\snapshot\...` before it leaves `fs` (yao-pkg/pkg#305) |

The entire archive is loaded once via `sea.getRawAsset('__pkg_archive__')` which returns a zero-copy `ArrayBuffer` reference to the executable's memory-mapped region. Individual files are extracted via `Buffer.subarray(offset, offset + length)` using the manifest's `offsets` map, then cached in a `Map` on first access. String results (when `encoding` is specified) are derived directly from the archive view; Buffer results are copied to prevent callers from corrupting the shared archive memory.

Expand Down Expand Up @@ -468,7 +469,7 @@ This keeps the VFS setup, shared patches, worker interception, and diagnostics a

## Shared Runtime Code

`prelude/bootstrap-shared.js` (~751 lines) contains runtime patches used by both bootstraps:
`prelude/bootstrap-shared.js` (~859 lines) contains runtime patches used by both bootstraps:

### Injection Mechanisms

Expand Down Expand Up @@ -505,6 +506,27 @@ This keeps the VFS setup, shared patches, worker interception, and diagnostics a

**`makeSymlinkResolver(symlinks, sep)`** — The symlink resolver for **both** modes (`findVirtualFileSystemKeyAndFollowLinks` in classic, `SEAProvider._resolveSymlink` in SEA). It follows parent components like POSIX, so a link at `node_modules/@scope/lib` also resolves `node_modules/@scope/lib/package.json` (#295). The longest matching key wins; that agrees with POSIX because the walker stores realpaths as targets (`toNormalizedRealPath`). More than `MAX_SYMLINK_DEPTH` hops, possible only in a hand-edited manifest, raises `ELOOP`.

### Symlink semantics in packaged binaries

Both bootstraps resolve symlinks on parent path components, so `require`, `fs.readFile` and friends reach files under a linked directory. The walker keys a link that sits inside a linked directory under the real directory (`reallib/inner.js`, not `lib/inner.js`) and records the directory link too, so both spellings resolve. Where they differ:

| | Traditional | Enhanced SEA |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `readdir({ withFileTypes: true })` | Reports a snapshot symlink as a link (`isSymbolicLink()` true, `isDirectory()`/`isFile()` false), matching real `readdir` | Same — the provider builds `Dirent`s from `manifest.symlinks`, using the shared `Dirent` in `bootstrap-shared.js` |
| `lstat` | Describes the link itself, agreeing with the dirent above | Same — `SEAProvider.lstatSync` gives the target's stat link semantics |
| `readlink` | Returns the absolute target from `SYMLINKS`; `EINVAL` for a path that exists but is not a link | Same, from `manifest.symlinks`, including the shared parent-resolution step |
| `realpath` | Follows the chain | Follows the chain |

> **Breaking change (both modes).** Snapshot symlinks used to look like what they point at. In traditional mode `readdir({ withFileTypes: true })` reported every entry as a plain file or directory (`Dirent.isSymbolicLink()` took an argument it is never called with, so it always returned `false`); in SEA mode it returned entries without the `Dirent` methods. `lstat` returned the target's stat in both. Both now report links as links, which is what Node does outside a packaged binary. For packaged apps whose snapshot contains symlinks (pnpm and workspace trees most of all, plus `node_modules/.bin`):
>
> - Recursive walkers that gate descent on `isDirectory()` and skip links by default (`glob` without `follow: true`, hand-written `isDirectory()` recursions) no longer descend into a symlinked directory. fast-glob and readdirp follow links by default and are unaffected.
> - A filter like `entries.filter((e) => e.isFile())` no longer matches a symlinked **file**.
> - `lstatSync(link).isDirectory()` / `.isFile()` are now `false`, `mode & S_IFMT` is `S_IFLNK`, and `size` is the byte length of the target path.
>
> `readlink` returns the walker's **absolute, fully resolved** target (`/snapshot/app/node_modules/.pnpm/...`, in the platform path form), not the raw link body on disk. Tools that re-create links from it (tar, archiver, a hand-written copy) write absolute `/snapshot/...` links. (`fs.cp` cannot copy out of the snapshot in either mode.)

> **How SEA reports links.** `@roberts_lando/vfs` used to route `fs.lstat` through `statSync`, which follows the link, and `fs.readlink` through `realpathSync`, which never reached the provider. [robertsLando/vfs#4](https://github.com/robertsLando/vfs/pull/4) routes both to `SEAProvider`, which answers from `manifest.symlinks`, the same record the resolver walks. Released in `@roberts_lando/vfs` 0.4.0.

**`setupProcessPkg(entrypoint)`** — Creates the `process.pkg` compatibility object with `entrypoint`, `defaultEntrypoint`, and `path.resolve()`.

**`installDiagnostic(snapshotPrefix)`** — Installs runtime diagnostics triggered by the `DEBUG_PKG` environment variable. Available in both traditional and SEA modes. The implementation lives in `prelude/bootstrap-shared.js` and is always present in the runtime bootstrap, but it is **only invoked when the binary was built with `--debug` / `-d`** — release builds omit the entrypoint call, so the diagnostic handler never runs and cannot expose the VFS tree contents.
Expand Down Expand Up @@ -613,11 +635,11 @@ With `node:vfs` and `"useVfs": true` in the SEA config, assets will be auto-moun

| File | Lines | Purpose |
| -------------------------------- | ----- | -------------------------------------------------------------------------------------------- |
| `prelude/bootstrap.js` | ~1964 | Traditional runtime bootstrap (fs/module/process patching) |
| `prelude/bootstrap-shared.js` | ~751 | Shared runtime patches (dlopen, child_process, process.pkg, diagnostics, symlink resolution) |
| `prelude/bootstrap.js` | ~2049 | Traditional runtime bootstrap (fs/module/process patching) |
| `prelude/bootstrap-shared.js` | ~859 | Shared runtime patches (dlopen, child_process, process.pkg, diagnostics, symlink resolution) |
| `prelude/sea-bootstrap.js` | ~74 | CJS wrapper: Module.runMain() (CJS) or vm.Script + USE_MAIN_CONTEXT_DEFAULT_LOADER (ESM/TLA) |
| `prelude/sea-bootstrap-core.js` | ~121 | Shared setup: VFS, patches, worker interception, diagnostics, perf start |
| `prelude/sea-vfs-setup.js` | ~577 | SEA VFS core: SEAProvider, archive loading, VFS mount, Windows patches |
| `prelude/sea-vfs-setup.js` | ~648 | SEA VFS core: SEAProvider, archive loading, VFS mount, Windows patches |
| `prelude/sea-worker-entry.js` | ~11 | Worker thread entry: requires sea-vfs-setup.js for VFS in workers |
| `scripts/build-sea-bootstrap.js` | ~50 | Build script: 2-step esbuild bundling (worker string + CJS main) |
| `lib/index.ts` | ~704 | CLI entry point, mode routing |
Expand Down
9 changes: 9 additions & 0 deletions lib/walker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -447,6 +447,15 @@ class Walker {
file = a.file;
realFile = a.realFile;

// A link inside a linked directory lives in the real one: key it there so
// its real path resolves too, and record the directory link it came through.
const dir = path.dirname(file);
const realDir = toNormalizedRealPath(dir);
if (realDir !== dir) {
await this.appendSymlink(dir, realDir);
file = path.join(realDir, path.basename(file));
}

if (!this.symLinks[file]) {
const dn = path.dirname(file);
this.appendFileInFolder({
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
"@babel/parser": "^7.23.0",
"@babel/traverse": "^7.23.0",
"@babel/types": "^7.23.0",
"@roberts_lando/vfs": "^0.3.3",
"@roberts_lando/vfs": "^0.4.0",
"@yao-pkg/pkg-fetch": "3.6.7",
"esbuild": "^0.28.1",
"into-stream": "^9.1.0",
Expand Down
Loading
Loading