Skip to content

Generate a Live Debugger symbol table from the build output - #582

Draft
watson wants to merge 7 commits into
masterfrom
watson/DEBUG-6327/generate-symbol-table
Draft

watson wants to merge 7 commits into
masterfrom
watson/DEBUG-6327/generate-symbol-table

Conversation

@watson

@watson watson commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Summary

DEBUG-6327: list the instrumented functions a build ships, with their repository path and original line range, so Live Debugger can upload them as symbols. The upload is #593; for now the only consumer is an opt-in debug dump. Without DD_LIVE_DEBUGGER_SYMBOL_TABLE_PATH, the plugin adds no hooks and no build time.

buildReport                     # one-off builds only, not watch mode or dev servers
  for each emitted JS chunk
    scanChunk                   # pair $dd_probes('<id>') … $dd_throw( like brackets; nesting = parent
    readChunkDebugId            # from the runtime bootstrap
    locateFunctions             # start: $dd_probes in the final source map
                                # end: last mapped token up to $dd_throw, because minifiers and
                                #   expression-body arrows leave the marker itself unmapped
  buildSymbolTable              # path from the ID: strip ?query, resolve Yarn virtual paths + symlinks,
                                #   relative to the git root; dedupe on (ID, file, range)
  serializeSymbolTable          # SymDB scopes: MODULE per file, nested FUNCTION scopes, language "browser"
  writeSymbolTable              # never inside the output directory

No output parsing and no transform-hook collection, so it works with persistent bundler caches and only lists functions that survive tree-shaking. Never fails the build: problems are logged as warnings.

Also: core's outputJson now resolves once the file is written, not when the data has been read. The new tests exposed the race.

Evidence

  • Before: nothing records which functions a build ships or where they are.
    After: symbolTable.integration.test.ts builds a fixture with esbuild, Rollup, Rspack, Vite and webpack, plus a minified Rspack production build: build rooted in a repo subdirectory, symlinked workspace package, Yarn PnP virtual path, loader query string on webpack and Rspack, multi-line arrow function with an expression body. E.g. for app/src/math.js:

    {
      "scope_type": "MODULE",
      "name": "app/src/math.js",
      "source_file": "app/src/math.js",
      "language_specifics": { "chunk_debug_ids": ["…"] },
      "scopes": [
        {
          "scope_type": "FUNCTION",
          "name": "add",
          "source_file": "app/src/math.js",
          "start_line": 1,
          "end_line": 6,
          "language_specifics": { "function_id": "src/math.js;add" },
          "scopes": [
            {
              "scope_type": "FUNCTION",
              "name": "double",
              "source_file": "app/src/math.js",
              "start_line": 2,
              "end_line": 4,
              "language_specifics": { "function_id": "src/math.js;double" }
            }
          ]
        }
      ]
    }
  • Before: collecting in the transform hook would miss every module restored from webpack's filesystem cache.
    After: a warm-cache webpack build (transform didn't run) produces the same table as the cold build.

  • web-ui (spa-rspack, production build with source maps, Live Debugger + RUM privacy, local plugin linked):

    Functions / files 605,487 / 94,051
    With a line range 99% (the rest are mostly compiler-injected helpers like _define_property)
    Yarn virtual-path functions 8,182, all with repository-relative sourceFile
    Table size 233 MB JSON, 12.9 MB gzipped
    Generation time 62–76 s on a ~37 min build (~3%); the build with the dump took 55 s longer wall-clock

    The first web-ui run had 45% of functions without a line range: minified expression-body arrows. The fallback lookups in locateFunctions fixed that; the integration test now covers it.

  • Live Debugger suite: 437/437. yarn typecheck:all and yarn cli integrity pass.

Merge Danger

Door: two-way

Nothing new runs unless DD_LIVE_DEBUGGER_SYMBOL_TABLE_PATH is set.

Blast Radius: minimal

  • Published packages gain a direct dependency on @jridgewell/trace-mapping 0.3.31, already installed through @jridgewell/remapping.
  • outputJson resolves slightly later for its other caller, the output plugin.
  • The transform now uses a constant for $dd_throw; generated code is unchanged.

🤖 Generated with Claude Code

Live Debugger needs to know which instrumented functions a build ships and
where they are in the repository, so the UI and MCP can copy a function's ID
from SymDB instead of computing it. This adds the generation step
(DEBUG-6327); the upload comes in DEBUG-6328.

After a one-off build, the plugin scans each emitted chunk for the
`$dd_probes('<id>')` call at the start of every instrumented function and the
`$dd_throw(` call at its end, pairs them like brackets, and maps both through
the chunk's final source map to get the original line range. It doesn't parse
the output or rely on the transform hook, so it works with persistent bundler
caches and only lists functions that survive tree-shaking.

File paths and display names come from the function ID. Query strings are
stripped, Yarn PnP virtual paths and symlinks are resolved, and paths are made
relative to the git root. The table also records the build's chunk debug IDs
and which chunks contain each file, when every instrumented chunk has one.

The table is serialized as SymDB scopes (a MODULE per file and nested FUNCTION
scopes, language `browser`). For now its only consumer is an opt-in debug dump
enabled by `DD_LIVE_DEBUGGER_SYMBOL_TABLE_PATH`; without it, the plugin adds no
hooks and no build time. Generation never fails the build and never writes
inside the output directory.

Also make core's `outputJson` resolve once the file is written instead of when
the data has been read, which the new tests exposed as a race.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Live Debugger Runtime Benchmark

Benchmark results were not produced. Check the workflow logs for details.

watson and others added 2 commits October 15, 2026 16:12
On a web-ui production build, 45% of the symbol table's functions had no
line range. The transform appends the closing code of an arrow function's
expression body without a source map segment, and minifiers leave some
rewritten tokens unmapped, so looking up the `$dd_throw` call alone often
found nothing.

Use the last mapped position of the same source between the two markers
instead. For block bodies that's still the closing brace. For expression
bodies, it's the last mapped token of the expression. On the same web-ui
build, functions without a line range drop to 1.8%.

The integration test fixture now has a multi-line arrow function with an
expression body, and a minified Rspack production build.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a web-ui production build, 1.8% of the symbol table's functions still
had no line range after the previous fix:

- Minifiers can also leave the `$dd_probes` call unmapped. Fall back to the
  first mapped token between the two markers for the start, like the end
  already falls back to the last one.
- A module bundled into many chunks can be mapped in some chunks and not in
  others, which left a copy without a line range next to the mapped ones.
  Drop such copies, and attach their nested functions to their parent.

Functions without a line range drop to 0.98% on the same build, mostly
helpers that compilers inject, such as `_define_property`, which have no
original source. Unit tests cover the lookups with hand-built source maps.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
watson and others added 3 commits October 16, 2026 10:35
Customers see the build logs, where "symbol table" is an internal term.
Call them Live Debugger symbols, like Datadog's documentation does for
symbol uploads, and remove noise from the contributing guide.

Environment: Datadog workspace

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Environment: Datadog workspace

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Master now guards probe calls with `globalThis.$dd_probes !== void 0 &&`.
The scanner still matches the bare `$dd_probes('<id>')` call, and the
symbol table integration test passes on the merged tree.

Environment: Datadog workspace

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant