Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion packages/plugins/live-debugger/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Developer notes for the Live Debugger plugin.

<!-- #toc -->
- [Development workflow](#development-workflow)
- [Function IDs](#function-ids)
- [Symbol table](#symbol-table)
- [Consumer build canary](#consumer-build-canary)
- [Fresh checkouts and CI](#fresh-checkouts-and-ci)
Expand Down Expand Up @@ -45,6 +46,18 @@ Generated code should keep the dormant runtime path small: call `$dd_probes(func

When adding or changing `liveDebugger` configuration, update [`src/types.ts`](./src/types.ts), [`src/validate.ts`](./src/validate.ts), [`src/validate.test.ts`](./src/validate.test.ts), and the consumer-facing [`README.md`](./README.md).

## Function IDs

Every instrumented function passes its ID to `$dd_probes`. Outside the plugin, IDs are opaque: Datadog stores them with the symbols and the Browser Debugger SDK matches probes on them, so the format can change here alone. [`src/transform/functionId.ts`](./src/transform/functionId.ts) generates them during the transform's traversal, and [`EXAMPLES.md`](./EXAMPLES.md#function-ids) describes the format.

- The path part is computed once per module by [`getFunctionIdPath`](./src/paths.ts): the query string is removed, Yarn PnP virtual folders are resolved, and the path is made relative to the git repository root, or to the build root outside a repository. Symlinks are only resolved for files outside the root, so most modules don't touch the file system.
- The name part has no line numbers, so a function keeps its ID when the code around it changes. Duplicate qualified names get a `#<n>` suffix in traversal order. The suffix is assigned before any skip check, so `namedOnly`, `functionTypes`, and skip comments don't change the IDs of other functions.
- A function's suffix changes when a function with the same qualified name is added before it. That's accepted, like a move.
- Names come from the code the plugin transforms, which earlier loaders may have compiled, not from the original source. Original names would need the input source map, which only webpack and Rspack provide, and decoding it for every module would cost build time.
- IDs are only unique within a module. Modules of the same file, such as a file imported with different queries, can share IDs.

Keep the generation O(functions) per module, with no extra parsing and no per-function file system calls.

## Symbol table

The plugin can list the instrumented functions that ship in a build, for upload to Datadog's Symbol Database. Each entry has the function ID, a display name, the path relative to the git repository root, and the original line range. The table lives in [`src/symbol-table`](./src/symbol-table).
Expand All @@ -62,7 +75,7 @@ How it works:
- After a one-off build, outside watch mode and dev servers, the plugin scans the text of every emitted JavaScript chunk, without parsing it. It doesn't use the transform hook, which doesn't run for modules restored from a persistent cache and doesn't know which functions tree-shaking removes.
- Every instrumented function has exactly one `$dd_probes('<id>')` call at the start of its body and one `$dd_throw(` call at its end. The scan pairs them like brackets, which also gives each function's parent.
- The transform anchors the `$dd_probes` call to the original body's opening brace, which gives the start line through the chunk's final source map. The end line is the last mapped position up to the `$dd_throw` call: for block bodies that's the closing brace. The code appended after an arrow function's expression body has no mapping, and minifiers leave some tokens unmapped, so the closest mapped token before it is used instead.
- The path and display name come from the function ID. Query strings are removed, Yarn PnP virtual paths and symlinks are resolved, and the path is made relative to the git root. Without a git repository, the table is skipped.
- The path and display name come from the function ID, whose path the transform already made relative to the git root. Without a git repository, the table is skipped.
- Each chunk's debug ID is read from the runtime bootstrap, when debug IDs are enabled, to record which chunks contain each file.

When changing instrumentation output, keep exactly one `$dd_probes` call at the start and one `$dd_throw` call at the end of every instrumented function, anchored to the body's braces. [`src/symbolTable.integration.test.ts`](./src/symbolTable.integration.test.ts) checks the table on every bundler.
Expand Down
54 changes: 35 additions & 19 deletions packages/plugins/live-debugger/EXAMPLES.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ A few conventions used throughout:
- [Arrow inside a super call argument](#arrow-inside-a-super-call-argument)
- [Arrow whose body is a super call](#arrow-whose-body-is-a-super-call)
- [Function expressions keep their own this](#function-expressions-keep-their-own-this)
- [Anonymous function IDs](#anonymous-function-ids)
- [Function IDs](#function-ids)
- [What is not instrumented](#what-is-not-instrumented)
<!-- #toc -->

Expand Down Expand Up @@ -323,7 +323,7 @@ class Button extends Component {
constructor(props) {
let $dd_t;
const handleClick = () => {
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/Button.js;handleClick');
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/Button.js;Button.constructor/handleClick');
try {
if ($dd_p0) $dd_p0 = $dd_entry($dd_p0, $dd_t);
const $dd_rv0 = this.props.onClick();
Expand All @@ -341,8 +341,7 @@ class Button extends Component {

An arrow nested in the `super(...)` argument is instrumented with the `$dd_t` receiver, and the
surrounding `super(...)` call is wrapped to capture the receiver. This arrow is anonymous, so its
ID uses the `<anonymous>@<line>:<column>:<index>` form (see [Anonymous function IDs](#anonymous-function-ids));
the exact numbers depend on the original source position.
ID names it `<anonymous>` (see [Function IDs](#function-ids)).

```js
// Before
Expand All @@ -357,7 +356,7 @@ class Widget extends Base {
constructor(items) {
let $dd_t;
($dd_t = super(items.map((x) => {
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/widget.js;<anonymous>@4:16:0');
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/widget.js;Widget.constructor/<anonymous>');
try {
if ($dd_p0) $dd_p0 = $dd_entry($dd_p0, $dd_t, {x});
const $dd_rv0 = x * 2;
Expand Down Expand Up @@ -388,7 +387,7 @@ class Widget extends Base {
constructor(args) {
let $dd_t;
const init = () => {
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/widget.js;init');
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/widget.js;Widget.constructor/init');
try {
if ($dd_p0) $dd_p0 = $dd_entry($dd_p0, $dd_t);
const $dd_rv0 = ($dd_t = super(args));
Expand Down Expand Up @@ -420,7 +419,7 @@ class Widget extends Base {
class Widget extends Base {
constructor(items) {
const double = function(x) {
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/widget.js;double');
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/widget.js;Widget.constructor/double');
try {
let $dd_rv0;
if ($dd_p0) $dd_p0 = $dd_entry($dd_p0, this, {x});
Expand All @@ -432,25 +431,42 @@ class Widget extends Base {
}
```

## Anonymous function IDs
## Function IDs

Named functions get the ID `<file-path>;<function-name>`. Anonymous functions — including
callbacks that are not assigned to a variable, property, or method — instead use:
A function's ID is the path of its file, relative to the git repository root, followed by `;`
and the function's qualified name. Without a git repository, the path is relative to the build
root instead.

```
<file-path>;<anonymous>@<line>:<column>:<sibling-index>
```
The qualified name is the function's own name, prefixed by the qualified name of the closest
named function or class around it: `<class>.` for methods and class properties, `<function>/`
otherwise. Getters and setters end with `:get` and `:set`, and functions without a name are
`<anonymous>`. Anonymous functions and classes don't prefix the names of the functions inside
them, and neither do object literals. For example, in `src/cart.js`:

where `<line>` and `<column>` are the function's position in the original source and
`<sibling-index>` disambiguates multiple anonymous functions that share the same parent node.
For example, the callback in `[1, 2].map((x) => x * 2)` might become:
| Function | ID |
| --- | --- |
| `function add() {}` | `src/cart.js;add` |
| `total` in `class Cart { total() {} }` | `src/cart.js;Cart.total` |
| The `size` getter in `class Cart { get size() {} }` | `src/cart.js;Cart.size:get` |
| The arrow in `class Cart { handler = () => {} }` | `src/cart.js;Cart.handler` |
| The callback in `function doubleAll(xs) { return xs.map((x) => x * 2); }` | `src/cart.js;doubleAll/<anonymous>` |
| `open` in `function make() { return class Box { open() {} }; }` | `src/cart.js;make/Box.open` |

When several functions of a file have the same qualified name, such as two callbacks in the same
function, the second one gets `#1`, the third one `#2`, and so on, in source order:

```js
let $dd_p0 = globalThis.$dd_probes !== void 0 && $dd_probes('src/utils.js;<anonymous>@1:11:0');
function load() {
// src/cart.js;load/<anonymous>
fetchItems().then((items) => render(items));
// src/cart.js;load/<anonymous>#1
fetchPrices().then((prices) => render(prices));
}
```

> Setting [`liveDebugger.namedOnly`](./README.md#livedebuggernamedonly) to `true` skips
> anonymous callbacks entirely, so only the `<file-path>;<function-name>` form is produced.
IDs don't depend on line numbers, so a function keeps its ID when the code around it changes. It
gets a new ID when it's renamed, when it moves to another file, function, or class, or when a
function with the same qualified name is added before it.

## What is not instrumented

Expand Down
2 changes: 1 addition & 1 deletion packages/plugins/live-debugger/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ Live Debugger also reads the build version from the top-level [`metadata.version
The Live Debugger plugin automatically instruments all JavaScript functions in your application at build time. It adds lightweight checks that can be activated at runtime without rebuilding your code.

Each instrumented function gets:
- A unique, stable function ID (format: `<file-path>;<function-name>`)
- A function ID that identifies the function and stays the same across builds until the function is renamed or moved
- A `$dd_probes()` call that returns active probes for that function (or `undefined` if none)
- Entry point tracking with parameter capture via `$dd_entry()`, which returns a handle identifying the current invocation
- Return value tracking with local variable capture via `$dd_return()`
Expand Down
10 changes: 8 additions & 2 deletions packages/plugins/live-debugger/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,15 @@ export const SKIP_INSTRUMENTATION_COMMENT = '@dd-no-instrumentation';
// The symbol table relies on it marking the end of every instrumented function's body.
export const LIVE_DEBUGGER_THROW_FUNCTION = '$dd_throw';

// Function IDs are `<build-root-relative path>;<function name>`.
// Function IDs are `<repository-relative path>;<qualified function name>`.
export const FUNCTION_ID_SEPARATOR = ';';
// Function name used in the IDs of anonymous functions, followed by their location.
// Separates a function's name from the name of the function or class that contains it.
export const SCOPE_SEPARATOR = '/';
// Separates the name of a method or class property from the name of its class.
export const CLASS_MEMBER_SEPARATOR = '.';
// Starts the suffix that tells apart functions with the same qualified name in a module.
export const DUPLICATE_SUFFIX_SEPARATOR = '#';
// Function name used in the IDs of anonymous functions.
export const ANONYMOUS_FUNCTION_NAME = '<anonymous>';

// Path where an opt-in debug dump of the symbol table is written.
Expand Down
13 changes: 12 additions & 1 deletion packages/plugins/live-debugger/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import type { SourceMapCompact, UnpluginBuildContext } from 'unplugin';

import { CONFIG_KEY, PLUGIN_NAME } from './constants';
import { createFileExtensionPattern, createFileFilter } from './filter';
import { getRepositoryRoot } from './paths';
import { getRuntimeBootstrap } from './runtime-bootstrap';
import { getSymbolTableHooks } from './symbol-table';
import { transformCode } from './transform';
Expand Down Expand Up @@ -43,6 +44,15 @@ export const getLiveDebuggerPlugin = (
let transformedFileCount = 0;
let totalFunctions = 0;
let totalFilesWithFunctions = 0;
let repositoryRoot: string | undefined;

// Looked up on the first transform, once the bundler has set the build root.
const getFunctionIdRoot = () => {
if (repositoryRoot === undefined) {
repositoryRoot = getRepositoryRoot(context.buildRoot) ?? context.buildRoot;
}
return repositoryRoot;
};

return {
name: PLUGIN_NAME,
Expand Down Expand Up @@ -74,10 +84,11 @@ export const getLiveDebuggerPlugin = (
}

try {
const functionIdRoot = getFunctionIdRoot();
const result = transformCode({
code,
filePath: id,
buildRoot: context.buildRoot,
repositoryRoot: functionIdRoot,
honorSkipComments: pluginOptions.honorSkipComments,
functionTypes: pluginOptions.functionTypes,
namedOnly: pluginOptions.namedOnly,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,21 @@
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2019-Present Datadog, Inc.

import { mkdir, outputFile, rm } from '@dd/core/helpers/fs';
import { getTempWorkingDir } from '@dd/tests/_jest/helpers/env';
import fsp from 'fs/promises';
import path from 'path';

import {
createSourceFileResolver,
getFunctionIdPath,
isInsideDirectory,
isOutsideRoot,
resolveYarnVirtualPath,
stripQuery,
} from './paths';

// Paths that don't exist, so resolving symlinks leaves them unchanged.
// A path that doesn't exist, so resolving symlinks leaves paths in it unchanged.
const REPOSITORY_ROOT = '/does-not-exist/repo';
const BUILD_ROOT = `${REPOSITORY_ROOT}/app`;

describe('stripQuery', () => {
const cases = [
Expand Down Expand Up @@ -72,38 +77,93 @@ describe('resolveYarnVirtualPath', () => {
});
});

describe('createSourceFileResolver', () => {
describe('getFunctionIdPath', () => {
const cases = [
{
description: 'make a path relative to the repository root',
input: 'src/a.js',
input: `${REPOSITORY_ROOT}/app/src/a.js`,
expected: 'app/src/a.js',
},
{
description: 'strip the query string',
input: 'src/Cart.vue?vue&type=script',
input: `${REPOSITORY_ROOT}/app/src/Cart.vue?vue&type=script`,
expected: 'app/src/Cart.vue',
},
{
description: 'resolve a workspace package outside the build root',
input: '../packages/lib/index.js',
expected: 'packages/lib/index.js',
},
{
description: 'resolve a Yarn virtual path',
input: '../.yarn/__virtual__/lib-virtual-abc/1/packages/lib/index.js',
input: `${REPOSITORY_ROOT}/.yarn/__virtual__/lib-virtual-abc/1/packages/lib/index.js`,
expected: 'packages/lib/index.js',
},
{
description: 'leave out a file outside the repository',
input: '../../shared/index.js',
expected: undefined,
description: 'keep a file outside the repository outside',
input: '/does-not-exist/shared/index.js',
expected: '../shared/index.js',
},
];

test.each(cases)('Should $description.', ({ input, expected }) => {
const resolveSourceFile = createSourceFileResolver(BUILD_ROOT, REPOSITORY_ROOT);
expect(resolveSourceFile(input)).toBe(expected);
expect(getFunctionIdPath(input, REPOSITORY_ROOT)).toBe(expected);
});

describe('with symlinks', () => {
let workingDir: string;
let repositoryRoot: string;

beforeAll(async () => {
const seed = `${Math.abs(jest.getSeed())}.function-id-path.${Date.now()}`;
workingDir = getTempWorkingDir(seed);
repositoryRoot = path.resolve(workingDir, 'repo');
const packageDir = path.resolve(repositoryRoot, 'packages/lib');
const packageFile = path.resolve(packageDir, 'index.js');
await outputFile(packageFile, 'export const lib = 1;');

// A link inside the repository, like a workspace package in node_modules.
const insideLink = path.resolve(repositoryRoot, 'app/node_modules/lib');
const insideLinkDir = path.dirname(insideLink);
await mkdir(insideLinkDir);
await fsp.symlink(packageDir, insideLink, 'dir');

// A link outside the repository.
const outsideLink = path.resolve(workingDir, 'linked-lib');
await fsp.symlink(packageDir, outsideLink, 'dir');
});

afterAll(async () => {
await rm(workingDir);
});

const symlinkCases = [
{
description: 'resolve a link outside the repository to its target',
input: 'linked-lib/index.js',
expected: 'packages/lib/index.js',
},
{
description: 'keep the path of a link inside the repository',
input: 'repo/app/node_modules/lib/index.js',
expected: 'app/node_modules/lib/index.js',
},
];

test.each(symlinkCases)('Should $description.', ({ input, expected }) => {
const moduleId = path.resolve(workingDir, input);
expect(getFunctionIdPath(moduleId, repositoryRoot)).toBe(expected);
});
});
});

describe('isOutsideRoot', () => {
const cases = [
{ description: 'accept a nested path', input: 'src/a.js', expected: false },
{ description: 'accept a name starting with dots', input: '..a.js', expected: false },
{ description: 'reject a parent path', input: '../a.js', expected: true },
{ description: 'reject a Windows parent path', input: '..\\a.js', expected: true },
{ description: 'reject the parent directory', input: '..', expected: true },
{ description: 'reject an absolute path', input: '/repo/a.js', expected: true },
];

test.each(cases)('Should $description.', ({ input, expected }) => {
expect(isOutsideRoot(input)).toBe(expected);
});
});

Expand Down
Loading
Loading