Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion packages/core/src/helpers/fs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,15 @@ export const outputJson = async (filepath: string, data: any) => {
const writable = fs.createWriteStream(filepath);
const readable = new JsonStreamStringify(data, undefined, 2);
const prom = new Promise<void>((resolve, reject) => {
readable.on('end', () => {
// Resolve once the data is on disk, not when it's been read.
writable.on('finish', () => {
resolve();
});

writable.on('error', (err) => {
reject(err);
});

readable.on('error', (err) => {
reject(err);
});
Expand All @@ -59,6 +64,15 @@ export const outputJson = async (filepath: string, data: any) => {
return prom;
};

// Return the canonical path, with symlinks resolved, or the input path if it doesn't exist.
export const realpathSync = (filepath: string) => {
try {
return fs.realpathSync(filepath);
} catch {
return filepath;
}
};

export const outputJsonSync = (filepath: string, data: any) => {
// FIXME: This will crash on strings too long.
const dataString = JSON.stringify(data, null, 4);
Expand Down
30 changes: 30 additions & 0 deletions packages/plugins/live-debugger/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Developer notes for the Live Debugger plugin.

<!-- #toc -->
- [Development workflow](#development-workflow)
- [Symbol table](#symbol-table)
- [Consumer build canary](#consumer-build-canary)
- [Fresh checkouts and CI](#fresh-checkouts-and-ci)
- [Excalidraw target](#excalidraw-target)
Expand Down Expand Up @@ -44,6 +45,35 @@ Generated code should keep the dormant runtime path small: call `$dd_probes(func

When adding or changing `liveDebugger` configuration, update [`src/types.ts`](./src/types.ts), [`src/validate.ts`](./src/validate.ts), [`src/validate.test.ts`](./src/validate.test.ts), and the consumer-facing [`README.md`](./README.md).

## Symbol table

The plugin can list the instrumented functions that ship in a build, for upload to Datadog's Symbol Database. Each entry has the function ID, a display name, the path relative to the git repository root, and the original line range. The table lives in [`src/symbol-table`](./src/symbol-table).

For now, the only consumer is an opt-in debug dump. Set `DD_LIVE_DEBUGGER_SYMBOL_TABLE_PATH` to a file path to generate the table and write it there:

```bash
DD_LIVE_DEBUGGER_SYMBOL_TABLE_PATH=/tmp/symbol-table.json yarn build
```

Without that variable, the plugin doesn't add any hook or build time. The dump is never written inside the bundler's output directory, and generation never fails the build: problems are logged as warnings.

How it works:

- After a one-off build, outside watch mode and dev servers, the plugin scans the text of every emitted JavaScript chunk, without parsing it. It doesn't use the transform hook, which doesn't run for modules restored from a persistent cache and doesn't know which functions tree-shaking removes.
- Every instrumented function has exactly one `$dd_probes('<id>')` call at the start of its body and one `$dd_throw(` call at its end. The scan pairs them like brackets, which also gives each function's parent.
- The transform anchors the `$dd_probes` call to the original body's opening brace, which gives the start line through the chunk's final source map. The end line is the last mapped position up to the `$dd_throw` call: for block bodies that's the closing brace. The code appended after an arrow function's expression body has no mapping, and minifiers leave some tokens unmapped, so the closest mapped token before it is used instead.
- The path and display name come from the function ID. Query strings are removed, Yarn PnP virtual paths and symlinks are resolved, and the path is made relative to the git root. Without a git repository, the table is skipped.
- Each chunk's debug ID is read from the runtime bootstrap, when debug IDs are enabled, to record which chunks contain each file.

When changing instrumentation output, keep exactly one `$dd_probes` call at the start and one `$dd_throw` call at the end of every instrumented function, anchored to the body's braces. [`src/symbolTable.integration.test.ts`](./src/symbolTable.integration.test.ts) checks the table on every bundler.

Known limitations:

- esbuild doesn't tell plugins whether it's watching, so its rebuilds always regenerate the table.
- In minified builds, the end line of an arrow function's expression body is the last line the minifier keeps mapped, which can be before the real end.
- Helpers that compilers inject, such as `_define_property`, have no original source, so they have no line range.
- On webpack and Rspack, files that reach the plugin without a source map from an earlier loader get shifted line numbers.

## Consumer build canary

Canary targets build real consumer projects with local Datadog build plugins.
Expand Down
2 changes: 1 addition & 1 deletion packages/plugins/live-debugger/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@
"dependencies": {
"@dd/core": "workspace:*",
"@jridgewell/remapping": "2.3.5",
"@jridgewell/trace-mapping": "0.3.31",
"chalk": "2.3.1",
"picomatch": "4.0.7"
},
"devDependencies": {
"@babel/parser": "7.24.5",
"@babel/traverse": "7.24.5",
"@babel/types": "7.24.5",
"@jridgewell/trace-mapping": "0.3.31",
"@types/picomatch": "4.0.2",
"magic-string": "0.30.21",
"typescript": "5.4.3"
Expand Down
13 changes: 13 additions & 0 deletions packages/plugins/live-debugger/src/constants.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,19 @@ export const PLUGIN_NAME: PluginName = 'datadog-live-debugger-plugin' as const;
// Skip instrumentation comment
export const SKIP_INSTRUMENTATION_COMMENT = '@dd-no-instrumentation';

// Runtime global that instrumented functions call when they throw.
// The symbol table relies on it marking the end of every instrumented function's body.
export const LIVE_DEBUGGER_THROW_FUNCTION = '$dd_throw';

// Function IDs are `<build-root-relative path>;<function name>`.
export const FUNCTION_ID_SEPARATOR = ';';
// Function name used in the IDs of anonymous functions, followed by their location.
export const ANONYMOUS_FUNCTION_NAME = '<anonymous>';

// Path where an opt-in debug dump of the symbol table is written.
// Until the symbol table upload exists, setting it is also what enables the symbol table.
export const SYMBOL_TABLE_PATH_ENV = 'DD_LIVE_DEBUGGER_SYMBOL_TABLE_PATH';

export const DEFAULT_FILE_EXTENSIONS = [
'.js',
'.jsx',
Expand Down
2 changes: 2 additions & 0 deletions packages/plugins/live-debugger/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import type { SourceMapCompact, UnpluginBuildContext } from 'unplugin';
import { CONFIG_KEY, PLUGIN_NAME } from './constants';
import { createFileExtensionPattern, createFileFilter } from './filter';
import { getRuntimeBootstrap } from './runtime-bootstrap';
import { getSymbolTableHooks } from './symbol-table';
import { transformCode } from './transform';
import type { LiveDebuggerOptions, LiveDebuggerOptionsWithDefaults } from './types';
import { validateOptions } from './validate';
Expand Down Expand Up @@ -116,6 +117,7 @@ export const getLiveDebuggerPlugin = (
}
},
},
...getSymbolTableHooks(pluginOptions, context, log),
buildEnd: () => {
if (totalFunctions > 0) {
log.info(
Expand Down
43 changes: 43 additions & 0 deletions packages/plugins/live-debugger/src/symbol-table/build-mode.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
// Unless explicitly stated otherwise all files in this repository are licensed under the MIT License.
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2019-Present Datadog, Inc.

import type { PluginOptions } from '@dd/core/types';

export type BuildModeTracker = {
hooks: Pick<PluginOptions, 'webpack' | 'rspack' | 'rollup' | 'vite'>;
// False in watch mode and on dev servers. esbuild doesn't tell plugins whether it's watching,
// so its builds always count as one-off builds.
isOneOffBuild: () => boolean;
};

// Track whether the build is a one-off build, as opposed to watch mode or a dev server.
export const createBuildModeTracker = (): BuildModeTracker => {
let isContinuousBuild = false;
let isCompilerWatching = () => false;

return {
hooks: {
webpack(compiler) {
isCompilerWatching = () => compiler.watchMode;
},
rspack(compiler) {
isCompilerWatching = () => compiler.watchMode;
},
rollup: {
buildStart() {
isContinuousBuild ||= this.meta.watchMode;
},
},
vite: {
configResolved(config) {
isContinuousBuild ||= config.command === 'serve';
},
buildStart() {
isContinuousBuild ||= this.meta.watchMode;
},
},
},
isOneOffBuild: () => !isContinuousBuild && !isCompilerWatching(),
};
};
61 changes: 61 additions & 0 deletions packages/plugins/live-debugger/src/symbol-table/index.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
// Unless explicitly stated otherwise all files in this repository are licensed under the MIT License.
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2019-Present Datadog, Inc.

import { existsSync, rm } from '@dd/core/helpers/fs';
import { getTempWorkingDir } from '@dd/tests/_jest/helpers/env';
import { getMockLogger, mockLogFn } from '@dd/tests/_jest/helpers/mocks';
import path from 'path';

import { generateSymbolTable, writeSymbolTable } from './index';

const PAYLOAD = { language: 'browser' as const, scopes: [] };

describe('Live Debugger symbol table', () => {
let workingDir: string;

beforeEach(() => {
const seed = `${Math.abs(jest.getSeed())}.symbol-table-unit.${Date.now()}`;
workingDir = getTempWorkingDir(seed);
});

afterEach(async () => {
await rm(workingDir);
});

test('Should not write the dump inside the output directory.', async () => {
const outDir = path.resolve(workingDir, 'dist');
const dumpPath = path.resolve(outDir, 'symbol-table.json');

await writeSymbolTable(PAYLOAD, dumpPath, outDir, getMockLogger());

expect(existsSync(dumpPath)).toBe(false);
expect(mockLogFn).toHaveBeenCalledWith(
expect.stringContaining('inside the output directory'),
'warn',
);
});

test('Should write the dump outside the output directory.', async () => {
const outDir = path.resolve(workingDir, 'dist');
const dumpPath = path.resolve(workingDir, 'symbol-table.json');

await writeSymbolTable(PAYLOAD, dumpPath, outDir, getMockLogger());

expect(existsSync(dumpPath)).toBe(true);
});

test('Should skip the table outside a git repository.', async () => {
const table = await generateSymbolTable({
outputs: [],
buildRoot: workingDir,
log: getMockLogger(),
});

expect(table).toBeUndefined();
expect(mockLogFn).toHaveBeenCalledWith(
expect.stringContaining("isn't in a git repository"),
'warn',
);
});
});
Loading
Loading