-
Notifications
You must be signed in to change notification settings - Fork 4.9k
Add Usage Insights canvas extension π€π€π€ #2939
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
davidkaya
wants to merge
10
commits into
github:main
Choose a base branch
from
davidkaya:dvidkaya-microsoft-add-usage-insights
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
5b49d51
Add Usage Insights canvas extension
davidkaya 45b50dd
Address Usage Insights review feedback
davidkaya 49ce227
Refine Usage Insights dashboard behavior
davidkaya 95f9a35
Keep Usage Insights live and accessible
davidkaya 7fda6aa
Correct Usage Insights metric edge cases
davidkaya 67b9610
Preserve zero-cost chart accuracy
davidkaya 3ba944f
Use non-overlapping token totals
davidkaya 6558a26
Keep Usage Insights snapshots consistent
davidkaya d524353
Use full Usage Insights author name
davidkaya 0aae213
Support current Usage Insights schemas
davidkaya File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # Usage Insights | ||
|
|
||
| A GitHub Copilot canvas extension for inspecting local session usage: | ||
|
|
||
| - current and historical AI-credit totals | ||
| - input, output, reasoning, cache-read, and cache-write tokens | ||
| - root-agent and per-sub-agent rollups | ||
| - model-call counts and durations | ||
| - recent and all-session aggregates | ||
| - highest-cost local sessions with in-canvas drill-down | ||
|
|
||
| The interface follows GitHub Copilot's native visual language and refreshes while | ||
| the canvas is open. | ||
|
|
||
| ## Privacy | ||
|
|
||
| All metrics are read locally from Copilot's session data. The extension: | ||
|
|
||
| - binds its renderer to `127.0.0.1` | ||
| - opens local session databases read-only | ||
| - does not send usage data to an external service | ||
| - keeps historical session drill-down inside the canvas | ||
|
|
||
| The published preview uses entirely synthetic sample data. | ||
|
|
||
| ## Installation | ||
|
|
||
| ```bash | ||
| copilot plugin install usage-insights@awesome-copilot | ||
| ``` | ||
|
|
||
| Reload extensions, then ask Copilot to open the **Usage Insights** canvas. | ||
|
|
||
| ## Canvas actions | ||
|
|
||
| ### `refresh` | ||
|
|
||
| Returns current metrics for an optional history range or selected session. | ||
|
|
||
| ```json | ||
| { | ||
| "range": "7d", | ||
| "sessionId": "optional-session-id" | ||
| } | ||
| ``` | ||
|
|
||
| Supported ranges are `24h`, `7d`, `30d`, and `all`. | ||
|
|
||
| ### `inspect_session` | ||
|
|
||
| Returns overall and per-agent metrics for one local session. | ||
|
|
||
| ```json | ||
| { | ||
| "sessionId": "required-session-id", | ||
| "range": "7d" | ||
| } | ||
| ``` | ||
|
|
||
| ## Structure | ||
|
|
||
| - `extension.mjs` - canvas declaration, loopback server, actions, and refresh events | ||
| - `stats.mjs` - read-only SQLite aggregation and sub-agent metadata resolution | ||
| - `renderer.mjs` - responsive, theme-aware HTML renderer | ||
|
|
||
| The extension intentionally has no `package.json`: `@github/copilot-sdk` is | ||
| resolved by the Copilot extension runtime, and SQLite uses Node.js's built-in | ||
| `node:sqlite` module. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - GitHub Copilot with extension canvas support | ||
| - a runtime version that provides `node:sqlite` | ||
| - local Copilot session data containing `assistant_usage_events` | ||
|
|
||
| ## Source | ||
|
|
||
| Contributed from | ||
| [`davidkaya/github-copilot-usage-insights`](https://github.com/davidkaya/github-copilot-usage-insights) | ||
| at commit `206d53faab1d8a3331de96e203d0206ad21845aa`. | ||
|
|
||
| ## License | ||
|
|
||
| MIT |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "name": "usage-insights", | ||
| "version": 1 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,206 @@ | ||
| import { createServer } from "node:http"; | ||
| import { homedir } from "node:os"; | ||
| import { join } from "node:path"; | ||
| import { createCanvas, CanvasError, joinSession } from "@github/copilot-sdk/extension"; | ||
| import { AgentMetadataStore, UsageInsightsStore } from "./stats.mjs"; | ||
| import { renderDashboardHtml } from "./renderer.mjs"; | ||
|
|
||
| const RANGE_VALUES = ["24h", "7d", "30d", "all"]; | ||
| const servers = new Map(); | ||
| const copilotHome = process.env.COPILOT_HOME || join(homedir(), ".copilot"); | ||
| const statsStore = new UsageInsightsStore(copilotHome); | ||
| const agentMetadata = new AgentMetadataStore(copilotHome); | ||
| let session; | ||
|
|
||
| function normalizeRange(value) { | ||
| return RANGE_VALUES.includes(value) ? value : "7d"; | ||
| } | ||
|
|
||
| function json(res, status, value) { | ||
| res.writeHead(status, { | ||
| "Cache-Control": "no-store", | ||
| "Content-Type": "application/json; charset=utf-8", | ||
| }); | ||
| res.end(JSON.stringify(value)); | ||
| } | ||
|
|
||
| function notifyCanvases() { | ||
| for (const entry of servers.values()) { | ||
| for (const client of entry.clients) { | ||
| client.write("event: refresh\ndata: {}\n\n"); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| async function buildDashboardData({ range = "7d", sessionId } = {}) { | ||
| if (!session) { | ||
| throw new CanvasError("session_not_ready", "The session metrics provider is still starting."); | ||
| } | ||
|
|
||
| const selectedSessionId = sessionId || session.sessionId; | ||
| const currentRuntime = | ||
| selectedSessionId === session.sessionId | ||
| ? await session.rpc.usage.getMetrics() | ||
| : undefined; | ||
| const metadata = await agentMetadata.get(selectedSessionId); | ||
|
|
||
| return statsStore.buildDashboard({ | ||
| currentSessionId: session.sessionId, | ||
| currentRuntime, | ||
| range: normalizeRange(range), | ||
| selectedSessionId, | ||
| agentMetadata: metadata, | ||
| }); | ||
| } | ||
|
|
||
| async function startServer(instanceId, defaults) { | ||
| const clients = new Set(); | ||
| const server = createServer(async (req, res) => { | ||
| const url = new URL(req.url || "/", "http://127.0.0.1"); | ||
|
|
||
| if (req.method === "GET" && url.pathname === "/") { | ||
| try { | ||
| const initialData = await buildDashboardData(defaults); | ||
| res.writeHead(200, { | ||
| "Cache-Control": "no-store", | ||
| "Content-Type": "text/html; charset=utf-8", | ||
| }); | ||
| res.end(renderDashboardHtml({ instanceId, defaults, initialData })); | ||
| } catch (error) { | ||
| res.writeHead(500, { | ||
| "Cache-Control": "no-store", | ||
| "Content-Type": "text/plain; charset=utf-8", | ||
| }); | ||
| res.end(error instanceof Error ? error.message : "Unable to load session metrics."); | ||
| } | ||
| return; | ||
| } | ||
|
|
||
| if (req.method === "GET" && url.pathname === "/api/stats") { | ||
| try { | ||
| const data = await buildDashboardData({ | ||
| range: url.searchParams.get("range") || defaults.range, | ||
| sessionId: url.searchParams.get("sessionId") || defaults.sessionId, | ||
| }); | ||
| json(res, 200, data); | ||
| } catch (error) { | ||
| json(res, 500, { | ||
| error: error instanceof Error ? error.message : "Unable to load session metrics.", | ||
| }); | ||
| } | ||
| return; | ||
| } | ||
|
|
||
| if (req.method === "GET" && url.pathname === "/events") { | ||
| res.writeHead(200, { | ||
| "Cache-Control": "no-cache", | ||
| Connection: "keep-alive", | ||
| "Content-Type": "text/event-stream", | ||
| }); | ||
| res.write("event: ready\ndata: {}\n\n"); | ||
| clients.add(res); | ||
| req.on("close", () => clients.delete(res)); | ||
| return; | ||
| } | ||
|
|
||
| json(res, 404, { error: "Not found" }); | ||
| }); | ||
|
|
||
| await new Promise((resolve, reject) => { | ||
| server.once("error", reject); | ||
| server.listen(0, "127.0.0.1", resolve); | ||
| }); | ||
| const address = server.address(); | ||
| const port = typeof address === "object" && address ? address.port : 0; | ||
| return { clients, server, url: `http://127.0.0.1:${port}/` }; | ||
| } | ||
|
|
||
| const usageInsightsCanvas = createCanvas({ | ||
| id: "usage-insights", | ||
| displayName: "Usage Insights", | ||
| description: "Inspect live token and AI-credit usage for the current session, its agents, and recent history.", | ||
| inputSchema: { | ||
| type: "object", | ||
| additionalProperties: false, | ||
| properties: { | ||
| range: { type: "string", enum: RANGE_VALUES }, | ||
| sessionId: { type: "string", minLength: 1 }, | ||
| }, | ||
| }, | ||
| actions: [ | ||
| { | ||
| name: "refresh", | ||
| description: "Return fresh metrics for the current or selected session and time range.", | ||
| inputSchema: { | ||
| type: "object", | ||
| additionalProperties: false, | ||
| properties: { | ||
| range: { type: "string", enum: RANGE_VALUES }, | ||
| sessionId: { type: "string", minLength: 1 }, | ||
| }, | ||
| }, | ||
| handler: async (ctx) => buildDashboardData(ctx.input), | ||
| }, | ||
| { | ||
| name: "inspect_session", | ||
| description: "Return the overall and per-agent metrics for one local session.", | ||
| inputSchema: { | ||
| type: "object", | ||
| additionalProperties: false, | ||
| required: ["sessionId"], | ||
| properties: { | ||
| sessionId: { type: "string", minLength: 1 }, | ||
| range: { type: "string", enum: RANGE_VALUES }, | ||
| }, | ||
| }, | ||
| handler: async (ctx) => buildDashboardData(ctx.input), | ||
| }, | ||
| ], | ||
| open: async (ctx) => { | ||
| let entry = servers.get(ctx.instanceId); | ||
| if (!entry) { | ||
| const defaults = { | ||
| range: normalizeRange(ctx.input?.range), | ||
| sessionId: ctx.input?.sessionId || "", | ||
| }; | ||
| entry = await startServer(ctx.instanceId, defaults); | ||
| servers.set(ctx.instanceId, entry); | ||
| } | ||
| return { | ||
| title: "Usage Insights", | ||
| status: "Live", | ||
| url: entry.url, | ||
| }; | ||
| }, | ||
| onClose: async (ctx) => { | ||
| const entry = servers.get(ctx.instanceId); | ||
| if (!entry) { | ||
| return; | ||
| } | ||
| servers.delete(ctx.instanceId); | ||
| for (const client of entry.clients) { | ||
| client.end(); | ||
| } | ||
| await new Promise((resolve) => entry.server.close(resolve)); | ||
| }, | ||
| }); | ||
|
|
||
| session = await joinSession({ canvases: [usageInsightsCanvas] }); | ||
|
|
||
| const initialEvents = await session.getEvents(); | ||
| agentMetadata.seed(session.sessionId, initialEvents); | ||
|
|
||
| session.on("assistant.usage", notifyCanvases); | ||
| session.on("session.usage_checkpoint", notifyCanvases); | ||
| session.on("subagent.started", (event) => { | ||
| agentMetadata.update(session.sessionId, event); | ||
| notifyCanvases(); | ||
| }); | ||
| session.on("subagent.completed", (event) => { | ||
| agentMetadata.update(session.sessionId, event); | ||
| notifyCanvases(); | ||
| }); | ||
| session.on("subagent.failed", (event) => { | ||
| agentMetadata.update(session.sessionId, event); | ||
| notifyCanvases(); | ||
| }); | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.