Skip to content

feat: source maps, maintenance windows, weekly summary, OpenTelemetry, a Connectors page with GitHub and MobilePay - #19

Merged
ExeQue merged 31 commits into
mainfrom
feat/roadmap-batch
Nov 11, 2026
Merged

ExeQue merged 31 commits into
mainfrom
feat/roadmap-batch

Conversation

@ExeQue

@ExeQue ExeQue commented Nov 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • Source maps (Symbolicate browser errors with uploaded source maps #13): POST /api/ingest/sourcemaps stores maps per project, bundle path and release. Browser frames are symbolicated before grouping, so issues stay stable across builds; the minified frames are a toggle away. Maps are pruned after 90 days unless they belong to one of the last 10 releases.
  • Maintenance windows (Maintenance windows, planned or reported by the app #14): planned in the UI/MCP or reported by the app (maintenance ingest event). Health checks and queues are muted, alerts are marked [Under maintenance] (webhooks get a maintenance object), occurrences and issues are flagged, and downtime up to the expected end is left out of uptime/SLO. A Maintenance overrun rule fires at the expected end and repeats until the window ends.
  • Weekly summary (Weekly summary alert rule #15): a new alert rule, off by default, sending a week's report on a set day and time — full in mail, short in chat, JSON in webhooks.
  • OpenTelemetry (Accept OpenTelemetry (OTLP/HTTP) traces, logs and exceptions #16): OTLP/HTTP traces and logs at /api/v1/opentelemetry/v1/*, protobuf or JSON, mapped onto the existing ingest batch. Adds google/protobuf and open-telemetry/gen-otlp-protobuf.
  • Connectors page: the overview moves from Data sources to its own page, always in the sidebar, with the services' icons, tags, search and a dialog listing every connection with its status. A service connected more than once gets a sidebar entry per connection.
  • GitHub connector (GitHub connector #17): pull requests, commits over time and contributors, CI, releases with their notes, lead time, stars/forks/watchers; with optional permissions also issues and Dependabot/code/secret scanning alerts. A connection can follow another branch (suggested from the repository), chooses which key figures its page shows, and every figure links to GitHub. Lists load more on demand; independent calls run in parallel.
  • Vipps MobilePay connector (Vipps MobilePay connector #18): revenue, transactions, refunds, fees, payouts, average order, refund and fee rates, the balance to be paid out, sales by hour and weekday, revenue per sales unit, and lists of transactions, largest sales and payouts. Days are fetched in parallel. Checked against a production ledger.
  • Docs: reference pages, the connectors grouped by category, README, guides and the docs sidebar.

Companion PR in the Laravel library: ExeQue/laravel-lumberjack (#3 source map upload command, #4 maintenance reporting).

Notes

  • Two migrations add rules to every team (maintenance_overrun on, weekly_summary off).
  • docker/php.ini raises post_max_size to 21M for source map uploads; the nginx example gets client_max_body_size 21m.
  • OTLP has not been run against a real collector yet; tests use the official generated classes and a JS-SDK-shaped JSON fixture.
  • ScreenQrTest occasionally runs out of the 128M worker memory inside the QR package; unrelated to this branch.
  • Not yet walked through in a browser: maintenance on the Health page, the issue filter, the Connectors dialog, the GitHub key figures and links, the MobilePay lists.

Checks

  • php artisan test --parallel: 2724 passed
  • pint, phpstan, npm run check, npm run types:check, npm run docs:build: clean

Closes #13, closes #14, closes #15, closes #16, closes #17, closes #18

POST /api/ingest/sourcemaps stores a map per project, bundle path and
release on the default disk. Browser frames are mapped to their original
source before grouping, so issues stay stable across builds; the minified
position is kept and can be shown on the issue page. Maps are pruned after
90 days unless they belong to one of the last 10 releases, and listed under
the project's settings.

Closes #13
One connection per repository with its own read-only fine-grained token,
checked permission by permission when connecting. Its page and data
sources show open pull requests, those waiting for review, merged pull
requests and failed workflow runs against the period before, open issues,
CI on the default branch and the latest release.

Closes #17
One connection per ledger, found from the merchant serial number and read
through the Report API with an access token that is cached until shortly
before it expires. Revenue, transactions, refunds, fees and payouts are
counted per whole ledger date in the ledger's currency, against the period
before; closed days are cached, since their figures never change.

Closes #18
…ke any repository address

Create token on GitHub opens a fine-grained token with the name, owner,
expiry and read-only permissions filled in. The repository field takes any
address under the repository, such as a pull request or a file on a branch.
A window covers one environment of a project, or all of them. It is
planned in the UI or over MCP, or reported by the app with a maintenance
start and end event. During a window health checks and queues are muted,
alerts are marked as sent under maintenance (and webhooks carry a
maintenance object), occurrences and new issues are flagged, and downtime
up to the expected end is left out of uptime and the SLO. A window that
runs past its expected end sends a Maintenance overrun alert, repeated
until it ends, and a recovery message when it does. The expected duration
and the repeat interval are set per project.

Closes #14
A team rule with project overrides, off by default, that sends a report of
the past week on a set day and time through the rule's channels: one
message per channel with a section per project. It covers requests, error
rate, p95 and Apdex against the week before, new issues and regressions,
the most frequent issues, the routes that got slower, uptime and outages,
maintenance windows, failed jobs and missed tasks, and the week's releases.
Mail gets the full report, chat channels a short version, webhooks JSON.

Closes #15
POST /api/v1/opentelemetry/v1/traces and /v1/logs take OTLP in protobuf or
JSON, plain or gzipped, with the project's server token, so an
OTEL_EXPORTER_OTLP_ENDPOINT of /api/v1/opentelemetry works as is. Server
spans become requests, consumer spans job runs, database spans queries,
exception events issues and log records log lines, all through the
existing ingest batch. Metrics are accepted and dropped.

Closes #16
Source maps and maintenance reporting as deploy steps, the weekly summary
and maintenance windows in the alerting guide, connector figures and the
maintenance state on status screens, OpenTelemetry for apps without a
library, and a body size for source map uploads behind nginx.
… GitHub

Commits over time, authors, the latest commits and the top contributors
from one fetch per period; pull request lead time; stars, forks and
watchers; lists of open pull requests. With the optional Issues and alert
permissions, which connecting probes and the token link now asks for:
issues opened and closed, open bugs, and open Dependabot, code scanning
and secret scanning alerts.
… Releases tab

Pull requests, workflow runs, releases, the latest commits and the top
contributors load more on demand, kept in the address, up to a cap per
list. Each release opens to its notes, rendered as Markdown.
…its figures in parallel

An optional branch, checked against GitHub, for CI status, failed runs and
every commit figure; empty follows the default branch. Independent calls
for the page, the connect checks, the issue counts and the security alerts
run through one pool.
… hour

Average order, refund and fee rates against the period before, the balance
waiting to be paid out, sales by hour of day and weekday, revenue per sales
unit, and lists of the latest transactions, largest sales and payouts with
load more. Uncached days are fetched in parallel; a failing day keeps the
others.
…s' own icons

The connectors overview moves from Data sources to its own page, opened
from Connectors in the sidebar, which is always there. Cards show each
service's icon, what it reads, its tags and its connections, connected
ones first, then by name; search and tag filters live in the address. A
service connected more than once gets a sidebar entry per connection.
The docs group the connectors by category.
…fault as the placeholder

The Connection tab's branch field shows the default branch as its
placeholder and offers the repository's branches in a datalist, loaded
after the page for owners and admins only.
…e Connectors cards

A connected card shows how many connections it has and how many fail;
the count opens a dialog listing them all with their status, when each was
last checked or the error it failed on, and a search for long lists.
… GitHub

Each key figure can be switched on or off per connection; only the choices
that differ from a figure's default are kept, so figures a new permission
unlocks, or stars once a repository turns public, follow their default.
Stars, forks and watchers are separate figures. Every figure links to the
matching page on GitHub. Data sources for screens are not affected.
@ExeQue

ExeQue commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner Author

TODO before this leaves draft

  • Walk through in a browser: maintenance windows on the Health page (plan, end, overrun badge), the issues' maintenance filter, the weekly summary rule settings, the Connectors page and its connections dialog, the GitHub page (key figures, links, branch field with suggestions, Load more, release notes, Security tab), the MobilePay page (figures, hour/weekday bars, lists).
  • Run OpenTelemetry ingest against a real collector (otlphttp exporter) and one SDK (e.g. JS or PHP) to confirm compatibility.
  • Reconnect GitHub with a token from Create token on GitHub and check the optional Issues and alert permissions end to end.
  • Companion PR in the Laravel library: feat: upload source maps and report maintenance mode laravel-lumberjack#5 (rebuilt on origin/main, using the exeque/lumberjack-agent package).
  • Tag releases only once both this PR and the library PR are done.

Follow-ups, not in this PR

  • Webhooks from GitHub (live CI/PR updates, workflow failure alerts, releases from GitHub) and from Vipps MobilePay (live payments, aborted payments).
  • Vipps MobilePay Recurring API (subscriptions, MRR).
  • ScreenQrTest occasionally runs out of the 128M worker memory inside the QR package.
  • lerd: raise postgres max_locks_per_transaction by default (parallel test suites with 16+ workers run out of shared memory).

- GitHub: connecting a repository again keeps its chosen key figures,
  drops the data read from the old branch when the branch changes, and
  renews a connection whose config cannot be decrypted.
- MobilePay: connecting no longer fails with a 500 when another
  connection of the project was saved under another APP_KEY.
- OpenTelemetry: an export with more than 5000 events is answered with
  413, as the agent's batches are; a server span without a status code
  is stored as 200, or 500 when its span status is an error.
- Maintenance: health checks and queue backlogs are muted only up to a
  window's expected end; an overrun alert queued before the window ended
  is still sent as an overrun. Open-window lookups use builder methods.
- Source maps: a path is checked against the column after it is
  normalized, concurrent uploads of the same map no longer collide, and
  a null source keeps the indexes of the others.
…or null sources

- Connecting a ledger again renews a connection of the same name whose
  keys cannot be decrypted (another APP_KEY), instead of adding a second.
- A frame mapped to a null source keeps its minified frame.
- An overrun alert counts its minutes up to the end once there is one.
…ctors

- A maintenance start the agent held back past the following end is
  ignored, and an end only closes windows that started before it.
- OTLP bodies inflate to at most 2 MB: protobuf is decoded in PHP at
  some 150 times the body's size in memory.
- Saving a maintenance window authorizes before validating.
- An ingest batch reads a project's maintenance windows once, not per
  occurrence; the overrun check reads each project's rule once a run.
- Weekly summaries are marked sent only once queued.
- A source map whose file cannot be written keeps no row (503).
- One GitHub connection per repository and project, by a partial
  unique index; a connect racing another renews that connection.
…d connectors

- Symbolication maps one error with at most 4 bundles (its top frames'),
  and decodes at most 20 maps per ingest batch: the browser key is public.
- Occurrences ask for maintenance per occurrence again, so an end in the
  same batch is seen.
- Each end's time is remembered for a day; a start older than it was held
  back by the agent and is ignored. A cancelled planned window no longer
  blocks starts; down, up, down within a second keeps the last start.
- A window planned wholly in the past is saved as ended; one moved to
  start after its end ends where it starts.
- The GitHub page and its settings say the token cannot be read instead
  of failing when the config was saved under another APP_KEY.
- Settings lists the newest 100 source maps with the total.
- Alert rule settings are validated per rule type.
- MobilePay connects hold a lock per ledger; a source map row another
  upload wrote the file for is kept; orWhereCovers without an
  environment covers every window; SourceMap gets a builder.
…run race

- The decoder reads the mappings in one pass without splitting them into
  arrays: a 20 MB map peaks at some 70 MB instead of hundreds, in under
  a second.
- Symbolication is one instance per worker, keeping decoded maps across
  batches within 128 MB, and decodes at most 20 maps a minute per
  project; past that, frames stay minified.
- Editing an open window never ends it; only a new window planned wholly
  in the past is saved as ended.
- expected_until is capped at a week, as expected_minutes is.
- A batch reads a project's maintenance windows once, and again after
  each maintenance event in it.
- An end and an overrun alert claim the window with conditional
  updates, so the ended message follows any overrun alert sent.
- A MobilePay connect waiting on the lock gets a message, not a 500.
- Source maps are addressed by uuid.
… path

Stored paths and frame paths are percent-decoded when normalized, so a
browser's /build/app[2]/my%20assets/app.js finds the map uploaded for
it. The upload's path length is checked as normalized, once.
…ndows moved to the past

- A map whose file could not be read is not kept decoded: the next error
  tries again. A replaced map's upload time moves only once its file is
  written, so a decoded copy of the old one is not taken for it.
- An overrun alert that runs after its window ended is dropped, so it
  never follows the ended message, whatever order the jobs run in.
- A planned window not started yet and moved wholly into the past is
  saved as ended, as a new one is.
- An occurrence stamped after now asks for its window on its own.
… map once a minute

A frame path like %FF decoded to a byte Postgres refuses, failing the
lookup of every frame in the error. A map that fails to decode is
tried again after a minute rather than on every error.
@ExeQue
ExeQue marked this pull request as ready for review November 11, 2026 20:33
@ExeQue
ExeQue merged commit 5e061a9 into main Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant