From 27ec884932b63de50c4cfda8393a70053485c466 Mon Sep 17 00:00:00 2001 From: Matt Hillsdon Date: Fri, 20 Nov 2026 15:42:36 +0000 Subject: [PATCH 1/5] Describe pages, routing and project storage in the technical overview docs/home-page.md has been the working notes for the home page and project storage and will be removed before this reaches main. This moves what should outlast it into the technical overview: how the pages and routing fit together, how projects are stored and saved, what happens across tabs and without IndexedDB, what each kind of import does, and the decisions that are not obvious from the code. It describes the code with the open pull requests against home-page merged, in particular #1431 for #project: links. --- docs/tech-overview.md | 97 +++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 94 insertions(+), 3 deletions(-) diff --git a/docs/tech-overview.md b/docs/tech-overview.md index 6f42d37ff..57e81afc9 100644 --- a/docs/tech-overview.md +++ b/docs/tech-overview.md @@ -14,14 +14,105 @@ The project is bundled using [Vite](https://vitejs.dev/). The test runner is [Vi We use Prettier to format code. Please set this up to format before committing changes, ideally on save (there's [a VS Code plugin](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode)). -The UI is split into different areas by the [workbench component](../src/workbench/Workbench.tsx). This component manages layout including expand/collapse of the sidebar and simulator and is a good starting point to navigate the codebase. The main areas are: +The app has three pages: a home page, a projects page and the editor. See [Pages and routing](#pages-and-routing). -1. The sidebar on the left, with Reference, Ideas, API and Project tabs as well as settings and help menus. -2. The main editing area with the text editor for the current file, header with the project title and the project action bar with buttons for key interactions. When a micro:bit device is connected over WebUSB, the serial area appears between the text editor and the project actions. +The editor's UI is split into different areas by the [workbench component](../src/workbench/Workbench.tsx). This component manages layout including expand/collapse of the sidebar and simulator and is a good starting point to navigate the codebase. The main areas are: + +1. The sidebar on the left, with Reference, Ideas, API and Files tabs as well as settings and help menus. +2. The main editing area with the text editor for the current file, a breadcrumb with the project name and the open file, and the project action bar with buttons for key interactions. When a micro:bit device is connected over WebUSB, the serial area appears between the text editor and the project actions. 3. The simulator on the right, with its own serial area and controls. The branding that you see on the [Foundation deployment](https://python.microbit.org/v/3) is not Open Source and is managed in a private GitHub project. This is swapped in via an alias for `theme-package` in the Vite config. +## Pages and routing + +Routing uses [react-router](https://reactrouter.com/). The routes are set up in [router.tsx](../src/router.tsx) and the URLs are built in [urls.ts](../src/urls.ts). + +- The home page is at the app's root and the projects page at `/projects`. Both are in [src/pages](../src/pages) and are built from components in `@microbit/ui-patterns`, which micro:bit CreateAI also uses, so the two apps present projects the same way. Page composition, the banner and the resource cards are specific to this app. +- The editor is at `/project`, with the sidebar's tab and position in the URL as `/project/:tab/:slug`. The editor used to be at the root, so its old `/:tab/:slug` URLs redirect. +- There is no project id in the URL. Which project a tab has open is kept in session storage. +- The editor route's loader chooses the project: the tab's current one, otherwise the most recently used, otherwise a new one with the starter program. The pages don't need a project, so they render without choosing one. + +The Foundation's website links to the editor with a program in the URL, as a `#project:` link. [project-link.ts](../src/fs/project-link.ts) reads it once at startup. It becomes a new project when the editor first opens (or replaces the single project where there is no database), and is then removed from the address so that a reload does not open it again. Such links point at the root, so the home page sends the user on to the editor while a link is still waiting to be opened. A link that cannot be read is dropped. + +When the editor is embedded in an iframe in controller mode, as micro:bit classroom does, the embedding page owns the project. The editor is then the only page, at the root, and there is no home or projects page. + +## Projects and storage + +Projects are stored in the browser, in IndexedDB, so a user can have several and come back to them. Nothing is stored on a server. + +The pieces: + +- [`ProjectsDatabase`](../src/fs/projects-db.ts) is the database, accessed with [idb](https://github.com/jakearchibald/idb). It has a `projects` store (id, name and a timestamp) and a `files` store keyed by project id and file name. +- [`FileSystem`](../src/fs/fs.ts) is what the editor reads and writes. It keeps an in-memory working copy of the open project and the MicroPython file system used to build a hex file. Its storage sits behind the `FSStorage` interface in [storage.ts](../src/fs/storage.ts). +- [`IndexedDBFSStorage`](../src/fs/indexeddb-storage.ts) is the `FSStorage` for one project in the database. +- [`Projects`](../src/project/projects.ts) lists, creates, opens, renames, duplicates and deletes projects, and keeps the file system on the open one. UI components go through it and never touch the database. + +### Opening and saving + +Opening a project loads its record and all its files in one read, then switches the file system's storage. Loading first means a failed read leaves the previously open project as it was. + +Edits are written through as they happen. There is no separate autosave and no "save" action for browser storage: + +- Writes are coalesced per file for 300ms, then flushed as one transaction. The project's timestamp is the time of the edit, so "last modified" does not depend on when the flush ran. +- Pending writes are also flushed when the page is hidden or unloading. A transaction that waits on a read, or on its requests finishing, does not survive the page unloading. So a flush never reads before it writes, and one from a hidden or unloading page commits explicitly. This is why the storage holds the project's name and writes the whole project record each time, rather than reading the record and updating it. +- A flush at any other time checks, in the same transaction, that the project still exists. +- A failed flush is not retried. The in-memory copy still has the edit, the user sees a toast that stays until dismissed ("Browser storage full" when the quota is exceeded), and the error is logged once per session. + +MicroPython itself is only needed to build a hex file or flash, not to edit. If it fails to load, projects still open and save, and loading is tried again the next time something needs it. + +### More than one tab + +Tabs tell each other about changes over a `BroadcastChannel`, naming the projects affected: + +- Every tab refreshes its list of projects. +- A tab with a changed project open reloads it. +- A tab with a deleted project open stops writing it, and if the editor is showing goes to the home page with a message. + +### Without IndexedDB, and embedded + +If IndexedDB is unavailable, for example in some private browsing modes, the editor falls back to how it worked before projects were stored: one project in session storage, lost when the tab closes. There is no project management, the pages redirect to the editor, and the user is warned before closing a tab with unsaved changes. + +In iframe controller mode the embedding page supplies and stores the project, and none of the above applies. + +A tab that was open on a version of the editor from before the database has its project in session storage. It is moved into the database once, at startup, and becomes that tab's current project. + +### Importing files + +A hex file is always a whole program. Other files are added to the open project from the editor, and make a new project from the pages. [project-import.tsx](../src/project/project-import.tsx) handles both. + +"Editor" below is the Files tab's Add files button or a drop on the editor. + +| Input | Iframe, editor | No IndexedDB, editor | Database, editor | Database, home or projects page | +| ---------------------------- | ------------------------------------------ | -------------------- | ----------------------------------------------------------- | ----------------------------------------------------------- | +| One hex | Replaces the project; asks first if edited | Same | New project named after the file; the editor switches to it | New project; the editor opens on it | +| Hex with other files | Error | Same | Same | Same | +| `.mpy` | Error | Same | Same | Same | +| One `.py` script | Added under its own name | Same | Same | New project named after the script, which becomes `main.py` | +| `main.py` | Replaces `main.py`; asks first | Same | Same | New untitled project with that `main.py` | +| Several files, or a module | Added; asks before overwriting any | Same | Same | New untitled project; starter `main.py` added if none | +| Idea from the Ideas tab | Replaces the project; asks first if edited | Same | New project named after the idea | n/a | +| `#project:` link at startup | Ignored; the host owns the project | Replaces the project | New project | n/a | +| Host `importproject` message | Replaces the project | n/a | n/a | n/a | + +### Design decisions + +Some choices that are not obvious from the code: + +- **The editor opens straight to code.** `/project` in a new tab opens the most recent project rather than asking the user to pick one. The editor has always been quick to get into, and some users bookmark it. +- **No project id in the URL.** The tab's project is kept in session storage instead. This may be revisited if projects are ever stored on a server. +- **Each deployment has its own database.** The database name includes the base path, so deployments that share an origin, such as production and beta, keep separate projects. +- **We don't ask the browser to persist storage.** `navigator.storage.persist()` prompts the user in Firefox and has no effect in Safari. Instead the home page says that projects are stored in this browser. +- **Whether a project has unsaved changes is not stored.** That flag exists to warn before work is lost, and a project in the database outlives the tab. The warning before closing a tab applies only without the database. +- **Imports never overwrite silently.** With the database, a hex or an idea makes a new project rather than replacing the open one. Without it there is only one project, so the editor asks before replacing one that has been edited: in micro:bit classroom a student's dropped hex would otherwise replace the teacher's starter program without a word. Adding files that would overwrite existing ones asks first in every mode. +- **A project changed in another tab is reloaded, not merged.** The last write wins. + +Known limits: + +- An edit may still be lost if the page unloads in the few milliseconds that an ordinary flush is mid-transaction. +- Switching project waits for MicroPython if it is still loading, which is only noticeable on a slow first visit. +- Browsers can evict stored data, for example when the device is short of space, and some do so for sites that have not been visited for a while. + ## Connecting to the micro:bit via WebUSB, flashing and hex files The [device folder](../src/device) contains WebUSB support for the micro:bit using [DAP.js](https://github.com/ARMmbed/dapjs). This includes an implementation of partial flashing which signficiantly speeds up subsequent flashing. From 9618b9438becce0527ffee351dfbbfad3a4c644d Mon Sep 17 00:00:00 2001 From: Matt Hillsdon Date: Fri, 20 Nov 2026 15:44:44 +0000 Subject: [PATCH 2/5] Note Safari's storage eviction and tie it to the projects tooltip --- docs/tech-overview.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/tech-overview.md b/docs/tech-overview.md index 57e81afc9..670912235 100644 --- a/docs/tech-overview.md +++ b/docs/tech-overview.md @@ -102,7 +102,7 @@ Some choices that are not obvious from the code: - **The editor opens straight to code.** `/project` in a new tab opens the most recent project rather than asking the user to pick one. The editor has always been quick to get into, and some users bookmark it. - **No project id in the URL.** The tab's project is kept in session storage instead. This may be revisited if projects are ever stored on a server. - **Each deployment has its own database.** The database name includes the base path, so deployments that share an origin, such as production and beta, keep separate projects. -- **We don't ask the browser to persist storage.** `navigator.storage.persist()` prompts the user in Firefox and has no effect in Safari. Instead the home page says that projects are stored in this browser. +- **We tell users where their projects are, rather than asking the browser to keep them.** `navigator.storage.persist()` prompts the user in Firefox and has no effect in Safari, so we don't call it. Instead an info tooltip by the projects row on the home page says that projects are stored in this browser and that the browser can remove them. Browsers do: see the known limits below. - **Whether a project has unsaved changes is not stored.** That flag exists to warn before work is lost, and a project in the database outlives the tab. The warning before closing a tab applies only without the database. - **Imports never overwrite silently.** With the database, a hex or an idea makes a new project rather than replacing the open one. Without it there is only one project, so the editor asks before replacing one that has been edited: in micro:bit classroom a student's dropped hex would otherwise replace the teacher's starter program without a word. Adding files that would overwrite existing ones asks first in every mode. - **A project changed in another tab is reloaded, not merged.** The last write wins. @@ -111,7 +111,7 @@ Known limits: - An edit may still be lost if the page unloads in the few milliseconds that an ordinary flush is mid-transaction. - Switching project waits for MicroPython if it is still loading, which is only noticeable on a slow first visit. -- Browsers can evict stored data, for example when the device is short of space, and some do so for sites that have not been visited for a while. +- Browsers can evict stored data, for example when the device is short of space. Safari removes it for a site that has not been visited in seven days of browser use. ## Connecting to the micro:bit via WebUSB, flashing and hex files From a1c392c8059c3b85681268921de78d5ec705974d Mon Sep 17 00:00:00 2001 From: Matt Hillsdon Date: Fri, 20 Nov 2026 15:48:37 +0000 Subject: [PATCH 3/5] Describe the close warning by what the user sees --- docs/tech-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tech-overview.md b/docs/tech-overview.md index 670912235..c4ca178ba 100644 --- a/docs/tech-overview.md +++ b/docs/tech-overview.md @@ -103,7 +103,7 @@ Some choices that are not obvious from the code: - **No project id in the URL.** The tab's project is kept in session storage instead. This may be revisited if projects are ever stored on a server. - **Each deployment has its own database.** The database name includes the base path, so deployments that share an origin, such as production and beta, keep separate projects. - **We tell users where their projects are, rather than asking the browser to keep them.** `navigator.storage.persist()` prompts the user in Firefox and has no effect in Safari, so we don't call it. Instead an info tooltip by the projects row on the home page says that projects are stored in this browser and that the browser can remove them. Browsers do: see the known limits below. -- **Whether a project has unsaved changes is not stored.** That flag exists to warn before work is lost, and a project in the database outlives the tab. The warning before closing a tab applies only without the database. +- **No warning before closing a tab.** The editor used to warn when a tab was closed with changes made since the last hex file was loaded or saved, because closing the tab lost them. With the database every edit is already stored, so there is nothing to warn about. The warning remains where there is no database. - **Imports never overwrite silently.** With the database, a hex or an idea makes a new project rather than replacing the open one. Without it there is only one project, so the editor asks before replacing one that has been edited: in micro:bit classroom a student's dropped hex would otherwise replace the teacher's starter program without a word. Adding files that would overwrite existing ones asks first in every mode. - **A project changed in another tab is reloaded, not merged.** The last write wins. From 47538d69a104b4c5eb9f32570fa04b77319390f9 Mon Sep 17 00:00:00 2001 From: Matt Hillsdon Date: Fri, 20 Nov 2026 15:49:08 +0000 Subject: [PATCH 4/5] Drop the storage persistence decision, keeping the limitation --- docs/tech-overview.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/tech-overview.md b/docs/tech-overview.md index c4ca178ba..12e07fe42 100644 --- a/docs/tech-overview.md +++ b/docs/tech-overview.md @@ -102,7 +102,6 @@ Some choices that are not obvious from the code: - **The editor opens straight to code.** `/project` in a new tab opens the most recent project rather than asking the user to pick one. The editor has always been quick to get into, and some users bookmark it. - **No project id in the URL.** The tab's project is kept in session storage instead. This may be revisited if projects are ever stored on a server. - **Each deployment has its own database.** The database name includes the base path, so deployments that share an origin, such as production and beta, keep separate projects. -- **We tell users where their projects are, rather than asking the browser to keep them.** `navigator.storage.persist()` prompts the user in Firefox and has no effect in Safari, so we don't call it. Instead an info tooltip by the projects row on the home page says that projects are stored in this browser and that the browser can remove them. Browsers do: see the known limits below. - **No warning before closing a tab.** The editor used to warn when a tab was closed with changes made since the last hex file was loaded or saved, because closing the tab lost them. With the database every edit is already stored, so there is nothing to warn about. The warning remains where there is no database. - **Imports never overwrite silently.** With the database, a hex or an idea makes a new project rather than replacing the open one. Without it there is only one project, so the editor asks before replacing one that has been edited: in micro:bit classroom a student's dropped hex would otherwise replace the teacher's starter program without a word. Adding files that would overwrite existing ones asks first in every mode. - **A project changed in another tab is reloaded, not merged.** The last write wins. From ed265676beb4741b9fa9958106c91f03704a872f Mon Sep 17 00:00:00 2001 From: Matt Hillsdon Date: Fri, 20 Nov 2026 17:07:00 +0000 Subject: [PATCH 5/5] Doc tweak Co-authored-by: Robert Knight <95928279+microbit-robert@users.noreply.github.com> --- docs/tech-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/tech-overview.md b/docs/tech-overview.md index 12e07fe42..cf4df7b7f 100644 --- a/docs/tech-overview.md +++ b/docs/tech-overview.md @@ -52,7 +52,7 @@ The pieces: Opening a project loads its record and all its files in one read, then switches the file system's storage. Loading first means a failed read leaves the previously open project as it was. -Edits are written through as they happen. There is no separate autosave and no "save" action for browser storage: +Edits are written through as they happen. There is no separate "save" action for browser storage: - Writes are coalesced per file for 300ms, then flushed as one transaction. The project's timestamp is the time of the edit, so "last modified" does not depend on when the flush ran. - Pending writes are also flushed when the page is hidden or unloading. A transaction that waits on a read, or on its requests finishing, does not survive the page unloading. So a flush never reads before it writes, and one from a hidden or unloading page commits explicitly. This is why the storage holds the project's name and writes the whole project record each time, rather than reading the record and updating it.