SpectrumJetson: Jetson Vision Coprocessor Setup

How FRC 3847 / 8515 turned an NVIDIA Jetson Orin Nano Super into a GPU-accelerated AprilTag vision coprocessor: what we built, why, and how to redo it. The detailed technical reference (exact versions, commit hashes, every measurement) is in docs/TECHNICAL.md.

🎓 Vision Training Course: Check out our interactive, 21-chapter visual guide on how the Jetson vision system works at spectrum3847.github.io/SpectrumJetson.

Thank you, Austin Schuh. The CUDA AprilTag detector at the heart of this build is Austin's work, first written for FRC 971 Spartan Robotics. He's the reason any of this works. He now develops it in RealtimeRoboticsGroup/aos (frc/orin). We build from his copy in frc971/bos; in 2026 we started inside FRC-Team-4143's CUDA PhotonVision. Full credits are at the end.

Built by Spectrum 3847 with Claude Opus 5.5 (Anthropic) in Claude Code, which did the research, code, patches, tests and docs alongside the team.

Last updated October 31, 2026.

Where we are now (JetPack 7, for 2027)

The Jetson now runs JetPack 7.2.1 (Jetson Linux r39.2.1, Ubuntu 24.04, CUDA 13.2), JDK 25, and our PhotonVision 2027 fork on WPILib 2027 alpha-8. Robot code is on WPILib 2027 alpha-8 too, with PhotonLib built from the same fork. We moved because the robot's 2027 PhotonLib can't read a 2026 PhotonVision's results, and PhotonVision 2027 can't run on JetPack 6 (its libraries need a newer Ubuntu). The story is in docs/HANDOFF-jetpack7.md.

Sections with a JetPack 6 note below are history from the October 2026 event build: the lessons hold, but the versions and some numbers are from the old setup.

Headlines

Measured on the bench, JetPack 7 (2026-10-30/31), 2 Thriftiest Cams on AprilTags:

What	Number
Frame rate	122 fps per camera at full resolution (1280x800)
Latency, capture to result	~15 ms median, p95 ~16–19 ms
PhotonVision's camera threads	~25% of one CPU core for 2 cameras (was ~50% before CUDA graphs)
Detection on the GPU	about 1.3–1.6 ms a frame
JPEG decode	on the Jetson's JPEG hardware (NVJPG), both engines, every camera
Power while idle	7.7 W, 71 °C with clocks lowered, against ~12 W at full clocks

• CUDA graphs (detector-11): the GPU detector now launches each frame's work as 3 pre-recorded "graphs" instead of ~67 separate calls. That halved the camera threads' CPU and cut median latency by 1.4 ms, with byte-identical detections.
• Both NVJPG engines on JetPack 7: a library mismatch in r39 left the second JPEG engine unused; our decoder fixes it in memory, only inside PhotonVision's process (342 instead of 194 frames/s in a test). Every camera now decodes on the hardware.
• Lower clocks when idle (spectrum-clocks, step 16): 20 s after the robot disables (or with no robot), the Jetson lowers its CPU and GPU clocks; any enable puts them back to full at once.
• Camera firmware from the browser: Settings has a Thriftiest Cam firmware card and a Swyft camera firmware card (see Camera firmware).
• Better calibrations: a full-screen calibration view that keeps partial boards at the image edges, and a study of why our old calibrations were weak at the edges (see Calibrating well).
• Checked against real matches: 6 recorded TXMAN1 matches (54,000 frames) replayed through the detector with each setting (see Field replay).
• Mount check: robot code now flags a camera whose mount is wrong (see Mount check).

From the 2026 event build (JetPack 6), still true in spirit:

• Set up for 4 AprilTag cameras on the USB-A ports, plus a 5th on a USB-C hub (tested: all 5 streaming). They share one USB 2.0 budget, which 5 cameras fill.
• Rewind: robot code can record every camera at 30 fps for 3% of one core.
• Accurate timing: frames are timestamped at mid-exposure and synced to the robot's clock.
• Match-ready: detecting tags about 20 s after power-on, an unplugged camera back in about 1 s, built for power cuts (see Storage), restarts itself after errors.
• Game pieces at 30 fps beside the AprilTag cameras (TensorRT). Not yet redone on JetPack 7.

Quick Start

For a new Jetson, or a spare SSD. You need an Orin Nano (Super) devkit, an NVMe SSD of at least 80 GB, 128 GB or more for Rewind recordings (not a DRAM-less budget drive: see Storage), a USB-C cable, Wi-Fi for the Jetson, and an Ubuntu 22.04 laptop (NVIDIA's flashing tools need 22.04 or 20.04, even for JetPack 7). The 80 GB is the 64 GiB system partition, the 2 GiB settings partition and at least 8 GiB of scratch.

1. Once per laptop: first edit config.env for your team (team number, Wi-Fi connection name, fan mode; L4T_VERSION=39.2.1 is the default). The next step bakes the Jetson's user and hostname into the image. Then scripts/host/01-prepare-bsp.sh downloads and unpacks NVIDIA's r39.2.1 flashing tools (about 3.3 GB) and creates the Jetson's account. After 01-prepare-bsp.sh --force (a fresh unpack), run step 2 again.
2. Prepare the image: scripts/host/prestage-rootfs.sh puts passwordless sudo, the laptop's saved Wi-Fi connection and its SSH key into the image, so nobody has to touch the Jetson after the flash.
3. Flash (about 7 minutes): put the Jetson in Force Recovery Mode (jumper FC REC to GND on the button header, pins 9-10, then power on; or sudo reboot forced-recovery on a running Jetson), then scripts/host/02-flash-nvme.sh. It erases the whole SSD, settings and recordings included, so it asks you to type ERASE first. If the flash fails with the SSD missing ("EMEM address decode error", no /dev/nvme0n1): unplug the power for 10 s, recovery mode again, and re-run. That's what fixed it for us.
4. Set up (unattended, about an hour): scripts/host/setup-jetson.sh. It waits for the Jetson to boot, copies everything over and runs scripts/jetson/install.sh there, which builds the GPU detector, installs PhotonVision and our fork's jar, and runs the robot tuning (09-robot-tuning.sh, steps 1–16). Optionally add --settings photon.sqlite to restore a saved setup. If a step fails, fix it and run the same command again: it carries on from that step. (--prebuilt installs a release's prebuilt libraries instead of building; there's no JetPack 7 bundle yet, and the installer refuses one built for another L4T or CUDA.)
5. Check: open PhotonVision at http://192.168.55.1:5800 and its Match Ready page, or run scripts/jetson/health-check.sh on the Jetson.
6. Before competition, once it's tuned: scripts/jetson/ro-root.sh on makes the system partition read-only (and saves the current settings as the fallback copy). To update anything later: ro-root.sh off, update, ro-root.sh on.

Prebuilt releases. A release's spectrum-jetson-prebuilt-<version>.tar.gz holds what the slow steps compile: the GPU detector, the TensorRT and JPEG-hardware libraries, the field-calibration tool, and our PhotonVision jar. Its MANIFEST names the repo commit it was built from, the L4T and CUDA versions, every patch, and each file's SHA-256; install.sh --prebuilt checks all of it. The latest, v2026.10.02, is a JetPack 6 bundle. To make a bundle after a from-source install: scripts/jetson/make-prebuilt-bundle.sh on that Jetson.

The rest of this README explains what each step does and why.

What's in this repo, and what the names mean

The pieces:

• PhotonVision fork (Spectrum3847/photonvision, branch spectrum-2027): the vision app that runs on the Jetson. Upstream PhotonVision 2027 plus our features; SPECTRUM.md there lists them. The code lives in that repo, not this one.
• GPU AprilTag detector (libspectrum-gpu-apriltag.so): finds tags on the Jetson's GPU in ~1.5 ms a frame. Austin Schuh's CUDA detector, plus our changes (patches/detector-NN), wrapped so PhotonVision can call it (detector/).
• SpectrumVision (robot-vision/): the robot-side Java library that reads the Jetson's results, checks them and fuses them into the robot's pose.
• This repo: everything else to turn a blank Jetson into a robot coprocessor, and to check it.

The folders:

Folder	What's in it
scripts/host/	Run on the laptop: flash the Jetson, back up and restore its SSD, set it up
scripts/jetson/	Run on the Jetson, in number order (01-…14), plus its helpers (spectrum-clocks, spectrum-rt, the fan, health check, camera firmware tools)
detector/	Our wrapper around the GPU detector (GpuDetectorJNI.cc), the hardware JPEG decoder, and replay tools that run recordings through the detector (detector-replay, fieldcal_detect)
patches/	Our changes to the GPU detector (detector-01…12) and the 2026 PhotonVision patches (photonvision-NN, kept for history: the 2027 fork carries them as commits)
kernel/	Our changes to Linux's USB camera driver (uvcvideo)
robot-vision/	The robot-side Java library: reads the Jetson's results, checks and fuses them into the robot's pose
calibrations/	Lens calibrations for our cameras, by camera serial
tests/	Bench and field tests, one folder each, with a run.sh
tools/	Bigger helpers: fake cameras, field calibration, the field model
docs/	The details: TECHNICAL.md (every measurement and decision), REWIND.md, research notes, reviews
training/	The student course website

Names you'll see:

• The GPU detector: 07-build-gpu-detector.sh builds it, 08-select-detector.sh gpu installs it, SPECTRUM_DETECTOR_* are its settings, and its log lines start detector stats and jpeg decoder.
• 971 / AOS / bos (old names): FRC 971 Spartan Robotics, their robot framework (AOS, now kept by Realtime Robotics Group), and frc971/bos, the repo the detector's code comes from. Until 2026-10-31 our files named the detector after them (lib971apriltag.so, bos, SPECTRUM_971_*, 971 stats); the old names still work, and older docs and logs use them. The full list is at the top of TECHNICAL.md.
• 4143: FRC 4143's CUDA PhotonVision, where the 2026 version of this started. 08-select-detector.sh gpu-4143 still installs their older detector build.
• Rewind: recording every camera during a match, to replay later.
• Match Ready: PhotonVision's page that says whether the Jetson is ready for a match.
• Quiet mode / idle mode: after a match or while disabled, the Jetson saves less to disk and runs the cameras slower; after 20 s disabled (or with no robot) it also lowers its clocks (spectrum-clocks).
• Far search: a slower second look for small, far-away tags.
• NVJPG: the Jetson's hardware JPEG decoder; camera frames are decoded there instead of on the CPU.
• CUDA graph: a batch of GPU work recorded once and replayed every frame with one call, which saves CPU.
• Thriftiest Cam / Swyft: the USB cameras we use.
• JetPack / L4T r39: NVIDIA's Jetson software (JetPack 7.2.1 = Jetson Linux r39.2.1, Ubuntu 24.04).

Overview

We turned an NVIDIA Jetson Orin Nano Super into a vision coprocessor that finds AprilTags on its GPU. It ran on team 8515's robot at the October 2026 off-season event (on JetPack 6), and is now set up for the 2027 season.

The robot controller is a SystemCore running WPILib 2027 robot code. The Jetson runs PhotonVision, the same software many FRC teams use on an Orange Pi. Ours is a special version that sends the AprilTag math to the GPU using a detector written by FRC team 971. The robot code talks to it through PhotonLib over NetworkTables, like any other PhotonVision camera.

Everything we did is scripted in this repo, so another Jetson can be set up the same way. These notes explain what we did and why, including the mistakes, so you can understand the system and not just copy commands.

How the pieces fit

A camera frame goes over USB into PhotonVision on the Jetson. The JPEG hardware decodes it, the GPU detector finds the tags, and PhotonVision sends the results to the robot over NetworkTables.

flowchart LR
  CAM["2-5x Thriftiest Cam<br/>USB 2.0, MJPEG 1280x800"] --> PV["PhotonVision 2027 fork<br/>(spectrum-2027)"]
  PV --> DET["GPU AprilTag detector<br/>libspectrum-gpu-apriltag.so"]
  DET --> PV
  PV -->|NetworkTables| SC["SystemCore<br/>WPILib 2027 robot code"]
  SC -->|time sync| PV

Loading

Piece	What we use
Computer	Jetson Orin Nano Super devkit (8 GB), booting from a 256 GB NVMe SSD, no SD card
Operating system	JetPack 7.2.1 (Jetson Linux r39.2.1, Ubuntu 24.04, kernel 6.8) with CUDA 13.2, cuDNN 9.20, TensorRT 10.16
Java	JDK 25
Power mode	MAXN SUPER (the fastest mode, which gives the board its "Super" name)
Cameras	2 Thrifty Bot Thriftiest Cams (docs) for AprilTags: OV9281, mono, global shutter, 1280x800, USB 2.0. Also a Swyft USB Camera.
Power	From the robot through a 15 V boost regulator board, which kept the Jetson running down to a 5 V input (tested 2026-09-25)
Vision software	Our PhotonVision fork, branch spectrum-2027: upstream PhotonVision 2027 on WPILib 2027 alpha-8, plus our features
Tag detector	Austin Schuh's current CUDA detector (from 971 / RealtimeRoboticsGroup), built from frc971/bos with our detector-NN patches. No WPILib inside it any more.
Game pieces	YOLO models on the GPU through TensorRT (our backend); deferred on JetPack 7 until we need it
Robot side	WPILib 2027 alpha-8, PhotonLib built from our fork, and SpectrumVision (robot-vision/)

Versions must match. PhotonLib checks a hash of the message layout and refuses results from a PhotonVision with a different one. Robot code and the Jetson must both be on our alpha-8 fork. (In 2026 the rule was the opposite: stay on PhotonLib alpha-2, because the Jetson ran a 2026 PhotonVision. See docs/MAINTENANCE-2027.md.)

Step 1: Flash JetPack onto the SSD

Flashing writes NVIDIA's operating system onto the Jetson's NVMe SSD and updates the bootloader stored on the board. It's done from an Ubuntu 22.04 laptop over a USB-C cable. The whole flash took about 7 minutes.

Why JetPack 7.2.1: PhotonVision 2027 needs a newer system library (glibc 2.38+) than JetPack 6's Ubuntu 22.04 has. JetPack 7.2.1 is the first to support the Orin Nano, with Ubuntu 24.04 and CUDA 13. (In 2026 we used JetPack 6.2.3, because the 4143 CUDA fork was built for it.)

1. Download and prepare NVIDIA's two files, the "BSP" (flashing tools) and the "sample root filesystem" (Ubuntu itself): scripts/host/01-prepare-bsp.sh downloads them into ~/nvidia/r39.2.1, unpacks everything and installs NVIDIA's flashing prerequisites. It also creates the Jetson's user account (spectrum3847, hostname photonvision-3847), so the first boot doesn't need a monitor and keyboard.
2. Put the Jetson in recovery mode:
   • Unplug power.
   • Jumper the FC REC and GND pins (pins 9 and 10) on the small 12-pin button header under the module. It's not the big 40-pin header.
   • Plug power back in, then remove the jumper.
   • Check with lsusb: the Jetson should show up as 0955:7523 NVIDIA Corp. APX.
3. Flash: scripts/host/02-flash-nvme.sh. The script temporarily stops Ubuntu's network manager from grabbing the Jetson's USB network connection, and temporarily opens the firewall for it. Both are common causes of failed flashes, and both are put back afterward. It erases the whole SSD (back up first: 04-backup-ssd.sh, rewind-pull.sh), so it asks you to type ERASE; YES=1 skips that. It also checks that r39 still keeps our 64 GiB system partition from growing.

Reading the flash log: "Waiting for target to boot-up" repeating for about 30 s is normal, and so are warnings like "backup GPT table is corrupt". The flash is only finished at "Flash is successful", so don't unplug the board before then. Our first JetPack 7 flash failed with the SSD missing ("EMEM address decode error"); a 10 s power cut and a re-run flashed cleanly. If it fails like that: power cycle, don't debug.

Step 2: First boot and verification

After flashing, the Jetson boots from the SSD and shows up on the laptop as a USB network device. The Jetson is 192.168.55.1, and the laptop gets 192.168.55.100.

1. Set up SSH keys so scripts can log in without a password: ssh-copy-id -i ~/.ssh/jetson_ed25519.pub spectrum3847@192.168.55.1.
2. Verify and enable MAXN SUPER: scripts/jetson/01-verify.sh checks three things. The root filesystem must be on the NVMe (/dev/nvme0n1p1), the software version must be R39.2.1, and the MAXN SUPER power mode must exist. It then switches to that mode (it may reboot to do it) and makes it the boot default. Afterward, all 6 CPU cores run at 1728 MHz.
3. Get it online. Over USB it has no internet, and its clock is wrong until it syncs, which makes package downloads fail. Wi-Fi is easiest: sudo nmcli --ask dev wifi connect <SSID>.
4. Install CUDA and Java with scripts/jetson/02-jetpack.sh, which runs apt install nvidia-jetpack and installs JDK 25. Flashing only installs the base OS, so CUDA, cuDNN and TensorRT come from this step.

Tip: never put a password in a script or a chat. On our Jetson the team account has passwordless sudo (/etc/sudoers.d/90-spectrum3847-nopasswd, added by the team so setup scripts can run over SSH); on a fresh Jetson, run sudo steps in a terminal where you type the password yourself.

Step 3: Build and install the vision software

install.sh runs all of these in order; here's what each does.

#	Part	Built where	Script	Notes
1	PhotonVision service	Jetson (installer)	jetson/03-photonvision.sh	Installs the systemd service that starts PhotonVision at boot. We then replace its jar with the fork.
2	GPU detector libspectrum-gpu-apriltag.so	Jetson	jetson/07-build-gpu-detector.sh, then 08-select-detector.sh gpu	Austin Schuh's current code (see below), our patches/detector-NN, and our JNI wrapper in detector/. It also builds the JPEG-hardware decoder (libspectrumnvjpg.so, gray and colour, both engines). 08 installs both and writes the detector's settings: --mwbd 20 and --mse 10 are the defaults we run. Settings you don't give keep their current value (--defaults resets them); it prints what changed.
3	PhotonVision fork jar	Laptop	host/03-build-photonvision-fork.sh, then jetson/06-install-fork-jar.sh	Branch spectrum-2027 of our fork. It builds on the laptop instead of taxing the Jetson; the Jetson runs it on Java 25. The build also bundles the training site (training/) as the sidebar's Documentation page, so the course works on the robot with no internet.
4	Camera driver with a bandwidth cap	Jetson	jetson/11-uvcvideo-payload-cap.sh --install	Needed for 3–4 cameras on the USB-A ports (see Performance). Built from NVIDIA's own r39 kernel source.
5	Robot tuning	Jetson	jetson/09-robot-tuning.sh	Steps 1–16, below. Step 15 (spectrum-rt, low-jitter camera threads) is installed but off; step 16 (spectrum-clocks, lower clocks when idle) is on.
6	USB controller watchdog	Jetson	jetson/14-usb-watchdog.sh --install	Resets the USB controller if the kernel says it died, so the cameras come back without a person. It never reboots (see Known issues).
7	TensorRT backend libspectrumtrt.so	Jetson	built by 07-build-gpu-detector.sh; install to /usr/lib	Game-piece detection. Models go in with jetson/12-install-yolo-model.sh. Deferred on JetPack 7.

04-build-allwpilib.sh and 05-build-gpudetector.sh build the 2026 setup's WPILib and 4143 detector (08-select-detector.sh gpu-4143). The current detector doesn't need WPILib.

Where the detector code comes from. FRC 971 (Spartan Robotics) wrote the CUDA AprilTag detector. Austin Schuh, its author, now maintains it in the RealtimeRoboticsGroup/aos repo and works with team 1868. We started with FRC-Team-4143's copy (GpuDetectorJNI), which dates from about August 2024. We switched to frc971/bos, which has Austin's current code with a CMake build. In a side-by-side test, the new detector found tags exactly as well as the old one, and 30–40% faster. On JetPack 7 we made it independent of WPILib (it builds the AprilTag C library in) and ported it to CUDA 13.

Why build on the laptop sometimes? The fork's jar is Java plus a web UI, with no native code, so it builds the same anywhere. The detector is native ARM and CUDA code, so it has to be built on the Jetson itself (or cross-compiled, which is more work).

Safe deploys. 06-install-fork-jar.sh refuses to install a jar that isn't a valid zip, and it keeps the previous working jar as photonvision.jar.prev. We added that after a truncated jar took PhotonVision down (see the bugs section).

Robot readiness. jetson/09-robot-tuning.sh prepares the Jetson for the robot: no automatic updates, desktop services a robot never uses turned off (printing, scanner, Docker, Bluetooth, ModemManager, firmware updates, the CSI camera daemon, update/man-db/anacron/e2scrub timers; avahi for .local names and udisks2 for USB drives stay on), headless boot, snapd off (it was adding 45 s to every boot), clocks locked at max on boot (and lowered while idle, step 16), USB autosuspend off for cameras, power-cut safety (data on the SSD within 3 s, the system log kept across power cuts), the cooling (the stock fan on NVIDIA's quiet profile, full speed, or none for a fanless heatsink plate: Settings > Robot state > Cooling, or FAN= here; see Fanless), the Wi-Fi reconnecting when NetworkManager gives up on it (only while its radio is on), a 30 s hardware watchdog (also while rebooting, where it was 10 minutes), reboot on kernel panic, PhotonVision restarted on any exit, OpenCV's worker threads sleeping instead of spinning, 1 s USB retries (a stuck camera held up the others on its hub for ~65 s each), and dropping a network connection whose other end has vanished after ~13 s instead of ~15 minutes (a dashboard laptop that left without closing kept its camera streams encoding for nobody). The system log keeps up to 2 GB. On JetPack 6 the Jetson then booted in 16.5 s instead of 57 s, with both cameras detecting about 20 s after power-on (to re-measure on JetPack 7). jetson/health-check.sh prints a PASS / WARN / FAIL readiness report you can run over SSH before a match.

Lower clocks when idle (step 16, spectrum-clocks; photonvision-90 in the fork). The clocks are locked at maximum so frame times stay steady. But between matches that's wasted heat: 20 s after the robot disables (or with no robot connected), PhotonVision runs spectrum-clocks throttle, and any enable runs spectrum-clocks full at once. Measured idle: 7.7 W and 71 °C, against ~12 W at full clocks. If PhotonVision isn't running for 30 s, a guard lowers them too. spectrum-clocks mode full (or the setting on Settings > Robot state) keeps full clocks always.

Deep idle (photonvision-99, spectrum-clocks deep). With no robot connected at all, no web UI open and no camera stream watched for 5 minutes (or PhotonVision stopped for 5 minutes), it goes further: every camera to 5 fps (SPECTRUM_DEEP_IDLE_FPS), CPU capped at 730 MHz, GPU at 306 MHz, the memory clock (EMC) free to drop instead of locked at 3199 MHz and the CPU idle states back on. All six cores stay online (offlining cores 2-4 was dropped 2026-11-01: deep idle with PhotonVision stopped measured 4.67 W either way, and full from deep got faster, ~0.3 s to ~0.1 s). A connected robot is never deep, so nothing between auto and teleop can be. A robot connecting or the UI opening ends it: full frame rate within 5 ms, clocks back to the normal throttle in ~30-50 ms. Measured on the bench (2 cameras, still at 122 fps): the deep clocks alone take 7.7 W / 69 °C to 6.3 W / 68 °C. SPECTRUM_DEEP_IDLE_SECONDS=0 turns it off; spectrum-clocks mode shallow keeps only the normal throttle; mode full keeps everything at full.

Camera threads pinned to cores (photonvision-102, on by default on the Jetson). Each tag camera's thread (the one that runs the tag detector, NAPA included) gets a core of its own: cores 2-5, in camera-name order. A fifth tag camera shares a core, and the log says so. The decode and TensorRT threads are left to Linux; putting them all on core 1 cost 1-3 ms of tag latency in tests. On the bench with 2 cameras, pinning neither helped nor hurt: tag latency, CPU and power stayed the same. It's meant for 4 cameras under load, and it does no harm with 2. Turn it off with SPECTRUM_PIN_CAMERAS=0 in a photonvision drop-in. Other settings: SPECTRUM_PIN_CAMERA_CORES (default 2-5), SPECTRUM_PIN_SHARED_CORE (default -1 = not pinned), SPECTRUM_PIN_OTHERS=1 (every other thread off the cameras' cores), SPECTRUM_PIN_FIFO (real-time priority, off). On the SystemCore it stays off unless set. TECHNICAL.md.

Fanless (heatsink plate)

Measured on JetPack 6 (2026-10-01); idle is now cooler with the clocks lowered (7.7 W, 71 °C).

With the stock heatsink's fan sealed under an aluminium plate and thermal tape (so it can't move air), Cooling: No fan (Settings > Robot state, photonvision-71; or FAN=off 09-robot-tuning.sh) keeps the fan stopped and takes it away from the kernel, which would otherwise switch it on at 74 °C. A Jetson with the stock heatsink and fan should be on a Fan setting: on No fan its fan stays stopped, the chip runs hot, and the cameras get capped (below). The choice applies at once and at every boot, and is kept on the settings partition, so a reinstall without FAN= keeps it (tests/fan-mode/run.sh checks each mode). This repo's config.env leaves FAN empty, and a fresh install with it empty is fanless, like ours. If the chip reaches 95 °C, spectrum-fan-guard sets a flag and PhotonVision caps every camera at 60 fps until it's under 88 °C (photonvision-57; Match Ready and Settings > Robot state show it). The chip itself throttles at 99 °C and shuts down at 104 °C.

Measured on the bench, 2026-10-01 (5 cameras, 4 on AprilTags at 122 fps, plate only, no airflow, lights on):

Load	Result
Full rate, continuously	89.4 °C after 18 min and still rising ~0.3 °C/min (stopped; projected ~95-98 °C). Never needed: matches are 2:30.
Idle at 30 fps (disabled), after that	Settles at 81 °C, 8.7 W
Three matches (auto 15 s + teleop 2:15, 8 min queue at idle between), from that idle	Peaks 86.2, 87.1, 87.6 °C: each match adds 4-5 °C, and the queue brings it back to ~83 °C. 11.6 W during a match.

That's the pessimistic case: at an event the robot is off except in queue, on the field and for short pit checks, so every match starts much cooler. tests/thermal/event-sim.sh --arm, then a power cut and a cold start, measures that case. Earlier plate tests that seemed to level off at ~73 °C were held there by the kernel switching the fan on at its 74 °C trip.

Fake cameras (testing without hardware)

scripts/jetson/fake-cameras.sh start [recording] makes virtual cameras (v4l2loopback) that play a Rewind recording frame for frame at its original rate, looping; with no recording given it generates one with AprilTags (detector/far_search_test --write-session). PhotonVision opens them like USB cameras, so the real path runs: cscore, our MJPEG decode, the GPU detector. While they run, PhotonVision uses throwaway settings in RAM (a copy of the real database's network and team settings, no cameras), so your real cameras' settings are never touched; fake-cameras.sh stop puts everything back, and so does a start that fails.

Uses: the browser test suite on a Jetson with no cameras plugged in (tests/ui), replaying a match recording through a new build, and bench load or thermal tests. Fake cameras have no exposure or gain controls and no USB bandwidth, so those features can't be tested this way. Needs sudo apt install v4l2loopback-dkms (install.sh --only fake-cameras).

Storage and power cuts

The first SSD died from power cuts (2026-09-30). The Jetson stopped booting: the kernel couldn't read the root filesystem's journal ("critical medium error", "JBD2: recovery failed"). The drive's own health report told the story: an Inland TN320 (a budget drive with no DRAM cache and no power-loss protection), 15 hours of use, 22 power cycles, 21 of them unsafe (power cut, no shutdown), and 285 media errors: areas of flash it could no longer read. Its wear was 0%.

We had tested power cuts and called them safe: ext4's journal keeps a filesystem consistent through a cut. But that only holds if the drive keeps what it already stored, and this one didn't. No filesystem can protect against a drive that corrupts its own flash, and robots lose power without warning all the time. Our test checked that the Jetson booted again, not the drive's error count, so it missed the damage building up. Lesson: when a test says "it survived", check what the hardware itself reports too.

So the SSD is now split three ways (02-flash-nvme.sh and scripts/jetson/10-data-partition.sh):

Area	Holds	Written	If a cut damages it
System, 64 GB	Ubuntu, CUDA, PhotonVision's program	only by updates. With ro-root.sh on it's mounted read-only under a RAM layer (NVIDIA's own overlay, nv_overlayfs_config): anything written to it is gone at the next boot	it isn't being written, so a cut can't damage it
Settings, 2 GB	pipelines, calibrations, field calibration, snapshots, robot-state switches	only when someone changes a setting, and straight to the SSD (sync, data=journal)	the Jetson runs the match on the last-good copy kept on the system partition (10-data-partition.sh --commit-settings); Match Ready says so
Scratch, the rest	Rewind recordings, PhotonVision's logs, the system log	all the time	the Jetson still boots and runs; recording pauses and logs go to RAM. Nothing is deleted: copy off what you need, then 10-data-partition.sh --reformat-scratch

Quiet mode (photonvision-56, Settings > Robot state): power cuts only damage an SSD while it's being written, and the robot is switched off a few minutes after every match. So 20 s after a match ends (field connected, teleop, then disabled; photonvision-83), or after the robot has been disabled for 60 s, the Jetson stops writing: Rewind stops, PhotonVision's log file closes (the log still shows in the UI), the system log moves to RAM, and the scratch partition is remounted read-only. The cameras keep running and tags are still found. A RAM log is lost when the robot is switched off, so PhotonVision also sends its log lines to the robot on /photonvision/jetson/log (photonvision-83): warnings and errors always, every line while quiet, at most 20 a second. Robot code that logs that topic keeps them in the match log. Any enable ends it: the partition is writable again about 80 ms later, and vision never waits on it (nothing on the match path writes to the SSD). A recording someone asks for always wins: field calibration, Record now, or robot code's /photonvision/rewind/record keeps quiet mode off, or ends it, for as long as it records. Robot code can also decide with /photonvision/jetson/quiet (true/false); the Jetson publishes quietNow and quietReason. tests/quiet-mode/run.sh plays a fake match and checks all of this: quiet 0.08 s after the match ended, a Record now ended it in 0.3 s, enable to writable in 0.08 s. A PhotonVision restart while quiet hands quiet mode over to the new process (photonvision-64, tests/quiet-mode/restart.sh). The 60 s is timed on a clock the Jetson's date doesn't move, so setting the date from the robot's when it connects doesn't start quiet mode early (photonvision-65, tests/quiet-mode/clock-jump.sh). If the partition can't be made writable again, PhotonVision stays in quiet mode, says why, and tries again every 5 s, and health-check.sh fails (tests/quiet-mode/leave-fails.sh).

Tested (tests/storage-fallback/run.sh scratch|settings, 2026-10-01): with either partition's superblock destroyed, the Jetson boots and PhotonVision serves. With scratch gone, its logs, the system log and Rewind go to RAM, and nothing lands on the system partition. With settings gone, it runs on the committed copy, with identical database contents. e2fsck -b 32768 then repairs the partition with its files intact. tests/ro-root/run.sh: with the read-only system on, PhotonVision and both partitions work, a file written to the system partition is gone after a reboot, and off makes it writable again (about 40 MB lands in the RAM layer per boot). Still to test: real power cuts on this layout, on a healthy SSD.

Choosing an SSD: use a known brand with a DRAM cache, or better an industrial drive with power-loss protection. Avoid DRAM-less budget drives. health-check.sh reports the drive's media errors and unsafe shutdowns; any media error means replace it. The installer saves the drive's health report at install time (~/install-logs/ssd-smart-at-install.txt).

Bugs we found and fixed

Most of these were found on the 2026 (JetPack 6) build; the fixes carried over to 2027. The detector-NN patches were called bos-NN then.

None of this code was written for our exact setup, so we found and fixed several real bugs. Each fix is a small patch file in patches/, applied automatically by the build scripts. They're good examples of how real systems fail.

Bug	What went wrong	Fix
Memory leak	The detector leaked two small matrices every time it decoded a tag. Over a long event it could run the Jetson out of memory.	Applied Austin's upstream fix (3e570d5a).
Detector slots	The C++ code had 10 detector slots and never reused them. The 11th pipeline change got handle -1 and then read past the start of an array, which is undefined behavior.	Reuse slots, check every handle, free detectors properly (gpudetector-03). A stress test creates and destroys 300 detectors.
Stale CUDA error	CUDA's error flag stays set until someone reads it. Newer CUDA libraries (CUB) fail on any leftover error, so one unchecked call broke a later, unrelated call on every frame.	Clear leftover errors before each frame, and check the calls that weren't checked.
One GPU error crashed everything	Austin's code aborts the whole program on any CUDA error, which takes PhotonVision down with it.	A CUDA error now skips one frame. PhotonVision restarts only if the whole GPU context is broken (cudaDeviceSynchronize fails for 1 s); a failure on one camera never takes the others down (detector-01 + detector/). Until the 2026-10-01 audit that held only for Detect() itself: a CUDA error in the mask upload, the graph recording or the far search escaped into Java and aborted the JVM. All of it is guarded now; a far-search failure only turns the far search off for 10 s.
Blank frames failed every time	With nothing to detect (covered lens, dark pit, plain wall), the detector launched a GPU kernel with 0 blocks, an invalid launch that broke the next step on every frame. Combined with the old watchdog, that restarted PhotonVision in a loop.	Return "no detections" early when there are no candidate blobs (detector-02). Reproduced and verified with tests/detector-frame-sizes: blank frames failed at 5 resolutions before, pass at all of them now.
Debug timing on every frame	With 4 cameras, PhotonVision used 2.2 CPU cores, over half of it inside the Linux kernel, and detection took 4.4 ms instead of 1.5. At the end of every frame, the detector asked the GPU for 22 timestamps to build a timing report that's only printed in debug mode, which is always off. All 4 cameras share one CUDA lock, so ~60 of these pointless calls per frame per camera had the cameras taking turns on that lock tens of thousands of times a second.	Record and read the timestamps only when the report is on (detector-03): detection 3.1 → 2.2 ms and 0.36 fewer cores with 4 cameras. Found by tracing which threads woke which (Linux ftrace) and pausing PhotonVision in gdb to see where they waited. Lesson: code that does nothing visible can still cost a lot, especially when it holds a shared lock.
62-second restart	Aborting triggered Ubuntu's crash reporter, which spent 28 s writing a 156 MB crash file before the restart could begin.	Exit cleanly instead, and turn off JVM core dumps. Worst-case vision outage went from about 62 s to about 8 s.
Blank AprilCudaTag tab	The fork's settings tab was written for an older version of the web framework (Vue 2) and couldn't render in Vue 3.	Ported it to Vue 3, showing only settings that actually do something (photonvision-01).
Missing Device Control card	PhotonVision used a brand-new browser feature (Intl.DurationFormat) to format the uptime. Firefox 130 doesn't have it, so the whole card, including the Restart button, disappeared.	Check for the feature and fall back (photonvision-03). Also: keep your browser updated.
Truncated jar	A deploy copied a 228 KB piece of a 76 MB jar. PhotonVision couldn't start, and systemd gave up after 5 tries.	The install script now refuses invalid jars and keeps the previous working one.
Colour decode hiding in cscore	Asking cscore for grayscale frames still decoded each JPEG to full colour first, then converted it: 8.9 ms a frame, most of a CPU core per camera.	We decode the camera's JPEG straight to gray in our detector library (photonvision-09): 2.6 ms. Both cameras went to 122 fps and CPU fell from 3.3 to 1.3 cores.
A camera stuck at 320x240, for real this time	The first fix (photonvision-27) never worked. The camera driver refused the 1280x800 format ("device busy") because the previous connection wasn't released yet. cscore recorded 1280x800 anyway, and PhotonVision's own log printed "Failed" on every successful start, which hid the real failure. The recovery only paused and restarted the stream, which never sends the format again.	Switch the camera to another mode and back, which really reopens it and sends the format twice (photonvision-37). The log now says "Failed" only on a real failure. Lesson: read the library's source before trusting what a function name says it does.
A camera's thread killed at startup	While PhotonVision was still adding cameras, one camera's thread read the camera list for the dashboard and crashed on it (ConcurrentModificationException). That camera never ran until the next restart.	A list that's safe to read while it's being added to, and the dashboard update can no longer kill a camera thread (photonvision-38).
A camera silently stuck at 320x240	At about 1 in 20 PhotonVision starts, cscore's own "restore the video mode" on connect raced PhotonVision's and left TopRight streaming 320x240, while cscore believed 1280x800. Our fast decoder saw the size mismatch, gave up after 30 frames (0.25 s), and fell back to cscore, which upscaled the 320x240 frames to 1280x800. Detection kept running at 122 fps on a blurry, stretched image: less range, 1.7 CPU cores instead of 0.5, and no error anyone would notice.	photonvision-27: a size mismatch never falls back. The frame is dropped, and after 1 s the camera is closed and reopened, so cscore applies the mode again. Setting the same mode again does nothing in cscore. Real decode failures still fall back, but it's retried every 10 s instead of until the next restart. The health check now fails any camera whose actual format isn't what PhotonVision set.
Frozen frames from the JPEG hardware	The Jetson has hardware for decoding JPEGs. The obvious way of calling NVIDIA's library, the way another FRC codebase does it, returned the first frame over and over, and reported success every time. A speed test looked great.	We found it by checking every decoded frame against the CPU decoder, pixel for pixel. We call the library differently now (detector/NvJpgDecoder.cc), and while running, one frame per camera every 2 s is re-decoded on the CPU and compared. Any difference switches the hardware off (tests/jpeg-hw/run.sh repeats the full check).
A memory leak in NVIDIA's JPEG library	NVIDIA's JPEG library leaked about 250 KB on every frame unless it's put in its "MJPEG" mode, which NVIDIA's own example code does but doesn't explain. PhotonVision grew to 5.6 GB within minutes, and Linux killed it twice. Our tests had only ever run for a minute at a time, so they never noticed.	One setting (mjpeg_decode) fixed it: memory now stays flat (8 minutes and ~116,000 frames measured). The test now fails if memory grows, and health-check.sh warns if PhotonVision uses over 2.5 GB or was killed for running out of memory. Lesson: run new code for a long time and watch its memory, not just its speed.
Lens model cut short	Calibration produces 8 lens-distortion numbers, but the fork only passed the first 5 to the CUDA detector. The detector uses them to straighten tag edges when it refines corners, so corners near the image edges came out slightly wrong.	A new setparams8 call passes all 8 (photonvision-04 + detector/). The log now shows "(8 dist coeffs)" for each camera.
Timers on the wrong clock	Quiet mode waits for the robot to be disabled for 60 s. It measured that with the date, and at an event the Jetson boots on the date it last saved, then jumps to the robot's when it connects. So quiet mode started one second after connecting. A jump the other way would have frozen a frame-rate-limited camera for as long as the jump.	Every timer uses a clock the date can't move (System.nanoTime, photonvision-65). tests/quiet-mode/clock-jump.sh jumps the date an hour: quiet mode now starts at 60 s, not 1. Lesson: the date is for showing people the time, not for measuring it.
A crash from a dashboard edit	Looking up the current pipeline also applied any pending pipeline switch, which frees the old pipeline's GPU detector. The dashboard's thread did that lookup for every edit, so a slider dragged at the moment robot code switched pipelines could free a detector the camera's own thread was still using: a crash, every camera down for ~8 s.	Only the camera's own thread switches pipelines now (photonvision-66). Lesson: a "get" that changes things is a trap for whoever calls it next.

Lesson: check results by measuring, not by assuming. The truncated-jar bug happened partly because we trusted a log line from the old process. Now every check looks at the running process's own ID.

Performance: what actually limits the frame rate

Now, on JetPack 7 (2026-10-30/31), 2 Thriftiest Cams at 1280x800:

Change	Result
Starting point after the move	122 fps each; min_white_black_diff was left at 5 by mistake: detect 2.2–2.6 ms, GPU ~45%, 4–11 ms spikes
--mwbd 20, the robot setting (now 08's default)	detect ~1.6 ms, GPU ~27%, ~1 ms lower median latency
Both NVJPG engines (libspectrumnvjpg.so fixes an r39 library mismatch in memory)	every camera on the JPEG hardware; 342 instead of 194 frames/s in a test
The rest of each frame's GPU work as CUDA graphs (detector-11)	CPU per frame ~4x lower in the detector; camera threads ~50% → ~25% of a core for 2 cameras; median latency −1.4 ms; detections byte-identical
Today	122 fps each, ~15 ms median latency (p95 ~16–19 ms)

PhotonVision's log prints each camera's latency (median, p95, max) every 10 s ("Loop timing"), and tests/perf-snapshot.sh and tests/bench-session/run.sh collect it.

The rest of this section is the 2026 story (JetPack 6). The lessons still hold.

We went from 33 fps to the cameras' full 122 fps, on two cameras at once. The GPU was never the bottleneck: the detector takes under 2 ms per frame, fast enough for 500+ fps. The real limits were the camera exposure, how PhotonVision reads frames, and how it decodes them.

Change	FPS (1 camera)	Latency	Why it mattered
Starting point (exposure 295)	33	44 ms	Exposure was 29.5 ms per frame, which caps the frame rate at 34 fps
Exposure 83 (8.3 ms)	61	20 ms	The camera can now deliver 120 fps. PhotonVision became the limit
Ask cscore for grayscale frames	62	18 ms	Same fps, but Java CPU dropped from 155% to 111% of a core. (Only half the fix: see below.)
Low Latency Mode off	~100	18 ms	PhotonVision stopped waiting for each frame and missing every other one
Two cameras, all of the above	92 and 104	~23 ms	Used 3.3 of 6 CPU cores
Decode the JPEG straight to gray ourselves (photonvision-09)	122 each	13 ms	cscore was secretly decoding every frame to full colour, then converting (8.9 ms a frame). Our decoder does gray only (2.6 ms).
OpenCV's worker threads sleep instead of spin (tuning step 9)	122 each	13 ms	2 cameras now use 1.3 of 6 cores, down from 3.3
Decode JPEGs on the Jetson's JPEG hardware (--jpeg nvjpg)	122 each	not measured	On the same bench scene, back to back: PhotonVision went from 0.85 to 0.52 cores. The hardware takes ~2.6 ms a frame whatever the scene; the CPU decoder is faster on plain scenes and slower on busy ones.
Dashboard stream only while someone is watching, at most 30 fps (photonvision-15)	122 each	not measured	With no browser open, PhotonVision went from 0.49 to 0.43 cores. It used to shrink, colour-convert and draw every frame for a stream nobody was watching.
Colour frames on the JPEG hardware too (photonvision-18)	120 (colour)	not measured	One colour camera at 120 fps (driver mode) beside an AprilTag camera: PhotonVision went from 1.12 to 0.59 cores. The pixels are identical to cscore's own decode.

Exposure units are a trap. USB cameras count exposure in the UVC standard's 100 µs units, so 295 means 29.5 ms, not 0.3 ms, and stock PhotonVision doesn't say so. We only found this by reading the camera's control directly with v4l2-ctl. Our build now shows milliseconds on the slider, e.g. "Exposure (5.0 ms)" (photonvision-10).

Shorter exposure, lower cutoff. Shorter exposure means less motion blur, but a darker image and a lower decision margin (how confidently a tag decoded): about 44 at 3 ms, about 118 at 8.3 ms. PhotonVision drops tags below its cutoff, 35 by default. We run exposure 50 (5 ms) with the cutoff at 15. The detector still finds real tags, and tag36h11 at Hamming distance 0 almost never gives false positives.

Lights flicker. Mains lighting flickers 120 times per second (every 8.3 ms). At exposures that aren't a multiple of 8.3 ms, each frame can catch a different part of the flicker and pulse in brightness. In our shop at 5 ms we measured no flicker (0.6% brightness change frame to frame) with tests/flicker-check/run.sh. Run it again under the event's lights; if frames pulse by several percent, go back to 83 (8.3 ms).

Measure before optimizing. We added a once-per-second stats line ("detector stats") to the detector (calls per second, milliseconds per frame, tags per frame, decision margin). It showed right away that the GPU was idle and the camera pipeline was the problem, which saved us from optimizing the wrong thing.

Four cameras share one GPU. With 4 cameras the GPU was only about a third busy (before the CUDA graph; now about a quarter), yet the slowest frames take 6–12 ms instead of ~3. That's queueing: a camera's GPU work sometimes waits behind another camera's, like a short line at a busy store. We tried two settings to shorten the wait: 32 hardware work queues for CUDA instead of 8, and letting the CPU sleep while the GPU works instead of spinning. The first test made the queues look like a big win, but repeating it showed the same settings vary by 2 ms from one restart to the next. Over 11 runs the queues were worth maybe 0.5 ms, and sleeping made no difference. Both are on anyway because neither costs anything (08-select-detector.sh sets the queues; details in TECHNICAL.md). Lesson: measure more than once before believing a speed-up. Also close the dashboard before measuring: 4 open camera streams added about half a CPU core and 1–2 ms to the slowest frames.

Four Thriftiest Cams at 122 fps (2026-09-29). The first time all four ran, PhotonVision used 2.2 CPU cores and the GPU was 40% busy. Three fixes later:

Change	Detect time	CPU	GPU
Starting point, 4 cameras	4.4 ms	2.2 cores	40–43%
Stop debug timing on every frame (detector-03, see the bugs table)	2.2 ms	1.5	35%
Find blob edges and drop the empty ones in one GPU step, not two (detector-04; the original author's own to-do)	1.9 ms	1.4	28%
Keep each frame on the GPU after the JPEG hardware decodes it, instead of copying it down and back up	1.3 ms	1.26	27%
Launch the first 13 GPU steps as one pre-recorded CUDA graph (detector-05)	1.2 ms	1.23	25.6%
Stop copying every gray frame in PhotonVision (photonvision-37)	1.2 ms	1.14	25.7%

Each frame now takes less time with four cameras than one camera did before. Five cameras also run at 122 fps each: 2.1 ms detection, 44% GPU, 1.9 cores, 11.1 W (facing bright ceiling lights, the hard case). The limit is now USB bandwidth: five capped Thrifties use 6,400 of the ~6,720 bytes the Jetson's USB 2.0 ports share, so a 6th won't fit. What's left of the CPU is mostly inside NVIDIA's own libraries: how CUDA waits for the GPU, and the JPEG hardware's driver, which reopens the graphics device for every frame.

Where the latency goes. A result reaches the robot about 14.5 ms after the middle of the exposure. We timed each step with the camera driver's own log: half the 5 ms exposure (2.5 ms), then 8.1 ms for the camera to send the frame, then 0.15 ms until our decoder starts, 2.8 ms of JPEG decode, and 1.2 ms of detection. The camera streams each frame out over about one frame period while its sensor reads out, so neither USB nor PhotonVision is what's slow. The detector-04 change was checked by replaying ~5,000 recorded frames: every one of 3,567 tag detections came out byte-for-byte identical.

More cameras. Each camera at its full 122 fps now costs about 0.6 of a CPU core (it was 1.4 before the decode fix), and less with the JPEG hardware, so 4 cameras should fit. The limit was USB bandwidth. With the stock driver each camera reserves ~196 Mbps whatever mode it runs, and the Jetson's whole USB 2.0 side holds two of those (see below).

We fixed that with a patched camera driver (scripts/jetson/11-uvcvideo-payload-cap.sh). It caps the Thriftiest Cam's reservation at 82 Mbps (UVC alternate setting 7), still about 1.4x the largest frame we've measured at 120 fps. Now 4 cameras fit on the USB-A ports, plus a 5th capped at 1600 bytes on a USB-C hub. Tested with 2 cameras: 122 fps each, every frame complete. We first estimated this would cost ~2 ms of latency (a 50 KB frame crossing USB in ~4.9 ms instead of ~2 ms). Measured with 4 cameras, it costs nothing: the camera itself spreads each frame over ~8 ms while its sensor reads out, and the capped link keeps up with that. It does not make timestamps less accurate: the driver stamps a frame when its first USB packet arrives. Java's memory isn't a concern: the heap peaked at 28 MB with zero garbage collections in 20 s.

There is only one USB 2.0 bus. The USB-C port doesn't add bandwidth (measured 2026-09-24). The Jetson has one USB controller, with one USB 2.0 bus (lsusb Bus 1) and one USB 3 bus (Bus 2). Every USB 2.0 port is a branch of Bus 1 and shares its budget:

• the four USB-A ports (one hub inside the Jetson);
• the USB-C port;
• the M.2 Key E slot (the Wi-Fi card's Bluetooth).

We tested by setting cameras' USB alternate settings directly, with PhotonVision stopped. The bandwidth figures are bytes reserved per 125 µs microframe.

Reserved	Where	Result
6120	two uncapped cameras on USB-A (3060 + 3060)	fits
6132	3072 on USB-C + 3060 on USB-A	fits, but a second 3060 on USB-A is then refused
6720	four at 1280 on USB-A + 1600 on USB-C	fits (what we run)
7400	3060 + 3060 + 1280, all on USB-A	refused
7520	four at 1280 on USB-A + 2400 on USB-C	refused

So about 6700 bytes fit, whichever ports the cameras are on (about 430 Mbps). With 4 cameras at 1280 and a 5th at 1600 it's full: a 6th USB 2.0 camera needs lower caps (check each camera still holds its frame rate), or one of these:

• A USB 3 camera: it streams on Bus 2, the USB 3 bus, which has its own budget. The USB-A ports are USB 3.2, and so is USB-C.
• A USB controller card in the spare M.2 slot: a second controller gives a second USB 2.0 bus.
  • The slot: the Orin Nano dev kit has an empty M.2 Key M 2230 slot, PCIe 3.0 x2 (NVIDIA's hardware layout, mark 11). The 2280 Key M slot holds our SSD.
  • Drivers: Key M slots carry only PCIe, not USB, so this needs a PCIe-to-USB controller card. The Jetson's kernel has the drivers (xhci-pci, plus the Renesas firmware loader).
  • Untested: most such cards are 2280 or need a riser, and the new ports must be mounted on the robot.
  • Not the Key E slot: its USB is Bus 1 again, and it holds the Wi-Fi card.

The Camera Matching page shows all of this (photonvision-32). A USB bandwidth card at the top draws the shared budget as a bar, one colour per camera, with what each actually sends filled in. Per camera it shows:

• FPS and Using: what the camera really sends, in MB/s and as a share of its allocation. At 90% or more it says "squeezed" (see below).
• Largest frame: the biggest recent frame, how many times it fits in one frame time at the camera's allocation ("fits 1.4x"), and how long it takes to cross USB.
• Allocation: what it reserves. Pick another from the list to change it:
  • it's saved for the next boot;
  • the camera is reset, like a replug, to apply it: about 2 s without frames;
  • choices that would go over the budget are greyed out, so to give one camera more, lower another first.

Each camera's card also gets a line like "USB bandwidth: 3.4 of 10.2 MB/s (port 1-2.4)".

What if a camera needs more than its allocation? It can't take more: the USB host lets it send only that many bytes per 125 µs. Its allocation is also guaranteed, so one camera never takes another's.

We tested what the camera itself does by giving two cameras less than they were sending (2026-09-24). Both kept their frame rate and compressed harder instead:

Camera	Allocation	fps	Average frame	Frame time on USB	Slots in use
TopRight (Thriftiest)	1280 B (10.2 MB/s)	121	47.5 KB	4.6 ms	—
TopRight	640 B (5.1 MB/s)	121	40.8 KB	8.0 ms	97%
Colour USB Camera, 640x480	256 B (2.0 MB/s)	30	36.6 KB	18 ms	76%
Colour USB Camera	128 B (1.0 MB/s)	30	31.5 KB	31 ms	96%

So a squeezed camera:

• sends lower-quality JPEGs, which may cost AprilTag range and precision (not yet measured with tags);
• adds latency, because each frame takes longer to cross USB.

We haven't pushed a camera so far that it drops frames.

Choosing an allocation:

• AprilTag cameras: keep them out of "squeezed" (under 90%), with "fits" at 1.3x or more.
• The cost of going smaller: latency. A 60 KB frame takes 5.9 ms to cross USB at 1280 bytes, and 8.0 ms at 944.
• Slower cameras have room to spare: a 30 fps game-piece camera or a 60 fps camera usually fits 2–6x.

The numbers come from our camera driver, which counts each camera's bytes and largest frames (kernel/uvcvideo-payload-cap.patch). Allocations set on the page are per USB port (e.g. 1-2.4:944) and override the per-model defaults from 11-uvcvideo-payload-cap.sh, which keeps them when reinstalled.

scripts/jetson/usb-bandwidth.py shows the same from the command line: what each camera reserves, which ports failed and why, and the cap command to fix it. The health check runs it too.

Timestamps mark mid-exposure (photonvision-13): the Jetson subtracts half the exposure from every frame's timestamp, so the robot shouldn't. On top of that, SPECTRUM_CAMERA_DELAY_US (the camera's own delay, readout and JPEG, before it sends) is 2500 µs by default (09-robot-tuning.sh, step 17; CAMERA_DELAY_US=N overrides): Team 100's rotating-disc measurement of the Thriftiest Cam in MJPEG, a starting value until our own spin test. The TXMAN1 replays (rotation method, tests/field-replay/RESULTS.md) put the whole camera-to-log shift at +8 to +10 ms (95% CI about ±3 ms); that fits 2.5 ms if the robot's logged pose lags by ~13–15 ms, which the replay can't check. Frames are timestamped by the camera's own clock (see Steadier timestamps below).

What other teams' vision systems do (Austin's AOS as run by 1868, 4646 and 254, 971's bos and cos, 6328's Northstar, EagleEye, Code Orange's MLTag, 4533's Whacknet), what an ideal system would have, the full profiling story, and future work: docs/VISION-RESEARCH.md.

Cameras and calibration

Both cameras use the same PhotonVision settings. Each one needs its own calibration at the resolution it will run at.

Camera settings (Dashboard, per camera). The camera's Input tab has a Tuning guide (photonvision-26): what to set, in order, with these values and why, and each control's tooltip gives its recommended value.

• The main point: short exposure on a robot. Auto exposure, or tuning by eye with the robot standing still, picks long exposures. Those blur as soon as the robot moves: spinning at 3 rad/s smears a tag about 11 px at 5 ms, and 44 px at 20 ms.

• Calibration is the exception: a long exposure is fine there because the board is still, but don't copy it into the vision pipelines.

• Type: AprilTagCuda

• Resolution: 1280x800 at 120 FPS, MJPEG. Don't use YUYV, which only manages 5 fps at this resolution.

• Auto Exposure off, Exposure 50 (5 ms), Brightness 100

• Camera Gain: the slider appears since photonvision-21 (upstream's Thriftiest Cam support), but the cameras' factory firmware reports no gain control, so there it does nothing. Thrifty Bot's published firmware 5.13 (experimental) restores gain; both robot cameras have it since 2026-10-29, flashed from the laptop through the Jetson without unplugging them (scripts/host/thriftiest-flash.sh, docs/THRIFTIEST-CAM-FIRMWARE.md). Test the slider before relying on it (docs/UPSTREAM-PORT.md).

• AprilTagCuda tab: decision margin cutoff 15

• Low Latency Mode off

• Processing Mode 3D and multi-tag on (Output tab), once the camera is calibrated. The robot's pose code needs both.

• Stream Resolution: small, to save CPU (it only affects the video you watch in the browser)

• AprilTag field layout: 2026 Rebuilt AndyMark until the robot connects. Robot code sends its own map, and the Jetson loads it, keeps it for the next boot and reports which one it's using (photonvision-74, below).

New cameras start with these settings automatically. A camera PhotonVision has never seen gets AprilTagCuda, 1280x800 MJPEG, exposure 50, decision margin 15, brightness 100, white balance 2800 K, Low Latency off and multi-tag on (TeamCameraDefaults in photonvision-06). 3D can't work without a calibration, so it starts off and turns itself on (with multi-tag) as soon as a calibration is saved or imported for the resolution the camera uses. It never turns 3D off. Existing cameras keep their saved settings.

Give each camera a serial of its own, and its name and calibration go with it (photonvision-86). Out of the box every Thriftiest Cam reports the same serial (01.00.00), so PhotonVision can only tell them apart by the USB port they're plugged into: each name (and its calibration) stays with its port, and a camera in the wrong port silently gets another camera's calibration. Store a serial in each camera once, from the laptop, robot disabled:

JETSON=10.85.15.15 scripts/host/thriftiest-flash.sh rename 2.1 SJ-TC01   # port, serial

Serials must start with SJ- (PhotonVision only trusts serials we stored) and be unique: label the camera with it. Settings > Thriftiest Cam firmware (photonvision-87) shows each camera's serial and firmware, warns about a camera that won't recover from a USB drop, and flashes the usbrecov firmware from the browser (docs/THRIFTIEST-CAM-FIRMWARE.md). At its next start PhotonVision moves that camera's saved settings and calibration from the port to the serial, and from then on finds it in any port (tested 2026-10-29: two cameras swapped ports, then moved to a third port; each kept its calibration). Cameras without one are still matched by port, as in the table below. The robot code uses the names, e.g. new PhotonCamera("TopLeft"), either way.

Swyft USB Camera (Realtek 0bda:3035, port 2.4, serial SJ-SW01 since 2026-10-30): every one ships with the same serial (200901010001), and its factory firmware won't go below 5 ms exposure. The robot's now runs Swyft's low-exposure firmware (0.2 ms) with a serial of its own, flashed from the Jetson by scripts/jetson/swyft-fw.py (docs/SWYFT-CAMERA-FIRMWARE.md).

Name	Physical port (looking at the ports)	USB hub port
TopLeft	top row, left	2.1
TopRight	top row, right	2.3
BottomLeft	bottom row, left	2.2
BottomRight	bottom row, right	2.4 (expected)

All the Jetson's USB 2.0 ports share one bandwidth budget: the four USB-A ports, the USB-C port, and anything on a hub (see Performance). Four cameras fit only with our capped camera driver installed (scripts/jetson/11-uvcvideo-payload-cap.sh --install; the health check shows which driver is loaded). With the stock driver, only two fit.

A calibration belongs to one physical camera and lens. With a serial stored in it, the calibration follows the camera; without one, if you move a camera to another port, recalibrate it there. One camera's calibration on another is off by about 3.8° (27 cm at 4 m) on our two: see TECHNICAL.md, "Cameras known by their serial".

Focusing a lens (photonvision-34): the Camera page's Focus card scores how sharp the image is, like Limelight's focus tool.

1. Turn on Measure, and point the camera at something detailed that stays still: a tag or the calibration board, about as far away as the tags you care about most.
2. Press Reset. Turn the lens slowly past the sharpest point, then back to where the centre reads 100%.
3. Check the 3x3 grid: each part of the image is shown against its own sharpest. A corner well below 100% when the centre is at 100% means a tilted lens or a soft corner.
4. Lock the lens (glue or silicone), then recalibrate.

The score depends on the scene and the light, so only compare numbers without moving the camera. It's measured only while the card is on, about 5 times a second, off the vision thread.

Copying settings between cameras (photonvision-25): no more photographing one camera's settings and typing them into another.

• Where: in the pipeline menu (☰ next to the pipeline name), Copy settings from… copies from any camera and pipeline into the one you're looking at.
• What: tick the groups: Camera (exposure, brightness, gain, white balance, stream resolution), AprilTag (decision margin and detector settings), 3D and multi-tag, Resolution, or Object detection.
• All cameras at once: it can also copy into the same pipeline number on every other camera.
• Never copied: orientation, names and each camera's exposure limits. Resolution is only copied between cameras with the same video modes.

Profiles (practice field, event field): give every camera the same pipelines at the same numbers, e.g. 0 = Event and 1 = Practice field.

• Making one (photonvision-46): in the new-pipeline dialog (☰ → +), Start from a copy of any camera's pipeline, and tick Create on every camera. Each camera gets it as its next number, and the message says which number each got; cameras that already have the same number of pipelines get the same number.
  • A copy from this camera is exact, like Duplicate.
  • A copy from another camera keeps each camera's own orientation, exposure limits and mask, and its resolution unless the two cameras offer the same video modes.
  • Only the camera on the dashboard switches to the new pipeline.
• Switching from the dashboard: ☰ → Switch every camera to pipeline N (the ⇄ icon) moves every camera to the number you're on. A camera without that number stays where it is, as do cameras in driver mode or calibrating; the message lists them, and warns when a camera's pipeline at that number has a different name.
• Switching automatically at events (photonvision-51): in Settings > Robot state, turn on Event pipeline when the field connects and pick the number. When the field (FMS) connects, every camera switches to it, once. After that anything sticks: a switch on the dashboard, robot code choosing a pipeline, or turning the setting off. The chooser lists what each camera has at each number, the report names any camera whose pipeline there is something else, and Switch now tries it without a field. health-check.sh prints the event pipeline and warns if a camera lacks it or has a different pipeline there.
• Numbers everywhere (photonvision-47): the Pipeline dropdown and the Camera Matching page show each pipeline as "N: name", so a mismatched profile is easy to spot.
• Robot code switches them all together too (the toggle is in issue #10), and the robot log records which one was active.
• Copy settings fills in the other cameras after tuning one.

More camera controls (photonvision-28): the Input tab now has Contrast, Gamma, Sharpness and Backlight Compensation, which stock PhotonVision doesn't show.

• Only the ones the camera has, with its own ranges; each tooltip gives the camera's default.
• Saved per pipeline. A pipeline that never set one uses the camera's default, re-applied on every switch.
• Leave them at the defaults unless you're testing. Sharpness is the one worth trying lower, since the camera's sharpening can put halos on tag edges.
• Copy settings includes them in the Camera group.

Leaving bad tags out of multi-tag (photonvision-22): if a tag is mounted wrong at an event, list it, and every camera's multi-tag solve ignores it.

• You list the bad tags, not the good ones.
• Where: type them on the Settings page's AprilTag Field Layout card (e.g. 7, 12, saved on the Jetson). Robot code can add more on /photonvision/excludedTags.
• Left-out tags are still reported as targets, with single-tag poses, and are dimmed in the tag table.

Measuring a camera's mount from the tags (photonvision-17):

• Where: with the robot level on the floor and 2+ tags in view, the Targets tab's Camera mount estimate shows the camera's height, pitch and roll on the robot, averaged over the last 100 samples with a ± spread.
• Why it works: while the robot is level, the camera's height, pitch and roll on the field are its mount's, wherever the robot is.
• Using it: the numbers use robotToCamera's axes and signs (negative pitch means tilted up), so they go straight into robot code.
• Yaw and X/Y also need the robot's pose. Robot code gets everything from /photonvision/<camera>/mount on NetworkTables, and at events it can compare the estimate with its configured mount to catch a bumped camera.
• Accuracy: it's only as good as the field's tag layout and a flat floor.

Calibration board settings (ChArUco, 5x5 markers, 30 mm squares, 22 mm markers):

Field	Value
Resolution	1280x800
Board Type	ChArUco
Tag Family	Dict_5X5_1000
Pattern Spacing (in)	1.181 (30 mm; this version uses inches)
Marker Size (in)	0.866 (22 mm)
Board Width (squares)	12
Board Height (squares)	9
Old OpenCV Pattern	off

The board is labeled "9x12", but PhotonVision needs width 12, height 9. With 9x12, every calibration failed with "Negative corner in reprojection error calc". We found the right setting with tests/charuco-board-check/check_board.py: it tried all four combinations on a live frame, and only 12x9 found corners (78 of 88).

Calibration tips:

• The board settings start at our board: ChArUco, 5x5 markers, 30 mm squares, 22 mm markers, width 12, height 9. Sizes are in millimetres (photonvision-23).
• After clicking Start, set Auto Exposure off and Exposure about 150. Calibration uses its own settings, and its default was nearly black. A long exposure is fine here because the board is held still; don't copy it into the vision pipelines.
• Use the full-screen view and cover the edges and corners: see Calibrating well for targets. Partial boards at the edge are good, not bad.
• Aim for a mean reprojection error under about 0.5 px (under 1 px is fine for FRC).
• If a calibration fails and gets stuck, restart PhotonVision (Settings → Restart Software) to clear the bad snapshots.
• A camera that cuts out only while the board is in view: its frames are too big for the USB bandwidth each camera gets (11-uvcvideo-payload-cap.sh: ~85 KB a frame at 120 fps, so four cameras fit). A board filling the view is 85-105 KB a frame; an ordinary scene is 45-67 KB. Calibration now runs at the slowest frame rate the chosen resolution offers (photonvision-70, ~340 KB a frame at 30 fps), and the camera goes back to its pipeline's mode afterwards. On an older build, the long exposure above does the same: it lowers the frame rate. Reported by another team, 2026-10-01.
• Board never found, or only some corners: tests/charuco-board-check/check_board.py tries both orientations and OpenCV's old and new marker layouts on a live frame (they differ for boards with an even number of rows), and says which matches: for example --squares 8 8 --square-mm 25.4 --marker-mm 19.05 --dict DICT_4X4_1000. Set the same on the calibration card (the old layout is its "Old OpenCV Pattern" switch).

Bench calibration results (checked with tests/calibration-check/check_calibration.py, which reads the saved calibration and reports error, outliers and coverage):

Camera	Snapshots	Corners kept	Mean error	fx	cx, cy
TopLeft (port 2.1)	43	97%	0.87 px	737.8	650.5, 362.4
TopRight (port 2.3)	41	96%	0.97 px	737.0	597.9, 371.6

(2026 bench results.) We couldn't get the error below 0.5 px handheld. The focal length came out at about 737 px in all three of camera 1's calibrations. We later found these calibrations never covered the image edges, and the "42% outliers" of our second try were good edge corners, not bad snapshots: see Calibrating well.

Camera firmware (Thriftiest Cam and Swyft)

Both camera makes can be flashed from PhotonVision's Settings page, with the cameras still plugged into the Jetson. A flash is refused while the robot is enabled.

	Thriftiest Cam firmware card	Swyft camera firmware card
Shows	each Thriftiest Cam once: its port, PhotonVision name, serial and firmware version, and a warning for firmware that won't recover from a USB drop	each Swyft on USB, by port and PhotonVision name, and the stored firmware images
Does	flashes firmware from the browser (we run the unofficial "usbrecov" 5.13, which comes back after a USB hub reset)	uploads, checks and flashes .rfw images (we run Swyft's low-exposure firmware, 0.2 ms), with an optional new serial
Serial	stored once from the laptop: scripts/host/thriftiest-flash.sh rename 2.1 SJ-TC01	set on the card while flashing (SJ-SW01)
Backs up	—	into swyft-firmware/backups on the settings partition
After	the camera comes back by itself	power-cycle the robot: a Swyft's video only comes back after its power is cut (a Jetson reboot isn't enough)
Command line	scripts/jetson/thriftiest-fw.py, scripts/host/thriftiest-flash.sh	scripts/jetson/swyft-fw.py
Details	docs/THRIFTIEST-CAM-FIRMWARE.md	docs/SWYFT-CAMERA-FIRMWARE.md

Why a serial matters: every Thriftiest Cam ships with the same serial (01.00.00) and every Swyft with 200901010001, so PhotonVision can only tell them apart by USB port. With a serial of its own, a camera keeps its name and calibration in any port (see below).

Calibrating well

The short version: cover the whole image, especially the edges and the four corners; keep partial boards; tilt the board; vary the distance. Our old calibrations were fine in the middle and weak at the edges, and it showed on the field.

What we found (tests/calibration-analysis/RESULTS.md, 2026-10-31). We checked our two robot cameras' calibrations against the 6 recorded TXMAN1 matches:

• Inside the area the snapshots covered, the calibrations fit to about 0.1 px. But no snapshot reached the outer ring of the image: the image-corner areas had 3–54 board corners each, where they need 150+.
• Just past the data, the 8-coefficient lens model folds over (its curve turns back), so a tag near the edge is projected to the wrong place. On the field, the error grew with distance from the centre exactly where coverage ended.
• Refitting the same snapshots with every corner kept and a model that doesn't fold cut the match error from median 1.21 → 0.97 px and p90 8.2 → 4.9 px. Those refits are in calibrations/*-refit.json, ready to import.
• The old advice "our second try had 42% outliers, so keep the board inside the image" was wrong: those "outliers" were good edge corners the centre-fitted model couldn't explain.

The full-screen calibration view (fork, photonvision-91; open it from the calibration card while calibrating). Built from that study, not yet tried on a real camera:

• The image at full size, live corners (green when still), and a coverage grid (10x8 cells, with the outer ring and the four corner blocks outlined) so you can see what still needs corners.
• "Move the board here" guidance: a ghost board where the next snapshot should go, plus quotas for far (≥1.2 m) and tilted (20–45°) shots.
• Quality readout after each solve: error, outliers and where they are, coverage, fx with its spread, and a red banner if the lens model folds. Save only works when the checks pass (or "save anyway" is ticked). Snapshots can be listed and deleted, and the result exported as JSON.
• Partial boards count: a ChArUco board hanging off the edge is kept when it shows 6+ corners across 2+ rows and columns. That's the only way to get corners into the extreme corners.
• Auto-capture: when the board is held still and adds new coverage (or fills a quota). Near-copies of earlier snapshots are skipped; Space takes one by hand.
• 8 lens coefficients are kept from the solve.
• An "edge-corner refit" option exists but is off until it's checked on the bench.

How to calibrate well:

Do	Target
Number of snapshots	about 60–100
Where	~40% centre and middle, ~60% aimed at the edges and corners. Every part of the image gets corners; get corners within ~10 px of the border
Tilt	20–45° about both axes in at least 30% of snapshots
Distance	0.4–1.5 m (far shots fill the corners with small, dense corners)
Exposure	Auto Exposure off, Exposure ~150 (the board is still, so blur isn't a problem; don't copy it into the vision pipelines)
Accept when	inlier error ≤0.25 px rms, no empty area, and the model doesn't fold

Keep the board flat and rigid (ours is a printed commercial panel). After any recalibration, check a tag at a measured 2.00 m reads 2.00 ±0.01 m.

Rewind: recording what the cameras saw

Our replacement for Limelight Rewind. While robot code sets /photonvision/rewind/record to true in NetworkTables, PhotonVision saves every camera's video to the Jetson's SSD at 30 fps. For real matches that means while enabled with the FMS attached; for a specific test, just around the test. Afterwards, scripts/host/rewind-pull.sh copies a recording to the laptop and turns it into videos. Each frame is stamped with the robot's clock, so the video lines up with the AdvantageKit log.

• It saves the camera's own JPEG frames. Nothing is decoded or re-encoded, so it costs 3% of one CPU core, and detection fps and latency don't change (measured).
• About 1 GB per match with 4 cameras. The oldest recordings are deleted past 100 GB.
• For bench tests there's a Record now switch in Settings → Rewind.
• To download a recording, click the download button next to it in Settings → Rewind. You get a zip with one video (.avi) per camera; VLC or Ubuntu's Videos app plays it.

Everything else, including the robot-code example and how to line video up with a log, is in docs/REWIND.md.

Field calibration: the event field's real tags, and every camera's mount

A page in PhotonVision's web UI (Field Calibration in the sidebar, photonvision-30). With the robot on and disabled, pushed by hand:

1. Tune camera settings (optional, about a minute per camera).
   • Each camera has its own card: its live stream, the tags it sees right now with their margins, a slider for every setting, and its own Tune. Turn the robot so that camera faces tags, near and far, and press Tune.
   • The Jetson steps it through exposure, gain, brightness, contrast, gamma and sharpness under the event's lights. It recommends the shortest exposure that still finds every tag (less motion blur), and changes the others only if they clearly help.
   • Set any setting by hand with its slider: it saves to the pipeline in use, like the Camera page. Tick a setting's lock to keep it while the rest are tuned.
   • Apply recommended saves that camera's results. Each camera keeps its results until it's tuned again, so the robot can be turned between cameras.
2. Record the spots. Push the robot to 10–20 spots and hold it still at each until the page says the spot counts (about 2 s, and it beeps). A field map shows which tags need more views, where each camera is, and a suggested next spot. It can also be shown in 3D, using FIRST's own field CAD. Each element is placed by its tags, so it matches the AndyMark or welded layout in use, and after a calibration it shows where things really are. The mouse works like Onshape's: right-drag turns, middle-drag or Ctrl + right-drag moves, the wheel zooms.
3. Solve. The Jetson replays the recording through the same GPU detector it uses in matches, then solves every tag's real pose and every camera's mount. It takes about a minute.
4. Results:
   • Tags: each tag marked as drawn or moved, with its offset.
   • Cameras: each mount compared with robot code's, plus a Java snippet.
   • Use this layout in PhotonVision: loads the corrected layout; Undo puts the old one back. A robot that sends its own field map replaces it as soon as it connects, so for matches put the corrected layout in robot code instead.

Robot code has to publish each camera's robotToCamera (a Transform3d struct) at /photonvision/<camera>/robotToCamera, from CAD or the last calibration. Without it, height, pitch and roll are still found, but x, y and yaw are only relative to one camera. The solved mounts come back at /photonvision/<camera>/fieldcal/robotToCamera.

How it works, the tests and the command-line version are in tools/fieldcal. The plan and the shop-test checklist are in docs/FIELD-CALIBRATION-PLAN.md.

Field replay: what 6 real matches say

We recorded 6 matches at TXMAN1 with Rewind (team 8515, 54,000 frames from two cameras, plus each match's robot log), then replayed them through the GPU detector on the Jetson with different settings (detector-replay, tests/field-replay/run.sh). Each tag sighting is checked against the other tags in the same frame (its "in-frame" error: median 1.3 px). Full results: tests/field-replay/RESULTS.md and tests/field-replay/accuracy/RESULTS.md.

Detector settings:

Setting	Verdict
max_line_fit_mse 10 (--mse)	Keep 10. 2.5 drops only 0.28% of sightings; 15–20 add almost nothing. It never moves a corner, only accepts or rejects.
min_white_black_diff 20 (--mwbd)	Keep 20. Doesn't change accuracy, and it's 30–40% faster than 5.
min_cluster_pixels 24	Keep 24.
Resolution 1280x800	Keep. 640x400 loses 42% of sightings and range (farthest tag 7.2 m vs 9.2 m).
Far search	Worth it: +4.5% sightings, and its finds are the most accurate (0.95 px median), for ~1.7 ms a frame. But on JetPack 7 each full-size search takes 5.9 ms (2.7 on JetPack 6), which made 1 frame in 10 late, so it's off by default until it's faster.
Edge filter (--edge, AOS-style)	Don't. At 10–50 px it removes 5–11% of sightings, 85–90% of them good.
Extra corner refinement (cornerSubPix, edge-line fit)	Don't. Both make corners worse than the detector's own edge refinement, which is worth 0.73 px and must stay on.

Where the error really is: a corner jitters only ~0.08 px when still, but the in-frame error is 1.3 px median and 9 px p90. That's systematic: the lens calibration at the image edges (see Calibrating well) and the field layout, not detector settings.

Robot side: no single filter separates bad detections from good ones. The 1.6 rad/s turn-rate gate our 2026 code used throws away more good sightings than bad (detections don't get worse when turning; the robot's pose does). Raise it to 3.5 rad/s or drop it. Frames also looked ~5–9 ms early against odometry: a bench spin test should confirm that before anyone compensates for it.

Camera mounts: solving each camera's mount from the matches showed the left camera's yaw ~3–6° off from robot code, and the right camera re-aimed between days (pitch −19° → −25°). Nothing noticed at the event, which is why we built the mount check.

Mount check (robot-vision)

A bumped or mis-measured camera moves every pose from that camera, while each detection still looks fine, so no filter catches it. MountCheck in SpectrumVision (robot-vision/, on by default) does:

• For each frame, it works out where the camera must be from the tags alone, and so where it must be mounted on the robot.
• Height, roll and pitch are checked directly (the robot is flat on the floor). x, y and yaw are checked against the other cameras (with 3+, the odd one out is the bad one) or the pose estimator.
• It only samples while the robot is slow, and uses the median of the last 400 samples. SUSPECT past 2° or 4 cm, BAD past twice that.
• It raises an alert (Elastic, AdvantageScope), publishes /SpectrumVision/mountCheck/…, and vision.health().mountCheck() is a pre-match check: disabled, robot still, each camera seeing tags for ~2 s.
• Replaying TXMAN1, it flagged 8515's left camera in 6 of 6 matches and the right in 5 of 6.

Details: robot-vision/README.md. To measure a mount precisely, use Field calibration.

Game-piece detection (TensorRT)

JetPack 7 (2027), measured 2026-11-01. Full guide: docs/OBJECT-DETECTION.md.

PhotonVision runs YOLO models (YOLOv8 / YOLO11, exported to ONNX) on the Jetson's GPU through TensorRT. That's our backend: photonvision-100 in the fork plus libspectrumtrt.so, built from detector/trt_yolo.cu. Robot code gets normal PhotonLib targets, with class and confidence. It's all off until you pick one of the pipelines below.

• Two pipeline types:

  • Object Detection: the camera only looks for game pieces. "Object FPS max" (30 by default) caps how often the model runs.
  • AprilTag + Object: one camera does both. Every frame still goes to the AprilTag detector, and the tags are published exactly as before, with no extra delay. Up to "Object FPS max" frames a second (15 by default) also go to the model. The game pieces show up as a second camera called <camera>-objects. In robot code: new PhotonCamera("front-objects"). Each result carries the time its own frame was taken.

• Tag FPS max (AprilTag tab): process fewer frames on a camera without changing its mode, e.g. 122 on the front and 60 on the sides. This saves the most GPU of any setting: 60 fps halved the GPU use (18% to 9% for 2 cameras). The cost is latency: the median went from 14.5 to 19 ms, because the frame we pick has usually waited a few ms already. Idle mode's lower rate still applies while the robot is disabled.

• Install a model: upload the .onnx in Settings > Object Detection, or run scripts/jetson/12-install-yolo-model.sh model.onnx "Name" Label1,Label2 640 384 on the Jetson. The first time a model is used, the Jetson builds a TensorRT "engine" from it. That takes about 5-13 minutes, so do it in the pit. After that it's cached and loads in under a second.

• One engine is shared by every camera. Frames that arrive within 2 ms of each other run together as a batch, which costs less per frame. The model also runs at a lower GPU priority than the AprilTag detector.

• Speed (YOLO11n, FP16, GPU at full clock, nothing else running):

  Input	1 frame	2 frames	4 frames
  640x640	5.1 ms	8.3 ms (4.2 each)	15.1 ms (3.8 each)
  640x384 (fits our 16:10 cameras)	3.5 ms	5.3 ms (2.6 each)	8.8 ms (2.2 each)

• With 2 cameras at 122 fps doing AprilTag + Object (YOLO11n 640x640):

  Objects per camera	Tag latency median / p95	GPU	Board power	Hottest
  none (tags only)	14.5 / 15.5 ms	18%	9.0 W	71 °C
  15 fps	14.7 / 16.0 ms	29%	10.5 W	73 °C
  30 fps	14.8 / 16.1 ms	41%	12.1 W	77 °C
  every frame (~67 fps)	15.8 / 17.1 ms	69%	15.6 W	84 °C

  At 15-30 objects a second per camera, the tags hardly notice (+0.2-0.3 ms). Don't run "every frame": it adds 1.3 ms to the tags and runs hot.

• Train your own model on our cameras' frames (from Rewind recordings), with about half the training images turned gray so one model works on both mono and colour cameras: docs/OBJECT-DETECTION.md. The 2026 FUEL models are in docs/GAME-PIECE-MODELS.md. Their 2026 engines don't load on JetPack 7, so re-install them from the ONNX.

• When the GPU runs short: first lower Tag FPS max on the side cameras, then try the GPU offload ladder (08-select-detector.sh gpu --gpu-level list). Last, switch on Tags on CPU (NAPA) in the AprilTag + Object pipeline: the tags move to the CPU, and the GPU is left to object detection (63% to 42% GPU with 2 cameras and yolo11n at 30 fps, the same tag latency). TECHNICAL.md explains all three.

• NAPA (by Zachary Wilson) is a fast CPU AprilTag detector. Use Detector: NAPA in the AprilTag pipeline on the SystemCore or other CPU-only computers. It finds more tags than AprilTag 3 for 1/7 of the CPU. Install libspectrum-napa-apriltag.so with scripts/jetson/07-build-napa-detector.sh. TECHNICAL.md.

Known issues

A Thriftiest Cam can get stuck after a USB hub reset, and only cutting its power fixes it (found 2026-09-25 with tests/usb-hub-reset/run.sh).

• What happens: when the Jetson's USB-A hub resets while the cameras stay powered, both Thriftiest Cams stop answering. We reset it in software; a static shock can do the same. The kernel logs "device descriptor read/64, error -110", then "unable to enumerate USB device".
• The Global Shutter cameras come back by themselves, in about 40 s: each stuck camera holds up the hub for ~20 s of retries first.
• What doesn't fix it:
  • switching the port's power off in software (the hub says it can, but the carrier board's USB 5 V is always on);
  • resetting the USB controller;
  • rebooting the Jetson: a reboot doesn't cut USB power either.
• What does: cutting the camera's power. Replug it, or power-cycle the robot.
• How you'll know: the camera's health/problem topic and the log say "not answering on USB port 1-2.1: replug it, or power-cycle the robot" (photonvision-33). The health check and usb-bandwidth.py name the port.
• 2026-10-29, firmware 5.13 doesn't fix it. Both robot cameras (orin-left on 1-2.1, orin-right on 1-2.3, bcdDevice 0513, the published 5.13 with gain, docs/THRIFTIEST-CAM-FIRMWARE.md) were streaming at 121 fps and the health check was READY. tests/usb-hub-reset/run.sh (hub off 2 s) and run.sh 1-2 10 (off 10 s) gave the same result: neither camera came back by itself. Each failed every descriptor read (device descriptor read/64 then read/8, error -110) for ~20 s, including the kernel's own port power cycle ("attempt power cycle", which doesn't cut the power), then "unable to enumerate USB device". 1-2.1 was retried first, then 1-2.3, so the hub was busy for ~40 s. Neither was in lsusb afterwards, and both stayed gone until they were replugged: more than a minute the first time, 2 min 20 s the second. After the replug both enumerated on the same ports with 5.13 and gain, and PhotonVision reopened them at 1280x800 and 120 fps within a second. The health check was READY again.
• 2026-10-29, the unofficial "usbrecov" 5.13 does come back. With orin-right (1-2.3) on the coach's patched 5.13 (two branches NOP'd, docs/THRIFTIEST-CAM-FIRMWARE.md) and orin-left (1-2.1) on the published 5.13 as the control, run.sh and run.sh 1-2 10: orin-right was streaming again by itself at 22.8 s and 21.5 s, and orin-left stayed stuck both times until replugged. Orin-right enumerated on its first try with no -110 errors; the ~20 s was the hub retrying the stuck orin-left first, Both robot cameras now run it, and with both patched the hub reset passes: both streaming again 2.2-2.5 s after the hub comes back. Not from Thrifty Bot: watch it on the robot.
• 2026-10-29, PhotonVision recovers a dropped port in about 3 s (photonvision-84, TECHNICAL.md). With the usbrecov firmware, when a camera's port drops (disabled at the hub, as Q13's EMI, or the camera gone from USB), PhotonVision switches that port's power off and on and reopens the camera: streaming again in 2.1-3.5 s, with the other cameras untouched. Before, nothing brought it back. Measured with tests/usb-port-glitch and tests/usb-recovery-bench; a real EMI or cable-shake drop and a driving robot are still to test.
• Our decision: keep the Jetson's own USB ports. We won't add a hub that can really switch its ports' power. Hot glue and strain relief make loose plugs, a likely cause, less likely. If a camera is out, power-cycle the robot between matches.

A software reboot hung twice (2026-09-25).

• What happened: after sudo reboot the Jetson reset (the fan dropped from full speed) but never finished booting. A power cycle fixed it.
  • The second time, both Thriftiest Cams were stuck (see above). We don't know about the first.
  • The log from the last 2 minutes before that reboot is missing.
• Suspected cause, not confirmed: the boot firmware waiting on the stuck cameras, since a reboot doesn't cut USB power.
• What's covered now:
  • A shutdown that hangs resets itself after 30 s: the hardware watchdog while rebooting, which was 10 minutes.
  • Nothing reboots automatically: the USB watchdog only resets the controller.
• Not covered: a hang in the boot firmware, before Linux starts the watchdog.
• What to do: if cameras are stuck, power-cycle instead of rebooting.
• To find the cause: with a DisplayPort monitor plugged in, run tests/usb-hub-reset/run.sh, then sudo reboot, and see where the boot stops.

PhotonVision may never connect to NT if it starts before robot code (reported by 6324, Oct 2026; not seen on our robot yet).

• What happens: PhotonVision comes up before the robot's NT server and never starts talking over NT. Restarting robot code fixes it.
• Likely cause: upstream #2398 (in patch 00) removed the loop that restarted the NT client every 5 s while it was disconnected. That loop was written for exactly this case ("if Photon starts before the robot NT server does, and both aren't static IP, it'll never connect"). Upstream removed it because the restarts seemed to break connections NT4 was still setting up. See docs/UPSTREAM-PORT.md.
• Not known yet: whether the client was stuck disconnected or showed connected with no data flowing, and whether the NT server address (team number or IP) or a static Jetson IP matters.
• Our decision: no patch for now. A watchdog that restarts the NT client after ~15 s disconnected, waiting longer between tries up to 60 s, is the likely fix if we see it. It has trade-offs (restarts can break a connection about to succeed, and every restart republishes topics), so we'll only add it with evidence.
• If you see it: save the PhotonVision log from that boot (the connection listener logs connects and disconnects), note the NT server setting and whether the Jetson's IP was static, and add it here. Restarting robot code, or pressing Save on PhotonVision's Settings page, gets it connected.

Troubleshooting quick reference

Symptom	Likely cause	What to do
lsusb shows no NVIDIA device	Not in recovery mode, or a charge-only USB-C cable	Redo the FC REC–GND jumper with power off. Try a USB-C cable you know carries data.
Flash hangs at "Waiting for target to boot-up" for minutes	NetworkManager or the firewall is interfering	Use 02-flash-nvme.sh, which handles both.
Camera doesn't show up (lsusb, no /dev/video*)	Loose cable, or plugged straight into the USB-C port	Reseat or swap the cable. Cameras work on USB-C through a hub (it shares the USB-A ports' bandwidth).
Kernel log: "new low-speed USB device … error -71 … unable to enumerate USB device"	The camera's data wires aren't connecting: cable, adapter, or a plug not fully in (seen with two cameras at once on 2026-09-24)	A camera that should be high-speed showing up as low-speed is a data-line problem. Reseat it, or swap the cable. scripts/jetson/usb-bandwidth.py and the health check name the port.
A camera is gone after a USB glitch, and its health says "not answering on USB port …" (kernel: "device descriptor read/64, error -110")	The camera is stuck: a USB hub reset leaves Thriftiest Cams like this (see Known issues)	Replug it, or power-cycle the robot. A Jetson reboot doesn't help: it doesn't cut USB power.
The Jetson doesn't come back after a reboot, and the fan never returns to full speed	The boot stopped before Linux started (seen twice; see Known issues)	Power-cycle it. If cameras were stuck, power-cycle instead of rebooting.
A 3rd, 4th or 5th camera won't start streaming ("No space left on device", "Not enough bandwidth" in the kernel log)	USB 2.0 bandwidth: a camera reserves bandwidth for its alternate setting, and every USB 2.0 port shares one budget (moving to USB-C doesn't help)	On the Camera Matching page, lower another camera's allocation in the USB bandwidth card. Or run scripts/jetson/usb-bandwidth.py: it prints the cap to install, e.g. CAP=1bcf:28c5:1280,32e4:0144:1280,32e4:62f0:1600 scripts/jetson/11-uvcvideo-payload-cap.sh --install. A camera it can't cap (the Razer Kiyo) takes half the budget by itself.
Low FPS (~34)	Exposure too long (the units are 100 µs; the slider shows ms)	Exposure 50–83 (5–8.3 ms).
Tags flicker in and out	Decision margin near the cutoff (dim light, or flickering light)	Run tests/flicker-check/run.sh. If frames pulse, use exposure 83; otherwise lower the cutoff a little. Retune on the field.
One camera shows no detections, and the log fills with "invalid JPEG image received"	The camera got stuck sending corrupt frames (seen once after rapid restarts)	PhotonVision now recovers it by itself (photonvision-29): it reconnects the camera after 3 s without usable frames, then resets it at the USB level, like a replug, 5 s later. Look for "no usable frames" in the log. If it keeps happening, replug that camera or restart PhotonVision.
Image nearly black during calibration	Calibration uses its own exposure settings	In the calibration card: Auto Exposure off, Exposure ~150.
Calibration fails ("Negative corner", null intrinsics)	Board width/height swapped	Width 12, height 9. Check with check_board.py. Restart PhotonVision to clear bad snapshots.
Settings page missing Device Control / Restart	Old browser (no Intl.DurationFormat)	Update the browser. Fixed in our patch too.
A camera's stream won't show	The browser's connection limit (each open stream holds one)	Close extra PhotonVision tabs, then Ctrl+Shift+R.
PhotonVision never shows up on NT after a boot; restarting robot code fixes it	PhotonVision started before robot code and its NT client got stuck (see Known issues)	Restart robot code, or press Save on PhotonVision's Settings page. Save the PhotonVision log first.
PhotonVision won't start ("corrupt jarfile")	A bad jar was installed	Reinstall a good jar with 06-install-fork-jar.sh, or copy back photonvision.jar.prev.
Robot code throws about a PhotonLib version or message mismatch	Robot PhotonLib and the Jetson's PhotonVision differ	Build robot code's PhotonLib from the same fork commit the Jetson runs (WPILib 2027 alpha-8).
Gradle build fails with a PKIX/SSL error	A network filter is blocking frcmaven.wpi.edu (it works from the shop as of 2026-10-30)	Check with curl -I https://frcmaven.wpi.edu; use another network if it's blocked.

How the tags look (photonvision-54): whenever a camera sees a tag big enough (24 px or more), PhotonVision measures the tag's white margin and black border, a few times a second. The Input tab shows it above the camera controls ("white 205, black 38 (contrast 167), 0% clipped: good"), with what to change when it isn't good: whites clipping (shorten exposure, or lower brightness or gamma), low contrast (lengthen exposure, watching motion blur), or blacks lifted (brightness too high). Match Ready's camera tiles and health-check.sh show it too. It explains what the Field Calibration tuner's scores can't. The Camera Gain slider is gone for cameras that have no gain control: the Thriftiest Cam is listed upstream as having gain, but ours has none, so the slider did nothing.

Settings snapshots (photonvision-53, Settings page): save every setting under a name ("Before Houston", "tuned exposure"): each camera's pipelines, settings and calibrations, plus Robot state and Rewind. They're kept on the Jetson (about 2 MB each). Restore brings every camera back to that snapshot and restarts PhotonVision; it saves the current settings first as "Before restoring …", so a restore can be undone. Download one to keep it off the Jetson. PhotonVision also saves one automatically the first time the field connects each day. Logs and calibration images are never touched.

Match Ready page (photonvision-52, in the sidebar): one screen before a match. A big READY / NOT READY from the same checks as health-check.sh (run by PhotonVision every 30 s while the page is open, or with Check again), anything that needs attention listed first, and a live tile per camera: pipeline number and name, fps, latency, targets in view, and whether it's calibrated at its resolution. Every check is underneath, by section.

Checking on it: run scripts/jetson/health-check.sh for a readiness report. journalctl -u photonvision -f shows PhotonVision's live log on the Jetson. The detector stats lines show frames per second, detection time and decision margin for each camera. The jpeg decoder lines (every 10 s) show which JPEG decoder is running and how its checks went. (Before 2026-10-31 these were 971 stats and 971 jpeg.)

Far-tag search (detector/far_search.cc, switch in photonvision-55; off by default on JetPack 7 until it's faster, see Field replay): the GPU detector normally looks for tags on a half-size image, so it misses tags under about 18–20 px (far away). When no camera has had a good view for 0.25 s (a good view = at least 2 near tags, 40 px or more with a decision margin of 30), the cameras start also searching their full-size images, one camera at a time and at most 30 a second across all of them. Far tags they find are then followed with small full-size crops. As soon as any camera has a good view again, it all stops, so while the pose is good it costs nothing.

• Range: in the bench test it finds tags down to 10 px instead of 18 px, 1.8× the distance (about 7 m → 12 m for a 6.5 in tag face-on, by the pinhole rule with fx 737; blur makes real ranges a bit shorter).
• Cost while searching (5 cameras, no tags in view, searching the whole time): GPU 16.5% → 19.4%, 9.7 → 10.1 W, average latency +0.2 ms. All cameras stay at 122 fps. Each full-size search is ~2.7 ms.
• Switch it off in Settings > Robot state (Far-tag search when the pose is weak), which also shows whether it's searching and what it has found. health-check.sh says whether it's on.
• Replay a recording with it: tests/far-search/replay.sh SESSION_DIR (on the Jetson) replays every camera of a Rewind session together, far search on and off, and reports the tags only the far search found. Replaying the bench recordings found a detector bug that could freeze a camera during a full-size search of a busy scene (fixed in detector-08).
• Tests: tests/far-search/run.sh on the Jetson renders real tags into frames and checks the range, that its corners match the normal search's (within 0.02 px), the policy, tracking a moving far tag (92 of 92 frames) and the budget. tests/ui checks the switch.

Idle while disabled (photonvision-49): while the Jetson is connected to a robot that's disabled, each camera processes 30 frames a second instead of 122. It's back to full rate within 5 ms of enable. With 5 cameras: 7.8 W disabled against 9.7 W enabled, less heat in the queue and between matches, and 30 results a second is still plenty to set the robot's starting pose. Off the robot (no NetworkTables connection) nothing changes, so tuning on the bench sees every frame. When the robot disables, PhotonVision also runs a garbage collection (27 ms), so the memory is cleaned up before the next match rather than during it. To test at full speed with the robot attached (photonvision-50): press Full speed on the dashboard's "Robot disabled: cameras idle at 30 fps" notice, or switch off Idle while the robot is disabled in Settings > Robot state. The same card sets the idle rate and shows what PhotonVision sees of the robot (connected, enabled, auto, FMS). It's saved across restarts, so health-check.sh warns while it's off.

Steadier timestamps: frames are timestamped by each camera's own clock (the USB video driver's hwtimestamps option) instead of when their first USB packet happens to be processed. The timestamps the robot receives now vary by 0.01 ms instead of 0.95 ms (up to 4 ms before), which matters when the robot is turning: 1 ms is 0.36° at 360°/s. 11-uvcvideo-payload-cap.sh --install sets it, and health-check.sh checks it.

1.2 ms lower latency from the camera driver: our USB video driver now looks at incoming packets every 2 ms instead of every 4 ms (urb_packets=16), so each finished frame reaches PhotonVision sooner: 12.98 → 11.76 ms from capture to result with 5 cameras, for about 0.1 core of CPU. Set by 11-uvcvideo-payload-cap.sh --install, checked by health-check.sh.

Hidden dashboard tabs stop their streams (photonvision-48): a dashboard tab that's behind another tab, or in a minimised window, disconnects its camera streams, and the Jetson stops encoding them. They reconnect when you look at the tab again. This also fixes streams that broke (http://undefined:…) when a dashboard reconnected to PhotonVision after a restart.

After installing a new build: run tests/ui/run.sh on the laptop. It drives the dashboard in Chrome and checks that every slider, switch and menu changes the page, PhotonVision and a second dashboard, and puts each one back. It also tests the mask editor, Start from, Create on every camera, Switch all and hidden-tab streams. It takes about a minute and a half and works on a temporary copy of a pipeline, so your settings aren't touched. See tests/ui.

Detector regression check (tests/regression/run.sh, on the Jetson): replays a fixed corpus of recordings through the detector as built now, far-tag search on as live, and compares every detection with the last accepted output (tests/regression/golden/). It runs before every prebuilt bundle (make-prebuilt-bundle.sh refuses on a change; SKIP_REGRESSION=1 to override).

• What it catches: any detection lost, gained or moved by more than 0.001 px, in any change to the detector: bos, our detector-* patches, the JNI build, far search, or new detector settings. Replays are bit-for-bit repeatable.
• The corpus (tests/regression/corpus.json, checksummed, so a damaged recording reports as that and not as a detector change):
  • fieldcal-synth: a rendered 4-camera drive with a real lens model (2,628 frames, 3,314 detections);
  • synthetic-near-far: near tags and a 14 px far tag (361 detections);
  • ceiling-4cam: four real cameras looking at the shop lights, no tags (false positives: 0).
  • Real tag footage and match recordings go in as we record them: run.sh --add NAME /opt/photonvision/rewind/sessions/SESSION --about "...".
• Where it lives: on the Jetson at /opt/photonvision/rewind/regression (scratch partition, beside Rewind's sessions; Rewind's quota never touches it), with a copy on the laptop at ~/spectrum-regression-corpus. --add holds quiet mode off while it writes.
• A change you meant (a detector improvement): read the report, then run.sh --bless, copy corpus.json and golden/ back to the laptop and commit them.
• Time: about 25 s for the whole corpus.
• Not covered: PhotonVision's Java side (poses, multi-tag, tag quality); the fork's tests and tests/tag-quality cover that.
• Checked by: a run with min white-black difference 5 instead of 20 reported 24 detections lost, 16 new and 3 moved in fieldcal-synth, and 0 false positives on the ceiling.

Robot code sees the same health on NetworkTables (photonvision-16), once a second:

• /photonvision/jetson/: GPU load, temperatures, fan speed, power, the JPEG decoder's state, and a throttle reason (OVER-CURRENT when the supply sags, HIGH TEMP, or clocks capped, photonvision-20). It's also on the Settings page as CPU Throttling.
• /photonvision/<camera>/health/: fps, pipeline time, latency, and failed JPEG decodes.
• The topics and suggested alerts are in docs/TECHNICAL.md and issue #10.
• tests/jetson-telemetry/run.sh prints them on the bench.
• The Settings page's Device Metrics also has a GPU Usage chart (photonvision-19).

How much to trust each tag (photonvision-61): every frame with tags also goes out on /photonvision/<camera>/tagQuality, a double array, just before the result it belongs to. PhotonLib's own message can't carry these: the robot checks its format.

• Layout: [sequenceID, tagCount, then per tag: fiducialId, decisionMargin, edgePx, undistortPx, reprojBestPx, reprojAltPx], in the result's target order. Match it to a PhotonLib result by result.metadata.sequenceID.
• edgePx: how close the tag's nearest corner is to the image edge. Distortion is strongest and the calibration weakest there, and the tag may be cut off. Trust falls off as this shrinks (Northstar and AOS use 25–50 px).
• undistortPx: how far the lens model moved the corner that moved most. Large means the pose relies heavily on the calibration being right out there.
• reprojBestPx, reprojAltPx: RMS pixel error of the tag's corners under its best and alternate single-tag poses. Single-tag ambiguity: keep a tag when best is clearly smaller than alt (AOS: under 0.4×), then pick the candidate closest to the gyro. NaN alt means there's no second candidate. For a tag used in multi-tag, both are under the multi-tag pose instead: a tag that disagrees with the others by several pixels is a bad tag or a bad layout.
• NaN whenever it can't be known: no lens calibration, or 3D off.
• Cost: 48 bytes a tag plus 16 a frame on the network (measured: 111 bytes against a 616-byte result, 18% more), and 3.9 µs a tag of Jetson CPU (tests/tag-quality Bench).
• Checked by: TagQualityTest in the fork. On PhotonVision's real test image the best error is 0.54 px. On two rendered tags through multi-tag it's 0.24 and 0.13 px, and moving one tag 5 cm in the field layout raises it to 4.8 px. tests/tag-quality/run.sh checks the live topic on the Jetson.

NetworkTables bandwidth. Results go to the robot over its wired Ethernet, which has plenty of room: about 0.5 KB a frame with one tag, 1.5 KB with four, so 2.5–7.4 Mbit/s at 5 cameras × 120 fps (measured overhead 107 bytes a frame with no tags; each tag ~294 bytes). What has to stay small is what crosses the radio to the driver station:

• Elastic is safe as it is: it only subscribes to topics with a widget, at its update period, never every value.
• AdvantageScope, on the field: set Live Mode to Low Bandwidth (only what's on screen). "Logging" pulls every value of everything, which is fine on the pit tether.
• AdvantageKit sends whatever robot code logs to every dashboard, so log one compact record per observation, not raw PhotonLib results: timestamp, camera, the multi-tag pose (or single-tag best and alternate), tag IDs and distances, and the tagQuality values. That's enough to replay a changed trust model against a match, at about a tenth of the size. Raw results are 0.3–0.9 MB/s at 5 cameras × 120 fps (2.4–7 Mbit/s over the radio with AdvantageScope in Logging mode, and 45–135 MB of log per match), and 81 of each tag's 294 bytes are always empty for tags (skew, the game-piece fields, minAreaRectCorners), plus 64 more duplicated for tags in multi-tag (the alternate pose and ambiguity), and 72 of the multi-tag result's own ~140 (its single solution is copied into the alternate fields). With 4 tags in multi-tag that's ~650 of ~1.5 KB a frame. Deeper replays (re-detecting tags) use Rewind's frames.
• Don't turn on "Publish protobuf" in the networking settings: it sends every result a second time.
• Robot loop time: NetworkTables' sending and receiving run on its own C++ thread, not in the robot loop. What does run in the loop is PhotonLib decoding results and addVisionMeasurement; if that shows up in loop time, read the cameras on a separate thread and hand the loop finished measurements.

Robot code: use SpectrumVision (robot-vision/), our robot-side library on top of PhotonLib, for WPILib 2026 (roboRIO) and 2027 (SystemCore). It includes the mount check. It reads every camera's frames, solves them (multi-tag, or one tag's position plus the gyro), gates out bad ones with named, tunable gates, gives each a standard deviation, and sends them to WPILib's or CTRE's estimator in capture-time order. It uses this Jetson's per-tag quality, health topics and excluded tags, and works with stock PhotonVision too. robot-vision/install.sh <robot project> 2026|2027 copies it in.

What robot code must do (from the 2026-10-01 audit, docs/AUDIT-2026-10.md; SpectrumVision handles the ones marked *):

• Match the Jetson's PhotonVision. Now: WPILib 2027 alpha-8, with PhotonLib built from our spectrum-2027 fork. (For the October 2026 event it was PhotonLib alpha-2 and WPILib alpha-6, against the 2026 PhotonVision.) PhotonLib refuses results whose message format differs: log PhotonPipelineResult.photonStruct.getInterfaceUUID() at startup and alert on a mismatch (* SpectrumVision alerts on any mismatch with the coprocessor's format). (docs/MAINTENANCE-2027.md)
• Use the Jetson's camera names (TopLeft, TopRight, …), exactly as the dashboard shows them.
• Call setEnabled(true) on every camera at robot init (*). A camera robot code disabled stays disabled if robot code restarts (PhotonVision keeps it in memory), and returns empty results.
• Drop the tags in /photonvision/excludedTagsActive (*). They're left out of multi-tag, but still sent as single-tag targets.
• Send the field map, and check the Jetson's (photonvision-74). Robot code publishes the map it uses: /SpectrumVision/FieldMap/Json (WPILib's AprilTagFieldLayout JSON), Hash and Name. The Jetson loads it only if the JSON hashes to Hash, saves it for the next boot, and reports the map it's actually using: /photonvision/jetson/FieldMap/Hash, Name, LoadedAt (unix seconds) and Error. Alert when its Hash isn't yours. The hash is defined in docs/TECHNICAL.md: the 2026 AndyMark field is 2792cd4e868967b8, welded ca28a7c1139ed099. Hash the JSON you publish, not the layout you loaded it from: WPILib 2027's JSON writer drops the sign of a -0.0, which changes the hash.
• Watch each camera's health/state (photonvision-81). It reads streaming, stalled, recovering, disconnected, gave-up, paused or starting, and heartbeat counts up every second. gave-up means the Jetson can't bring that camera back: after its USB resets and port power toggles (photonvision-84), a Thriftiest Cam in that state needs a power cycle. With the usbrecov firmware a toggle usually brings it back first, in about 2 s. A dead camera sends no results, so isConnected() goes false (*: SpectrumVision alerts on gave-up and problem).
• Log the Jetson's diagnostics (photonvision-82, /photonvision/jetson/; the table is in docs/TECHNICAL.md). They cover input voltage, USB trouble with the kernel's line, the Ethernet link, NT disconnects, boot ID, clock source, CPU, RAM and SSD health. Log the whole /photonvision/jetson/ table: the Jetson's own logs don't survive quiet mode. Publish /photonvision/clock/unixMs too, so clockSource reads robot (*: SpectrumVision alerts on a voltage dip, USB trouble, an Ethernet drop, a reboot, an unset clock, SSD errors and high RAM).
• The event profile wins at field connect. With "switch to the event pipeline when the FMS connects" on, a pipeline robot code chose before the field connected is replaced. Set it again after DriverStation.isFMSAttached() if robot code picks pipelines.
• After a NetworkTables reconnect, results in the first ~1 s can carry an unsynced timestamp: skip results whose metadata.timeSinceLastPong is large (*: the timeSynced gate).
• The robot's control word: both formats work. A 2026 roboRIO publishes /FMSInfo/FMSControlData, and WPILib 2027 publishes the /FMSInfo/ControlWord struct (photonvision-62). Before this, PhotonVision read a 2027 robot as always disabled, so cameras idled at ~30 fps through matches.

Camera settings go in the robot log too (photonvision-24), so you can tell what a camera was set to in any match.

• Per camera: /photonvision/<camera>/settingsJson holds the full current pipeline settings, the video mode, which lens calibration is in use, and the camera's controls as actually set, including ones the UI doesn't show.
• Jetson-wide: /photonvision/jetson/settingsJson holds the PhotonVision build, the detector and decoder settings, the tags left out of multi-tag, and a fingerprint of the field layout.
• Cost: both are rebuilt every 5 s but only sent when something changes, and AdvantageKit records them at the start of every log.
• Rewind: each recording's session.json gets the same snapshot.

Every change in the UI is saved right away (within about a second), with no undo and no "unsaved changes" warning (upstream issue #2614). So at an event:

• Before experimenting, duplicate the pipeline (☰ next to the pipeline name) and try things on the copy. Switch back if it doesn't work out.
• If a good setup gets lost, the settings from any earlier match are in that match's robot log (settingsJson above) and in any Rewind recording's session.json. Nothing restores them in one click: re-enter them from there, or use Copy settings from… if another camera still has good ones.
• Only one person tunes a camera at a time. Two dashboards now stay in sync (photonvision-41), but two people dragging the same slider will still fight.

Where everything lives, and what's left

The detailed technical reference, with exact versions, commits and measurements, is docs/TECHNICAL.md. This README is the overview. What this setup lacks compared with Limelight 4, and what the robot code has to do about it (MegaTag 1/2, gyro heading), is in docs/LIMELIGHT-COMPARISON.md. Other teams' vision systems and our performance work are in docs/VISION-RESEARCH.md. What we took from upstream PhotonVision, and what to test, is in docs/UPSTREAM-PORT.md.

Folder	What's in it
scripts/host/	Run on the laptop: prepare and flash the Jetson (01, 02), build the PhotonVision fork jar (03), back up and restore the SSD (04, 05), copy and export Rewind recordings (rewind-pull.sh, rewind-export.py)
scripts/jetson/	Run on the Jetson, in order: verify (01), CUDA and Java (02), PhotonVision service (03), allwpilib for the old 4143 detector (04), 4143 detector (05), install jar (06), GPU detector (07), pick detector (08), robot tuning (09, steps 1–16), data partitions (10), camera driver bandwidth cap (11), install a YOLO model (12), field-calibration replay tool (13), USB controller watchdog (14), plus health-check.sh, usb-bandwidth.py (what each camera reserves on USB, and the fix), spectrum-clocks, spectrum-rt, and the camera firmware tools
patches/	Our fixes to other people's code, applied by the build scripts
robot-vision/	SpectrumVision, the robot-side library (PhotonLib wrapper: solvers, gates, trust, alerts, testing), WPILib 2026 and 2027 builds; robot-vision/README.md
detector/	Our JNI wrapper and CMake build for Austin's current CUDA detector (and the MJPEG decoders, CPU and hardware, the TensorRT object detector, and the replay tools detector-replay and fieldcal_detect, which run Rewind recordings through the detector)
tools/fieldcal/	Field calibration: tag positions and camera mounts from a recording of the robot pushed to still spots (README)
tools/fieldmodel/, assets/field-models/	The 3D field model for the Field Calibration page, converted from FIRST's field CAD (README)
kernel/	Our patch to Linux's USB camera driver (bandwidth cap), built by 11-uvcvideo-payload-cap.sh
tests/	tests/run-all.sh runs every test that needs no one at the bench (about 30 min, a table at the end). Dashboard browser tests (tests/ui), a fake robot for robot-state features (tests/fake-robot), quiet mode (tests/quiet-mode: a match, a restart, a date jump, a failed remount), Rewind's numbering, delete and NetworkTables reconnects (tests/rewind-sessions), a SystemCore rehearsal (tests/systemcore-rehearsal: the real WPILib 2027 robot runtime on the laptop, the Jetson connected to it), garbage-collection measurement (tests/jvm-gc), frame timestamps (tests/uvc-timestamps, tests/fake-robot/timestamps.sh), far-tag search (tests/far-search), detector stress test, live A/B and fault-injection test, ChArUco board checker, calibration checker, JVM memory check, Rewind on/off test, power-cut test, camera unplug test, USB hub reset test, robot clock test, flicker check, CPU profiler, performance snapshot, telemetry and mount-estimate check
docs/	The technical reference, Rewind, the Limelight 4 comparison, vision research, the upstream PhotonVision port, the game-piece models, and the original handoff document that started the project

What's left (2027):

• TensorRT game pieces on JetPack 7 (TensorRT 10.16): deferred until we need it.
• Recalibrate both cameras with the full-screen view, covering the edges and corners (Calibrating well); until then, import the refits in calibrations/*-refit.json.
• Field calibration on a real field: the event's tag positions and every camera's mount (Field calibration). The field layout is the other half of the remaining error.
• Replay the robot side: the pose estimator (EKF) and the multi-camera solver (MultiCameraSolver in robot-vision/, one pose from every corner of every camera; 0.2 cm median in sim) against the TXMAN1 recordings and logs.
• Timing spin test: a still tag, the robot turning with the gyro's own sample timestamps logged, to check the default SPECTRUM_CAMERA_DELAY_US=2500 (Team 100's figure) against our cameras. The match replays' rotation method gives +8 to +10 ms total but can't separate the camera from the robot's log latency.
• Faster far search on JetPack 7 (5.9 ms a search, against 2.7 on JetPack 6), then turn it back on.
• The rest of the fork's 2027 port (SPECTRUM.md).

Still to measure at the bench (JetPack 7): tests/bench-session/run.sh walks through these with prompts, a time limit on every step, and puts every setting back.

• 4 and 5 cameras at 122 fps: CPU, GPU, latency, temperature (only 2 measured so far).
• CUDA graphs on/off and far search on/off, interleaved, with tags in view.
• PhotonVision's 3D mode on vs off: CPU and latency.
• spectrum-rt (low-jitter camera threads, off by default) on vs off, with and without CPU load.
• USB recovery timing with the usbrecov firmware (opt-in step: it cuts camera links).
• Boot time and first detection after power-on, and a software reboot that doesn't hang.
• The full calibration view on a real camera, then the regression corpus with real tag footage.

History: the October 2026 event checklist (JetPack 6). Kept as a record; versions and some numbers are from the 2026 setup.

• Calibrate both cameras at 1280x800 (done on the bench; redo on the robot)
• Deploy the jar with the Device Control and 8-coefficient fixes
• Robot tuning: no auto-updates, headless boot, clocks locked, USB autosuspend off, power-cut safety (data on the SSD within 3 s, system log kept across power cuts)
• Cooling on Settings > Robot state: no fan (ours, the fresh-install default), NVIDIA's quiet profile for a stock fan (775 rpm at 43 °C on the bench) or full speed, 30 s hardware watchdog, reboot on kernel panic, PhotonVision always restarted
• Camera unplug test: the camera detects again ~1 s after it's plugged back in; the other camera is unaffected (tests/camera-replug/run.sh)
• With 4 cameras (2026-09-29): a camera pulled for 10 s detects again 1.3 s after it's back in, and one yanked and pushed back (out 1 s) loses ~1.5–2 s in all. The other 3 never dropped below 119.7 fps. Two cold power cycles: all 4 detecting 21–22 s after power-on, clean both times
• Power-cut test: pulled the plug mid-recording. No filesystem errors, the log survived, PhotonVision came back healthy, 1.4 s of video lost (tests/power-cut/run.sh). But after 21 cuts the SSD itself failed (see Storage); the test must also check the drive's media-error count
• Power-cut test on the split storage, checking the SSD's media errors after every cut
• Reboot test: tuning survives a reboot; boot 57 s → 16.5 s, first detection ~20 s after power-on
• Name the cameras after their ports (TopLeft, TopRight; BottomLeft/BottomRight when added)
• Rename the Global Shutter cameras BottomLeft (port 2.2) and BottomRight (port 2.4), if they stay
• Wi-Fi / Bluetooth switches in PhotonVision (Bluetooth off; Wi-Fi off before events)
• Static IP 10.85.15.15 on Ethernet (set in PhotonVision: Settings > Networking)
• Rewind: record every camera to the SSD when robot code asks (bench-tested, no fps cost)
• Jetson sets its date from the robot's clock when it has no internet (photonvision-08; robot code publishes /photonvision/clock/unixMs, issue #10)
• Test on the robot network with the SystemCore (NetworkTables, time sync, PhotonLib reading results, Rewind's robot-clock timestamps, the Jetson's date from the robot). Include pulling the Ethernet cable for 5 s mid-test: NetworkTables and time sync should reconnect by themselves
• Hot glue the USB connectors and add strain relief. Label each camera with its stored serial (SJ-TC01, ...); a camera without one in the wrong port silently swaps calibrations
• Lock each lens after focusing (glue or silicone, like 6328), then recalibrate. Vibration can turn a lens and spoil its calibration
• Turn off Wi-Fi for competition (Bluetooth is already off)
• Write the vision subsystem in 2026-FM-SystemCore using the AndyMark field layout, with photonlib kept at alpha-2
• Check temperatures with the Jetson mounted on the robot (44 °C on the bench with the fan at full speed, 43 °C on the quiet profile; fanless on the plate, see Fanless above)
• Decode speedup: both cameras at 122 fps, 13 ms latency, 1.3 of 6 CPU cores
• Upstream PhotonVision v2026.3.4 fixes, setEnabled() support, OpenCV leak fixes (docs/UPSTREAM-PORT.md)
• Frame timestamps moved to mid-exposure (photonvision-13); the camera's own delay is still to be measured with the robot spin test
• Game-piece detection: TensorRT backend, FUEL model working (76 fps uncapped)
• Game-piece pipelines capped at 30 fps by default (no measurable effect on AprilTag cameras)
• JPEG hardware decode for gray and colour cameras, with pixel checks and a memory-leak test (--jpeg nvjpg, tests/jpeg-hw/run.sh)
• Dashboard stream only while someone is watching, capped at 30 fps (photonvision-15)
• Game-piece colour camera on the robot
• USB bandwidth: capped camera driver so 4 cameras fit on USB-A (alt 7, tested with 2: 122 fps, no bad frames)
• 5 cameras on USB 2.0 (4 on USB-A, 1 on a USB-C hub): the whole USB 2.0 side shares one budget, measured
• USB bandwidth card on Camera Matching (photonvision-32): allocated vs used per camera, and a per-camera allocation setting
• Check photonvision-31 live: the health/problem topic, and no USB resets while a camera lacks bandwidth
• Measure whether a squeezed AprilTag camera (lower JPEG quality) loses range or precision
• 4 cameras on the USB-A ports (2 Thriftiest, 2 Global Shutter) and a colour camera on a USB-C hub, all streaming
• Re-measure with tests/perf-snapshot.sh with every camera running
• USB hub reset test (tests/usb-hub-reset/run.sh): the Global Shutter cameras come back by themselves; the Thriftiest Cams stay stuck until their power is cut (Known issues)
• USB controller watchdog (14-usb-watchdog.sh), tested with a faked dead controller: it resets the controller and reports which cameras came back
• Log flood fixed (photonvision-33): a camera failure logged 20,000 lines a minute. The system log also keeps up to 2 GB now, and the hardware watchdog covers reboots (30 s)
• Find why a software reboot can hang: repeat it with a DisplayPort monitor plugged in (Known issues)
• Focus score on the Camera page (photonvision-34), like Limelight's
• Target yaw is positive to the right (clockwise), the opposite of WPILib's angles. PhotonVision's own code and tests say so, but PhotonLib's PhotonTrackedTarget comments and the "getting target data" docs page say left. photonvision-36 corrects those in our fork and labels the Targets tab's columns; aiming code needs turn = -kP * target.getYaw() (training chapter 19)
• Retune exposure and decision margin on the event field, and run tests/flicker-check/run.sh under its lights. Compare decision margin 15 against AOS's 50 on Rewind footage (false positives)
• Later, if we need it: AprilTags and game pieces in one pipeline on the same camera (plan in docs/VISION-RESEARCH.md)
• Field calibration mode: push the robot by hand to 10–20 spots, then solve for the event's real tag positions, every camera's mount and the best camera settings (docs/FIELD-CALIBRATION-PLAN.md). Test in the shop first.
  • Solver: tools/fieldcal. On synthetic recordings: tags to a few mm, camera height to 4 mm, pitch and roll to 0.03°
  • Runs on the Jetson, replaying the recording through the 971 GPU detector at ~400 fps (scripts/jetson/13-build-fieldcal-detect.sh --install)
  • A Field Calibration page in PhotonVision (photonvision-30): tune camera settings, guided recording, solve, results, apply/undo the layout. Tested on the bench through the API (a synthetic recording solved, apply and undo round-trip exact); the page itself not yet looked at
  • Robot code publishes each camera's robotToCamera at /photonvision/<camera>/robotToCamera
  • A 3D view of the field (Onshape mouse controls) with simplified 2026 elements placed by their tags
  • The real field CAD in the 3D view (tools/fieldmodel, 2.1 MB): FIRST's STEP converted, each element placed by its tags, the perimeter by the field's size, so it fits the AndyMark and welded layouts
  • Shop test on our half field, including the settings tuning with real tags
  • Recalibrate the lenses with board views right into the corners: our calibrations can't model the outer 1–4% of the image
• The camera controls PhotonVision's UI didn't show (contrast, gamma, sharpness, backlight compensation) are on the Input tab (photonvision-28), at the camera's defaults unless set
• Measure whether contrast, gamma or sharpness help (decision margin, far-tag range, corner jitter): the field-calibration settings sweep
• A stuck camera recovers by itself (photonvision-29): reconnect, then USB reset. Tested on the bench with its test hook
• Bad tags left out of multi-tag, calibration with 100 auto snapshots and our board as the default, camera settings in the robot log, copy settings between cameras (patches 22–25)
• Test the detector's max_line_fit_mse at 2.5 (upstream #2138: rejects tags cut off at the image edge) against the default 10 with tags in view: range and edge behaviour. Then set it with 08-select-detector.sh gpu --mwbd 20 --jpeg nvjpg --mse 2.5
• Test whether the Camera Gain slider does anything on our cameras (photonvision-21)
• Re-test the detector's thread count with several tags per camera, the field case (tests/detector-threads/run.sh, switches live; with one tag it made no difference)
• Robot code, in progress: Rewind starting itself on FMS matches, named by match (issue #10 8a)
• Robot code: drop a bad tag at run time from the dashboard (issue #10 11b and 12). The Jetson side is done (photonvision-22)
• Robot code: the rest of issue #10's robot-side items (trusting tags less near the image edge, single-tag checks, sanity gates, standard deviations, alerts on the Jetson's health topics)
• Full backup image of the SSD with PhotonVision's settings (scripts/host/04-backup-ssd.sh: 8.7 GB, 7 min). Keep it on the team drive, never GitHub (it holds the Wi-Fi password and SSH keys)
• GitHub release v2026.09.24: the PhotonVision jar, TensorRT backend, camera driver and settings
• Clone a spare SSD from the backup (scripts/host/05-restore-ssd.sh)
• Field map from the robot (photonvision-74): run tests/field-map/run.sh on the Jetson, then check on the robot that /photonvision/jetson/FieldMap/Hash matches robot code's and stays put through a Jetson power cycle
• Diagnostics (photonvision-82) on the Jetson: re-run 09-robot-tuning.sh (smartctl), then check every /photonvision/jetson/ topic in the table has a value. Unplug and replug a camera, and check usbEvents and lastUsbEvent change. Unplug the Ethernet for 2 s, and check ethCarrierChanges goes up by 2. Pull the input down on a bench supply until the carrier's 5 V regulator drops out, and check inputVoltageMinV (normally about 4.95 V: it's the module's 5 V input, not the 12 V supply). Run export-match.sh after a practice match.
• Dead cameras (photonvision-81) on the Jetson. Check health/state and heartbeat with the stuck-camera hook (stalled → recovering → gave-up after 4 USB resets), with a camera unplugged (disconnected, then gave-up after 10 s), and with a camera wedged by tests/usb-hub-reset. Also check that no rawBytes arrive from a dead camera.

Credits and licenses

This setup stands on other people's work. We link to or patch their code rather than copy it into this repo, except where noted. Each project keeps its own license.

Code we build on:

Project	What we use	License
PhotonVision	The vision software itself. We merged release v2026.3.4 (patch 00) and ported the server side of setEnabled (#2484, #2499) and OpenCV leak fixes (#2511).	GPL-3.0
FRC-Team-4143/photonvision	The CUDA version of PhotonVision our 2026 build started from (commit d8c9e8e); its CUDA pipeline is ported into our 2027 fork. Also their earlier GpuDetectorJNI detector build.	GPL-3.0
Austin Schuh and FRC 971's CUDA AprilTag detector	The GPU detector, from RealtimeRoboticsGroup/aos (Apache-2.0), built from frc971/bos third_party/971apriltag. Our detector-* patches (called bos-* before 2026-10-31) change it.	Apache-2.0 (aos); bos has no license file
WPILib allwpilib	cscore and ntcore run inside PhotonVision. In 2026 we built v2026.2.1 on the Jetson for the detector; the current detector doesn't link it.	BSD-3-Clause
Linux kernel uvcvideo	Our kernel/uvcvideo-payload-cap.patch modifies the USB camera driver: on JetPack 7 from NVIDIA's own kernel 6.8 source (on JetPack 6, the stock v5.15.199 from the stable kernel's GitHub mirror).	GPL-2.0
libjpeg-turbo	Grayscale MJPEG decode in detector/GpuDetectorJNI.cc (the system library).	IJG / BSD-style
NVIDIA JetPack, CUDA, TensorRT	The OS, GPU toolkit and inference engine. Hardware JPEG decode uses JetPack's libnvjpeg and the Jetson Multimedia API headers. Not redistributed.	NVIDIA licenses
Ultralytics	Exporting YOLO models to ONNX on the laptop. Not redistributed.	AGPL-3.0

Models:

• FUEL YOLO11n by Team 2826 Wave Robotics. It's the model PhotonVision ships for 2026; Wave gave PhotonVision permission to include it.
• FUEL YOLO26n by Project516 (AGPL-3.0), downloaded for comparison.

We don't redistribute either; see docs/GAME-PIECE-MODELS.md.

Field model:

• Source: assets/field-models/2026-rebuilt.glb, the 3D field on the Field Calibration page, is converted from FIRST's official 2026 field CAD (FE-2026-rev-rebuilt-playing-field.step, from the Playing Field page) by tools/fieldmodel. It's simplified, and the FUEL and fasteners are left out.
• Terms: © FIRST. FIRST publishes the CAD for teams but states no license for it. AdvantageScope redistributes field models converted from the same CAD in the same way. If FIRST asks, we'll remove it and have the build convert it locally instead.

Ideas and research we learned from (no code copied):

• EagleEye by Scythe-Engineering (grayscale-only decode). It's PolyForm Noncommercial, so we took ideas only.
• Team 3476 Code Orange's ML-assisted AprilTags ("MLTag").
• Team 4533's Whacknet (coprocessor constrained solve).
• 971's bos and cos (TensorRT YOLO, hardware JPEG decode).
• Mechanical Advantage 6328's Northstar.
• Many Chief Delphi threads, linked in docs/VISION-RESEARCH.md.

This repo is licensed under the GPL-3.0, following PhotonVision. The kernel patch in kernel/ is GPL-2.0, like the Linux driver it changes. The scripts, tests, docs and our own code (detector/, and the new files in our patches) were written by Spectrum 3847 with Claude Opus 5.5 (Anthropic), working in Claude Code.