Skip to content

backend

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

backend

README.md

backend

Go HTTP API for Lumen, the self-hosted, invite-only music library.

Features

  • Auth — Argon2id password hashing, HTTP-only cookie sessions backed by Postgres, first-run admin seeding (ADMIN_USERNAME / ADMIN_PASSWORD; if the password is empty a random one is written, mode 0600, to ADMIN_PASSWORD_FILE (default $MUSIC_PATH/.initial-admin-password, or <user config dir>/lumen/initial-admin-password if that isn't writable), and the seeded admin must reset it on first login).
  • Invite-only registration — admins mint invite tokens (role, max uses, expiry); users register via /register?token=….
  • Library — admin-managed music roots, filesystem scanning and ingest with a watcher + rescan support, metadata extraction, cover art, search.
  • Playback — ranged audio streaming (scrub-friendly), play history/stats, cross-device activity over an authenticated WebSocket with REST/Postgres snapshot recovery and addressed remote-control commands (see docs/playback-websocket.md). Local streaming serves the original audio format; client-profile transcoding is not supported.
  • Playlists & favorites — CRUD plus per-user state.
  • Sharing — public /share/… link-preview pages (what Discord and chat apps scrape), /embed/… embeddable players, and server-rendered preview images (see fonts/). Share/cover URLs are HMAC-signed with COVER_SIGN_KEY.
  • Importers — pollers that pull new files into the library from external sources: Filen share links (via the bundled Node helper in filen-downloader/), ArtistGrid, Lastshare, and a tracker-API pin scanner (see docs/api-tracker.md for that API's reference).

Layout

cmd/server/           entrypoint
internal/
  auth/               argon2id, session store, bootstrap admin seeder
  config/             env-driven config
  db/                 pgx pool + embedded migrations
  httpapi/            chi router, middleware, handlers
  ingest/             scanner, watcher, rescan
  library/            library metadata store
  playlists/          playlist store
  preview/            share/embed preview + image rendering
  storage/            storage interface + local FS implementation
  invites/ users/     invite + user stores
  filen/ artistgrid/  external importers
  lastshare/ apitracker/ pinscan/
  musicroots/         admin-managed scan roots
fonts/                fonts bundled for preview image rendering
filen-downloader/     Node 22 helper the Docker image bundles for Filen links

Run (local)

Requires Go 1.26+ and a Postgres 16 instance:

docker run --rm -e POSTGRES_PASSWORD=mlib -e POSTGRES_USER=mlib -e POSTGRES_DB=mlib -p 5432:5432 postgres:16

DATABASE_URL=postgres://mlib:mlib@localhost:5432/mlib?sslmode=disable COOKIE_SECURE=false go run ./cmd/server

Migrations run automatically on boot. COOKIE_SECURE=false is needed over plain HTTP so the session cookie isn't dropped. All knobs are listed in .env.example.

Configuration is validated strictly at startup: booleans must be accepted by Go's strconv.ParseBool, durations must parse and be greater than zero, and TRUSTED_PROXIES entries must be IP literals or CIDR ranges. Invalid values stop startup with the variable name instead of silently falling back. Check existing deployment environment files before upgrading.

Non-admin personal uploads are capped per user by PERSONAL_UPLOAD_QUOTA_MB (default 20480, i.e. 20 GiB; 0 disables the cap).

Enabled music-root paths are cached per backend process and refreshed when roots are added, enabled, disabled, or removed through that process's API. Changes made directly in Postgres or through another backend process require a restart of this process to refresh its snapshot. Playback still checks track access in Postgres and opens and stats the original file on each request.

Rescanning the library

POST /api/admin/library/rescan skips global files whose path, size, and nanosecond modification time match their last successful ingest. Fingerprints persist across restarts; existing files receive them on their first successful ingest after upgrading. Files with recorded errors, incomplete metadata, deleted tracks, or a moved canonical path are read again. Reachable nested roots are scanned once; separately configured roots inside dot-directories remain separate scans.

Use POST /api/admin/library/rescan?force=true to read every file again, for example after replacing a file while preserving its size and modification time, or to retry artwork processing. Both routes require an admin session. GET /api/admin/library/rescan reports unchanged alongside the existing progress counters; unchanged files also count toward dedup and processed.

Migration 0023 repairs listening timestamps from play history and temporarily locks play-write tables. Migration 0024 adds the nullable ingestion fingerprint columns. Migrations 0025–0029 build catalog and recent-track indexes concurrently in separate statements. Index creation can extend startup for large libraries while allowing writes to continue during each build.

Test / lint

gofmt -l .
go vet ./...
go test ./...

Database-backed tests need a disposable PostgreSQL database. The test role should be able to create databases so packages can use isolated sibling databases:

PGPASSWORD=mlib createdb --host=localhost --username=mlib mlib_test
LUMEN_REVIEW_TEST_DATABASE_URL=postgres://mlib:mlib@localhost:5432/mlib_test?sslmode=disable go test ./...

Docker

The Dockerfile builds a static binary and ships it on Alpine with ffmpeg (previews and TIDAL remuxing), Node (Filen helper), and the preview fonts. The deployment compose at the repo root builds it with context: ./backend.

Track pagination

GET /api/tracks accepts sort=recent|title|artist|album|duration (default recent). Sorting is applied to the full visible result set before limit and offset, with a track-ID tie breaker.

Search types

GET /api/search?q=radiohead&type=album searches albums. type accepts all, track, album, or artist; song and plural names are also accepted. Omitting type preserves the original track-only behavior. The shared client provides api.search / api.searchPage (default all) and keeps api.searchTracks / api.searchTracksPage track-only for playlist pickers.

The response contains tracks, albums, and artists arrays (including empty arrays for excluded types), sources, next_offsets, and optional warnings. Local results respect the viewer's library visibility. Remote albums and artists include source: "tidal", source_id, a namespaced id, and optional cover_url. Open them with /api/tidal/albums/{source_id} or /api/tidal/artists/{source_id}. The artist endpoint returns top songs and releases from the proxy's bounded artist aggregation.

sources=local,tidal is the default; either source can be selected alone. limit is per source and type (default 25, maximum 50). all may therefore return up to six times limit. A source failure leaves successful results in the response and adds a warning; retry the search to retry failed sources. Unconfigured TIDAL is skipped by default; an explicit TIDAL source request returns a configuration warning.

next_offsets contains only streams that may have more results. Track streams retain the local / tidal keys; album and artist streams use local_album, tidal_album, local_artist, and tidal_artist. Continue with streams set to exactly the returned keys and a <key>_offset parameter for each value:

/api/search?q=radiohead&type=all&limit=25&streams=tidal,tidal_album&tidal_offset=25&tidal_album_offset=50

Keep the query, type, sources, and limit unchanged across pages. An empty next_offsets means exhausted (streams= explicitly requests no streams). The combined result count is never a per-stream offset. Legacy track-only callers can still continue with only the returned sources and their local_offset / tidal_offset values.

TIDAL album and artist search requires the updated bundled hifi-api/lumen_hifi.py extension (/lumen/search/albums and /lumen/search/artists). Restart the Compose hifi-api service when deploying this change. These endpoints query full collections rather than capped top hits, using the proxy's existing credential handling. Independently hosted proxies need the same extension to support these search types.

Remote playback queue sync

Web, desktop, and mobile players publish their actual queue order alongside WebSocket activity.update messages. devices.snapshot includes the latest queue for each connected device, so controllers can join playback already in progress. Queue snapshots stay in memory for that connection and are cleared when playback is cleared or the device disconnects; no database migration is required.

Snapshots fit within the 64 KiB WebSocket message limit, including the activity payload. They contain up to 50 tracks around the current track, plus the window offset, full queue length, current index within the window, shuffle/repeat settings, and a queue revision. The window advances with playback. The jump_to command uses an absolute index, track ID, and queue revision to select a track without replacing or reshuffling the target's queue. The target rejects selections from an outdated queue. Queue sync requires an updated backend and both clients; older clients can continue sending activity without a queue.