From 999baf0708c77bfc99f4aae28bed281951f25ea5 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 28 Oct 2026 10:49:57 +0000 Subject: [PATCH] docs: add Run Your Own Superhero, a self-hosting guide for the full stack A new section under Build that walks an operator through every service behind superhero.com: - Node, middleware and compiler: ae_mdw from snapshots, the paths the API reads, and where the endpoints are set. - API and indexer: build, configuration, first-time database setup (tested on PostgreSQL 16 against superhero-api main), indexing progress and the HTTP surface. - Charts and history: which charts are computed on read, which are rebuilt on a schedule, AE price data, and how a new instance fills in. - Web app: build- and run-time settings, the CSP, passkeys and the domain-bound features. - Chat relay: groups_relay, TLS, the admin key shared with the API. - Governance backend, contracts (public or your own), keys and integrations, and operations with a launch checklist. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_018Az8TKjLpFXw9ouAdoyLDs --- pages/_meta.json | 1 + pages/self-hosting/_meta.json | 12 ++ pages/self-hosting/api-and-indexer.mdx | 199 ++++++++++++++++++++++ pages/self-hosting/chain-access.mdx | 121 +++++++++++++ pages/self-hosting/chart-history.mdx | 85 +++++++++ pages/self-hosting/chat-relay.mdx | 127 ++++++++++++++ pages/self-hosting/contracts.mdx | 129 ++++++++++++++ pages/self-hosting/governance-backend.mdx | 82 +++++++++ pages/self-hosting/index.mdx | 90 ++++++++++ pages/self-hosting/integrations.mdx | 113 ++++++++++++ pages/self-hosting/operations.mdx | 102 +++++++++++ pages/self-hosting/web-app.mdx | 185 ++++++++++++++++++++ 12 files changed, 1246 insertions(+) create mode 100644 pages/self-hosting/_meta.json create mode 100644 pages/self-hosting/api-and-indexer.mdx create mode 100644 pages/self-hosting/chain-access.mdx create mode 100644 pages/self-hosting/chart-history.mdx create mode 100644 pages/self-hosting/chat-relay.mdx create mode 100644 pages/self-hosting/contracts.mdx create mode 100644 pages/self-hosting/governance-backend.mdx create mode 100644 pages/self-hosting/index.mdx create mode 100644 pages/self-hosting/integrations.mdx create mode 100644 pages/self-hosting/operations.mdx create mode 100644 pages/self-hosting/web-app.mdx diff --git a/pages/_meta.json b/pages/_meta.json index 9526da9..a2b244a 100644 --- a/pages/_meta.json +++ b/pages/_meta.json @@ -25,6 +25,7 @@ "agents": "AI Agents", "api-reference": "API Reference", "contracts": "Smart Contracts", + "self-hosting": "Run Your Own Superhero", "-- chain": { "type": "separator", "title": "Underlying Chain" diff --git a/pages/self-hosting/_meta.json b/pages/self-hosting/_meta.json new file mode 100644 index 0000000..557dd39 --- /dev/null +++ b/pages/self-hosting/_meta.json @@ -0,0 +1,12 @@ +{ + "index": "Overview", + "chain-access": "Node, Middleware & Compiler", + "api-and-indexer": "API & Indexer", + "chart-history": "Charts & History", + "web-app": "Web App", + "chat-relay": "Chat Relay", + "governance-backend": "Governance Backend", + "contracts": "Contracts", + "integrations": "Keys & Integrations", + "operations": "Operations" +} diff --git a/pages/self-hosting/api-and-indexer.mdx b/pages/self-hosting/api-and-indexer.mdx new file mode 100644 index 0000000..ee295c6 --- /dev/null +++ b/pages/self-hosting/api-and-indexer.mdx @@ -0,0 +1,199 @@ +--- +title: "API & Indexer" +description: "Build and run superhero-api: its datastores, configuration, first-time database setup, indexing progress and HTTP surface." +--- + +# API & Indexer + +`superhero-com/superhero-api` is one NestJS process that does all of the backend work: + +- **Indexer.** Walks the chain through the middleware and stores what Superhero needs in + PostgreSQL: posts, tips, tokens, trades, DEX swaps, governance, identity links. +- **API.** REST under `/api`, Socket.IO channels for live token data, and a notifications + namespace. +- **Jobs.** Trending scores, token performance, leaderboards, DEX summaries, AE price history, + popular-feed ranking, holder syncs and push delivery, on cron schedules and Redis-backed queues. +- **Integrations.** X verification and rewards, `.chain` name sponsoring, identity links and + token-gated chat rooms, each switched on by its own keys. + +## Requirements + +| Component | Version / notes | +| --- | --- | +| Node.js | 20 (the Docker image uses `node:20-alpine`) | +| PostgreSQL | 16, no extensions | +| Redis | Any current version; keys are prefixed with the network id | +| Middleware and node | See [Node, Middleware & Compiler](/self-hosting/chain-access) | + +## Build + +The build installs two packages straight from GitHub (`bctsl-sdk` and `dex-contracts-v2`) and +compiles `bctsl-sdk` during install. + +**Docker.** The `Dockerfile` reads a GitHub token from a BuildKit secret for those installs. Any +token works, including a read-only one: + +```bash +git clone https://github.com/superhero-com/superhero-api.git +cd superhero-api +DOCKER_BUILDKIT=1 docker build --secret id=GITHUB_TOKEN,env=GITHUB_TOKEN -t superhero-api . +``` + +**Without Docker.** Point Git at HTTPS for GitHub, then install and build: + +```bash +git config --global url."https://github.com/".insteadOf "ssh://git@github.com/" +git config --global --add url."https://github.com/".insteadOf "git@github.com:" +npm ci +npm run build +``` + +The image runs `npm run migrate:prod:locked`, which applies pending migrations under a Postgres +advisory lock, and then starts `node dist/main` on port 3000. + +## Configure + +Start from `.env.example` in the repository. The settings below are the core; feature keys are in +[Keys & Integrations](/self-hosting/integrations). + +| Variable | Default | What it sets | +| --- | --- | --- | +| `NODE_ENV` | — | Set `production` | +| `AE_NETWORK_ID` | `ae_mainnet` | `ae_mainnet` or `ae_uat`: selects chain endpoints, contracts and the Redis key prefix | +| `APP_PORT` | `3000` | Listen port | +| `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASSWORD`, `DB_DATABASE` | port `5432` | PostgreSQL connection | +| `DB_POOL_MAX`, `DB_POOL_MIN` | `40`, `5` | Connection pool | +| `REDIS_HOST`, `REDIS_PORT`, `REDIS_PASSWORD` | `localhost`, `6379` | Redis connection | +| `TRUST_PROXY` | unset | Set `1` (or your proxy's CIDR) behind a reverse proxy, so rate limits see client IPs | +| `ALLOWED_ORIGINS` | all origins | Comma-separated CORS allowlist for REST and Socket.IO | +| `DEBUG_ENABLED` | `false` | `true` adds info-level logs to the default error logs | +| `ENABLE_SWAGGER` | — | `true` serves the OpenAPI UI at `/api` in production | + +Chain endpoints are set in code, in `src/configs/network.ts`; see +[Point the API at it](/self-hosting/chain-access#point-the-api-at-it). Contract addresses are set +per network in code too; see [Contracts](/self-hosting/contracts). + +### Indexer settings + +| Variable | Default | What it sets | +| --- | --- | --- | +| `SYNC_INTERVAL_MS` | `3000` | How often the backfill loop checks for work | +| `BULK_MODE_BATCH_BLOCKS` | `1000` | Blocks per step while far from the tip | +| `BACKFILL_BATCH_BLOCKS` | `50` | Blocks per step close to the tip | +| `MICRO_BLOCKS_PARALLEL_BATCH_SIZE` | `4` | Concurrent micro-block fetches | +| `FETCH_JSON_TIMEOUT_MS` | `30000` | Timeout for each middleware request | +| `REORG_DEPTH` | `100` | Key blocks re-checked every 10 minutes for reorgs | + +With a middleware on the same network, raise `MICRO_BLOCKS_PARALLEL_BATCH_SIZE` to speed up the +first backfill. + +## Create the database + +On a new database, create the schema once before the first normal start. The schema comes from the +API's entities; the migrations then add what the entities do not describe. These steps were +checked against `main` on PostgreSQL 16. + +**1. Build the schema from the entities.** Start the API once in schema-sync mode, with the +indexer off: + +```bash +docker run --rm -p 3000:3000 --env-file .env.api \ + -e NODE_ENV=development -e DB_SYNC=true -e DISABLE_MDW_SYNC=true \ + superhero-api node dist/main +``` + +When `curl localhost:3000/api/stats` answers, stop the container. Schema sync works only outside +`NODE_ENV=production`, so this is a one-off step. + +**2. Record the migrations the entities already cover.** These ten describe tables and columns the +sync just created: + +```sql +CREATE TABLE IF NOT EXISTS migrations ( + id SERIAL PRIMARY KEY, + timestamp BIGINT NOT NULL, + name VARCHAR NOT NULL +); +INSERT INTO migrations (timestamp, name) VALUES + (1718900000001, 'TgrTokenColumns1718900000001'), + (1718900000002, 'TgrCommunityRoom1718900000002'), + (1718900000003, 'TgrRoomMembership1718900000003'), + (1718900000004, 'TgrRoomNotificationPreference1718900000004'), + (1718900000005, 'TgrRoomMessageSeen1718900000005'), + (1718900000006, 'TgrTokenBalance1718900000006'), + (1718900000007, 'TgrRoomBackfillState1718900000007'), + (1718900000008, 'TgrTokenRoomId1718900000008'), + (1718900000009, 'TgrRoomMembershipEvent1718900000009'), + (1718900000036, 'TokenPerformanceTable1718900000036'); +``` + +**3. Start normally.** The container applies the remaining migrations, including the social graph +tables and the query indexes, and starts the API: + +```bash +docker run -d --restart=always --name superhero-api -p 3000:3000 \ + --env-file .env.api -e NODE_ENV=production superhero-api +``` + +From here on, every start applies new migrations automatically before the API comes up. + +## Indexing + +On first start the indexer reads the middleware's tip height and runs two loops: + +- **Live sync** subscribes to the middleware WebSocket and indexes new blocks as they arrive. +- **Backfill** walks from the tip back to the start of the chain in batches, storing blocks and the + transactions any plugin cares about. + +Some data arrives ahead of the backfill: all posts are pulled from the middleware shortly after +boot, and new tokens are pulled from the factory every 10 minutes. Trades, DEX swaps and chart +history fill in as the backfill reaches them; token history begins at the factory's deployment +height. + +### Progress and health + +| Endpoint | What it tells you | +| --- | --- | +| `GET /api/stats` | `currentBlockNumber`, `lastSyncedBlockNumber`, `remainingBlocksToSync`, `mdwConnected`, uptime. Cheap; use it as the liveness probe. | +| `GET /api/mdw/health` | Backfill status (`running` or `complete`), remaining blocks and percentage, live-sync lag, and each plugin's synced height | +| `GET /api/v2/mdw/sync-state` | The raw sync state rows | + +The indexer is caught up when `backwardSync.status` is `complete` and `liveSync.lag` is a few +blocks. + +### After downtime + +Every 10 minutes the indexer re-checks the last `REORG_DEPTH` key blocks, about five hours of +chain at the default 100. If the API was down for longer, set `REORG_DEPTH` above the number of +key blocks it missed, start it, let one validation pass run, then set it back. + +## HTTP surface + +| Path | What it serves | +| --- | --- | +| `/api/*` | REST; the full list is in the [API Reference](/api-reference) | +| `/socket.io/` | Socket.IO: `token-updated`, `token-history`, `social-graph-updated`; namespace `/notifications` | +| `/graphql` | GraphQL (introspection off in production) | +| `/api/bull-board` | Job queue dashboard for operators | + +At the reverse proxy: + +- Pass WebSocket upgrades for `/socket.io/`. +- Keep `/api/bull-board` on your internal network or behind authentication. +- Set `TRUST_PROXY` on the API to match the proxy. + +## Running it + +- **One instance.** The indexer, the cron jobs, the Socket.IO broadcasts and the payout queues + are designed for a single process. See [Scaling](/self-hosting/operations#scaling) for more. +- **Restart policy.** The process exits on an unexpected error so a supervisor restarts it + cleanly; run it with `--restart=always` or the equivalent. +- **Factory changes.** On every start the API keeps only the tokens of the factory it is + configured with. Change the factory on a fresh database, or expect the token tables to be rebuilt. + +## Related + +- [Charts & History](/self-hosting/chart-history): what the jobs above compute and when charts fill in. +- [Keys & Integrations](/self-hosting/integrations): X, push, rewards and the other feature keys. +- [Operations](/self-hosting/operations): backups, monitoring and upgrades. +- [Architecture Overview](/concepts/architecture): plugins, reorg handling and the live-update path. diff --git a/pages/self-hosting/chain-access.mdx b/pages/self-hosting/chain-access.mdx new file mode 100644 index 0000000..79cffc0 --- /dev/null +++ b/pages/self-hosting/chain-access.mdx @@ -0,0 +1,121 @@ +--- +title: "Node, Middleware & Compiler" +description: "Run the æternity node and middleware the API indexes from, and where the Sophia compiler fits." +--- + +# Node, Middleware & Compiler + +Everything Superhero shows comes from the æternity chain. The API reads it through two services: +the **node**, for chain state and broadcasting, and the **middleware** (`ae_mdw`), for indexed +transaction history and a live WebSocket feed. + +## Public endpoints or your own + +| Option | Good for | +| --- | --- | +| Public `https://mainnet.aeternity.io` and `/mdw` | Trying the stack, development, a small instance. This is the API's default. | +| Your own `ae_mdw` | Production. The first backfill reads the full transaction history from the middleware, and a local middleware makes it fast and keeps your instance independent. | + +## What the API uses + +| Service | Path | Used for | +| --- | --- | --- | +| Middleware REST | `/mdw/v3/status`, `/v3/key-blocks`, `/v3/transactions`, `/v3/contracts`, `/v3/aex9/*`, `/v3/names/*` | Backfill, token holders, contract logs, names | +| Middleware WebSocket | `/mdw/v2/websocket` (`Transactions`, `KeyBlocks`, `MicroBlocks`) | Live indexing | +| Node REST | `/v3/accounts/*`, `/v3/names/*`, dry runs, broadcasting | Balances, portfolio history, backend transactions | + +Portfolio history reads historical balances from the node, so run a full node with garbage +collection off, which is the `ae_mdw` default. + +## Run `ae_mdw` + +`aeternity/ae_mdw` runs the node and the middleware in one container: the node on port 3013, the +middleware on 4000 and the middleware WebSocket on 4001. + +The quickest route is the official installer, [`aeternity/get-aeternity`](https://github.com/aeternity/get-aeternity). +It sets up `ae_mdw` with Docker Compose and can download the node and middleware snapshots for +you. Check the `fork_management.network_id` in the generated `aeternity.yaml`: `ae_mainnet` for +mainnet, `ae_uat` for testnet. + +To set it up by hand, follow the +[middleware setup guide](https://github.com/aeternity/ae_mdw/blob/master/docs/middleware_setup.md): + +```bash +mkdir -p data/mnesia data/mdw.db && chown -R 1000 data +# Extract a full node snapshot into data/mnesia and a middleware snapshot into data/mdw.db +docker compose up -d # from the ae_mdw repository +curl -s localhost:4000/v3/status +``` + +Snapshots are published by the æternity team; `get-aeternity` downloads the current full node +snapshot (`main_v1_full_latest.tar.zst`) and middleware snapshot (`mdw_main_latest.tar.zst`). The +middleware needs the **full** node database, not the light one. + +For hardware, follow the æternity +[hardware requirements](https://github.com/aeternity/aeternity/blob/master/docs/hardware-requirements.md); +`get-aeternity` recommends a 1 TB SSD. Apply the host settings in the middleware setup guide +(open-file limit and `sysctl` values) before the first start. + +### Expose it the way the API expects + +The API addresses the node at `/`, the middleware at `/mdw` and the middleware WebSocket at +`/mdw/v2/websocket` on one host. Put a reverse proxy in front of the container: + +| Public path | Container port | +| --- | --- | +| `/` | `3013` (node) | +| `/mdw/` | `4000` (middleware REST) | +| `/mdw/v2/websocket` | `4001`, path `/v2/websocket` | + +Or point the API straight at `http://mdw-host:3013`, `http://mdw-host:4000` and +`ws://mdw-host:4001/v2/websocket` on a private network. + +### Point the API at it + +The API's chain endpoints live in `src/configs/network.ts`, one block per network: + +```ts +export const NETWORK_MAINNET: INetwork = { + url: 'https://chain.example.com', + networkId: NETWORK_ID_MAINNET, + middlewareUrl: 'https://chain.example.com/mdw', + explorerUrl: 'https://aescan.io', + compilerUrl: 'https://v8.compiler.aepps.com', + websocketUrl: 'wss://chain.example.com/mdw/v2/websocket', + name: 'Mainnet', +}; +``` + +`AE_NETWORK_ID` (`ae_mainnet` or `ae_uat`) selects the block. Rebuild the API after editing. The +web app has its own copy of these endpoints; see +[The compiled network block](/self-hosting/web-app#the-compiled-network-block). + +### Health + +`GET /v3/status` on the middleware reports `node_height`, `mdw_height` and `mdw_synced`. The +middleware is ready when `mdw_synced` is `true` and `mdw_height` is within a block or two of +`node_height`. + +## The Sophia compiler + +The web app and the API work from committed contract interfaces (ACIs) and precompiled bytecode, +so neither compiles Sophia at runtime. You need a compiler only to deploy contracts from source, +for example the DEX deploy script. + +For deployments, use `https://v8.compiler.aepps.com`, or run your own: + +```bash +docker run -p 3080:3080 aeternity/aesophia_http:v8.0.0 +``` + +## Explorer + +Transaction links go to [aeScan](https://aescan.io), which reads the same public chain. Set the web +app's `EXPLORER_URL` to use another one. aeScan is open source (`aeternity/aescan`) and can run +against your own middleware through its `NUXT_PUBLIC_*` settings. + +## Related + +- [API & Indexer](/self-hosting/api-and-indexer): what the API does with these endpoints. +- [Networks & Endpoints](/aeternity/networks): the public æternity endpoints. +- [Explorers](/aeternity/explorers): aeScan and the other explorers. diff --git a/pages/self-hosting/chart-history.mdx b/pages/self-hosting/chart-history.mdx new file mode 100644 index 0000000..5368495 --- /dev/null +++ b/pages/self-hosting/chart-history.mdx @@ -0,0 +1,85 @@ +--- +title: "Charts & History" +description: "How the API builds token candles, DEX charts, portfolio history, AE price history and rankings, and how each fills in on a new instance." +--- + +# Charts & History + +Every chart in Superhero is served by the API from data its indexer stores. There is no separate +charting service and no history import: once the indexer has the trades, the charts follow. This +page shows where each chart comes from and how it fills in on a new instance. + +## Computed when you ask + +These are built from indexed trades on every request, with a short cache, so they are complete as +soon as the trades they cover are indexed. + +| Chart | Endpoint | Built from | +| --- | --- | --- | +| Token candles (OHLCV) | `GET /api/tokens/{address}/history?interval=` | Bonding-curve trades (`transactions`) | +| Token price series | `GET /api/tokens/{address}/transactions` | Bonding-curve trades | +| Token previews and sparklines | `GET /api/tokens/preview/{address}`, `.../sparkline.svg` | Bonding-curve trades | +| DEX pair candles | `GET /api/dex/pairs/{address}/history` | DEX swaps (`pair_transactions`) | +| DEX token price | `GET /api/dex/tokens/{address}/history` | The token's deepest WAE pool | +| Portfolio and PnL | `GET /api/accounts/{address}/portfolio/history`, `.../pnl-chart.svg` | Trades, key blocks and historical balances from the node | + +The live chart updates arrive on the `token-history` Socket.IO channel the moment the indexer +stores a trade. + +Portfolio history asks the node for an account's balance at past heights, so it needs a node that +keeps historical state (a full node with garbage collection off, the `ae_mdw` default) and the key +blocks for the period, which the backfill provides. + +## Rebuilt on a schedule + +| Data | Schedule | Endpoint | +| --- | --- | --- | +| Token performance (price change, highs and lows per window) | Every 5 minutes | `GET /api/tokens/{address}/performance` | +| Token ranks | Every 5 minutes | Token list ordering | +| Leaderboard snapshots (7d, 30d, all time) | Every 10 minutes | `GET /api/accounts/leaderboard` | +| DEX token prices and pair summaries | Every 10 minutes (`DEX_PRICE_SYNC_CRON`) | `/api/dex/*/summary` | +| Trending scores | Every 2 hours for active tokens, every 6 hours for the rest, and at startup | Token list `order_by=trending_score` | +| Popular feed | Every 10 minutes, in Redis | `GET /api/posts/popular` | +| Daily analytics | Hourly | `/api/analytics/*` | + +Each of these runs at startup or on its first tick, so a new instance shows them within minutes of +the trades being indexed. Nothing needs a manual rebuild. + +## AE price and fiat values + +The API reads AE prices from the public CoinGecko API every 10 minutes; no key is needed. + +| Data | Endpoint | Source | +| --- | --- | --- | +| Current rates | `GET /api/coins/aeternity/rates` | CoinGecko simple price, stored in `coin_prices` | +| Price history, up to 365 days | `GET /api/coins/aeternity/history` | CoinGecko market chart, stored in `coin_historical_prices` | +| Market data | `GET /api/coins/aeternity/market-data` | CoinGecko markets | + +The API also ships a bundled AE price file, used until the first CoinGecko refresh completes. Allow +outbound HTTPS to `api.coingecko.com`. + +**How fiat values are stored.** AE amounts are exact, straight from the chain. When the indexer +stores a bonding-curve trade, it also stores the trade's fiat values (`buy_price.usd`, +`market_cap.usd` and the other currencies) at the AE rate of that moment. On an instance that +indexes live, that is the rate at trade time. On a fresh backfill, historical trades take the rate +from the time they were indexed. AE-denominated charts are exact either way; for fiat charts over +history, backfill early and keep the instance running. + +## How a new instance fills in + +| Time after first start | What you see | +| --- | --- | +| Seconds | Posts (pulled straight from the middleware), the factory, collections, AE rates | +| Minutes | New tokens from the factory, trending tags, token info and holders as their sync jobs run | +| As the backfill reaches each height | Trades, candles, performance, leaderboards, DEX swaps and pool charts, portfolio history | +| Backfill complete | Everything, with the live loop keeping it current | + +Watch `GET /api/mdw/health` for the backfill's progress; token history starts at the factory's +deployment height, so it completes before the walk reaches the early chain. + +## Related + +- [API & Indexer](/self-hosting/api-and-indexer): the indexer and its settings. +- [Web App](/self-hosting/web-app#charts-in-the-app): which component draws which chart. +- [WebSocket](/api-reference/websocket): the `token-history` channel. +- [Trending & Topics](/api-reference/trending): the trending-score formula. diff --git a/pages/self-hosting/chat-relay.mdx b/pages/self-hosting/chat-relay.mdx new file mode 100644 index 0000000..963f466 --- /dev/null +++ b/pages/self-hosting/chat-relay.mdx @@ -0,0 +1,127 @@ +--- +title: "Chat Relay" +description: "Run the Nostr relay behind Superhero chat, connect the API as its room admin, and point the web app at it." +--- + +# Chat Relay + +Superhero chat is Nostr. Direct messages and profiles go to any relay in the app's relay list, and +token-gated community rooms are NIP-29 groups on a relay the API manages. superhero.com runs +[`verse-pbc/groups_relay`](https://github.com/verse-pbc/groups_relay), a Rust relay with NIP-29 +groups and NIP-42 authentication, behind Caddy for TLS. + +## What the relay handles + +| Feature | Nostr kinds and NIPs | +| --- | --- | +| Profiles and direct messages | Kind `0`, kind `4` (NIP-04) | +| Authentication | NIP-42 `AUTH` (kind `22242`) | +| Room messages | Kinds `9` and `11` with an `h` tag naming the room | +| Room state | Kinds `39000` (metadata), `39001` (admins), `39002` (members) | +| Room administration by the API | Kinds `9007` create, `9002` edit metadata, `9000` add member, `9001` remove member, `9006` set roles, `9008` delete | + +Each community room is a NIP-29 group whose id is the token's sale address. The API creates the +room, keeps its member list in step with token balances and the community's moderation settings, +and the relay enforces membership. + +## Run `groups_relay` + +The repository ships a `Dockerfile` and a `compose.yml` that mount `./config` and `./db` and check +`/health`. + +```bash +git clone https://github.com/verse-pbc/groups_relay.git +cd groups_relay +docker compose up -d +``` + +Configure it with `config/settings.yml` (plus `settings..yml` and `settings.local.yml`) or +with `NIP29__relay__*` environment variables: + +| Setting | Set it to | +| --- | --- | +| `relay_secret_key` | A new secret key, in hex. This key is the relay's admin; replace the sample value in the repository. | +| `relay_url` | The public URL clients use, e.g. `wss://relay.example.com`. NIP-42 checks the client's relay tag against it. | +| `local_addr` | Listen address; default `0.0.0.0:8080` | +| `db_path` | LMDB data directory; default `/app/db` | + +```bash +NIP29__relay__relay_secret_key=<64 hex chars> +NIP29__relay__relay_url=wss://relay.example.com +``` + +The relay serves plain WebSocket on port 8080, with `/health` for probes and `/metrics` for +Prometheus. + +## TLS in front + +Put a TLS proxy in front of the relay. [Caddy](https://caddyserver.com) issues and renews the +certificate automatically and proxies WebSockets without extra configuration: + +```caddyfile +relay.example.com { + @metrics path /metrics + respond @metrics 404 + reverse_proxy 127.0.0.1:8080 +} +``` + +Keep `/metrics` on your internal network, as above. Whatever proxy you use, make sure certificate +renewal is automatic and monitored: browsers only connect to the relay over valid TLS. See +[Operations](/self-hosting/operations#tls-certificates). + +Check it from outside: + +```bash +curl -s -H 'Accept: application/nostr+json' https://relay.example.com +``` + +The response is the relay's NIP-11 document, including its `pubkey` (the admin key) and +`supported_nips` (`1, 9, 11, 29, 40, 42, 70`). + +## Connect the API + +| API variable | Value | +| --- | --- | +| `TG_RELAY_URL` | The relay's public URL, the same as `relay_url` | +| `TG_BOT_NSEC` | The relay's secret key in `nsec1…` form | +| `TG_ROOM_ADMINS` | Optional: npubs or hex keys added as admins of every room | +| `TG_BACKFILL_ON_BOOT` | `true` to create rooms for existing tokens at startup; the sweep resumes where it left off and skips rooms that exist | + +`TG_BOT_NSEC` is the same key as the relay's `relay_secret_key`: hex on the relay, `nsec` for the +API. Convert between them with any Nostr tool, for example `nak encode nsec `. At startup the +API reads the relay's NIP-11 `pubkey` and confirms it matches. + +Use the public `wss://` hostname for `TG_RELAY_URL` too, so the API's NIP-42 authentication matches +`relay_url`. + +With both variables set, the API provisions rooms and membership; with either unset, chat rooms +stay off and everything else runs as normal. Check the wiring: + +```bash +curl -s https://api.example.com/api/rooms/config +# {"relay_url":"wss://relay.example.com","admin_pubkey":"…"} +``` + +## Point the web app at it + +Set the web app's runtime `NOSTR_RELAY_URLS` to your relay (a comma-separated list is fine): + +```bash +NOSTR_RELAY_URLS=wss://relay.example.com +``` + +The server adds it to the CSP and to the client's relay list. Include the API's `TG_RELAY_URL` in +this list: the app opens community rooms only on a relay it is configured for. See +[Web App](/self-hosting/web-app#runtime-variables). + +## Back it up + +The relay keeps everything in its LMDB directory (`db_path`). Back it up with the relay stopped, or +use the `export_import` tool bundled in the image. + +## Related + +- [Chat](/social/chat): how chat works for users, DMs and rooms. +- [Moderation](/social/moderation): the community settings the API applies to rooms. +- [API & Indexer](/self-hosting/api-and-indexer): the service that manages the rooms. diff --git a/pages/self-hosting/contracts.mdx b/pages/self-hosting/contracts.mdx new file mode 100644 index 0000000..ead349b --- /dev/null +++ b/pages/self-hosting/contracts.mdx @@ -0,0 +1,129 @@ +--- +title: "Contracts" +description: "Use Superhero's public contracts or deploy your own, and where each address is configured in the API, the web app and the agent skill." +--- + +# Contracts + +Superhero's contracts are public and permissionless on æternity. A self-hosted instance can use +them as they are, so your users share posts, tokens and liquidity with everyone on Superhero, or +deploy its own for a separate community. You can mix the two: for example, your own factory for a +private token universe, with the public DEX for liquidity. + +## At a glance + +| Contract | Using the public one | Deploying your own | +| --- | --- | --- | +| Tipping v3 (posts) | Your feed and superhero.com's are the same feed | Deploy the precompiled bytecode; a separate feed | +| Community factory (tokens, DAOs, affiliation) | Same tokens, sales and DAOs as superhero.com | `deployCommunityFactory` from `bctsl-sdk`, then register collections | +| DEX (factory, router, WAE) | Shared liquidity; recommended | `dex-contracts-v2` deploy script; starts with empty pools | +| Governance registry | æternity's community-wide polls | Deploy the precompiled registry and run your own governance backend | +| AddressLink (identity links) | Read existing links | Your own contract and provider namespaces, for writes | + +The public addresses are in [Deployed Addresses](/contracts/addresses). `AE_NETWORK_ID=ae_uat` +switches the API to the testnet set. + +## Tipping v3 + +Posts are calls to `post_without_tip` on the Tipping v3 contract +([`aeternity/tipping-contract`](https://github.com/aeternity/tipping-contract), tag +`v5.0.0-alpha2`). It has no owner and an `init()` without arguments. Precompiled bytecode is in +`generated/bytecode_hashes.json`, so you can deploy it with the æternity SDK and its ACI, without a +compiler. + +| Where | Setting | +| --- | --- | +| API | `src/social/config/post-contracts.config.ts`, per network | +| Web app | Runtime `CONTRACT_V3_ADDRESS` | +| Agent skill | `CONTRACT_ADDRESS` in `superhero-post.mjs`, `superhero-comment.mjs`, `superhero-read.mjs` | + +## Community factory + +The factory deploys each community's token, bonding curve, sale, DAO and management contract in +one transaction. Its own `init` creates the AffiliationTreasury, the shared bonding curve and the +ProtocolDAO token. Source: [`superhero-com/bcl-contracts`](https://github.com/superhero-com/bcl-contracts). + +**Deploy with the SDK.** [`bctsl-sdk`](https://github.com/bctsl/bctsl-sdk) (`v1.0.0`) deploys the +precompiled factory: + +1. `deployCommunityFactory(aeSdk, protocolDaoTokenName)` deploys the factory and calls `init`. +2. `createCollection(name, allowedNameLength, allowedNameChars)` registers each collection, the + naming rules for its tokens. Registering a collection takes exactly 555 AE, which the contract + burns. See [Collections & Naming](/tokens/collections-and-naming). +3. Record the deployment key-block height and read `affiliation_treasury()` and + `protocol_dao_token()` for the addresses below. + +| Where | Setting | +| --- | --- | +| API | `BCL_FACTORY` in `src/configs/contracts.ts`: `address`, `deployed_at_block_height`, `bctsl_aex9_address` (ProtocolDAO token), `affiliation_address` (treasury), `collections` | +| Web app | Reads the factory from `GET /api/factory`; set `INVITATIONS_CONTRACT` in `src/utils/constants.ts` to your treasury | +| Agent skill | `FACTORY_ADDRESS` in `superhero-token-create.mjs` and `superhero-invite.mjs` | + +Leave `collections` as `{}` to load them from the chain; add entries to give them descriptions. The +indexer starts token history at `deployed_at_block_height`. The API keeps only the tokens of the +factory it is configured with, so set the factory before the first backfill. + +## DEX + +The DEX ([`aeternity/dex-contracts-v2`](https://github.com/aeternity/dex-contracts-v2)) is shared +by every æternity app; using the public deployment gives your users its pools and liquidity from +day one. To deploy your own, for testnet or a local chain: + +```bash +SECRET_KEY=... NETWORK_NAME=testnet FEE_TO_SETTER=ak_... \ +COMPILER_URL=https://v8.compiler.aepps.com make deploy +``` + +It deploys the pair model, the factory, WAE and the router. + +| Where | Setting | +| --- | --- | +| API | `src/dex/config/dex-contracts.config.ts`, per network | +| Web app | `DEX_FACTORY`, `DEX_ROUTER`, `DEX_WAE`, `DEX_AEETH` in the `NETWORKS` block of `src/config.ts` (build time) | + +## Governance registry + +The registry indexes polls and holds the global delegation map. It has no admin, and anyone can +register a poll. The mainnet registry is æternity's community-wide one; most instances use it. + +For a private poll set, deploy `Registry_Compiler_v6.aes` from +[`aeternity/aepp-governance`](https://github.com/aeternity/aepp-governance) +(`governance-contracts/generated/bytecode_hashes.json`) with `init()`, and run your own +[governance backend](/self-hosting/governance-backend) against it. + +| Where | Setting | +| --- | --- | +| API | `src/plugins/governance/config/governance.config.ts`: address and start height | +| Web app | Runtime `GOVERNANCE_CONTRACT_ADDRESS` | +| Governance backend | `CONTRACT_ADDRESS` | +| Agent skill | `AE_GOVERNANCE_REGISTRY` | + +## AddressLink + +Identity links (an X handle, a Nostr key, a bio, a website, a preferred `.chain` name) are written +to the AddressLink contract under **provider namespaces**. Each namespace belongs to one account, +and only that account can write under it. Superhero's backend wallet owns `x`, `nostr`, `site`, +`bio` and `prefaens` on its contract: the user signs, the backend submits. + +To offer identity links on your instance, deploy your own AddressLink contract (the Superhero team +provides it on request), then call `register_provider` for each of `x`, `nostr`, `site`, `bio` and +`prefaens` from your backend wallet. + +| Where | Setting | +| --- | --- | +| API | `ADDRESS_LINK_CONTRACT_ADDRESS` and `ADDRESS_LINK_SECRET_KEY` (the provider wallet, which also pays for the link transactions) | + +See [AddressLink](/contracts/address-link) for the entrypoints and +[Verifying Your X Account](/identity/x-verification) for the flow. + +## Social graph (optional) + +Follow lists come from a SocialContract when `SOCIAL_GRAPH_CONTRACT_ADDRESS` and +`SOCIAL_GRAPH_NETWORK_ID` are set on the API. The API checks the deployed bytecode against +`src/plugins/social-graph/aci/SocialContract.build.json` before it reads the contract. + +## Related + +- [Deployed Addresses](/contracts/addresses): the public addresses and deployment heights. +- [Calling Contracts](/contracts/calling-contracts): deploying and calling with the æternity SDK. +- [CommunityFactory](/contracts/community-factory): the factory's entrypoints. diff --git a/pages/self-hosting/governance-backend.mdx b/pages/self-hosting/governance-backend.mdx new file mode 100644 index 0000000..7eb4316 --- /dev/null +++ b/pages/self-hosting/governance-backend.mdx @@ -0,0 +1,82 @@ +--- +title: "Governance Backend" +description: "Run the æternity governance server that computes stake-weighted poll results for the /voting screens." +--- + +# Governance Backend + +The `/voting` screens read polls and their stake-weighted results from the æternity governance +server, `governance-server` in [`aeternity/aepp-governance`](https://github.com/aeternity/aepp-governance). +It watches the governance registry and every poll contract, resolves delegations, and serves the +results. It keeps a cache in Redis and recomputes everything from the chain, so it has no database +to manage. + +The public instance is `https://governance-server-mainnet.prd.service.aepps.com/`. Use it, or run +your own next to your middleware. + +## Run it + +Build the image from the repository root, because the Dockerfile copies the compiled poll contracts +from `governance-contracts/`: + +```bash +git clone https://github.com/aeternity/aepp-governance.git +cd aepp-governance +docker build -f governance-server/Dockerfile -t governance-server . + +docker run -d --restart=always -p 3010:3000 \ + -e CONTRACT_ADDRESS=ct_ouZib4wT9cNwgRA1pxgA63XEUd8eQRrG8PcePDEYogBc1VYTq \ + -e NODE_URL=https://chain.example.com/ \ + -e MIDDLEWARE_URL=https://chain.example.com/mdw \ + -e WEBSOCKET_URL=wss://chain.example.com/mdw/websocket \ + -e REDIS_URL=redis://redis:6379 \ + governance-server +``` + +`governance-server/docker-compose.yml` runs the server with its Redis in one step. + +| Variable | Required | What it sets | +| --- | --- | --- | +| `CONTRACT_ADDRESS` | yes | The governance registry. Mainnet: `ct_ouZib4wT9cNwgRA1pxgA63XEUd8eQRrG8PcePDEYogBc1VYTq`; testnet: `ct_2nritSnqW6zooEL4g2SMW5pf12GUbrNyZ17osTLrap7wXiSSjf` | +| `NODE_URL` | yes | æternity node | +| `WEBSOCKET_URL` | yes | Middleware WebSocket, v1 path (`/mdw/websocket`) | +| `REDIS_URL` | yes | Redis for the cache | +| `MIDDLEWARE_URL` | recommended | Middleware REST; with it, the server finds poll creations and votes through `/v2/txs` quickly | +| `PORT` | no | Default `3000` | +| `SHORT_CACHE_TIME`, `LONG_CACHE_TIME`, `KEEP_HOT_INTERVAL` | no | Cache tuning; defaults 120 s, 8 h, 60 s | + +Check it: + +```bash +curl -s https://governance.example.com/version +curl -s https://governance.example.com/pollOrdering +``` + +## Routes the app uses + +| Route | Returns | +| --- | --- | +| `GET /pollOrdering[?closed=true]` | The poll list for `/voting` | +| `GET /pollOverview/{poll}` | A poll with its stake-weighted results | +| `GET /votesState/{poll}` | Current votes and stakes | +| `GET /delegatedPower/{account}[?poll=]` | An account's own and delegated voting power | +| `GET /accountPollVoterAuthor/{account}` | Polls an account voted in or created | +| `POST /contractEvent` | A nudge after a vote or delegation, so the cache refreshes immediately | +| `GET /version` | Server version | + +## Point the clients at it + +| Client | Setting | +| --- | --- | +| Web app | Runtime `GOVERNANCE_API_URL`, and `GOVERNANCE_CONTRACT_ADDRESS` for the registry | +| Agent skill | `AE_GOVERNANCE_BACKEND_URL` and `AE_GOVERNANCE_REGISTRY` | + +The API indexes governance on its own as well (`/api/governance/votes`, `/api/governance/delegations`) +for history; the registry address and start height are in +`src/plugins/governance/config/governance.config.ts`. + +## Related + +- [Governance & Voting](/governance): polls, delegation and stake weighting. +- [The Governance Registry](/governance/registry): the contract this server watches. +- [Contracts](/self-hosting/contracts#governance-registry): using the shared registry or your own. diff --git a/pages/self-hosting/index.mdx b/pages/self-hosting/index.mdx new file mode 100644 index 0000000..5cc2aeb --- /dev/null +++ b/pages/self-hosting/index.mdx @@ -0,0 +1,90 @@ +--- +title: "Run Your Own Superhero" +description: "Every service behind superhero.com, what each one does, and the order to bring them up on your own servers." +--- + +# Run Your Own Superhero + +The web app, the API and the services they run with are open source, and the contracts they use +are public on æternity. You can run the whole stack under your own domain: the web app, the API and +its indexer, the chart history, the chat relay and the governance backend, on Superhero's public +contracts or your own. + +This section walks through each service: what it does, how to configure it, and how to tell it is +healthy. + +## The moving parts + +```mermaid +flowchart LR + U["Browser / PWA"] --> W["Web app
superhero"] + U -->|"REST + Socket.IO"| A["API + indexer
superhero-api"] + U -->|"Nostr wss"| R["Chat relay
groups_relay"] + U -->|"HTTPS"| G["Governance backend
governance-server"] + U -->|"sign + broadcast"| N["æternity node"] + W -->|"page metadata, sitemap"| A + A --> PG[("PostgreSQL 16")] + A --> RD[("Redis")] + A -->|"REST + WebSocket"| M["Middleware
ae_mdw"] + A -->|"room admin (NIP-29)"| R + A -->|"AE price"| CG["CoinGecko"] + G --> M + G --> RG[("Redis")] + M --- N +``` + +| Service | Repository | What it does | Page | +| --- | --- | --- | --- | +| æternity node + middleware | `aeternity/ae_mdw` | Chain access and indexed transaction history | [Node, Middleware & Compiler](/self-hosting/chain-access) | +| API and indexer | `superhero-com/superhero-api` | Indexes the chain into Postgres; serves REST, Socket.IO, notifications and rewards | [API & Indexer](/self-hosting/api-and-indexer) | +| Chart history | part of the API | Token candles, DEX charts, portfolio history, AE price history, trending scores | [Charts & History](/self-hosting/chart-history) | +| Web app | `superhero-com/superhero` | The React app and its production server | [Web App](/self-hosting/web-app) | +| Chat relay | `verse-pbc/groups_relay` | Nostr relay for direct messages and token-gated community rooms | [Chat Relay](/self-hosting/chat-relay) | +| Governance backend | `aeternity/aepp-governance` | Stake-weighted poll results for `/voting` | [Governance Backend](/self-hosting/governance-backend) | +| Contracts | several | Posts, tokens, DAOs, DEX, identity links, governance | [Contracts](/self-hosting/contracts) | +| Keys and integrations | — | X, web push, mobile push, GIFs, payout wallets | [Keys & Integrations](/self-hosting/integrations) | + +## Two ways to run it + +**Your own front end on the shared network.** Build the web app for your domain and point it at +`https://api.superhero.com`. You get your own brand and domain on day one, with the same posts, +tokens and chat as superhero.com. Only [Web App](/self-hosting/web-app) applies. + +**The full stack.** Run every service yourself. Your API indexes the chain into your own database, +so you control the data, the uptime and the features you switch on. You can keep using the public +contracts, which means your users share posts, tokens and liquidity with everyone on Superhero, or +deploy your own for a separate community. See [Contracts](/self-hosting/contracts). + +## Order of bring-up + +1. **Chain access.** Run `ae_mdw` (node and middleware in one container) from a snapshot, or start + with the public endpoints. See [Node, Middleware & Compiler](/self-hosting/chain-access). +2. **Datastores.** PostgreSQL 16 and Redis for the API, plus a Redis for the governance backend. +3. **API and indexer.** Create the database schema once, start the API, and let the indexer + backfill. See [API & Indexer](/self-hosting/api-and-indexer). +4. **Chat relay.** Run `groups_relay` behind TLS and give the API its admin key. See + [Chat Relay](/self-hosting/chat-relay). +5. **Governance backend.** See [Governance Backend](/self-hosting/governance-backend). +6. **Web app.** Build for your domain and point it at the services above. See + [Web App](/self-hosting/web-app). +7. **Keys and integrations.** Add the X app, web push keys and anything else you want to offer. See + [Keys & Integrations](/self-hosting/integrations). +8. **Operations.** TLS renewal, backups, monitoring and the launch checklist. See + [Operations](/self-hosting/operations). + +## What to expect + +- **The first backfill takes time.** The indexer walks the whole chain once. Posts and tokens + appear first through fast pulls; trades and chart history fill in as the walk reaches them. + Start from a node and middleware snapshot and run them on the same network as the API to keep + this as short as possible. +- **One API instance is the simple setup.** The indexer, crons, sockets and payout queues are + designed to run in a single process. See [Scaling](/self-hosting/operations#scaling). +- **Domain-bound features move with your domain.** Passkey wallets, the X OAuth callback and web + push are tied to the domain they run on. See [Web App](/self-hosting/web-app#your-domain). + +## Related + +- [Architecture Overview](/concepts/architecture): how the contracts, indexer and clients fit together. +- [Deployed Addresses](/contracts/addresses): the public contracts on mainnet and testnet. +- [Networks & Endpoints](/aeternity/networks): the public æternity endpoints. diff --git a/pages/self-hosting/integrations.mdx b/pages/self-hosting/integrations.mdx new file mode 100644 index 0000000..2f34fd3 --- /dev/null +++ b/pages/self-hosting/integrations.mdx @@ -0,0 +1,113 @@ +--- +title: "Keys & Integrations" +description: "The optional features of a Superhero instance and the keys that switch each one on: X, rewards, names, push, GIFs, trending tags and the agent skill." +--- + +# Keys & Integrations + +The core of Superhero (posts, tokens, trading, DAOs, charts, governance, chat) runs with the +settings from the previous pages. The features below each add one external service or one wallet, +and each switches on when its keys are set. Keep every key and private key in your secret store, +not in the repository. + +## X (Twitter) + +Linking an X account and the X reward programs use an app in the +[X developer portal](https://developer.x.com). + +| Where | Setting | What it is for | +| --- | --- | --- | +| X app | Callback URL `https://app.example.com/profile/x/callback` | The OAuth return to your web app | +| Web app (build) | `VITE_X_OAUTH_CLIENT_ID` | Shows "Connect X" and starts the OAuth flow | +| API | `X_CLIENT_ID`, `X_CLIENT_SECRET` | OAuth 2.0 with PKCE, to confirm the account the user signed in with | +| API | `X_CLIENT_SCOPE` | Default `users.read tweet.read` | +| API | `X_API_KEY`, `X_API_KEY_SECRET` | App-only access for the reward checks: profiles, followers and posts. Use an X API plan that includes user and post lookups (Basic or above). | + +Writing the link on-chain also needs an AddressLink contract with your provider namespaces; see +[Contracts](/self-hosting/contracts#addresslink). + +## Rewards programs + +The X reward programs pay AE from wallets you fund. All payouts stay off until you switch them on: +`PROFILE_REWARDS_DISABLED=false` arms payouts, and each program then needs its own flag and wallet. + +| Program | Settings | +| --- | --- | +| Onboarding reward | `PROFILE_X_ONBOARDING_REWARD_ENABLED`, `PROFILE_X_ONBOARDING_REWARD_AMOUNT_AE`, `PROFILE_X_ONBOARDING_REWARD_PRIVATE_KEY` | +| Posting rewards | `PROFILE_X_POSTING_REWARD_ENABLED`, `PROFILE_X_PERPOST_REWARD_ENABLED`, `PROFILE_X_PERPOST_REWARD_PRIVATE_KEY`, `PROFILE_X_FOLLOWER_TIERS` | +| Streak bonus | `PROFILE_X_REWARD_STREAK_BONUS_ENABLED`, `..._LENGTH`, `..._AMOUNT_AE`, `..._PRIVATE_KEY` | +| Invite milestones | `PROFILE_X_INVITE_MILESTONE_THRESHOLD`, `PROFILE_X_INVITE_MILESTONE_REWARD_AMOUNT_AE`, `PROFILE_X_INVITE_MILESTONE_REWARD_PRIVATE_KEY`, `PROFILE_X_INVITE_LINK_BASE_URL` | +| Eligibility | `PROFILE_X_REWARD_MIN_FOLLOWERS`, `PROFILE_X_REWARD_DAILY_CAP_HOURS` | + +Each `*_PRIVATE_KEY` is a hot wallet: fund it with what the program needs for the period ahead and +top it up on a schedule. Payouts run from one API instance; see +[Scaling](/self-hosting/operations#scaling). The programs are described in +[Rewards](/rewards/rewards-program). + +## `.chain` names + +`PROFILE_CHAIN_NAME_PRIVATE_KEY` is the sponsor wallet that claims `.chain` names for your users, +sets their pointer and transfers each name to the user. See [Claim Your .chain Name](/getting-started/claim-name). + +## Notifications + +| Channel | Settings | +| --- | --- | +| Browser push | `VAPID_PUBLIC_KEY`, `VAPID_PRIVATE_KEY`, `VAPID_SUBJECT` (a `mailto:` or `https:` URL). Generate the pair with `npx web-push generate-vapid-keys`. | +| Mobile push (Expo) | `EXPO_ACCESS_TOKEN`, optional; set it for your Expo project's limits | +| Notification feed | Built in; served from the API's `/notifications` Socket.IO namespace | + +The web app fetches the public VAPID key from the API, so push works on your domain as soon as the +API has the pair. Rate and retention settings (`NOTIF_*`, `WEB_PUSH_*`, `EXPO_*`) are listed in +`.env.example`. See [Notifications](/social/notifications). + +## GIFs, images and media + +| Feature | Setting | +| --- | --- | +| GIF search | `GIPHY_API_KEY` on the API; the composer searches through `/api/giphy/search` | +| Stock images | Openverse, from the browser; no key | +| Video meetings | `JITSI_DOMAIN` on the web app; default `meet.jit.si` | + +Posts store media as URLs, so there is no upload storage to run. + +## Trending tags and DEX listings + +`TRENDING_TAGS_API_KEY` (16 characters or more) protects two operator endpoints: +`POST /api/trending-tags`, where an external collector submits trending topics, and +`PATCH /api/dex/tokens/{address}/listed`, which lists a token on the DEX screens. Send it as +`x-api-key` or a bearer token. + +## Affiliation sign-in + +The invite and affiliation flows can confirm a social account with `GITHUB_CLIENT_ID` and +`GITHUB_CLIENT_SECRET`, and `GOOGLE_CLIENT_ID` and `GOOGLE_CLIENT_SECRET`, alongside X. See +[Affiliation](/rewards/affiliation). + +## AE price data + +AE rates and price history come from the public CoinGecko API with no key. Allow outbound HTTPS to +`api.coingecko.com`. See [Charts & History](/self-hosting/chart-history#ae-price-and-fiat-values). + +## The agent skill + +The [agent skill](/agents) reads three settings from the environment for governance: +`AE_NODE_URL`, `AE_GOVERNANCE_REGISTRY` and `AE_GOVERNANCE_BACKEND_URL`. Its other scripts keep +their endpoints and addresses as constants at the top of each file: + +| Constant | Scripts | +| --- | --- | +| `NODE_URL` | `superhero-post`, `-comment`, `-token-create`, `-token-swap`, `-invite`, `-name`, `-wallet` | +| `API_BASE` / `https://api.superhero.com` | `superhero-comment`, `-portfolio`, `-transactions`, `-trending`, `-token-create`, `-token-swap` | +| `MIDDLEWARE_URL` | `superhero-read`, `-name` | +| `CONTRACT_ADDRESS` (Tipping v3) | `superhero-post`, `-comment`, `-read` | +| `FACTORY_ADDRESS` | `superhero-token-create`, `-invite` | +| `INVITE_BASE_URL` | `superhero-invite` | + +Set them to your stack in your copy of the skill. + +## Related + +- [API & Indexer](/self-hosting/api-and-indexer): the core settings. +- [Operations](/self-hosting/operations): keeping wallets and keys safe in production. +- [Verifying Your X Account](/identity/x-verification): the X flow from the user's side. diff --git a/pages/self-hosting/operations.mdx b/pages/self-hosting/operations.mdx new file mode 100644 index 0000000..93b5c1d --- /dev/null +++ b/pages/self-hosting/operations.mdx @@ -0,0 +1,102 @@ +--- +title: "Operations" +description: "TLS, health checks, backups, upgrades, scaling and a launch checklist for a self-hosted Superhero." +--- + +# Operations + +## TLS certificates + +Browsers reach four of your services directly: the web app, the API (REST and Socket.IO), the chat +relay and the governance backend. Each needs a valid certificate at all times; a lapsed certificate +on the relay, for example, stops chat for every user. + +- Terminate TLS at a proxy that renews automatically, such as Caddy or Traefik, or run `certbot` + with its renewal timer enabled. +- Monitor expiry on every public hostname and alert well before the date. +- After changing a proxy or host, check each endpoint from outside: + +```bash +for h in app.example.com api.example.com relay.example.com governance.example.com; do + echo | openssl s_client -connect $h:443 -servername $h 2>/dev/null | openssl x509 -noout -enddate +done +``` + +## Health checks + +| Service | Probe | Healthy when | +| --- | --- | --- | +| Web app | `GET /robots.txt` | `200` | +| API | `GET /api/stats` | `200` and `mdwConnected: true` | +| API indexer | `GET /api/mdw/health` | `liveSync.isActive: true` and a small `liveSync.lag`; after the first backfill, `backwardSync.status: complete` | +| Middleware | `GET /mdw/v3/status` | `mdw_synced: true` | +| Chat relay | `GET /health` on port 8080, NIP-11 document on the public URL | `OK`; `pubkey` matches the API's `TG_BOT_NSEC` | +| Governance backend | `GET /version` | `200` | + +The API logs errors only by default; set `DEBUG_ENABLED=true` for info logs. It also logs a +`[StabilizationChecklist]` line every five minutes and, with chat rooms on, a `[TgrMetrics]` line +every minute, both easy to alert on. + +## Backups + +| Store | What to back up | Notes | +| --- | --- | --- | +| API PostgreSQL | Regular `pg_dump` or continuous archiving | The index rebuilds from the chain, but a restore is far faster than a full backfill | +| Chat relay LMDB | The `db_path` directory | Stop the relay or use its `export_import` tool | +| Redis (API, governance) | Optional | Queues and caches; both services rebuild them | +| Secrets | Database passwords, the relay key, VAPID pair, X credentials, every wallet key | In your secret store, with access logged | + +## Upgrades + +1. Read the release notes of each repository you run. +2. Build the new images. +3. Restart the API: it applies new migrations under a Postgres advisory lock, then starts. +4. Restart the web app. +5. Check the health endpoints above. + +If the API was down for more than about five hours, see +[After downtime](/self-hosting/api-and-indexer#after-downtime). + +## Scaling + +The API is built to run as **one instance**. In that process: + +- the indexer's backfill, live sync and reorg validation run once; +- the cron jobs (trending, leaderboards, analytics, price sync) run once; +- Socket.IO broadcasts reach every connected client; +- reward and `.chain` payouts use one transaction queue per wallet. + +Scale it vertically, and scale PostgreSQL first: most of the work is queries. The response cache +is in memory, so give the process room for it, and put a CDN or caching proxy in front of the +public, cacheable reads such as token lists and sparklines if traffic grows. + +## Security checklist + +- `/api/bull-board` reachable only from your network or behind authentication. +- The relay's `/metrics` kept internal. +- `NODE_ENV=production` on the API: Swagger stays off unless `ENABLE_SWAGGER=true`, and GraphQL + introspection is off. +- `TRUST_PROXY` matches your proxy, so rate limits apply per client. +- `ALLOWED_ORIGINS` set to your web app's origin. +- PostgreSQL and Redis on a private network, with passwords. +- Every `*_PRIVATE_KEY` and `ADDRESS_LINK_SECRET_KEY` wallet funded for its purpose only. +- A new `relay_secret_key`, not the sample from the relay repository. + +## Launch checklist + +- [ ] Middleware synced (`mdw_synced: true`) and reachable from the API. +- [ ] Database created as in [Create the database](/self-hosting/api-and-indexer#create-the-database); API healthy. +- [ ] Backfill past the factory's deployment height, so tokens, trades and charts are complete. +- [ ] Relay up behind TLS; `/api/rooms/config` shows your relay and admin key. +- [ ] Governance backend answering `/pollOrdering`. +- [ ] Web app built with your `VITE_WEBAUTHN_RP_ID`, running with your API, relay and governance + URLs; the CSP report endpoint quiet. +- [ ] Passkey sign-up, a post, a trade, a chat message and a vote tested end to end on your domain. +- [ ] Certificates on automatic renewal, with expiry monitoring. +- [ ] Database backups running, and a restore tested. + +## Related + +- [Run Your Own Superhero](/self-hosting): the overview. +- [API & Indexer](/self-hosting/api-and-indexer): settings and the HTTP surface. +- [Chat Relay](/self-hosting/chat-relay): TLS and the admin key. diff --git a/pages/self-hosting/web-app.mdx b/pages/self-hosting/web-app.mdx new file mode 100644 index 0000000..2147ba5 --- /dev/null +++ b/pages/self-hosting/web-app.mdx @@ -0,0 +1,185 @@ +--- +title: "Web App" +description: "Build the Superhero web app for your own domain, point it at your own services, and serve it with its production server." +--- + +# Web App + +The web app (`superhero-com/superhero`) is a React and Vite single-page app served by a small +Express server. The server injects runtime configuration, renders per-page titles and social +cards, sets a strict Content Security Policy and builds the sitemap. Most settings can change at +runtime through environment variables; a few are compiled in, so plan one build per domain and +network. + +## Requirements + +- Node.js `^20.19.0` or `>=22.12.0`. The Docker image uses Node 20. +- Access to `github.com` at install time: `npm ci` pulls the contract and SDK packages + (`bctsl-contracts`, `bctsl-sdk`, `dex-contracts-v2`, `tipping-contract`) straight from GitHub. +- A domain you control, with TLS in front of the server. + +## Build + +```bash +git clone https://github.com/superhero-com/superhero.git +cd superhero +npm ci +VITE_NETWORK=ae_mainnet \ +VITE_WEBAUTHN_RP_ID=app.example.com \ +npm run build +``` + +The output is `dist/`. The build also writes a static `public/sitemap.xml`; the server replaces it +at runtime with a live one. + +### Build-time variables + +| Variable | What it sets | Default | +| --- | --- | --- | +| `VITE_NETWORK` | `ae_mainnet` or `ae_uat`; selects the compiled network block | `ae_mainnet` | +| `VITE_WEBAUTHN_RP_ID` | The passkey Relying Party ID: your registrable domain | `superhero.com` | +| `VITE_SUPERHERO_API_URL` | Your API, compiled in; takes precedence over the runtime value | none | +| `VITE_NOTIFICATIONS_API_URL` | A separate API for the notification feed, socket and web push | the main API | +| `VITE_X_OAUTH_CLIENT_ID` | Your X app's OAuth client id; enables "Connect X" | none | +| `VITE_NOSTR_RELAY_URLS` | Chat relays, compiled in; meant for local development | none | + +For a deployed build, prefer the runtime variables below for the API and relays, so the CSP and +the client always agree. + +### The compiled network block + +The SDK's node, the Sophia compiler, the middleware used by some views and the DEX contract +addresses come from the `NETWORKS` block in `src/config.ts`. To use your own node, middleware or +compiler everywhere in the app, edit that block and rebuild: + +| Key in `NETWORKS` | Mainnet value today | +| --- | --- | +| `NODE_URL` | `https://mdw.wordcraft.fun` | +| `MIDDLEWARE_URL` | `https://mdw.wordcraft.fun/mdw` | +| `compilerUrl` | `https://v7.compiler.aepps.com` | +| `DEX_FACTORY`, `DEX_ROUTER`, `DEX_WAE`, `DEX_AEETH` | See [Deployed Addresses](/contracts/addresses) | + +Then add the new origins to `CONNECT_SRC_ALLOWLIST` in `server/lib/csp.cjs`, or set the matching +runtime variables (below), so the CSP allows them. `npm run check:csp-origins` checks the built +bundle against the policy. + +### Building with Docker + +The repository's `Dockerfile` builds and serves the app on port 80. It forwards three build +arguments to Vite: `VITE_WEBAUTHN_RP_ID`, `VITE_X_OAUTH_CLIENT_ID` and +`VITE_WALLET_CONNECT_PROJECT_ID`. `.env` files are not copied into the image, so add an `ARG` and +`ENV` line for any other `VITE_*` variable you want baked in, such as `VITE_NETWORK` for a testnet +image. + +```bash +docker build \ + --build-arg VITE_WEBAUTHN_RP_ID=app.example.com \ + --build-arg VITE_X_OAUTH_CLIENT_ID=your-x-client-id \ + -t superhero-web . +``` + +## Run + +```bash +docker run -d --restart=always -p 3000:80 \ + -e NODE_ENV=production \ + -e SUPERHERO_API_URL=https://api.example.com \ + -e SUPERHERO_WS_URL=wss://api.example.com \ + -e NOSTR_RELAY_URLS=wss://relay.example.com \ + -e SITEMAP_ORIGIN=https://app.example.com \ + superhero-web +``` + +Without Docker, run `node server/index.cjs` from the repository root after the build. + +### Runtime variables + +The server writes these into `window.__SUPERCONFIG__` for the client and adds their origins to the +CSP `connect-src`, so one variable covers both. + +| Variable | What it sets | +| --- | --- | +| `SUPERHERO_API_URL` | Your API: REST, the Socket.IO channels and the server's own page metadata | +| `BACKEND_URL` | Older name for the API URL; set it to the same value | +| `SUPERHERO_WS_URL` | The `wss://` origin of your API, so the CSP allows the socket | +| `NOSTR_RELAY_URLS` | Comma-separated `wss://` chat relays; see [Chat Relay](/self-hosting/chat-relay) | +| `NODE_URL`, `MIDDLEWARE_URL` | Node and middleware for the views that read runtime config | +| `GOVERNANCE_API_URL` | Governance backend; see [Governance Backend](/self-hosting/governance-backend) | +| `GOVERNANCE_CONTRACT_ADDRESS` | Governance registry contract | +| `CONTRACT_V3_ADDRESS` | Tipping v3 contract for posts and tips | +| `EXPLORER_URL` | Block explorer for transaction links | +| `JITSI_DOMAIN` | Video meetings domain; default `meet.jit.si` | +| `POPULAR_FEED_ENABLED` | `true` or `false` | +| `SITEMAP_ORIGIN` | Origin written into the live sitemap | +| `PORT` | Listen port; default `80` | + +### What the server does + +- **Page metadata.** For every HTML request it sets the title, Open Graph, Twitter card, + canonical link and JSON-LD. Posts, profiles and token pages read their data from + `SUPERHERO_API_URL`. +- **Content Security Policy.** A per-request nonce, `connect-src` built from a fixed allowlist + plus the origins of the runtime variables above, and a report endpoint at `/_csp-report`. +- **Sitemap.** `/sitemap.xml` is rebuilt from the API at startup and every six hours, using + `SITEMAP_ORIGIN`. +- **Agent-readable pages.** `/post/.md`, `/users/
.md` and `/trends/tokens/.md` + serve Markdown for crawlers and agents, and `/hubs` serves plain crawl pages. + +For a liveness probe, request a static file such as `/robots.txt`. + +## Your domain + +### Passkeys belong to their domain + +Passkeys are bound to the Relying Party ID they were created under. Build with +`VITE_WEBAUTHN_RP_ID` set to your domain, and your users create passkey wallets on your site. + +Passkey wallets created on superhero.com stay with superhero.com, and the browser keeps each +site's wallet vault in that site's own storage. Users who want the same account on your domain +bring their backup, or connect with the Superhero Wallet extension or app, which works on any +domain. + +Replace `public/.well-known/apple-app-site-association` with your own iOS app id, or remove it if +you have no app. + +### Domain strings to update + +| File | Set it to | +| --- | --- | +| `src/seo/Head.tsx` (`CANONICAL_ORIGIN`) | Your origin, for canonical and Open Graph URLs in the client | +| `public/robots.txt` | Your sitemap URL | +| `public/llms.txt`, `public/index.md` | Your site's name and links | +| `scripts/generate-sitemap.cjs` | Your origin, for the static fallback sitemap | + +### Behind a TLS proxy + +The server builds canonical and Open Graph URLs from the request protocol. Behind a reverse proxy +that terminates TLS, add `app.set('trust proxy', 1)` in `server/index.cjs` so those URLs use +`https://`. The server sends HSTS when it sees `X-Forwarded-Proto: https`. + +### Connect X + +Register `https://app.example.com/profile/x/callback` as a callback URL in your X developer app, +and build with that app's client id in `VITE_X_OAUTH_CLIENT_ID`. The client secret lives on the +API; see [Keys & Integrations](/self-hosting/integrations). + +## Charts in the app + +| Chart | Library | Data | +| --- | --- | --- | +| Token candles | lightweight-charts | `GET /api/tokens/{sale}/history`, live on the `token-history` socket channel | +| Token sparklines | SVG image | `GET /api/tokens/preview/{sale}/sparkline.svg` | +| DEX pool candles | lightweight-charts | `GET /api/dex/pairs/{pair}/history`, live on `token-history` for `PairTransaction::{pair}` | +| DEX token price | lightweight-charts | `GET /api/dex/tokens/{token}/history` | +| Portfolio and PnL | recharts | `GET /api/accounts/{address}/portfolio/history` | +| Trend cloud | visx | `GET /api/trending-tags` | + +All of it comes from your API, so the charts fill in as your indexer catches up. See +[Charts & History](/self-hosting/chart-history). + +## Related + +- [Run Your Own Superhero](/self-hosting): the overview and the order to bring things up. +- [API & Indexer](/self-hosting/api-and-indexer): the service the web app reads from. +- [Chat Relay](/self-hosting/chat-relay): `NOSTR_RELAY_URLS` and the relay behind it. +- [Using Testnet](/getting-started/testnet): building for `ae_uat`.