Add a configuration reference page

content/2.getting-started/2.installation.md

@@ -65,3 +65,14 @@ docker run -d \
 Once this has started, Caddy will issue an SSL certificate for your domain and you'll be able to immediately access the Postal web interface and login with the user you created in one of the previous steps.
 
 ![Image](/screenshots/Screen-Shot-2021-07-29-23-26-18.23-Qwv2DD40v4jMEoaHtE.png)
+
+## Next steps
+
+Once you can log in:
+
+1. Create an organization and a mail server.
+2. Add the domain you want to send from and follow the [Sending domains](/features/sending-domains) page to verify it and publish its SPF, DKIM and return path records.
+3. Create an **SMTP** or **API** [credential](/features/smtp-authentication) and send a test message.
+4. Point the `smtp` section of `postal.yml` at your new mail server so that Postal can send its own notification e-mails, then `postal restart`.
+
+The full set of commands provided by the `postal` helper is documented on [The postal command](/getting-started/postal-command) page.

content/2.getting-started/4.dns-configuration.md

@@ -99,7 +99,7 @@ You may wish to replace <code>~all</code> with <code>-all</code> to make the SPF
 
 ## Return Path
 
-The return path domain is the default domain that is used as the `MAIL FROM` for all messages sent through a mail server. You should add DNS records as below.
+The return path domain is the default domain that is used as the `MAIL FROM` for all messages sent through a mail server, so bounces and auto-responses are delivered here. It is also the domain Postal signs messages with (using your `signing.key`) whenever a sending domain does not yet have a working DKIM record of its own. You should add DNS records as below.
 
 <table>
   <thead>
@@ -133,15 +133,15 @@ The return path domain is the default domain that is used as the `MAIL FROM` for
     <tr>
       <td>postal._domainkey.rp.postal.example.com</td>
       <td>TXT</td>
-      <td>Value from <code>postal default-dkim-record</code></td>
+      <td>Value from <code>postal default-dkim-record</code> (the <code>postal</code> selector is the value of <code>dns.dkim_identifier</code>)</td>
     </tr>
   </tbody>
 </table>
 
 
 ## Route domain
 
-If you wish to receive incoming e-mail by forwarding messages directly to routes in Postal, you'll need to configure a domain for this just to point to your server using an MX record.
+If you wish to receive incoming e-mail by forwarding messages directly to routes in Postal (each route has a unique `{token}@routes.postal.example.com` address - see [Routing incoming e-mail](/features/routing-incoming-email)), you'll need to configure a domain for this just to point to your server using an MX record.
 
 <table>
   <thead>
@@ -186,6 +186,10 @@ If you would like to make use of Click and Open Tracking then you should set up
   </tbody>
 </table>
 
+## Records for each sending domain
+
+The records above are for the Postal installation itself. Each domain you send mail *from* additionally needs its own SPF, DKIM and (optionally) return path and MX records, which Postal generates for you and checks automatically. These are described on the [Sending domains](/features/sending-domains) page.
+
 ## Example Postal Configuration
 
 In your `postal.yml` you should have something that looks like the below to cover the key DNS records.

content/2.getting-started/8.postal-command.md

@@ -0,0 +1,83 @@
+---
+title: The postal command
+description: 'Reference for the postal helper command installed from the postalserver/install repository.'
+category: Installation
+---
+
+The `postal` command used throughout these docs is a small Bash script from the [installation helper repository](https://github.com/postalserver/install), which the [pre-requisites](/getting-started/prerequisites) page has you clone to `/opt/postal/install` and symlink to `/usr/bin/postal`. It wraps Docker Compose so that you don't need to remember the container commands.
+
+All commands operate on a `docker-compose.yml` in `/opt/postal/install`, which is generated for you from a template (`templates/docker-compose.v3.yml`) the first time you run `bootstrap`, `upgrade` or any command that needs it. Docker Compose is run with the project name `postal`, so the containers are named `postal-web-1`, `postal-smtp-1`, `postal-worker-1` and so on.
+
+## Running Postal
+
+| Command | Description |
+|---|---|
+| `postal start` | Start all services in the background (`docker compose up -d`). Extra arguments are passed through, e.g. `postal start web`. |
+| `postal stop` | Stop and remove the containers (`docker compose down`). |
+| `postal restart` | Restart all containers. Required after changing `postal.yml`. |
+| `postal status` | Show the running containers (`docker compose ps`). |
+| `postal logs [service]` | Show logs for all services or one of `web`, `smtp`, `worker`. Extra arguments are passed through, e.g. `postal logs -f worker`. |
+| `postal bash [service]` | Open a shell inside a running service container. |
+| `postal dc [args]` | Run any other `docker compose` command against the Postal project, e.g. `postal dc pull`. |
+
+## Setup and upgrade
+
+| Command | Description |
+|---|---|
+| `postal bootstrap hostname [path]` | Create initial configuration in `path` (default `/opt/postal/config`): `postal.yml` from the example file with your hostname and a random `rails.secret_key` filled in, a `Caddyfile`, and a 1024-bit RSA `signing.key`. Existing files are never overwritten. Also generates `docker-compose.yml` for the latest release. |
+| `postal initialize` | Pull the image and run `postal initialize` inside a temporary container to create the main database and load the schema. |
+| `postal upgrade [version]` | Upgrade to the given version (or the latest release). See [Upgrading](/getting-started/upgrading). |
+| `postal upgrade-db` | Run database migrations only, without pulling a new image or restarting. Useful after restoring a database from another installation. |
+| `postal set-version x.x.x` | Regenerate `docker-compose.yml` for a specific version without pulling or restarting anything. |
+
+## Other tools
+
+These run the corresponding command from the [container image](/other/containers#other-commands) in a temporary `runner` container.
+
+| Command | Description |
+|---|---|
+| `postal make-user` | Interactively create a global administrator user (prompts for e-mail address, first name, last name and password). |
+| `postal default-dkim-record` | Print the DKIM TXT record to publish at `postal._domainkey.{return path domain}`. |
+| `postal test-app-smtp address` | Send a test e-mail using the `smtp` section of your configuration. |
+| `postal console` | Open a Rails console against your installation. |
+| `postal version` | Print the version of Postal in the configured image. |
+
+## Options
+
+| Option | Description |
+|---|---|
+| `--version x.x.x` | With `bootstrap` or `upgrade`, use this version instead of looking up the latest release on GitHub. |
+| `--no-git-pull` | With `upgrade`, skip updating the helper repository first. |
+| `--dev` | Print the commands that would be run instead of running them. |
+
+Looking up the latest release requires `curl` and `jq` and makes an unauthenticated request to the GitHub API, which is rate limited. If you hit the limit, pass an explicit version.
+
+## Customising the installation
+
+### Number of workers
+
+To run more than one worker container, add a `docker-compose.override.yml` (see below) that sets the number of replicas for the `worker` service. This survives upgrades, whereas `postal start --scale worker=3` is reset to one worker the next time `postal upgrade` runs.
+
+```yaml
+services:
+  worker:
+    deploy:
+      replicas: 3
+```
+
+### Overriding the compose file
+
+Docker Compose automatically merges `docker-compose.override.yml` from the same directory. Use this for changes such as [log drivers](/features/logging#redirecting-logs-to-the-host-syslog), extra environment variables or resource limits, so that they survive upgrades (the generated `docker-compose.yml` is replaced on every upgrade).
+
+### Hooks
+
+If a `hooks` directory exists in `/opt/postal/install`, the Bash script with the matching name in it is run at various points. Available hook names are `pre-start`, `post-start`, `pre-stop`, `post-stop`, `pre-restart`, `post-restart`, `pre-initialize-pull`, `pre-initialize`, `post-initialize`, `pre-upgrade-pull`, `post-upgrade-pull`, `pre-upgrade-db`, `post-upgrade-db`, `post-upgrade`, `pre-bootstrap`, `post-bootstrap` and `set-postal-version`.
+
+## The generated compose file
+
+For reference, the v3 template starts the following services, all using host networking and mounting `/opt/postal/config` at `/config`:
+
+* `web` - `postal web-server`
+* `smtp` - `postal smtp-server` with the `NET_BIND_SERVICE` capability
+* `worker` - `postal worker`
+* `runner` - a `tools` profile service used by the helper for one-off commands; it is not started by `postal start`

content/3.features/mail-server-settings.md

@@ -0,0 +1,101 @@
+---
+title: Mail Server Settings
+description: 'Modes, limits, retention, held messages, the suppression list and other per-server settings.'
+category: Features
+---
+
+Each mail server within an organization has its own settings, found under **Settings** in the server menu. Some settings are only visible to [global administrators](/features/users-and-permissions) under **Advanced Settings**.
+
+## Server settings
+
+| Setting | Description |
+|---|---|
+| **Name** | A display name, unique within the organization. |
+| **Permalink** | A short identifier (letters, digits and hyphens) used in SMTP usernames (`org-permalink/server-permalink`) and URLs. It cannot be changed after the server is created. |
+| **Mode** | `Live` or `Development`. See below. |
+| **IP pool** | Only shown when [IP pools](/features/ip-pools) are enabled. The pool that outgoing mail from this server is sent from unless an IP pool rule matches. |
+| **Postmaster** | The contact address included in bounce messages Postal generates when an incoming message cannot be delivered. Defaults to `postmaster@` the message's domain. |
+
+### Live and Development mode
+
+In **Live** mode all mail is routed normally. In **Development** mode every outgoing and incoming message is placed in the held queue with the note "Server is in development mode." instead of being delivered to recipients or endpoints. Messages are still parsed, inspected and visible in the web interface, and count towards the server's send limit. Individual held messages can be released manually from the web interface, which delivers them despite the mode.
+
+If you only want to hold messages from a particular application or environment rather than the whole server, set the **hold** option on that application's [credential](/features/smtp-authentication#holding-messages-from-a-credential) instead.
+
+## Spam
+
+The **Spam threshold** and **Spam failure threshold** for incoming mail are set here. See [Spam & Virus Checking](/features/spam-and-virus-checking#classifying-spam).
+
+## Retention
+
+Each server has three retention settings which are shown on the **Retention** page and can be changed by a global administrator under **Advanced Settings**. They are enforced by a background task which runs once a day at 03:00 (server time, normally UTC).
+
+| Setting | Default | Description |
+|---|---|---|
+| **Raw message retention days** | 30 | How many days the raw content of messages (headers, bodies and attachments) is kept. Raw data is stored in one table per day, and whole days are removed once they are older than this. After removal the message still appears in lists and searches but its content and attachments can no longer be viewed, and a queued message whose raw data has been removed will fail with "Raw message has been removed". |
+| **Raw message retention size** | 2048 MB | The total disk space raw message data may use. When exceeded, whole days are removed starting with the oldest until usage is under the limit. |
+| **Message retention days** | 60 | How many days message metadata (the message record itself, its deliveries, clicks, loads and spam checks) is kept. Older messages are deleted entirely. |
+
+Leaving any of these blank disables that limit ("Indefinitely" / "No limit"). The **Retention** page also shows the current total size of the server's message database.
+
+## Send limit
+
+A global administrator can set a **Send limit** for a server under **Advanced Settings**. This is the maximum number of outgoing messages accepted in a rolling 60 minute window; the current usage is shown on the **Send Limit** page. Incoming messages are counted but not limited.
+
+* When the volume reaches **90%** of the limit, the server is marked as *approaching* its limit.
+* When the volume reaches the limit, every further outgoing message is held with the note "Message held because send limit (N) has been reached." until the volume drops. Releasing a held message while the server is still over the limit holds it again.
+
+Once a minute Postal checks for servers that have recently approached or exceeded their limit and, at most once per hour for each state, e-mails every user in the organization and triggers the `SendLimitApproaching` / `SendLimitExceeded` [webhook events](/developer/webhooks#send-limit-events). The e-mails are sent using the `smtp` section of your Postal configuration.
+
+## Held messages
+
+Messages that are held are not delivered but remain visible under **Messages → Held** where they can be released (re-queued for delivery) or the hold cancelled. Each held message triggers a `MessageHeld` webhook. A message may be held for any of the following reasons:
+
+| Reason | Details recorded | Can be released manually? |
+|---|---|---|
+| Server is in Development mode | "Server is in development mode." | Yes |
+| Credential is set to hold | "Credential is configured to hold all messages authenticated by it." | Yes |
+| Recipient is on the suppression list | "Recipient (…) is on the suppression list (reason: …)" | Yes |
+| Send limit reached | "Message held because send limit (…) has been reached." | Only once the volume has dropped below the limit |
+| Server or organization is suspended | "Mail server has been suspended…" | No - it will be held again until unsuspended |
+| Incoming spam on a route set to Quarantine | "Message placed into quarantine." | Yes |
+| Incoming mail on a route set to Hold | "Message has been accepted but not sent to any endpoints." | Yes (marked as Processed) |
+
+Held messages expire after the number of days set by `postal.default_maximum_hold_expiry_days` (default **7**). An hourly task cancels the hold on expired messages, recording a `HoldCancelled` delivery with the note "The hold on this message has been removed without action." The message is not delivered.
+
+## Suppression list
+
+Each server maintains a suppression list of recipient addresses that Postal will not deliver to, viewable under **Messages → Suppressions**. Addresses are added automatically when Postal is sending **outgoing** mail:
+
+* **Too many hard fails** - a permanent (`5xx`) rejection from the recipient's mail server when there has already been at least one other hard fail to the same address in the previous 24 hours.
+* **Too many soft fails** - the message has been retried the maximum number of times (`postal.default_maximum_delivery_attempts`, default 18) without success.
+
+Incoming bounce messages do **not** add addresses to the list.
+
+While an address is on the list, new messages to it are held (see above) rather than attempted. Releasing such a message manually bypasses the list, and if the delivery then succeeds the address is removed from the list. Entries are otherwise removed automatically after `postal.default_suppression_list_automatic_removal_days` (default **30**) days.
+
+## Advanced settings (administrators only)
+
+| Setting | Description |
+|---|---|
+| **Send limit** | See above. |
+| **Allow sender header** | Permits any `From` address as long as a `Sender` header contains an address on one of the server's verified domains. See [From/Sender validation](/features/smtp-authentication#fromsender-validation). Applies to SMTP and the API. |
+| **Privacy mode** | When enabled, the `Received` header Postal adds to submitted messages omits the submitting client's IP address, reverse DNS hostname and HELO name, leaving only `by {hostname} with SMTP/HTTP; {date}`. |
+| **Log SMTP data** | Log the full content of messages submitted using this server's credentials in the SMTP server log. Debugging only. See [Logging](/features/logging#smtp-server-logging). |
+| **Outbound spam threshold** | Enables spam scanning of outgoing messages; messages scoring at or above this are failed. Blank disables outbound scanning. See [Spam & Virus Checking](/features/spam-and-virus-checking). |
+| **Message retention days**, **Raw message retention days**, **Raw message retention size** | See [Retention](#retention). |
+
+### Suspending a server
+
+An administrator can **suspend** a server from **Advanced Settings** by entering a reason. While suspended:
+
+* Every message processed for the server (incoming and outgoing) is held with the reason "Mail server has been suspended".
+* The SMTP server rejects `RCPT TO` for the server's routes and credentials with `535 Mail server has been suspended`.
+* API requests using the server's credentials return the `ServerSuspended` error.
+* All users in the organization are e-mailed about the suspension.
+
+Organizations also carry a suspension flag which suspends all of their servers at once; there is no interface for this, but it can be set from `postal console` (`Organization.find_by(permalink: "my-org").update(suspended_at: Time.now)`). Use **Unsuspend server** to restore normal operation; held messages must then be released manually.
+
+## Deleting a server
+
+Deleting a server (**Settings → Delete**, which requires typing the server's name to confirm) marks it as deleted immediately and hides it from the interface. The server's message database is dropped by an hourly background task shortly afterwards. This cannot be undone.