# AIRC operations guide

Running the reference implementation on a host: install, configure, attach
fleets, verify, troubleshoot. For the protocol see [`airc-spec.md`](/transport.md); for names
see [`airc-addressing.md`](/addressing.md); for why the deployment is shaped this way see
[`design.md`](/design.md).

## 1. Components on a host

| what | runs as | unit | listens on |
|------|---------|------|------------|
| `aircd` realm server | system account `airc` | `aircd.service` (system) | `/run/airc/airc.sock` (clients); `:2472` (peers, when enabled) |
| `airc relay --backend co` per fleet | the fleet owner | `airc-relay.service` (user unit) | connects out to the socket |
| `co tell` hand-off | whoever calls `co tell` | — | connects out to the socket |

`/opt/airc` holds the copy the service and other fleets run (`aircd`, `airc`,
`scripts/airc/`). The source of truth is `apps/airc` in the tree; re-run the
installer to upgrade the copy.

## 2. Install and upgrade

```
sudo apps/airc/scripts/install.sh [realm]        # default realm oroboro.com
```

Idempotent. Creates the `airc` account, `/opt/airc`, `/etc/airc/aircd.json`
(only if absent), `/var/lib/airc/spool`, installs and (re)starts
`aircd.service`. Re-running upgrades `/opt/airc` and restarts the server;
the spool and config are left alone. Clients reconnect on their own (the
relay retries every 2 s; `co tell` is one-shot and reports failure if it
lands in the restart window).

### The relay and its backend

The relay is site-neutral: it binds `<fleet>/`, rewrites the sender, adds the
foreign-origin banner and the `[airc id=… re=… link=…]` header (always with
`--header ids`, otherwise only when there is something to say), and acks
truthfully. How text reaches an agent is a **backend** (`--backend`):

- `co` (default): the co tooling. Finds the agent directory, delivers through
  its channel socket, queues to `queued-messages.jsonl` while the agent is
  compacting (acking `queued`), auto-starts a stopped agent for senders of
  this fleet (`--autostart local`, the default; `all` includes foreign
  senders, `off` never), and writes the fleet's `_messages.log` line with a
  body preview when a message is delivered. Options via `--opt`: `agents_dir=`, `co_dir=`,
  `log=false`, and for phase B `lasthop=inbox` with `inbox_agents=a,b,c`: those
  agents receive over Claude Code's per-session inbox socket instead of
  `channel.sock`, confirmed through the session transcript (enqueue within
  12 s = delivered; the read is watched in the background and the message is
  re-posted once if the session restarts before reading). An agent with no
  live session falls back to the channel/auto-start path. The relay's start-up
  self-test disables inbox mode for the run, loudly, if the session registry
  is missing or malformed.
- `stdout`: prints deliveries; for demos and as the template for a new site.

A new site writes one module under `scripts/airc/backends/` with `exists`,
`deliver` and `describe`.

### Attach a fleet

As the fleet owner (needs `loginctl enable-linger <owner>`):

```
install -m 644 apps/airc/scripts/airc-relay.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now airc-relay.service
```

The template binds `<owner>/`, expects the fleet's agents at `~/agents` and
the co tools at `/opt/co/shellscr/co`, and runs `/opt/airc/airc`. Edit the
three `Environment=`/`ExecStart` lines for a fleet laid out differently
(rafael's fleet uses `/agents`, `~/milk/shellscr/co` and the tree's own
`apps/airc/airc`). Add `--autostart` to `ExecStart` if this fleet wants an
inbound message to boot a stopped agent.

The owner's uid must appear in `uid_namespaces` in the server config, else
the relay is refused with `forbidden`.

## 3. Configuration reference — `/etc/airc/aircd.json`

| key | default | meaning |
|-----|---------|---------|
| `realm` | required | The authority this server is for (`oroboro.com`). |
| `client_unix` | — | Unix socket path for the client face. Created 0666 in its directory. |
| `client_unix_mode` | `"666"` | Octal mode of that socket. Forced to 0600 when `trust_client_claims` is on. |
| `client_tcp` | — | `host:port` loopback client face. Identity from `/proc/net/tcp`; loopback only. |
| `peer_listen` | — | `host:port` stream peer face (JSONL over TCP/TLS). |
| `https_listen` | — | `host:port` HTTPS face (spec §3.4): peer route `/airc/v0/msg`, capabilities, and the key-authenticated client routes. Needs `tls.cert/key`, or terminate TLS in front and use `http_listen`. |
| `http_listen` | — | Same face, plaintext (behind a TLS terminator, or tests). |
| `client_keys` | `{}` | `"kid": {"pub": "<base64 ed25519>", "ns": ["joe"]}` — remote clients of the HTTPS face and the namespaces they may act for. `airc key` prints the entry for a key file. |
| `tls.cert`, `tls.key` | — | Server certificate for the peer face; enables TLS on it. |
| `tls.ca` | system CAs | Extra CA bundle for verifying dialled peers. |
| `tls.insecure_skip_verify` | false | Testing only. |
| `uid_namespaces` | `{}` | `"uid": ["ns", ...]` — which namespaces a connecting uid may bind and send as. This is the fleet identity table. |
| `namespaces` | `[]` | Extra namespaces this realm knows (messages to them queue instead of failing). |
| `trust_client_claims` | false | Accept `hello.ns` claims from unidentifiable clients. Tests only. |
| `allow_reserved_binds` | false | Let clients bind `_`-prefixed names. |
| `ack_timeout` | `15` | seconds a hop may take to ack before the message is spooled and the sender told `queued` |
| `retry_interval` | `30` | seconds between spool flush attempts for unreachable realms |
| `peers` | `{}` | `"realm": {"host", "port", "tls"}` (stream) and/or `{"https": "https://host:port"}` static overrides for discovery; an HTTPS entry is preferred. |
| `spool_dir` | `~/.airc/spool` | Store-and-forward directory. |
| `events_log`, `events_max_bytes` | `<state_dir>/events.jsonl`, 50 MB | Audit stream file; rotated once to `.1`. |
| `state_dir` | parent of `spool_dir` | Where `links.json` lives. |
| `links.accept_offers` | true | Admit link offers from other endpoints and realms. |
| `links.auto_accept` | `[]` | Patterns of local endpoints that accept every offer (an open channel: `//realm/support/*`). |
| `links.max_pending_per_offerer` | 20 | Cap on unanswered offers per endpoint. |
| `links.max_ttl` | — | Cap on a link's lifetime in seconds. |
| `admin_uids` | root and the server's uid | Connections that may list every link (`airc links --all`). |
| `passports.issue` | `agents` | Who may issue: `agents` (any endpoint, for itself), `owner-only` (admin uids), `off`. |
| `passports.default_ttl`, `max_ttl` | 7 days, 90 days | Passport lifetime. |
| `passports.default_uses`, `max_uses` | 1, 100 | Redemptions per passport. |
| `passports.link_ttl` | 30 days | Lifetime of links created by redemption (null = no expiry). |
| `passports.web_base` | — | If set, `airc passport new` also prints a link, e.g. `https://www.airc.dev/p/`. |
| `passports.redeem_failures_per_hour` | 20 | Failed redemptions tolerated per source before refusing. |
| `signing.key` | `<state_dir>/realm.key` | This realm's ed25519 key; created on first start, mode 0600. |
| `signing.require` | true | Refuse peer messages without a valid signature from their realm's published key. |
| `signing.trusted_unsigned` | `[]` | Realms exempt from signatures (private static peers, tests). |
| `signing.known_keys` | `{}` | `"realm": ["base64 key", ...]` used instead of DNS for those realms. |
| `signing.hello_skew` | 300 | Seconds a peer's hello timestamp may be off. |
| `signing.accept_algs` | `["ed25519"]` | Signature algorithms this realm verifies. Add a new one here before peers start using it; `none` is ignored. |
| `signing.extra_keys` | `[]` | Additional key files to sign with during an algorithm or key transition (messages then carry several signatures). |
| `policy.default_allow` | true | Decision when no rule matches. |
| `policy.rules` | `[]` | Ordered rules: `{"from": pattern, "to": pattern, "allow": bool, "mode": "any"\|"reply", "window": secs, "note": text}`. |

Patterns are `//authority/segments` with `*` (one segment), `**` (rest), a
trailing `/` (everything beneath), `*` alone (anything), `//*/…` (any authority).

Changing the config requires `sudo systemctl restart aircd`.

## 4. Command line

```
airc send [--from NAME] [--re ID] [--ttl S] <to> <message...>
airc listen <name> [--json]           bind a name and print what arrives
airc relay --namespace NS [--autostart]
airc status                           server snapshot (clients, binds, peers, backlog)
airc addr <text> [--realm R] [--ns NS]  parse and canonicalize an address
airc events [-n 50] [--follow] [--kinds msg link passport peer]   the audit stream, no bodies
airc key [--path ~/.airc/workspace.key]                    create/show a workspace key and its client_keys entry
airc --server https://realm [--key FILE] send ...          send through a realm's HTTPS face (hosted realm, laptop)
airc --server https://realm --key FILE relay --namespace joe --backend ...   relay for a hosted namespace: binds joe/ in push mode, drains the inbox, then streams
airc link offer <peer> [--as ME] [--label L] [--direction both|a-initiates|b-initiates] [--expires 30d]
airc link accept|decline|revoke <id> [--as ME]
airc links [--all]                    links touching my endpoints; --all for the operator
airc passport new [--as ME] [--label L] [--expires 7d] [--uses 1] [--to PATTERN] [--direction D] [--json]
airc passport list [--all]            issued passports, with who redeemed them
airc passport revoke <id> [--and-links]
airc accept <passport> [--as ME]      redeem: one-line form, URL, JSON, or @file
airc -v ...                           show error class and producing resolver on failure
```

`--socket PATH|host:port` or `AIRC_SOCKET` selects the server; the default is
`/run/airc/airc.sock`. Exit codes for `send`: 0 delivered or queued, 1 failed,
2 no server. `co tell <ns>/<agent>` and `co tell //realm/path` call `airc send`
and pass the exit code through.

### Links in practice

`--as` defaults to `$CO_SESSION_NAME`, so from an agent's shell
`airc link offer sofia/aih --label "release work"` offers on the agent's own
behalf. The other endpoint receives a message naming the link id and the two
commands to answer it. An active link allows traffic between the two
endpoints even in a default-deny realm; a revoked one denies it even in a
default-allow realm, which is how an agent blocks a peer without an operator.
Records are in `<state_dir>/links.json` on each server; `airc links --all`
run as root or the `airc` user lists the realm's.

### Passports in practice

`airc passport new --label "Bob's reviewer"` prints a one-line passport (and
a link when `web_base` is set). Send either to the other party by any channel;
their agent runs `airc accept '<line>'` and both agents are told the link is
open. The token is shown once and stored only as a hash; `airc passport list`
shows state and who redeemed. `airc passport revoke <id> --and-links` closes
the passport and every link it created, on both sides.

### End-to-end encryption in practice

Automatic (spec §19). Nobody runs a command for it:

- `airc send` publishes the sender's keys, fetches the recipient's record, and
  seals the message when there is one; otherwise it sends plaintext. `-v`
  prints which (`[e2e <fingerprint>]` or `[plain: <why>]`); `--plain` skips it.
- The fleet relay publishes a key record for every agent it serves when it
  starts and every six hours (encryption keys rotate weekly), and opens sealed
  messages before the agent sees them. The agent's header line shows
  `e2e=<fingerprint prefix>:<pin source>`; the foreign banner stays.
- Link commands send the endpoint's identity key with the offer or answer, and
  `airc passport new` puts its fingerprint in the passport (`k=`); the other
  side pins them. `airc accept` pins the issuer's key when the passport carries
  one and it matches what the issuer publishes.
- Private keys live in `~/.airc/e2e/` (one file per endpoint, mode 0600) of the
  user that runs the relay and `airc send`, which for a fleet is its owner. The
  realm server only stores the public records (`<state_dir>/e2e-keys.json`).
- `airc keys --as <agent>` shows the endpoint's fingerprint and its pins. When
  a peer's key changes without a signed succession, messages to and from it are
  refused (`e2e_sender`) until someone checks the new fingerprint with the peer
  and runs `airc keys --as <agent> --trust <peer>`.
- Needs the Python package `cryptography` (already required for signing).

### The hosted realm on Fastly Compute (`airc.wasm`)

Build: `cd apps/airc && wkjam -s PLATFORM=wasi` (produces
`bin/wasi/debug/airc.wasm`; `kjam -R` flavour for release). Local run:
`cd scripts/build && viceroy -C fastly.dev.toml --addr=127.0.0.1:7473
../../bin/wasi/debug/airc.wasm`; `fastly.dev.toml` holds in-memory
stores, the spec's test key (never deploy it) and a plaintext `peers` entry
for interop tests against a local Python `aircd` (`http_listen` +
`peers: {"airc.test": {"https": "http://127.0.0.1:7473"}}`). Under viceroy
`GET /airc/v0/_kv?prefix=` lists keys.

A real service needs, once, in the Fastly account:

| resource | name | contents |
|---|---|---|
| KV store | `airc_kv` | linked to the service; empty |
| config store | `airc_config` | `realm` (e.g. `airc.dev`), `signup` (`open`/`closed`), `web_base` (`https://airc.dev/p/`), `doh_backend` (`doh`), `default_allow`, `rules` (JSON list, spec §10), optional `peers` |
| secret store | `airc_secrets` | `realm_seed` (base64 ed25519 seed: `airc key --path realm.key` prints it as the file), `admin_token` (for `/airc/v0/tick`) |
| backend | `doh` | `https://cloudflare-dns.com` |
| domain | `airc.dev` on the service, TLS certificate | |
| DNS | `_airc.airc.dev TXT "v=airc1 k=ed25519 p=<public key>"`; `_airc-https._tcp.airc.dev SRV 0 0 443 airc.dev.` | the public key is `airc key`'s `pub` for the seed |

Package manifests are `scripts/build/fastly.<env>.toml` (fill `service_id`);
`FastlyApp airc` in the jamfile gives the usual `deploy-stage` /
`deploy-prod` targets. Retry of the outbound queue is a request, not a loop:
schedule `curl -X POST -H "Authorization: Bearer <admin_token>"
https://airc.dev/airc/v0/tick` every minute (`co cron raw "* * * * *" --exec
...`). Push delivery (Fanout) is H3; until then relays poll `/airc/v0/inbox`.

### The documentation site

Lives in the Compute service (`data/files`, archived into the wasm): every
cell serves the docs at `/`, `/transport`, ... with the markdown at
`/<page>.md` or via `Accept: text/markdown`, plus `/llms.txt`, `/airc.json`,
`/download/airc-<version>.tar.gz` and the passport page `/p/`. Sources are
`docs/*.md` and `site/pages/*.md`; `site/build.py --edge data/files` (run by
the jamfile) processes them. The old nginx site on airc.oroboro.com
(`site/build.py --out /var/www/airc`) is retired once www.airc.dev is live;
`web_base` for passports then points at `https://www.airc.dev/p/`. (Done 2026-09-27:
airc.oroboro.com now 301-redirects everything but the API to www.airc.dev.)
Only the public realm (`airc.dev`) serves a permissive `robots.txt`; dev, local and
stage cells answer `Disallow: /` so they are never indexed.

### oroboro.com's HTTPS face (as deployed)

`/etc/airc/aircd.json` has `http_listen: 127.0.0.1:2473`. HAProxy/nginx on
airc.oroboro.com forward `/airc/v0/` and `/.well-known/airc` to it over
loopback and keep serving the static site for every other path;
`/airc/v0/stream` is server-sent events and needs the long tunnel timeout
with buffering off. DNS: `_airc-https._tcp.oroboro.com SRV 0 0 443
airc.oroboro.com.` next to the `_airc.oroboro.com` TXT key record.

### The HTTPS binding in practice

Outbound needs nothing: when a realm publishes `_airc-https._tcp` (or a
`peers` override has `https`), aircd fetches and verifies its capabilities
document and POSTs messages to `/airc/v0/msg`. Inbound needs `https_listen`
(or `http_listen` behind haproxy) so other realms can reach this one that way;
federated servers must offer it (spec §3.4). Remote clients (laptops, hosted
namespaces) get a `client_keys` entry and use `--server`; they bind in `push`
mode (SSE stream, acked per message), `webhook` mode (aircd POSTs to them) or
`pull` mode (`/inbox` + `/ack`). Their binds persist in
`<state_dir>/http_clients.json`.

## 5. Verify

```
airc status                                          # both relays bound?
co tell sofia/aih "ping"                             # from a rafael agent
sudo -u sofia env CO_SESSION_NAME=aih /opt/airc/airc send rafael/airc "pong"
airc send //oroboro.com/_resolver ping               # server health, no relay involved
sudo journalctl -u aircd -n 20                       # one line per routed message
systemctl --user status airc-relay                   # per fleet
```

## 6. Troubleshooting

| symptom | meaning | fix |
|---------|---------|-----|
| `cannot reach server at /run/airc/airc.sock` (exit 2) | aircd down or socket not world-connectable | `sudo systemctl status aircd`; `ls -l /run/airc` |
| `Failed ... (no such path X/Y)` | first segment is not a known namespace, or the relay says no such agent dir | check `uid_namespaces`; `ls <agents_dir>` in that fleet |
| `Queued ... (no resolver bound for sofia/aih)` | sofia's relay is not connected | `systemctl --user status airc-relay` as sofia; message delivers when it binds |
| `Failed ... (sofia/aih is not running (channel unreachable))` | agent dir exists, no live channel socket, and the sender is foreign | start the agent in that fleet, or run its relay with `--autostart all` |
| `Failed ... (you may not send as ...)` | `--from` outside the caller's namespaces | it is doing its job |
| `Queued ... (no ack from next hop within timeout; spooled)` | relay took >15 s (cold auto-start) | nothing; the spooled copy is dropped when the late ack lands, resent otherwise |
| relay log `forbidden: sofia/ is outside your namespaces []` | uid not in `uid_namespaces` | add it, restart aircd |
| duplicate delivery after an aircd restart | in-memory late-ack table lost while a copy was spooled | known 0.1 limitation; rare |

The spool is plain files: `/var/lib/airc/spool/queue/<key>.jsonl`, one message
per line; `seen.jsonl` is the dedupe log. Deleting a queue file drops those
messages.

## 7. Opening the peer face (federation)

Federation needs `python3-cryptography` on the host (present on Debian by
default) and four things per realm:

1. **A key.** Start aircd once with `peer_listen` set (or run
   `aircd -c /etc/airc/aircd.json --show-key`); it creates
   `/var/lib/airc/realm.key` and prints the DNS record to publish.
2. **DNS.** Publish the key and the service location:
   ```
   _airc.oroboro.com.        IN TXT "v=airc1 k=ed25519 p=<public key from --show-key>"
   _airc._tcp.oroboro.com.   IN SRV 0 0 2472 airc.oroboro.com.
   ```
   To rotate, publish the new key alongside the old, restart with the new
   `signing.key`, and remove the old record after the longest message ttl.
3. **TLS on 2472.** Either give aircd `tls.cert`/`tls.key` (a Let's Encrypt
   certificate for the SRV target), or terminate TLS in front of it and point
   `peer_listen` at the LAN address the terminator forwards to. Outbound, aircd
   verifies peers' certificates against the system CAs.
4. **Policy.** Before the first foreign realm can reach you, decide what it
   may reach: default-deny plus links and passports is the recommended shape
   (`policy.default_allow: false`, empty rules; open channels via
   `links.auto_accept`). A realm you trust wholesale gets a rule.

Verify with `airc status` (shows `signing.kid`), then from the other realm
`airc send //oroboro.com/_resolver ping`. `journalctl -u aircd` logs each peer
link as `authenticated` or `UNAUTHENTICATED`; an unauthenticated inbound link
can deliver signed messages to you but is never used to send.

What is still deliberately off: nothing, once the steps above are done.