How evrd fits together
One Go process holds your IRC connections and serves your clients. Run it on its own with a file on disk, next to Postgres and ScyllaDB, or as a cluster of nodes that share the work and take over when one fails.
One process, three kinds of storage
Native clients attach to the IRC listener, while the web app talks to the HTTP and WebSocket API. Both share the connection to each upstream network. Accounts go to a SQL database, chat history to a separate store, and metrics out over OpenTelemetry.
Where state lives
| State | Stored in | Shared by nodes? |
|---|---|---|
| Accounts, sessions, client passwords, networks | Postgres (or SQLite) | Yes with Postgres; SQLite is one node only |
| Chat history | ScyllaDB (or Pebble) | Yes with ScyllaDB; Pebble is one node only |
| Which node owns which user | Postgres nodes, user_leases | Yes |
| Live connections to IRC networks | The user's owner node | Forwarded |
| Attached IRC clients, web chat sockets | The user's owner node | Forwarded |
Why history has its own store
Chat history never goes into Postgres. The history directive picks the backend:
- ScyllaDB, shared by every node. Each buffer gets one partition per month, read newest first. Writes are queued (up to 16k lines), so a slow database doesn't stall IRC.
- Pebble, an embedded store on local disk, for one evrd process per directory.
evrctl history copy <from> <to> copies history between them.
Three ways to run it
Every shape uses the same binary. The db and history directives pick the backends. Nodes that share a Postgres share the users between them, and internal_listen (with cluster_secret, internal_url, internal_irc_listen and internal_irc_addr) lets them forward clients to each other.
listen ircs://:6697 api :8098 db sqlite:/var/lib/evrd/evrd.db history pebble:/var/lib/evrd/history public_url https://bnc.example.org irc_host bnc.example.org
listen ircs://:6697 api :8098 db postgres://evrd:…@postgres:5432/evrd history scylla://scylla:9042/evrd secret_key_file /etc/evrd/secret.key otel_endpoint http://victoriametrics:8428/opentelemetry/v1/metrics metrics_url http://victoriametrics:8428
# the single-node lines (listen, api, public_url, …), plus: db postgres://…/evrd history scylla://db1,db2/evrd # must be shared internal_listen :8099 internal_irc_listen :6699 cluster_lease 20s secret_key_file /etc/evrd/secret.key # the same key on every node # per node, in the environment: EVRD_NODE_ID=node-a EVRD_INTERNAL_URL=http://evrd-a.internal:8099 EVRD_INTERNAL_IRC_ADDR=evrd-a.internal:6699 EVRD_CLUSTER_SECRET=<same long secret everywhere>
Any node is a front door
In a cluster each user is owned by exactly one node, which runs their networks and serves their clients. A client can land on any node, and that node forwards it to the owner. Nodes need no message bus, only one extra hop.
The client connects to the balancer
The balancer passes IRC through as plain TCP, so TLS on 6697 terminates at evrd. It can't route by user, because IRC only reveals the username in PASS once the connection is already open.
Any node answers
node-b accepts the connection, answers CAP and reads the registration: PASS, NICK and USER.
Find the owner
node-b looks up who holds alice's lease in Postgres. Which node gets a lease is decided by rendezvous-hashing the user ID over the live nodes, so every node agrees, and when a node joins or leaves only about 1/N of users move.
Splice to the owner
node-b connects to node-a's internal IRC port, sends EVRD-PROXY <secret> <client ip> <caps>, replays the registration and any bytes already received, then splices the two sockets. node-a authenticates alice as usual. The web app works the same way: alice's API requests and her chat WebSocket are reverse-proxied to node-a's internal HTTP port. Only signing in, the public config and the admin pages about everyone are served wherever they land.
When a node goes away
Ownership is a lease in Postgres. Nodes heartbeat every 5 s and count as live if they beat within 15 s. Leases last 20 s. Step through a crash, or switch to a rolling deploy, where the node drains on SIGTERM instead.
Normal operation
node-a owns alice. It runs her Libera.Chat connection, serves her HexChat and renews her lease with one batched statement per heartbeat.
node-a dies
Alice's upstream connection and her client connection drop with it. The balancer's short connect timeout and health checks (passive in nginx, active in Traefik) take node-a out of rotation, so new connections don't wait a minute on a dead node.
node-a drains
On SIGTERM a node stops its users' connections and releases their leases straight away, instead of letting them run out.
The lease expires, node-b takes over
node-a renewed alice's lease every 5 s, so it runs out 15–20 s after the crash. On its next tick after that, the next node in her rendezvous order acquires it with a conditional update and bumps the epoch. A node that has let her go answers anything still forwarded to it with “moving, try again”, and the retry lands on node-b.
The next tick picks her up
node-a released the lease and left the live set as it shut down, so node-b acquires alice on its next tick, within 5 s, without waiting for anything to expire, and starts reconnecting her networks at the paced connect_rate.
Back online on node-b
On the dev stack, after a docker kill of a node, a user's network was connected again on the other node 19 s later. Alice's client reconnects through any node and lands on the new owner. Networks see a disconnect and a reconnect, because a TCP connection to an IRC server can't move between machines.
node-a comes back empty
node-b now owns alice. Once the new node-a has been up 10 s, gradual rebalancing hands its share of users back, at most 50 per tick from each node.
Fencing: a node that can't renew its leases, for example because it lost Postgres, stops its users' connections at half the TTL, before anyone else may take over. As long as the nodes' clocks agree, two nodes never run the same user. A short Postgres outage is better than split-brain.
Give every user an address of their own
Large networks throttle and cap connections per source address. Without help, every user on a node would connect to Libera.Chat from that node's single address.
connect_rate 5 # most connects/second to one server, per node upstream_bind 192.0.2.10 192.0.2.11 # spread users over these (stable per user) upstream_prefix6 2001:db8:42::/64 # every user's own IPv6 address (IPv6 servers) bindhost 192.0.2.20 vhost.example.org # addresses users may pick # a prefix needs Linux and a local route: ip -6 route add local 2001:db8:42::/64 dev lo
What each one does
- connect_rate spaces connects to each IRC server host per node, and reconnect backoff is jittered, so a takeover or netsplit recovery doesn't flood a network.
- upstream_bind spreads users over a list of source addresses, with each user keeping the same one on that node. Per node (EVRD_UPSTREAM_BIND).
- upstream_prefix6 gives every user their own IPv6 address from a routed prefix when connecting over IPv6 (IPv4 servers still use upstream_bind). Per node (EVRD_UPSTREAM_PREFIX6); needs Linux.
- bindhost is ZNC's BindHost. It lists addresses users may choose from, and admins can lock a user's choice. Like connect_rate, it can also be set for every node at once in Admin → Settings.
Per-address caps on large networks are usually the first real limit. Talk to the networks you care about about limits for your ranges.
Upgrade without anyone leaving IRC
A connection to an IRC server lives in the process that opened it, so restarting a bouncer normally makes every user quit and rejoin every channel. On a single node, a second, much smaller process holds those connections instead, and evrd can come and go underneath it.
What happens on a restart
- evrd stops. It detaches from every connection instead of sending QUIT. The holder keeps them open, answers the servers' PINGs and buffers whatever arrives.
- The new evrd starts and asks the holder for each network's connection. It gets back the nick, the server's 005 lines and the buffered lines, so it skips registration and doesn't rejoin anything.
- It catches up: buffered messages go into history, and NAMES and TOPIC rebuild each channel. Clients reconnect to evrd and replay what they missed, as usual.
Switching a network off or deleting it still sends QUIT. If the holder isn't running, evrd connects directly, as it always has.
evrd-links -socket /var/evrd/run/links.sock \
-chroot /var/evrd -user _evrd
# evrd.conf
links_socket /var/evrd/run/links.sock
The holder is small and rarely changes, so it rarely needs a restart. Restarting it does drop the connections. The OpenBSD installer sets it up and restarts it only when a release changed it. It's for a single node. A cluster moves users between nodes instead (Failover).
Start small, grow when you need to
The shapes differ only in configuration, so you can start with the simplest one and move up when you need to.
| Single file | Single node + databases | Cluster | |
|---|---|---|---|
| evrd processes | 1 | 1 | 2 or more |
| Accounts (db) | SQLite | Postgres | Postgres, shared |
| History (history) | Pebble | ScyllaDB or Pebble | ScyllaDB, shared |
| Load balancer | None | None | nginx or Traefik ≥ 3.7 |
| A node fails | Down until restart | Down until restart | Users move in ≈ 20 s |
| Deploying a new version | Restart; with evrd-links, nobody leaves IRC | Restart; with evrd-links, nobody leaves IRC | Rolling, drain on SIGTERM |
| Good for | One person, a Pi or a small VPS | A group on one machine | A community with no machine to lose |
# stop evrd first when copying from a Pebble store
evrctl history copy pebble:/var/lib/evrd/history scylla://db1,db2/evrd
Before you go multi-node
- History has to be shared, so clusters need ScyllaDB.
- Every node needs the same secret_key and cluster_secret. Keep both out of database backups.
- Leave settings_write_conf off, so the database is the one place admin settings live.
- Keep the internal ports (:8099, :6699) on a private network.