Cheryl (game server)
Cheryl runs the game servers, managed by CubeCoders AMP, plus the friend group’s chat and voice. Since 2026-11-17 it is a VM on figgis; the old VPS is being retired:
| Cheryl (the VM) | Old Cheryl (cheryl-old, being retired) |
|
|---|---|---|
| What | VM 102 on figgis (Proxmox at OVH, Hillsboro OR) | Its own OVHcloud VPS |
| Size | 8 vCPU, 48 GiB RAM, 160 GiB disk, Ubuntu 26.04 | 6 vCPU, 11 GB RAM, 96 GB disk, Ubuntu 26.04 |
| Public IP | 51.81.175.16 (an OVH additional IP with its own virtual MAC) |
40.160.144.240 (amp.spelvin.dev) |
| Tailnet | cheryl, 100.107.153.62, tag:cheryl |
cheryl-old, 100.67.110.77, tag:cheryl |
| SSH | ssh cheryl (through figgis; nothing else can reach port 22) |
ssh amp / ssh cheryl-old (by IP) |
| Built by | docker-infra/ansible (vms, guest_base, docker, amp roles) |
By hand |
| Runs | AMP + Enshrouded, Mattermost (talk.spelvin.dev, behind the host’s nginx), Mumble (voice.spelvin.dev), Dozzle agent. All moved 2026-11-17 |
Nothing live: its Enshrouded, Mattermost, Mumble and arr are stopped (kept for rollback; arr moved to Pam). Still running: the old AMP panel, nginx, a Dozzle agent |
Everything has moved off old Cheryl (2026-11-17); it can be cancelled once the moved services
have been used for a few days (see TO-TEST.md in docker-infra).
Chat and voice on Cheryl: Mattermost and Mumble deploy there from Lana’s checkout
(DOCKER_HOST=ssh://cheryl), as they did to old Cheryl; their volumes were copied over
as-is. nginx and certbot on the host come from the playbook’s web_proxy role (the
certificate was copied from old Cheryl and renews there). talk and voice.spelvin.dev
point to 51.81.175.16 (DNS only). Mattermost’s calls back to Woodhouse pass Cloudflare’s
Bot Fight Mode because they carry Mattermost’s own user agent; plain curl from these IPs is
challenged.
Cheryl: AMP
Section titled “Cheryl: AMP”- Install: the playbook’s
amprole runs CubeCoders’ installer unattended, then binds the panel (ADS01) to127.0.0.1:8080and publishes it to the tailnet withtailscale serve --tcp 8080. The panel is tailnet-only (http://cheryl:8080/, useradmin, password inansible/secrets.enc.yml); old Cheryl’s panel was public. - Game servers run in Docker (AMP’s
Defaults.UseDocker, imagecubecoders/ampbase:debian, host networking), one container per instance namedAMP_<instance>. Their console output is the container log, so it shows in Dozzle. Enshrouded runs under Wine inside the container. - Licence: AMP Standard, activated by hand (it also has to be in
ADSModule.Defaults.NewInstanceKeybefore new instances can be created). The key allows 5 machines; it is inansible/secrets.enc.yml. - Firewall: Proxmox’s per-VM firewall on figgis, inbound DROP plus
firewall_ininansible/inventory/host_vars/figgis.yml: SSH from figgis only, UDP 15637 for Enshrouded. A new game’s ports go there. The installer’siptables-persistentand AMP’s own firewall timer are removed by the role.
| AMP instance | What | Listens on |
|---|---|---|
ADS01 |
The panel, and the API in front of every instance | 127.0.0.1:8080 (tailnet tcp:8080) |
Enshrouded01 |
Enshrouded, “Dos Amigos”, 4 slots | 127.0.0.1:8081 (AMP), UDP 15637 public |
Moving an instance (how Enshrouded moved): create it fresh from the same template on the
new panel, let it download, stop it, then carry over the old instance’s App.AppSettings
and other changed keys in GenericModule.kvp, its LocalFileBackupPlugin.kvp,
scheduleEvents.json/scheduleTimes.json, and the game’s own files (for Enshrouded,
enshrouded_server.json, which holds the join password, and savegame/). S3 keys must
be set again through the panel or API: AMP encrypts them per instance, so copied values
can’t be read and uploads silently don’t happen.
Old Cheryl
Section titled “Old Cheryl”What runs here
Section titled “What runs here”| AMP instance | What | Listens on |
|---|---|---|
ADS01 |
The AMP controller (the panel, and the API in front of every instance) | 127.0.0.1:8080 |
Enshrouded01 |
Enshrouded, stopped since 2026-11-17 (moved to the VM; start-at-boot off) | 127.0.0.1:8081 |
| arr (Docker) | SABnzbd, Sonarr, Radarr, Prowlarr | 127.0.0.1:8085, 8989, 7878, 9696, published to the tailnet by tailscale serve |
| mattermost (Docker) | Mattermost Team Edition + its Postgres | 127.0.0.1:8065, public as https://talk.spelvin.dev through the host’s nginx |
| mumble (Docker) | Mumble voice server | 64738 TCP and UDP, public (server password required) |
Instance data is under /home/amp/.ampdata/instances/. The Enshrouded world is
small (a few MB in enshrouded/2278520/savegame); nearly all of the instance’s
11 GB is game files that Steam can download again.
Chat and voice (Discord replacement)
Section titled “Chat and voice (Discord replacement)”Being set up from 2026-10-28 to replace Discord for the friend group, chosen because Mattermost’s bot features (slash commands, buttons, ephemeral replies, image posts, pins) cover what Woodhouse needs, and Mumble passes a good microphone through with all processing off.
- Mattermost at
https://talk.spelvin.dev(chat.is Open WebUI). nginx on the host proxies to it, the same nginx that serves the AMP panel; the site file is copied inmattermost/nginx-talk.spelvin.dev.conf. Not behind Cloudflare Access, because its apps can’t pass an Access login; its own accounts are the gate and new ones need an invite. Let’s Encrypt certificate by certbot (2026-10-28), renewed by certbot’s timer like the panel’s. - Mumble at
voice.spelvin.dev, port 64738 (a DNS-only record: Cloudflare’s proxy can’t carry it), public because voice needs UDP that Lana’s Tunnel can’t carry. Server password and SuperUser password are inmumble/secrets.enc.env. - Both deploy from Lana like arr (
deploy-mattermost.yml,deploy-mumble.yml) and are on the update agent’s list. Woodhouse runs on both Mattermost and Discord until the move is done (see its README); the update agent posts to both.
Docker and the arr stack
Section titled “Docker and the arr stack”amp has no git checkout and no CI key of its own. Lana drives its Docker daemon instead:
deploy-arr.yml and the update agent run docker compose in Lana’s checkout of arr/ with
DOCKER_HOST=ssh://amp. amp is an alias in Lana’s ~/.ssh/config (key
~/.ssh/amp_docker, over Tailscale), and on amp that key is restricted in authorized_keys
to command="docker system dial-stdio",restrict, so it can reach Docker but not a shell.
ubuntu is in the docker group (it already had passwordless sudo).
Config is in /opt/arr/config/<app>; downloads are in /srv/data on the root disk. The
library isn’t here: Sonarr and Radarr write it to Pam over SMB.
Disk is the constraint. The 96 GB root volume is shared by the OS, the game files and
the download staging area (~80 GB free with nothing in flight). SABnzbd pauses below 20 GB
free, and Sonarr/Radarr reject releases over 20/25 GB. SABnzbd’s unpack and repair run at
nice -n 19 / ionice -c2 -n7; its download speed is not capped, and on 2026-10-23 a full
speed download plus unpack while the game was busy (load above 7) starved SSH for a few
minutes. Details: Media stack.
Controller watchdog
Section titled “Controller watchdog”AMP 2.8.0.8’s controller (ADS01) occasionally crashes during its hourly refresh of the
game-template library (an unhandled ArgumentException, “Destination is too short”, while
reading git’s output; seen 2026-09-30 and 2026-10-17), and nothing restarts it. The game
keeps running, but the panel returns 502 and the Woodhouse bot can’t reach the server.
A systemd timer, amp-ads-watchdog.timer, runs /usr/local/sbin/amp-ads-watchdog every
3 minutes: if nothing answers on 127.0.0.1:8080 it runs ampinstmgr start ADS01 as the
amp user. It never touches game instances. Restarts are logged in
journalctl -u amp-ads-watchdog. Not part of docker-infra; set up by hand on 2026-10-18.
How it’s reached
Section titled “How it’s reached”This box is the exception to “nothing has a port exposed to the internet”. It is not behind the Cloudflare Tunnel or Access:
- The game needs its UDP ports open directly on the public IP; a Tunnel can’t carry that.
- The AMP panel is served by nginx on 80/443 at
amp.spelvin.dev(a Let’s Encrypt certificate, proxying to127.0.0.1:8080), with AMP’s own login as the only gate. Putting it behind Access with Authentik, like the apps on Lana, is an open improvement. - SSH is key-only: root login and password login are off
(
/etc/ssh/sshd_config.d/10-hardening.conf), and fail2ban runs the stocksshdjail. If key login ever breaks, the provider’s console is the way back in. - The arr web UIs are bound to localhost and published to the tailnet only, by
tailscale serveTCP forwards on 8085, 8989, 7878 and 9696. amp has no host firewall (no ufw), so a Docker port published without an address would be on the public IP. Cheryl also hosts the Tailscale Servicessvc:nzb,svc:sonarr,svc:radarrandsvc:prowlarr(HTTP on 80 to the same ports); see Networking. - The AMP API for automation goes over Tailscale only.
tailscale servepublishes the controller’s port 8080 to the tailnet, so Lana can callhttp://100.67.110.77:8080without AMP itself listening on anything but localhost.
Discord controls
Section titled “Discord controls”Woodhouse, the bot on Lana, exposes /enshrouded status,
restart and update. It logs in to AMP as discord_bot at http://100.107.153.62:8080
(Cheryl’s VM, since 2026-11-17). Restart and update need a Discord role and a confirmation.
The account’s permissions are broader than “Enshrouded only”: its role has
Instances.*, Core.AppManagement.* and <Enshrouded01 id>.Core.* (copied as they were on
old Cheryl). Narrowing them is an open task. See the
woodhouse README.
Backups
Section titled “Backups”AMP’s backups upload to Garage on Pam: endpoint
http://100.109.83.48:3900 over Tailscale, region garage, bucket game-backups, key
amp-backups (that bucket only; shown again with garage key info amp-backups --show-secret
on Pam). Enshrouded01 takes a backup every hour and keeps the newest (replace oldest). On
the VM the first upload was checked on 2026-11-17 (backup “Migrated from old Cheryl”,
pinned). A restore has not been tried.
Gaps to fill in
Section titled “Gaps to fill in”- Old Cheryl’s AMP panel is still public at
amp.spelvin.devwith only AMP’s login in front, until that box is retired. discord_bot’s AMP permissions (above).- No restore test of an AMP backup yet.