Troubleshooting
A machine does not come up
Section titled “A machine does not come up”Each machine’s group says InService when the machine is serving. What its boot said is kept in
CloudWatch, even for machines that are gone:
tofu output boot_log # by role, the command that tails its machines' boot logsaws autoscaling describe-scaling-activities --auto-scaling-group-name <group> --max-items 3A machine refuses to start on a document that names another release than its image’s. That
means the image and spin_version disagree: the images the module names are those of
spin_version unless image_ids names others.
A shell on a machine
Section titled “A shell on a machine”No machine runs SSH. The control plane’s and the proxy’s machines take a shell with Session Manager, on the one in service. A runner takes no Session Manager shell: its image carries no agent and its role grants none, so its serial console, its boot log and its metrics say what it does, and one that fails is replaced. When a runner has to be looked at from inside, an administrator opens a recorded root shell on it through the control plane.
aws autoscaling describe-auto-scaling-groups \ --query "AutoScalingGroups[].[AutoScalingGroupName,join(',',Instances[?LifecycleState=='InService'].InstanceId)]" \ --output textaws ssm start-session --target <instance-id>On a runner, when its metrics, logs and console have not answered:
spin admin runner listspin admin shell open <runner-id> --reason "<why>" --incident--incident opens the shell on the runner as it is, its workspaces still running; without it the
runner is emptied first. Either way the session is recorded and the runner is retired once the
shell ends, so it costs the runner.
Spin OS has the tools to look with — tcpdump, strace, ss, nft, conntrack, dig,
bpftool, htop, iostat — because nothing can be installed on a machine.
The dashboard does not answer
Section titled “The dashboard does not answer”dig +short app.<domain>should answer with the proxy’s address. If not, the domain’s nameservers are not the zone’s yet.- The proxy gets its certificate a few minutes after DNS resolves. Port 80 must be reachable for Let’s Encrypt.
- If you set
proxy_allowed_cidrs, you must be on one of those networks.
A workspace waits for a host
Section titled “A workspace waits for a host”A workspace waits in Waiting for a host while no runner has room for it, and says why.
- No runner is up. The control plane has asked for one. The first runner takes a few minutes to boot and register.
- The runner cannot start. The account’s spot vCPU quota is too low, or spot has no capacity
for the types named.
describe-scaling-activitieson<name>-runnerssays which. See Before you start. - Every runner takes no new workspace. A runner that was drained stays drained until it is activated:
spin admin runner listspin admin runner activate <runner-id>max_runnersis reached, and the runners are full.
A workspace does not start, or misbehaves
Section titled “A workspace does not start, or misbehaves”spin workspace status <name> # which party disagrees: control plane, supervisor, host, volumespin workspace console <name> # what the machine printed while it bootedspin workspace history <name> # every change of state, with the reasonA workspace that failed three times is FAILED until it is started again. The state names the
part that failed.
An administrator can follow one start from the request to the guest in Admin → Traces,
searching for span_name:workspace.start.
A workspace cannot reach something
Section titled “A workspace cannot reach something”spin workspace egress <name>It lists what the workspace reached, what was refused, and the lookups that failed, with why. A destination refused by policy is fixed with a policy (spin workspace retag), not with the
network. The record lives only while the workspace runs: start it and try again before reading
it.
A runner is wedged
Section titled “A runner is wedged”Replace its machine: terminate it without lowering the group’s size, and the group starts another, which joins as a new host. Drain it first if it holds workspaces you want kept.
aws autoscaling terminate-instance-in-auto-scaling-group \ --instance-id <i-...> --no-should-decrement-desired-capacityInstallations run Spinbox v20261027.01; the reference is generated from spin v20261114.01 (51f7e0dea). This page as Markdown, for a model or a terminal.

