PatchRail

Runner resource exhaustion — exit code 137 / OOMKilled

Log signatures

Process completed with exit code 137
OOMKilled
JavaScript heap out of memory
No space left on device
signal: killed

What’s happening

The runner — not your program — hit a hard resource ceiling. Exit code 137 is the giveaway: it means the process received SIGKILL (128 + 9), which on Linux almost always means the kernel OOM killer fired. ENOSPC and "No space left on device" are the disk-side equivalents, frequently caused by accumulated Docker layers, package caches, or test artifacts on long-lived runners. The trap: the visible error is often downstream — a test "mysteriously failed" because the process it depended on was silently killed.

Fix playbook

  1. 1Search the log for 137, signal: killed, or OOMKilled before the first test failure. If present, stop debugging the test.
  2. 2Rerun while measuring: wrap the heavy step in /usr/bin/time -v (peak RSS) and print df -h before and after.
  3. 3Reduce peak usage first: lower test parallelism (pytest -n2, jest --maxWorkers=2), cap heap (NODE_OPTIONS=--max-old-space-size=4096), split the build into stages.
  4. 4Free disk: prune Docker (docker system prune -af), delete caches you don't restore, clean /tmp between steps.
  5. 5Only then consider a bigger runner class — it's the most expensive fix and hides growth, it doesn't stop it.

Prevention

  • Add a step that prints df -h and available memory at job start. When exhaustion happens again you'll have a baseline in the log instead of a mystery.

Triage every red build, not just this one.

This page is one of 31 failure classes the open-source patchrail CLI matches from a raw log — locally, with secrets redacted first. If a log stumps it, open an issue; if it saves you a debugging morning, a star helps other maintainers find it.

★ Star on GitHubpipx install patchrail