A missing swap file caused hours of downtime and a corrupted MySQL database. Learn why cloud VPS servers need swap configured and how to add it in six commands.
Last weekend, a client’s WordPress site went dark for hours. The database had corrupted, MySQL was stuck in a crash loop, and the error logs were full of gibberish. It took a full rebuild to bring it back online.
The root cause? A missing swap file on a 2GB server.
This post explains what happened, why it matters, and how to prevent it happening to you.
A Missing Swap File Cost Hours of Downtime
The server was a DigitalOcean droplet running a standard LAMP stack: Ubuntu 24.04, Apache, MySQL 8, PHP 8.2. It had been running fine for months.
On Sunday morning at 08:07, the site stopped responding. WordPress showed “Error establishing a database connection” - the classic sign that MySQL has fallen over.
When I investigated, MySQL was in a restart loop. It would start, immediately crash, and systemd would try again. The restart counter showed 13,851 attempts.
The kernel logs told the real story:
Out of memory: Killed process 1045049 (mysqld) total-vm:1837208kB
The server had run out of RAM. Linux killed MySQL to free up memory. But MySQL was mid-write when it died, corrupting the InnoDB system tablespace. Every restart attempt failed because the database files were damaged.
Why the Server Ran Out of Memory
Two things combined to cause this.
First, the server had been targeted by a brute force attack. A single IP address had made over 1,000 requests to wp-login.php in one night, trying to guess the admin password. Each request spawned PHP processes and database connections, gradually eating into available memory.
Second - and this is the critical bit - the site was running Wordfence. The security plugin detected the attack and tried to sync the attack data to its servers. At 08:07, it triggered roughly 25-30 simultaneous HTTP requests to process this sync.
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 200 "WordPress; https://example.com"
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 500 "WordPress; https://example.com"
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 500 "WordPress; https://example.com"
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 500 "WordPress; https://example.com"
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 500 "WordPress; https://example.com"
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 500 "WordPress; https://example.com"
0.0.0.0 - "POST /?wordfence_syncAttackData HTTP/1.1" 500 "WordPress; https://example.com"
... 20+ more ...
Whoops! 🥹
On a server with 2GB of RAM and no swap file, that was enough to tip it over the edge. The kernel’s out-of-memory killer did exactly what it’s designed to do: it killed the biggest memory consumer to save the system.
Unfortunately, that was MySQL. 🤣
What Is Swap and Why It Matters
Swap is disk space that Linux uses as overflow when RAM runs low. It’s slower than RAM, but it provides a crucial buffer.
Without swap, when memory fills up, the kernel has only one option: start killing processes. It picks the biggest offender, terminates it immediately, and hopes for the best. There’s no graceful shutdown, no chance to finish writing data.
With swap, the system has breathing room. It can move less-used memory pages to disk, slowing things down rather than crashing them. MySQL might become sluggish under memory pressure, but it won’t be killed mid-transaction.
The irony is that DigitalOcean, AWS, and most cloud providers don’t create swap files by default. They assume you’ll provision enough RAM or manage this yourself. For a 2GB droplet running WordPress, MySQL, and a memory-hungry security plugin, that assumption is optimistic. This is one of those hosting and infrastructure details that’s easy to overlook until it causes a crisis.
How We Recovered the Database
The recovery process took about 90 minutes:
-
Enable InnoDB recovery mode - MySQL has a built-in recovery option that skips corrupted pages. Adding
innodb_force_recovery = 1to the config allowed it to start in a degraded, read-only state. -
Dump the databases - With MySQL limping along, we ran
mysqldump --all-databasesto extract everything. This succeeded, giving us a clean 70MB backup. -
Rebuild the data directory - We moved the corrupted files aside, ran
mysqld --initialize-insecureto create fresh InnoDB system files, and restored from the dump. -
Recreate the database user - The restore included the data but not the user credentials WordPress needed. A quick
CREATE USERandGRANT ALL PRIVILEGESsorted that.
Adding a Swap File
The actual fix for the underlying problem:
sudo fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
Six commands. A few seconds to run. Would have prevented hours of downtime.
Additional Hardening
While we were at it, we addressed a few other issues:
Fail2ban for WordPress - The brute force attack should have been blocked at the firewall. We added a fail2ban jail that automatically bans any IP making more than five POST requests to wp-login.php within ten minutes. This stops attacks before they reach PHP, saving both CPU and memory.
Disabled binary logging - MySQL was writing binary logs that had grown to 1.5GB. These are useful for replication and point-in-time recovery, but this single-server WordPress site needed neither. Disabling them with skip-log-bin prevents unnecessary disk usage.
Tuned Wordfence - The security plugin that triggered the crash wasn’t configured for a resource-constrained server. Enabling “low resource scanning”, reducing the memory limit from 256MB to 128MB, and trimming live traffic logging all help reduce its footprint. Like keeping WordPress updated, configuring plugins properly is ongoing maintenance that prevents problems.
How Much Swap Do You Need?
For most WordPress servers:
- 1-2GB RAM: Add 1-2GB swap
- 4GB RAM: Add 1-2GB swap
- 8GB+ RAM: Add 2GB swap as a safety net
It’s not about regularly using swap - if you’re swapping heavily, you need more RAM. It’s about having a buffer that prevents catastrophic failure when something unexpected happens.
Check Your Server Now
SSH into your server and run:
free -h
If the swap line shows 0B, you’re vulnerable to the same failure. Adding swap takes under a minute and persists across reboots.
This isn’t theoretical. A missing swap file, a security plugin doing its job, and a brute force attack combined to take a production site offline for hours and required a full database rebuild to recover.
The fix cost nothing and took six commands. The outage cost hours of downtime and hours of recovery work.
Check your servers. 🥸