Backups That Need No Babysitting: A Containerised Database Backup Job

Every long-running project arrives at the same unglamorous requirement: something has to copy the database, put it somewhere other than the machine running it, and keep doing so without anyone remembering to check. That request recurs across engagements, so NotAnother Software built one tool for it rather than a script per project: a containerised backup job, now at v2.0.0, whose working part is a single Bash script run as the container’s command.

The Challenge

Backups only attract attention on the day they have failed, which makes silence the thing to design against. The first version, kept in the repository, shows what goes wrong without that discipline: one MySQL database dumped to local disk and nothing more — no upload, no retention, no second engine. Worse, all of its work sat inside a guard testing that the output directory did not exist, immediately after mkdir -p had created it — so the body never ran. The job exited cleanly having produced nothing, in a container set to restart on failure — and there was no failure to see.

Our Solution

The current job reverses that assumption. The dump and the upload each check their exit status and call exit 1 on failure rather than carrying on, and a failing database aborts the whole run instead of being quietly skipped, so the exit code a scheduler sees is honest. Each step narrates itself, so the log alone shows how far a run got. The container runs once, declared restart: "no", with no loop or timer inside it — the cadence belongs to whatever schedules it.

Ordering matters too: for each database the loop dumps, uploads, deletes the local copies and only then prunes the bucket, so a run that fails to dump or upload exits before it can delete a good backup on its way out.

Taking a Copy Without Taking the Service Down

Every flag of the mysqldump invocation answers an operational condition. --single-transaction with --lock-tables=false takes a consistent snapshot without locking the tables, so the live database keeps serving traffic during the copy. --quick streams rows out instead of buffering an entire result set in memory. --triggers, --routines and --events capture behaviour and not merely table data, --default-character-set=utf8mb4 stops text arriving mangled, and --set-gtid-purged=OFF keeps replication state out of what should be a portable file.

Despite the project’s name, the same job handles PostgreSQL: a DB_TYPE switch routes the run to pg_dump, writing a mode-600 ~/.pgpass and pointing PGPASSFILE at it, so the credential travels in a private file rather than on a command line. One image, built from ubuntu:24.04, carries both clients. DB_NAMES is read as a comma-separated list and looped, because a host rarely holds one database.

Predictable by Design

Nothing about a target is baked into the image, which carries the script and the client binaries: account settings arrive as container environment and per-server settings in a small file mounted over the script’s .env. The storage endpoint is among them, so any S3-compatible provider works.

Each dump is compressed with gzip -9 and uploaded to a deterministic key — bucket, prefix, host, database, date, then a file stamped with database name and timestamp — so the bucket can be browsed by hand without an index. That layout is load-bearing: retention works against the same path. For each database the job lists the date prefixes beneath its key, sorts them in reverse and deletes everything past BACKUP_RETENTION_DAYS, the ISO date format making the reverse sort chronological. Pruning happens in the bucket on every run, with no lifecycle policy to drift, and local files are removed once the upload succeeds, so the container keeps no state.

Even the build verifies itself: before the AWS CLI is unzipped, the Dockerfile imports a pinned public key committed to the repository and checks the installer’s detached signature with GPG, so a mismatch fails the build rather than baking an unverified binary into an image trusted with production data.

Conclusion

The interesting engineering in a backup job is not the dump command; it is everything arranged around it so the job needs no supervision — ordering the steps so a failure cannot cascade, keeping no state between runs, verifying the binaries at build time, letting the exit code speak. Together they separate a chore automated once from a tool that can be pointed at the next database server without a rewrite.