Skip to content
Published on

Continuous Backups and Local Dev from Production with Shippy

Authors

Shippy is a single-binary deployment tool for Composer-based PHP projects — part one of this series covers the normal shippy deploy path for TYPO3 and Pimcore. This post covers the other half of what it ships with: shippy backup and shippy gitlab:upload, and how to turn them into an unattended nightly backup pipeline — plus a recipe for restoring one of those backups straight into a local DDEV environment.

What shippy backup produces

shippy backup <hostname>

connects to the host over the same SSH config used for deploys, and writes backup-<hostname>-<timestamp>.zip (default: current working directory) containing a database dump and a set of files pulled from the remote shared/ directory. Nothing is installed on the server for this — it reuses the SSH connection and, for the database, either explicit credentials or auto-detection.

Configuration lives in a backup: block in .shippy.yaml:

backup:
  output: ./backups # Local directory for the ZIP (default: cwd)

  # Paths inside the remote shared/ directory to include
  files:
    - .env
    - public/fileadmin/
    - public/uploads/

  database:
    # auto   - try `typo3 configuration:show` (TYPO3 v14+), then .env, then settings.php
    # dotenv - read DB_HOST/DB_DATABASE/DB_USERNAME/DB_PASSWORD from .env
    # typo3  - same as auto but skips the generic .env attempt
    # manual - explicit driver/host/port/name/user/password below
    credentials: auto

    exclude_tables:
      - 'cache_*'
      - 'cf_*'
      - 'sys_log'
      - 'be_sessions'

    options:
      single_transaction: 'true' # MySQL: consistent dump without locking tables

With credentials: auto on TYPO3 v14+, Shippy runs ./vendor/bin/typo3 configuration:show DB/Connections/Default on the server to read the live, authoritative database configuration — not a guess based on .env, the actual resolved config TYPO3 itself would use. It falls back to parsing .env keys or config/system/settings.php only if that command is unavailable (older TYPO3, or a non-TYPO3 project, where credentials: dotenv or manual apply instead).

exclude_tables matters more than it looks: without excluding cache_*, cf_* and similar, a TYPO3 backup ZIP balloons with tables that reconstruct themselves automatically on the next cache:flush anyway. Excluding session tables (be_sessions) is a minor but real privacy consideration for a backup that might land somewhere less locked-down than the production database.

Options at the command line: --output/-o to override the directory, --skip-database and --skip-shared to produce a partial archive, --verbose for per-file output.

Automating it: a nightly pipeline

shippy gitlab:upload pushes any file to the GitLab Generic Packages registry of the project's git origin — project path and host are auto-detected from the origin remote, no extra config. Authentication resolves in order: --token flag, then GITLAB_TOKEN, then CI_JOB_TOKEN — the last of which GitLab CI sets automatically inside a job, so a scheduled pipeline needs zero manually-managed secrets for the upload step itself:

# .gitlab-ci.yml
nightly_backup:
  stage: backup
  image: ghcr.io/ochorocho/shippy:latest
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
  script:
    - shippy backup production
    - shippy gitlab:upload backups/backup-production-*.zip

rules: if: $CI_PIPELINE_SOURCE == "schedule" means this job only runs from a pipeline schedule, not on every push — set one up under Project → Build → Pipeline schedules (e.g. 0 2 * * * for 02:00 daily). The job still needs SSH access to the production host, exactly like a deploy job: an SSH private key as a masked CI/CD variable, written to disk in a before_script before shippy backup runs.

Once it's run at least once, archives show up under Project → Deploy → Package Registry, grouped by <package-name>/<package-version> — --package-name and --package-version default to the project name and a timestamp, both overridable per upload.

TIP

GitLab's Generic Packages registry has no built-in expiry. A nightly job accumulates one ZIP per day indefinitely unless something prunes it — either a cleanup policy on the registry, or a second scheduled job that deletes packages older than N days via the Packages API.

Restoring a backup into DDEV for local development

This part isn't a Shippy command — there's no shippy restore. It's what the backup ZIP composes into once you have it, and the actual reason continuous backups are worth having beyond disaster recovery: a fast path to local development against real data.

1. Download and inspect the archive. Either from the GitLab package registry UI, or via the Packages API for scripting it:

unzip -l backup-production-20260812020000.zip

Check what's actually inside before assuming a filename — the database dump and the files: paths from the backup: config are both in there, but confirm the dump's exact filename and extension first.

2. Import the database dump into DDEV. ddev import-db reads a .sql, .sql.gz or .zip file directly:

cd my-typo3-project   # DDEV project root
ddev import-db --file=/path/to/extracted/dump.sql

3. Copy the shared files in. For TYPO3, that's public/fileadmin/ and public/uploads/ — whatever was listed under backup.files:. ddev import-files handles a directory or an archive:

ddev import-files --source=/path/to/extracted/fileadmin

4. Point local config at the DDEV database. DDEV's own .env/config.yaml already sets up local DB credentials (db/db/db against the db service by convention) — a project's .env.example or .ddev/config.yaml has the exact values, since they're fixed per DDEV's own conventions rather than anything Shippy touches.

ddev launch

From here it's a real local copy of production — actual content, actual uploaded assets, actual editorial state — without a second SSH hop or a manual mysqldump while trying to remember which tables to skip. Because the backup already excluded cache_*/cf_*/session tables, the import is smaller and TYPO3 rebuilds its caches fresh on first request, which is what you want locally anyway.

Why the same ZIP serves both purposes

Deliberately reusing one artifact for disaster recovery and local dev refresh means the backup path gets exercised constantly instead of only in an emergency. A backup nobody has restored from is a backup whose restore process is untested — pulling last night's archive into DDEV every time someone needs fresh local data means a broken backup (missing table, wrong credentials, a .env value that changed) surfaces as an annoyance during normal work, not as a surprise during an actual incident.

Links