docker-rsync-scheduler

README

docker-rsync-scheduler

Push local directories to a remote host over rsync-and-ssh on a schedule — structured logs, no metrics, no open ports.

What it does

Reads a YAML config defining N sync jobs. For each job it runs rsync over ssh to push a local directory one-way to a remote host. Every run emits structured slog lines (logfmt, UTC timestamps); in built-in scheduling mode these go to the container's PID-1 stream for collection by a log aggregator (Alloy, Promtail) and alerting via Grafana or similar, whereas an external sync pass triggered via docker exec emits to the exec caller instead (see the Scheduling modes observability caveat).

• One-way mirror of each configured local directory to a [user@]host:/path
• Per-job --delete, --chown=uid:gid, and exclude patterns
• Empty-source guard: a job whose top-level source directory is missing or empty is skipped, so a wholly vanished or unmounted source cannot let --delete wipe the matching remote tree. The guard probes the local dir against the built-in global excludes only (not per-job excludes), so on a delete: true job whose own excludes could match every entry, set max_delete (rsync --max-delete=N) as a backstop.
• Built-in interval scheduler, or hand scheduling to an external scheduler (cron, Ofelia, etc.) via the sync subcommand
• File-marker healthcheck — unhealthy when any job fails, recovers on the next clean pass
• Logs only: no Prometheus exporter, no HTTP server, no listening socket

Architecture

• Scheduler your way. Ships with a self-contained Go interval scheduler (no external cron, systemd timers, or orchestrator scheduling needed), or hands the cadence to an external scheduler via the sync subcommand. See Scheduling modes.
• Overlap lock. A single advisory flock serialises every sync pass so the built-in ticker and an external sync exec never run two passes at once. See Scheduling modes.
• Subcommands. daemon (PID 1, the default command; dispatches built-in vs external based on SYNC_INTERVAL), sync (one pass, exit 0 if all jobs succeed, 1 if any fail), and health (the Docker probe). The built-in startup pass, the interval pass, and the sync subcommand share one sync-pass function.
• No shell, validated config. Each job runs via exec.CommandContext with an explicit argument slice, and every config field is validated at startup. See Security.
• Bounded resources. Per-job timeout via context (default 10m, override with SYNC_TIMEOUT); captured rsync stderr is bounded to 1 MB so a chatty subprocess cannot OOM the container.
• Health. File-marker pattern via github.com/cplieger/health — the marker is set after each pass and probed by the health subcommand.

Quick start

The image is published to both GHCR (ghcr.io/cplieger/docker-rsync-scheduler) and Docker Hub (cplieger/docker-rsync-scheduler) — identical contents, use whichever you prefer.

services:
  rsync:
    image: ghcr.io/cplieger/docker-rsync-scheduler:latest
    container_name: rsync
    restart: unless-stopped
    environment:
      LOG_LEVEL: "info"
      CONFIG_PATH: "/config/config.yaml"
      SYNC_INTERVAL: "6h"   # Go duration; "off" disables the built-in scheduler
      SYNC_TIMEOUT: "10m"
    volumes:
      - ./config.yaml:/config/config.yaml:ro
      - ./id_ed25519:/keys/id_ed25519:ro
      - /srv/source/certs:/sources/certs:ro

Scheduling modes

The container runs in one of two modes, selected by SYNC_INTERVAL.

Built-in scheduler (default)

Set SYNC_INTERVAL to a Go duration (6h, 1h, 30m, …). The container runs a sync pass at startup and then every interval. This is the zero-dependency default; nothing else is required. On an unset or unparseable (non-sentinel) value it falls back to 6h.

External scheduler

Set SYNC_INTERVAL=off (aliases: disabled, 0). The container stays running but idle, and you trigger each pass out-of-band by exec'ing the sync subcommand:

docker exec rsync docker-rsync-scheduler sync

The pass runs once and exits; its exit code is non-zero on failure, and it updates the same health marker the long-running container reports. This lets a central scheduler own the cadence.

Observability caveat (external mode). A sync triggered via docker exec runs as a child process, not the container's PID 1. Docker's logging driver captures only PID 1's stdout/stderr, so a socket-based log collector (for example Alloy's Docker log discovery) does not see the exec child's output — the sync cycle complete heartbeat, sync failed, skip empty source, and per-job lines instead go to the exec caller (for example the Ofelia job result). The Loki-based alerts described under Healthcheck (heartbeat-absence, skipped count, skip empty source) therefore fire only in built-in scheduling mode. In external mode, read per-run outcomes from the exec scheduler's own job-failure reporting and rely on the container health marker (which the sync subcommand does update) as the container-level signal.

Example with Ofelia labels:

services:
  rsync:
    image: ghcr.io/cplieger/docker-rsync-scheduler:latest
    container_name: rsync
    restart: unless-stopped
    environment:
      LOG_LEVEL: "info"
      CONFIG_PATH: "/config/config.yaml"
      SYNC_INTERVAL: "off"   # disable built-in loop; Ofelia drives it
      SYNC_TIMEOUT: "10m"
    labels:
      ofelia.enabled: "true"
      ofelia.job-exec.rsync-sync.schedule: "@every 6h"
      ofelia.job-exec.rsync-sync.command: "docker-rsync-scheduler sync"
      ofelia.job-exec.rsync-sync.no-overlap: "true"
    volumes:
      - ./config.yaml:/config/config.yaml:ro
      - ./id_ed25519:/keys/id_ed25519:ro
      - /srv/source/certs:/sources/certs:ro

Overlapping passes are prevented in both modes by an advisory file lock (flock) on /tmp/.docker-rsync-scheduler.lock, so a manual docker exec pass that races a scheduled one (or the built-in ticker) will skip rather than run a second concurrent pass. Ofelia's no-overlap is still recommended to avoid queuing redundant triggers.

Configuration reference

Environment variables

Variable	Description	Default	Required
CONFIG_PATH	Path to the YAML config inside the container	/config/config.yaml	No
SYNC_INTERVAL	Built-in scheduler cadence as a Go duration (e.g. 6h, 1h, 30m). The first pass runs at startup; subsequent passes fire every interval thereafter. Set to off (or disabled/0) to disable the built-in scheduler and trigger passes externally — see Scheduling modes. Falls back to 6h on an unset or unparseable (non-sentinel) value.	6h	No
SYNC_TIMEOUT	Per-job rsync timeout as a Go duration (e.g. 10m, 1h). Falls back to the default on unset or unparseable values.	10m	No
LOG_LEVEL	Log level: debug, info, warn, or error	info	No

Config schema (config.yaml)

A ready-to-edit config.example.yaml ships in the repo — copy it to config.yaml and edit. The container fails fast with a clear error if the config is missing or invalid.

jobs:
  - name: certs                          # required, unique, used as a log key
    local: /sources/certs                # required, absolute path inside the container
    remote_host: root@192.0.2.10         # required, [user@]host (DNS, IPv4, or IPv6 literal)
    remote_path: /srv/certs              # required, absolute path on the remote
    remote_uid: 1000                     # optional; with remote_gid -> rsync --chown=uid:gid
    remote_gid: 1000                     # optional
    ssh_key: /keys/id_ed25519            # required, private key path inside the container
    delete: true                         # optional, default false -> rsync --delete when true
    max_delete: 100                      # optional; with delete -> rsync --max-delete=N (abort if a pass would delete > N files)
    excludes: ["**/locks", "**/*.lock", "logs"]  # optional, per-job exclude patterns

The remote_host field is [user@]host, where host is a DNS hostname, an IPv4 address, or an IPv6 literal. Write IPv6 literals as the bare address (2001:db8::1 or user@2001:db8::1) — the brackets rsync's host:path syntax needs are added for you. A host containing a colon that is not a valid IP (a trailing colon, or an incomplete address) is rejected at startup so it can't be misread as rsync's daemon-mode :: separator. Link-local IPv6 with a zone id (fe80::1%eth0) is not supported; use a global or ULA address, or define an ssh_config Host alias and reference the alias name.

Every job also receives a fixed set of global excludes: .stfolder, .stversions, .DS_Store, Thumbs.db. Each job is pushed with rsync -rlptD (archive minus owner/group/ACL/xattr) plus --stats, the per-job and global excludes, and the -e "ssh -i <key> -o StrictHostKeyChecking=accept-new -o BatchMode=yes -o ConnectTimeout=10" transport.

Volumes

Mount	Description
/config/config.yaml	The YAML config (mount read-only). Override the path with CONFIG_PATH.
/config/known_hosts	Optional SSH known_hosts file (mount read-only). When present, enables strict host-key pinning instead of TOFU. See SSH host-key verification.
/keys/<name>	SSH private key(s). Mount read-only; the host file must be mode 0600.
(your sources)	The local directories referenced by your jobs. Mount read-only.

Healthcheck

The built-in healthcheck (docker-rsync-scheduler health) checks for a marker file that is set after each sync pass: healthy when the most recent pass had zero failed jobs, unhealthy when any job failed. Empty-source skips count as success. The container recovers automatically on the next clean pass — no restart required. In built-in mode it begins unhealthy, runs one pass at startup, and transitions accordingly, so size healthcheck.start_period for the time the initial pass may take. In external mode the container starts healthy (idle, nothing has failed) and each triggered sync updates the marker.

Because an empty source is skipped as a success, a job whose source silently becomes empty (for example a read-only bind mount that failed to mount and Docker materialised as an empty directory) keeps the container healthy and never logs at level=ERROR — it is invisible to both the error-level and heartbeat-absence alerts. Each skip emits a level=WARN msg="skip empty source" line and the sync cycle complete heartbeat carries a skipped count; add a Loki alert on a persistently non-zero skipped (or skipped == jobs) across several consecutive passes, or on the recurring skip empty source warning, to catch a vanished source before the remote mirror goes stale.

HEALTHCHECK --interval=60s --timeout=5s --retries=3 --start-period=120s \
    CMD ["/usr/local/bin/docker-rsync-scheduler", "health"]

Alerting

docker-rsync-scheduler has no metrics endpoint; its operational state is in its logs (structured slog in logfmt). Ship the container's logs to Loki (Grafana Alloy's Docker log discovery does this with no configuration) and evaluate these with Loki's ruler; firing alerts deliver through your Alertmanager exactly like Prometheus metric alerts. These rules assume the built-in scheduler is running; see the caveat after the group for external-trigger deployments.

groups:
  - name: docker-rsync-scheduler
    rules:
      - alert: RsyncSchedulerSyncFailed
        expr: |
          sum by (container) (count_over_time(
            {container="rsync"} |= `sync failed` [15m]
          )) > 0
        for: 0m
        labels:
          severity: warning
        annotations:
          summary: "docker-rsync-scheduler failed a sync job"
          description: >
            A job logged "sync failed": its rsync exited non-zero (an ssh or
            transport error, a remote-side failure, or a per-job timeout; the
            line carries rsync_exit, timed_out, and a bounded stderr tail).
            That job's remote mirror is now stale. Check the remote host, the
            ssh key, and connectivity.
      - alert: RsyncSchedulerStalled
        expr: |
          absent_over_time({container="rsync"} |= `sync cycle complete` [8h])
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "docker-rsync-scheduler has not completed a sync pass in 8h"
          description: >
            docker-rsync-scheduler logs a "sync cycle complete" line at the end
            of every pass that runs; the built-in scheduler runs one at startup
            and then every SYNC_INTERVAL (default 6h). None in 8h while the
            container is up means the scheduler is wedged or dead, which a
            fault-only ruleset misses because a stalled scheduler emits no
            "sync failed" line either. Restart the container.

With the built-in scheduler (SYNC_INTERVAL set to a duration) the sync runs in the container's PID 1, so its logs reach Loki. In external-trigger mode (SYNC_INTERVAL=off) each pass runs in a docker exec child, not PID 1, so its logs never reach the container's log stream and neither rule can see them; drive alerting from your external scheduler's own job-failure reporting and the container health marker instead (see Scheduling modes). The "sync cycle complete" line is emitted whether a pass finished clean or with failures, so the stall rule is a pure deadman for a scheduler that has stopped running, while per-job failures are caught by the fault rule.

Thresholds and the severity label are starting points: size the stall window to your SYNC_INTERVAL (the 8h default assumes the 6h built-in interval), adjust the container selector (or job / service, depending on your log collector) to your deployment, and route by whatever labels your Alertmanager uses. To also catch a source that has silently gone empty (skipped as a success, so it never trips the fault rule), see the skipped / skip empty source note under Healthcheck.

Security

No network listener, no HTTP server, no exposed ports. The image ships openssh-client only — no sshd. Each job is executed with an explicit argument slice via exec.CommandContext; the -e "ssh ..." value is a single argument that rsync splits internally, so nothing is passed through a shell. Config is validated at startup: required fields present, names unique, local/remote_path absolute, remote_host matched against a strict pattern, the ssh key readable, and every field rejected if it contains shell metacharacters or control characters (defense-in-depth, since the arg-list exec already prevents interpretation).

SSH host-key verification

By default the container uses StrictHostKeyChecking=accept-new (Trust On First Use). This lets a fresh deploy connect without pre-provisioning host keys, but trusts the first key it sees.

For stricter security, mount a read-only known_hosts file at /config/known_hosts. When this file is present the container switches to StrictHostKeyChecking=yes with an explicit UserKnownHostsFile, rejecting connections to any host whose key does not match the pinned entry. This prevents MITM attacks at the cost of requiring the operator to maintain the known_hosts file.

Generate it from your remote:

ssh-keyscan -t ed25519 192.0.2.10 > known_hosts

Then mount it into the container:

volumes:
  - ./known_hosts:/config/known_hosts:ro

Tool	Result
govulncheck	No vulnerabilities in call graph
golangci-lint (gosec, gocritic)	0 issues
trivy	Inherits the Alpine base image scan
gitleaks	No secrets detected
hadolint	Clean

Why it runs as root. The container runs as root by design: it must read host-owned source files (e.g. a host UID like 1000) across multiple bind mounts. A fixed non-root USER would break this. Mount sources read-only and use a dedicated, least-privilege SSH key on the remote.

Dependencies

All dependencies are updated automatically via Renovate. Base images and Go modules are pinned by digest/version; the rsync/openssh-client apk packages are installed unpinned so they track the digest-pinned base.

Dependency	Source
golang	Go
alpine	Docker Hub
rsync	Alpine
openssh-client	Alpine

Runtime Go modules: github.com/cplieger/health, github.com/cplieger/scheduler, github.com/cplieger/slogx, and gopkg.in/yaml.v3.

Credits

This project packages rsync (GPL-3.0) and the OpenSSH client (BSD) into a container image. All credit for those tools goes to their upstream maintainers.

Contributing

Issues and pull requests are welcome. Please open an issue first for larger changes so the approach can be discussed before implementation.

Disclaimer

This project is built with care and follows security best practices, but it is intended for personal / self-hosted use. No guarantees of fitness for production environments. Use at your own risk.

This project was built with AI-assisted tooling using Claude Opus and Kiro. The human maintainer defines architecture, supervises implementation, and makes all final decisions.

License

This project is licensed under the GNU General Public License v3.0.

Documentation

Overview

Package main runs a scheduled rsync-over-ssh daemon that pushes local directories to a remote host.