Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

freshdock documentation

freshdock is a health-gated Docker container auto-updater in a single Rust binary, driven by container labels and environment variables. New here? Start with the quickstart.

The freshdock.dev site covers why freshdock exists and how it compares with Watchtower; this book covers how it works, down to every label, flag, and environment variable.

Getting started

Reference

Project & process

Runnable example stacks: examples/compose/.

Quickstart

Get freshdock watching a container in about a minute. No config file is required: freshdock is driven by container labels and environment variables, notifications included.

1. Install

Pick one (full options in the README):

cargo install freshdock                          # from crates.io
# or
docker pull ghcr.io/turbootzz/freshdock:latest   # container image

2. See what freshdock would do (read-only)

Label a container to opt it in, then run check, which never changes anything:

services:
  web:
    image: nginx:1.27
    labels:
      - "freshdock.enable=true"   # opt in; mode defaults to watch
freshdock check

You get a table of opted-in containers and whether each has an update available. The CLI reference explains what the status cells mean.

3. Run the daemon

freshdock is opt-in and defaults enabled containers to watch (notice updates, never restart). Start the scheduler:

docker run -d \
  --name freshdock \
  --restart unless-stopped \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  ghcr.io/turbootzz/freshdock:latest run

This is the minimal-watch.yml example. A read-only socket is enough for watch.

To avoid labelling every container, FRESHDOCK_WATCH_ALL=true includes them all unless they opt out. See watching every container.

4. Let it update something

Switch a container to an updating mode and give the daemon a writable socket:

    labels:
      - "freshdock.enable=true"
      - "freshdock.mode=nightly"   # check at 04:00 daily; recreate if newer
      - "freshdock.notify=true"    # if you've configured a notifier

Updates are health-gated with automatic rollback: a broken new image is reverted, not left running.

Next steps

Coming from Watchtower?

Watchtower was archived in December 2025. freshdock is a from-scratch successor, so the concepts map closely but the spelling differs. This page translates the labels and flags you already know. For a feature scorecard against Watchtower, see freshdock.dev/watchtower-alternative.

The biggest difference: freshdock is opt-in. Watchtower updates every container unless you exclude it; freshdock ignores every container unless you set freshdock.enable=true. An enabled container with no explicit mode defaults to watch (detect and notify, never restart), so nothing is recreated until you ask with a mode like live or nightly. One env var flips this back, see keeping Watchtower’s opt-out model.

Config is environment-first, like Watchtower: fleet-wide settings, registry credentials, run flags, and notification targets are all environment variables, so a container deployment needs no config file. Notification targets take a shoutrrr-style URL, so WATCHTOWER_NOTIFICATION_URL=discord://token@id becomes FRESHDOCK_NOTIFY_OPS_URL=discord://token@id almost verbatim. See the configuration reference.

Keep your labels

freshdock reads the com.centurylinklabs.watchtower.* labels themselves, so an existing fleet usually needs no relabelling: swap the updater container and go. A freshdock.* label always wins when both are present, so you can migrate label-by-label at your own pace.

What’s honoured directly:

Watchtower labelEffect in freshdock
com.centurylinklabs.watchtower.enable=trueSame as freshdock.enable=true. The container lands on freshdock’s safe default mode (watch, or [settings] default_mode), so it will not auto-update like Watchtower did until you give it an active mode.
com.centurylinklabs.watchtower.enable=falseAn explicit opt-out: never enabled, never re-run as a compose one-shot, never repaired as a network-namespace sidecar (an absent label allows both of those). Under FRESHDOCK_WATCH_ALL it is the exclusion label.
com.centurylinklabs.watchtower.monitor-only=trueSame as freshdock.mode=watch; beats [settings] default_mode.
com.centurylinklabs.watchtower.lifecycle.pre-update / post-updateSame as the freshdock.lifecycle.* hooks.
...lifecycle.pre-update-timeout / post-update-timeoutHonoured in Watchtower’s unit (minutes, converted; 0 = unlimited). The freshdock.lifecycle.*-timeout labels count seconds.

Not supported, logged once and ignored: no-pull, depends-on, scope, lifecycle.pre-check / post-check. freshdock always pulls before recreate and has no per-cycle check hooks. Dependency ordering is not read from the Watchtower label: inside a Docker Compose project freshdock reads Compose’s own depends_on graph instead, and rolls the project out as one unit. See Compose projects.

Keeping Watchtower’s opt-out model

Set FRESHDOCK_WATCH_ALL=true (or [settings] watch_all = true) and freshdock treats every running container as enabled unless it opts out, the way Watchtower did. Pair it with FRESHDOCK_DEFAULT_MODE to say how those containers update; without it they land on watch and nothing is recreated.

services:
  freshdock:
    image: ghcr.io/turbootzz/freshdock:latest
    command: ["run"]
    environment:
      FRESHDOCK_WATCH_ALL: "true"         # every container, unless it opts out
      FRESHDOCK_DEFAULT_MODE: "nightly"   # 04:00 daily, in place of a global WATCHTOWER_SCHEDULE
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped

  db:
    image: postgres:16
    labels:
      - "com.centurylinklabs.watchtower.enable=false"   # still an exclusion

The exclusion labels you already have keep working: com.centurylinklabs.watchtower.enable=false, freshdock.enable=false, and freshdock.mode=off all opt a container back out. Explicit freshdock.* labels are unaffected: a freshdock.mode=weekly label still wins over FRESHDOCK_DEFAULT_MODE.

freshdock excludes its own container from this, so it never tries to update itself. Detection and the custom-hostname caveat are covered in watching every container.

Label translation

If you prefer clean labels, or need the finer-grained knobs, this is the native spelling:

Watchtower labelfreshdock labelNotes
com.centurylinklabs.watchtower.enable=truefreshdock.enable=trueOpt in.
com.centurylinklabs.watchtower.enable=false (with global watch)omit the labels, or freshdock.mode=offfreshdock ignores unlabelled containers, so there is usually nothing to disable. Keep the label on a compose one-shot or a network-namespace sidecar you never want touched. With FRESHDOCK_WATCH_ALL on, keep the exclusion label (either spelling works).
com.centurylinklabs.watchtower.monitor-only=truefreshdock.mode=watchDetect and notify, never pull or recreate.
(no per-container schedule)freshdock.mode=nightly/weekly/monthly + freshdock.schedule=<cron>Scheduling is per container in freshdock, not a single global cron.
com.centurylinklabs.watchtower.lifecycle.pre-updatefreshdock.lifecycle.pre-updateExec in the old container, but stricter: any non-zero exit (not only 75), a timeout, or a failed exec skips the update. Timeout labels count seconds in freshdock, minutes in Watchtower. See lifecycle hooks.
com.centurylinklabs.watchtower.lifecycle.post-updatefreshdock.lifecycle.post-updateBest-effort exec in the new container after the health gate.
com.centurylinklabs.watchtower.lifecycle.pre-check / post-check(no equivalent)freshdock has no per-cycle check hooks.
com.centurylinklabs.watchtower.no-pull=true(no equivalent)freshdock always pulls before recreate; there is no “recreate without pull”.
com.centurylinklabs.watchtower.depends-on(no equivalent)The label itself is ignored. Inside a Compose project freshdock uses Compose’s native depends_on graph, which needs no extra labels, see Compose projects. Outside one, containers are processed independently.

Flag / environment translation

Watchtower flag / envfreshdock equivalentNotes
--interval / WATCHTOWER_POLL_INTERVALfreshdock run --interval <seconds> or FRESHDOCK_INTERVALCadence for live/watch containers.
--schedule / WATCHTOWER_SCHEDULE (global cron)per-container freshdock.mode + freshdock.schedulefreshdock schedules each container on its own mode.
--monitor-only / WATCHTOWER_MONITOR_ONLYfreshdock.mode=watchPer container, not global.
--label-enable / WATCHTOWER_LABEL_ENABLE(the default)freshdock is label-gated out of the box; freshdock.enable=true is required. Set FRESHDOCK_WATCH_ALL=true for the opt-out behaviour you get without this flag in Watchtower.
--enable-lifecycle-hooks / WATCHTOWER_LIFECYCLE_HOOKS(not needed)Setting a freshdock.lifecycle.* label is the opt-in; there is no global switch.
(no global default mode)[settings] default_mode or FRESHDOCK_DEFAULT_MODESets the fallback mode for enabled containers with no freshdock.mode label. A freshdock.mode label still wins per container.
--cleanup / WATCHTOWER_CLEANUP[settings] cleanup = true, FRESHDOCK_CLEANUP=true, or freshdock.cleanup=true per containerOff by default. Removes the replaced image after a healthy update; add [settings] prune_dangling = true (or FRESHDOCK_PRUNE_DANGLING=true) for a daemon-wide dangling prune. The replaced container archive is always removed regardless.
--remove-volumes / WATCHTOWER_REMOVE_VOLUMES(no equivalent)freshdock never removes volumes; recreate preserves all mounts.
--rolling-restart / WATCHTOWER_ROLLING_RESTART(not applicable)freshdock recreates one container at a time and health-gates each.
--notifications + WATCHTOWER_NOTIFICATION_URL (shoutrrr)FRESHDOCK_NOTIFY_<NAME>_URL (shoutrrr-style URL) or a [notifications.<name>] tableNear drop-in: discord://token@id, telegram://token@telegram?chats=id, smtp://..., or https://.... No file required. Add FRESHDOCK_NOTIFY_<NAME>_TRIGGERS to filter events. See notifications.
WATCHTOWER_NOTIFICATIONS_LEVEL / per-event configper-target triggers = ["available","succeeded","failed"]Subscribe each target to the events it cares about.
REPO_USER / REPO_PASS (registry auth)FRESHDOCK_REGISTRY_* env (or a [registry.<name>] table)Per-registry credentials. An env token alone is enough; no file needed.
DOCKER_HOSTDOCKER_HOSTThe same standard Docker env var, with the same schemes: unix://, tcp://, http://, https://, ssh://. Unset, freshdock falls back to /var/run/docker.sock and then to Podman’s sockets.

A worked example

Watchtower (global daily updates, one excluded container, Discord notifications):

docker run -d --name watchtower \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e WATCHTOWER_SCHEDULE="0 0 4 * * *" \
  -e WATCHTOWER_NOTIFICATION_URL="discord://token@id" \
  containrrr/watchtower
services:
  db:
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

The freshdock equivalent: schedule and notify opt-in move onto the containers as labels, and the Discord target is a single env var, with no file at all.

services:
  freshdock:
    image: ghcr.io/turbootzz/freshdock:latest
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      # near-verbatim from WATCHTOWER_NOTIFICATION_URL=discord://token@id
      - "FRESHDOCK_NOTIFY_OPS_URL=discord://<token>@<id>"
      - "FRESHDOCK_NOTIFY_OPS_TRIGGERS=succeeded,failed"
    command: ["run"]
  app:
    image: ghcr.io/example/app:latest
    labels:
      - "freshdock.enable=true"
      - "freshdock.mode=nightly"      # 04:00 daily by default
      - "freshdock.notify=true"
  db:
    image: postgres:16
    # no freshdock.* labels, so it is ignored entirely (no need to "disable" it)

See examples/compose/ for complete stacks that pass docker compose config.

Troubleshooting

Symptom-first fixes for the most common first-run issues. Each section links to the reference page with the full story.

Contents


freshdock can’t reach the daemon / wrong socket

freshdock probes, in order: DOCKER_HOST (any scheme: unix://, tcp://, http://, https://, ssh://), then /var/run/docker.sock, then Podman’s sockets ($XDG_RUNTIME_DIR/podman/podman.sock, /run/user/$UID/podman/podman.sock, /run/podman/podman.sock). The startup log says which family answered and which API version was negotiated:

INFO freshdock::docker: connected to the local Docker socket
INFO freshdock::docker: negotiated Docker API version api_version=1.53

If those lines are missing, freshdock never reached a daemon; it fails at connect time rather than part-way through a cycle. Point it at the right socket with DOCKER_HOST=unix:///path/to/podman.sock (or a tcp:// endpoint). DOCKER_HOST, once set, is authoritative: there is no fallback to the local socket if that endpoint is dead.

In CI, the live test suites skip themselves when no daemon answers. Set FRESHDOCK_LIVE_REQUIRED=1 (as the repo’s own live gate job does) to turn that skip into a failure, so a missing daemon can’t pass as a green run.

See Deployment: which socket freshdock uses.

permission denied on the Docker socket

freshdock can’t reach /var/run/docker.sock. As a container, make sure the socket is mounted; as a host binary, the user needs to be in the docker group (or run via systemd with SupplementaryGroups=docker).

See Deployment: Docker socket permissions.

no CA certificates were found

The TLS root store is empty, so no HTTPS call to a registry or a notification target could be verified. check and run refuse to start; recreate builds no HTTP client, so it is unaffected. The official image ships a bundle. For a custom image, install ca-certificates (Debian/Ubuntu/Alpine package name) or mount the host bundle:

--mount type=bind,src=<host bundle>,dst=/etc/ssl/certs/ca-certificates.crt,readonly

The host path differs per distribution: Debian/Ubuntu/Alpine /etc/ssl/certs/ca-certificates.crt, Fedora/RHEL /etc/pki/tls/certs/ca-bundle.crt, openSUSE /etc/ssl/ca-bundle.pem.

If SSL_CERT_FILE or SSL_CERT_DIR is set, only that location is read and the usual system paths are skipped, so a stale value in either produces this error on a host that does have a bundle. Unset them, or point them at a real one.

freshdock sees my container but never updates it

Almost certainly the container is in watch mode, the default for an enabled container with no freshdock.mode label. watch detects and notifies, but never pulls or restarts; that is the opt-in-by-design safety net, not a bug. To update, set an updating mode:

    labels:
      - "freshdock.enable=true"
      - "freshdock.mode=nightly"   # or live / weekly / monthly

Or change the fleet-wide fallback with [settings] default_mode / FRESHDOCK_DEFAULT_MODE.

See Scheduling & update modes.

A scheduled run (nightly/weekly/monthly), and any run that found, applied or failed an update, ends with one summary line. A live/watch poll where nothing happened logs it at debug only, so it doesn’t repeat every few minutes; set RUST_LOG=freshdock=debug to see it for those too:

INFO freshdock::scheduler: scheduler: run summary checked=3 up_to_date=2 unknown_digest=1 available=0 updated=0 not_updated=0 pinned=0 failed=0
  • up_to_date for a container you expected to move usually means its tag is a fixed version (5.1.3). freshdock follows the digest behind the tag you run and never switches to a newer tag; change the tag yourself to upgrade.
  • unknown_digest means freshdock has nothing local to compare against, typically a locally built image. For a container you labelled, a warning naming it is logged the first time.
  • not_updated is an update that was due but rolled back or was deferred by a hook; failed is an inspect, registry or recreate error. The lines above the summary say which.
  • pinned is covered below.

My container doesn’t appear in check at all

The freshdock.enable=true label is missing (or typo’d). freshdock is opt-in: an unlabelled container is invisible by design, and a misspelled label name is indistinguishable from no label, so there is no error for it. Verify with:

docker inspect --format '{{json .Config.Labels}}' <container> | grep freshdock

See Configuration: labels.

check reports an update that never goes away

Older versions compared upstream against a single entry of the local image’s RepoDigests. Docker records one image under several manifest digests once a multi-arch index is republished without a change to your platform’s manifest, so the comparison could never match again: update? = yes survived docker pull, freshdock recreate, and every scheduled run.

Upgrade past #74. On an older build, drop the stale digest references:

docker image inspect <image> --format '{{json .RepoDigests}}'
docker rmi <repo>@sha256:<stale digest>

See freshdock check.

A container is reported as pinned (no check)

Its image is pinned to a digest (repo@sha256:<digest>), so there is no moving tag to follow and freshdock will never update it. Switch to a tag (repo:1.27) if you want updates.

Earlier releases also showed this for a container whose tag had moved on locally, typically a second container on the same tag right after the first one was updated. The container is now probed by the reference it was created from, so that case reports a real update? verdict and is updated like any other.

A container created from an image id rather than a tag is genuinely pinned. That includes a sidecar freshdock repaired while its tag pointed at a newer image than the one it ran (see the sidecar section), and any sidecar an earlier release repaired. Put it back on a tag with docker compose up -d <service>, or recreate it from its tag by hand.

See Configuration: pinned images.

Updates fail with a read-only socket

A socket mounted :ro is enough for check and watch, but an updating mode (live / nightly / weekly / monthly) has to stop, create, and start containers, which needs a writable socket mount (drop the :ro).

See Deployment: socket read-only vs writable.

A sidecar on network_mode: container:X lost its network

It shouldn’t any more. Recreating X necessarily destroys the network namespace its sidecars share, so freshdock repairs them. The rules:

  • Before stopping X, freshdock finds every running container whose HostConfig.NetworkMode is container:X, and re-creates each one afterwards. This also happens after a rollback, since restarting the restored container gives it a fresh namespace too.
  • The repair runs before X’s -old- archive is removed, because Docker refuses to remove a container whose namespace a running container still holds.
  • Compose’s network_mode: service:X becomes container:<id of X> on disk. That dead id is rewritten to the new container’s id on the way through. A literal name reference is left alone, and after a rollback nothing is rewritten, because the restored container owns its original id again.
  • The sidecar needs no freshdock labels: it is not being updated, only repaired, so the freshdock.enable gate does not apply to it.
  • It is re-created from its own tag while that tag still points at the image it runs, so a labelled sidecar stays managed. Once the tag has moved locally, it is re-created from the exact image ID instead, so a repair can never sneak an upgrade in; a warning says it is now pinned. No health gate and no lifecycle hooks run.
  • A sidecar that explicitly opts out with freshdock.enable=false or freshdock.mode=off is skipped, with a warning naming it. It keeps a dead namespace until you restart it yourself. An absent label is not an opt-out, since repairing unlabelled bystanders is the point.
  • freshdock itself is skipped, with a warning, when deployed with network_mode: container:<vpn> and pointed at that same container. Stopping it would kill the daemon mid-cycle, so restart the freshdock container by hand afterwards.
  • Re-attachment is best-effort. A failure logs failed to re-attach network-namespace dependent, naming the container; the update itself still stands.
  • When the failure lands after the sidecar was renamed away, freshdock renames it back and starts it again, so a failed repair leaves a running container rather than a stopped -old- one. If even that fails, the warning names the archive you have to recover by hand.
  • Known limits: sidecars referencing X by an id prefix shorter than 12 characters are not recognised, and dependency chains are not followed transitively.

See Manual test: network-namespace dependents.

My app updated but its database migration didn’t run

A one-shot migrate service sits in exited (0) after its last run, so a per-container updater never sees it. The app image moves, the app is recreated, and the migration stays on the old code: new application code against an old schema. Nothing crashes, so no health gate catches it.

freshdock handles this natively as long as the migration is wired up the normal Compose way:

  web:
    depends_on:
      migrate:
        condition: service_completed_successfully

That condition is what marks migrate as a one-shot the project waits on. freshdock then re-runs it before recreating web, even though migrate has no freshdock labels of its own.

If it still doesn’t run, check in this order:

  • Is the condition actually service_completed_successfully? A bare depends_on: [migrate] (or condition: service_started) does not mark it as a one-shot, and freshdock will not touch an unlabelled container without it.
  • Is compose_aware on? It is by default; FRESHDOCK_COMPOSE_AWARE=0 or [settings] compose_aware = false disables it.
  • Does the migration opt out? freshdock.enable=false, com.centurylinklabs.watchtower.enable=false, or freshdock.mode=off on the one-shot are honoured, one-shot or not.
  • Do the two share an image? The rollout is triggered by a container freshdock watches, and only members on that same image are updated. In the usual pattern migrate and web both run app:latest, which is what makes the link.
  • Was it still running? A one-shot that is mid-run is left alone rather than stomped; it is picked up on the next cycle.

Full rules: Compose projects.

A compose rollout aborted

rollout ABORTED: shop-migrate-1 exited with code 1. The services after this
point were not touched and are still running their previous image.

This is the safety behaviour, not a bug: a one-shot the project waits on did not exit 0, so freshdock stopped rather than starting new code against a migration that did not complete. Your stack is still running its previous image.

The failed container is kept on purpose, and so is the archive of its previous instance, since the logs are the only record of what went wrong:

docker logs shop-migrate-1

Fix the cause, then re-run it with freshdock recreate <the labelled service>, or wait for the next scheduled cycle. Nothing is rolled back automatically: the command already ran and may have applied part of its work, so restoring the previous container object would change nothing about your database while throwing the evidence away. See when a one-shot fails.

Where are the logs?

  • Container: docker logs freshdock
  • systemd: journalctl -u freshdock

Verbosity is controlled by RUST_LOG (default info); RUST_LOG=freshdock=debug is the useful next step. Secrets (tokens, passwords, webhook URLs) are redacted at every level, including trace.

See Configuration: environment variables.


Still stuck? Open an issue with the freshdock check output and a RUST_LOG=freshdock=debug log excerpt.

Configuration reference

This is the single source of truth for everything freshdock reads. Environment variables are the primary way to configure a deployment and override the file per field, so a container never has to mount a freshdock.toml. The one thing they can’t express is a registry whose host has dots in it, since that name can’t be spelled as an env-var name.

SourceCovers
Labels on each containerper-container behaviour: what to update and when
Environment variablesfleet-wide settings, registry credentials, notification targets, the run flags
freshdock.toml (optional)credentials for a registry host with dots; notification targets you’d rather keep in a file

New here? Start with the quickstart. For the command flags, see the CLI reference.

Contents


Labels

freshdock is opt-in: a container with no freshdock.enable=true is ignored entirely. (That default can be inverted fleet-wide, see watching every container.) All behaviour is driven by these Docker labels, set in compose under labels: or with docker run --label.

LabelValuesDefaultMeaning
freshdock.enabletrue / falsefalse (true under watch_all)Master switch. Without true, the container is invisible to freshdock and every other label is ignored. Under watch_all an absent label counts as enabled (the other labels then apply as usual) and false is the explicit opt-out.
freshdock.modelive / nightly / weekly / monthly / watch / offwatch (or [settings] default_mode)How and when this container updates. See scheduling.
freshdock.schedule5-field cronthe mode’s defaultOverride the cron for a calendar mode. Ignored for live / watch / off. See cron syntax.
freshdock.notifytrue / falsefalseEmit notifications for this container’s update events. Requires a configured [notifications.*] target. See notifications.
freshdock.cleanuptrue / false[settings] cleanup (else false)After a healthy update, remove the image the old container ran. Overrides the global [settings] cleanup. See health & cleanup.
freshdock.lifecycle.pre-updateshell command(none)Exec’d (sh -c) in the old container before it is stopped; a non-zero exit, timeout, or exec failure skips the update. See lifecycle hooks.
freshdock.lifecycle.pre-update-timeoutseconds60Time budget for the pre-update hook; 0 disables the timeout.
freshdock.lifecycle.post-updateshell command(none)Exec’d (sh -c) in the new container after it passes the health gate. Best-effort; a failure never fails the update. See lifecycle hooks.
freshdock.lifecycle.post-update-timeoutseconds60Time budget for the post-update hook; 0 disables the timeout.

Values are case-insensitive and tolerate surrounding whitespace. An invalid value is reported with the offending label named.

Notes:

  • Watchtower labels are read too: com.centurylinklabs.watchtower.enable, com.centurylinklabs.watchtower.monitor-only, and the watchtower lifecycle.pre-update/post-update hook labels (whose timeouts are in minutes) are honoured as fallbacks, so a migrated fleet needs no relabelling. A freshdock.* label always wins over its watchtower counterpart. See coming from Watchtower.
  • With freshdock.enable=true and no freshdock.mode, the mode is watch (detect and notify, never mutate). Change that fleet-wide fallback with [settings] default_mode or FRESHDOCK_DEFAULT_MODE; an explicit freshdock.mode label always wins.
  • freshdock.schedule refines only the calendar modes (nightly / weekly / monthly). live and watch are polled on the daemon’s run --interval instead and ignore the label. See scheduling.
  • The post-update health timeout and the grace period for containers without a healthcheck are hardcoded: not configurable by label, file, or env. See health & rollback: timings.
  • A container whose image is pinned to a digest (repo@sha256:...) has no moving tag to follow. freshdock reports it as pinned (no check) and never updates it.

Compose projects

A container in a multi-service Docker Compose project is not updated on its own. freshdock reads the project’s com.docker.compose.* labels, re-runs the one-shots the project waits on (condition: service_completed_successfully) first, then updates the rest in depends_on order. A failed one-shot aborts the rollout. The label gate still applies, with one exception: an unlabelled one-shot that the project waits on is re-run anyway. Explicit opt-outs and watch mode still win.

On by default; [settings] compose_aware = false or FRESHDOCK_COMPOSE_AWARE=false turns it off. [settings] one_shot_timeout (FRESHDOCK_ONE_SHOT_TIMEOUT) sets how long a one-shot may run, default 600 seconds. Full rules in Compose projects.


Watching every container

[settings] watch_all (or FRESHDOCK_WATCH_ALL=true) inverts the opt-in gate, the way Watchtower worked: every running container counts as enabled unless it opts out. An absent freshdock.enable label then means enabled, not ignored. The default is false.

Three labels opt a container back out:

LabelValueEffect
freshdock.enablefalseInvisible again, as without watch_all.
com.centurylinklabs.watchtower.enablefalseSame, so exclusions from a Watchtower fleet keep working.
freshdock.modeoffNever updated. It still appears in freshdock check with mode off.

A container enabled this way takes its mode from default_mode (FRESHDOCK_DEFAULT_MODE), or watch when that is unset. Turning watch_all on therefore never restarts anything by itself; it starts updating once default_mode names an updating mode. Explicit labels always win: a container with freshdock.mode=live keeps live, and freshdock.enable=true behaves as it did before.

freshdock skips its own container when it enables containers this way, so the daemon never tries to update itself. It recognises itself by the Docker default hostname (the short container id). If you give the freshdock container a custom hostname, add freshdock.mode=off to it, or set freshdock.enable=true and an updating freshdock.mode on it if you do want it updated. The same applies when freshdock runs with network_mode: container:<name>: it then carries the namespace owner’s hostname, so label both containers explicitly in that setup. The skip is evaluated per invocation, so a freshdock check run outside the daemon’s container still shows a row for it.

freshdock recreate <name> follows the same rules: with watch_all on, an unlabelled container passes the gate, while freshdock.enable=false and freshdock.mode=off are still refused.

A runnable stack is in watch-all.yml.


Environment variables

Environment variables override the file per field: a _TOKEN variable on its own replaces the file token while keeping the file username. For registry and notification targets, <NAME> is the table name, upper-cased, with - replaced by _.

VariableSets / overridesNotes
FRESHDOCK_CONFIGconfig file pathThe --config flag wins over it.
FRESHDOCK_REGISTRY_<NAME>_USERNAME[registry.<name>] username<NAME> is an alias (DOCKERHUB, GHCR, QUAY, LSCR). Hosts with dots can’t be expressed unambiguously; configure those in the file.
FRESHDOCK_REGISTRY_<NAME>_TOKEN[registry.<name>] tokenA token, with or without a username, is enough to create a registry entry from the environment alone.
FRESHDOCK_NOTIFY_<NAME>_URLdeclares a notification targetA shoutrrr-style URL (discord://, telegram://, smtp://, or https://) creates the target from the environment alone. A bad URL warns and is skipped.
FRESHDOCK_NOTIFY_<NAME>_TRIGGERSan env-declared target’s triggersOptional comma list of available,succeeded,failed; omit for all three. Pairs with the _URL variable.
FRESHDOCK_NOTIFY_<NAME>_BOT_TOKENa Telegram target’s bot_tokenOverrides the secret on an already-declared target (file or _URL).
FRESHDOCK_NOTIFY_<NAME>_PASSWORDan SMTP target’s passwordSame: overrides the secret on an already-declared target.
FRESHDOCK_DEFAULT_MODE[settings] default_modeOne of live/nightly/weekly/monthly/watch/off. An invalid value warns and the file value (else watch) applies.
FRESHDOCK_WATCH_ALL[settings] watch_alltrue/false/1/0, case-insensitive. Treats every running container as enabled unless it opts out. See watching every container.
FRESHDOCK_COMPOSE_AWARE[settings] compose_awaretrue/false/1/0, case-insensitive. On by default. Rolls a Compose project out as one unit. See Compose projects.
FRESHDOCK_ONE_SHOT_TIMEOUT[settings] one_shot_timeoutWhole number of seconds a compose one-shot may run before the rollout gives up. Default 600. An invalid value warns and the file value applies.
FRESHDOCK_CLEANUP[settings] cleanuptrue/false/1/0, case-insensitive. An invalid value warns and the file value applies.
FRESHDOCK_PRUNE_DANGLING[settings] prune_danglingSame boolean forms as FRESHDOCK_CLEANUP.
FRESHDOCK_INTERVAL, FRESHDOCK_TICK, FRESHDOCK_STOP_TIMEOUTthe run flags of the same nameThe flag wins over the env var. An invalid value is a startup error (it is the flag). See the CLI reference.
NO_COLOR--no-colorAny non-empty value disables colored output.
RUST_LOGlog verbositye.g. info, freshdock=debug, trace. Default info.
DOCKER_HOSTDocker daemon endpointunix://, tcp://, http://, https:// or ssh://. Unset: /var/run/docker.sock, then Podman’s sockets. See deployment.

freshdock --help prints the same override list (after_long_help).


The optional freshdock.toml file

freshdock runs without a file. Reach for one only when you need something environment variables can’t express:

  1. Registry credentials for a custom host with dots (e.g. registry.example.com), whose name can’t be spelled as an env-var name.
  2. A notification target you’d rather keep in a file, though these can be declared from the environment too (see FRESHDOCK_NOTIFY_<NAME>_URL and Declaring targets from the environment).

When present, it is resolved in this order:

  1. --config <path> flag
  2. $FRESHDOCK_CONFIG
  3. ./freshdock.toml in the working directory

An explicit path (flag or env) that doesn’t exist is an error; a missing default ./freshdock.toml is fine (you get an empty config). Secrets in the file are redacted in all log output, even at RUST_LOG=trace. Registry tokens, Telegram bot tokens, and SMTP passwords can come from environment variables instead; a webhook or Discord URL stays in the file unless the whole target is declared with FRESHDOCK_NOTIFY_<NAME>_URL.

The file has three top-level tables, all optional: [settings], [registry.*], and [notifications.*]. A key freshdock does not know is a startup error naming the key and its line, so a typo like watch_al cannot silently turn a setting off. The environment layer is more forgiving: an unrecognised FRESHDOCK_* variable is a startup warning naming the variable, and freshdock carries on.

[settings]

Fleet-wide defaults. Every key is optional, and each has an environment variable that overrides it.

[settings]
default_mode   = "watch"   # fallback mode; invalid value falls back to watch
watch_all        = false   # true: every running container counts as enabled unless it opts out
                           # (enable=false, watchtower enable=false, mode=off)
compose_aware    = true    # roll a Compose project out as one unit
one_shot_timeout = 600     # seconds a compose one-shot may run before the rollout gives up
cleanup          = false   # remove the replaced image after a healthy update
prune_dangling   = false   # daemon-wide dangling-image prune after each update (no per-container label)
KeyEnv varTypeDefaultNotes
default_modeFRESHDOCK_DEFAULT_MODEstring (a mode name)unset, so watchApplied to enabled containers without a freshdock.mode label. A freshdock.mode label always overrides it.
watch_allFRESHDOCK_WATCH_ALLboolfalseEnables every running container unless it opts out, and gives those containers default_mode (else watch). See watching every container.
compose_awareFRESHDOCK_COMPOSE_AWAREbooltrueTreats a Compose project as one update unit: re-runs the one-shots it waits on, updates in depends_on order, and aborts if a one-shot fails. See Compose projects.
one_shot_timeoutFRESHDOCK_ONE_SHOT_TIMEOUTinteger (seconds)600How long a compose one-shot may run before the rollout aborts. Raise it for slow migrations.
cleanupFRESHDOCK_CLEANUPboolfalseDefault for freshdock.cleanup. Best-effort; a shared image in use elsewhere is kept, and a cleanup failure never fails the update.
prune_danglingFRESHDOCK_PRUNE_DANGLINGboolfalseDaemon-wide; prunes untagged images after a success. Best-effort. No per-container label.

[registry.<name>]

One table per registry. <name> may be a friendly alias (dockerhub, ghcr, quay, lscr) or a literal host ("registry.example.com"); both fold onto the same registry as the matching image reference. For the four aliases you can skip the file entirely and set FRESHDOCK_REGISTRY_<NAME>_TOKEN (and optionally _USERNAME) instead.

[registry.ghcr]
username = "octocat"          # any non-empty value works for a GHCR PAT
token    = "ghp_xxx"          # personal access token (read:packages for GHCR)

[registry.dockerhub]
username = "myuser"           # required for Docker Hub
token    = "dckr_pat_xxx"

[registry."registry.example.com"]
token    = "..."              # username optional
KeyTypeRequiredNotes
usernamestringdepends on registryDocker Hub needs the real account name; GHCR and most others accept any non-empty value with a PAT.
tokenstring (secret)yesPassword or personal access token. Redacted in logs.

For per-registry guidance (PAT scopes, the alias list, a smoke test, and what’s out of scope), see registry-auth.md.

[notifications.<name>]

One table per target, selected by type. A target can equally be declared from the environment with FRESHDOCK_NOTIFY_<NAME>_URL; the file form below is for when you’d rather keep it in config. Every target may set an optional triggers list to subscribe to a subset of events; omit it (or use []) to receive all three (available, succeeded, failed). Payload formats and the event/mode matrix are documented in notifications.md.

[notifications.ops-webhook]
type = "webhook"
url  = "https://example.com/hooks/freshdock"
# triggers omitted: all of available, succeeded, failed

[notifications.discord]
type        = "discord"
webhook_url = "https://discord.com/api/webhooks/123/abc"
triggers    = ["succeeded", "failed"]

[notifications.tg]
type      = "telegram"
bot_token = "123456:ABC-DEF"          # or FRESHDOCK_NOTIFY_TG_BOT_TOKEN
chat_id   = "987654321"
triggers  = ["failed"]

[notifications.email]
type     = "smtp"
host     = "smtp.example.com"
port     = 587                        # optional; defaults per tls: 587/465/25
username = "[email protected]"    # username + password together, or neither
password = "s3cr3t"                   # or FRESHDOCK_NOTIFY_EMAIL_PASSWORD
from     = "[email protected]"
to       = ["[email protected]"]      # non-empty list
tls      = "starttls"                 # starttls (default) | implicit (465) | none
triggers = ["succeeded", "failed"]

Per-type keys:

typeKeysNotes
webhookurl (secret)Generic JSON POST.
discordwebhook_url (secret)Posts a coloured embed.
telegrambot_token (secret), chat_idPlain-text message via the Bot API.
smtphost, port?, username?, password? (secret), from, to (list), tls (="starttls")tls is "starttls" | "implicit" | "none" (plaintext, dev only, logged as a warning). An omitted port defaults from the mode: 587 (starttls), 465 (implicit), 25 (none); set it explicitly for a relay on another port (e.g. mailpit on 1025). The legacy starttls = true|false maps to starttls/implicit; keeping both keys is fine when they agree, a contradictory pair is an error. username+password must be set together or both omitted (anonymous relay).

All targets also accept triggers = ["available", "succeeded", "failed"] (subset allowed).


A complete example

File-free (environment only)

A deployment with a private GHCR image, cleanup on, and no notifications needs no file:

FRESHDOCK_DEFAULT_MODE=nightly
FRESHDOCK_CLEANUP=true
FRESHDOCK_REGISTRY_GHCR_USERNAME=octocat
FRESHDOCK_REGISTRY_GHCR_TOKEN=ghp_xxx

(Set these under environment: in compose, Environment= in a systemd unit, or export in a shell.)

With a file (for notifications)

To keep a notification target in the file instead of a FRESHDOCK_NOTIFY_<NAME>_URL, declare it there and keep the secret in the environment:

# freshdock.toml
[settings]
default_mode   = "watch"
cleanup        = true
prune_dangling = false

[registry.ghcr]
username = "octocat"
token    = "ghp_xxx"

[notifications.discord]
type        = "discord"
webhook_url = "https://discord.com/api/webhooks/123/abc"
triggers    = ["succeeded", "failed"]

A copy-paste starting point with every section commented out lives at freshdock.toml.example in the repository root. Runnable compose stacks live in examples/compose/.

CLI reference

freshdock has three subcommands: check (read-only), recreate (one container, manual), and run (the scheduler daemon). freshdock --help and freshdock <command> --help print the same information at the terminal.

Global options

Available on every subcommand.

OptionDefaultMeaning
--no-colorcolour onDisable ANSI colour. There is no terminal detection, so set it (or NO_COLOR to any non-empty value) when writing to a log file or another non-interactive sink.
--config <PATH>see belowPath to an optional freshdock.toml.

The config file is optional: you need one only for a registry host whose name has dots in it, since everything else, notification targets included, has an environment variable. Resolution order is --config <PATH>, then $FRESHDOCK_CONFIG, then ./freshdock.toml; an explicit path that is missing is an error, a missing default file is fine. See the configuration reference.

RUST_LOG controls log verbosity (default info; try freshdock=debug or trace). Secrets are always redacted.


freshdock check

freshdock check

Read-only. Lists every enabled container (freshdock.enable=true, or every container that does not opt out under watch_all, the daemon’s own container excepted), resolves the latest digest once per unique image, and prints a status table. It never pulls, stops, or recreates anything. The dedupe conserves Docker Hub’s anonymous budget of 100 pulls / 6 h; manifest fetches count as pulls.

The table has six columns: container, image, mode, current digest, latest digest, and update?. The update? column is yes (a newer digest exists), no (up to date), or -/? when no comparison was possible. When a digest can’t be resolved, latest digest shows the reason instead:

latest digest valueMeaning
a short digest (e.g. sha256:ab12cd...)The resolved upstream digest; compare with update?.
pinned (no check)Image is pinned to a digest, so there is no moving tag to follow.
auth required (set credentials)The registry needs credentials that aren’t configured. See registry-auth.
network unavailableThe registry couldn’t be reached; nothing is assumed.
error: container inspect failed: ...The daemon could not be asked what the container runs. A local read failure, not a registry one.
error: ...The probe failed for another reason (the message follows).

Docker can record one local image under several manifest digests: pulling a tag whose multi-arch index was republished without a change to your platform’s manifest adds the new index digest beside the old one. freshdock reports no when the upstream digest is any of them, and shows that digest as current digest.

A container can also be behind its own tag, when an earlier update moved the tag on and this one still runs the image it pointed at before. Its row shows - for current digest, since the superseded image can carry no digest of its own, and yes for update? even though that digest is already in the local store. The update then recreates it from the image the tag resolves to now.

Examples:

freshdock check                 # render the table
freshdock --no-color check      # ANSI-free, for logs
RUST_LOG=freshdock=debug freshdock check   # include registry rate-limit info

freshdock recreate <NAME>

freshdock recreate <NAME>

Manually update one container by name or ID: inspect, pull, stop, rename, create, start, then health-gate the new container and roll back to the previous one if it fails. A configured pre-update lifecycle hook runs between pull and stop and can skip the update (the command reports the skip and exits 0).

ArgumentMeaning
<NAME>Name or ID of the running container to recreate.

It ignores the scheduling modes (live, nightly, weekly, monthly, watch), since those drive the scheduler and not manual intent, but the enable gate and the off opt-out still apply. A container that is not enabled (no freshdock.enable=true, unless watch_all is on) or that opts out with freshdock.enable=false, com.centurylinklabs.watchtower.enable=false, or freshdock.mode=off is refused: nothing is touched and the command exits 1, so a script can tell a refusal from a completed update. Any other mode, watch included, is allowed because you typed the command yourself. Exit 1 means only that: an update that was rolled back exits 0 (the previous container is running again), and so does a hook skip.


freshdock run

freshdock run [--interval <SECS>] [--tick <SECS>] [--stop-timeout <SECS>]

The scheduler daemon. Each tick it lists running containers, parses their labels, and acts on the ones that are due: live/nightly/weekly/monthly are updated (health-gated, with rollback); watch is report-only. Runs in the foreground until SIGINT/SIGTERM, then finishes the in-flight container and exits. See scheduling for the timing model.

OptionEnv varDefaultMeaning
--interval <SECS>FRESHDOCK_INTERVAL300Poll cadence for live and watch containers.
--tick <SECS>FRESHDOCK_TICK60Scheduler loop granularity. Calendar (cron) modes are evaluated once per tick, so this bounds how late a fire can be.
--stop-timeout <SECS>FRESHDOCK_STOP_TIMEOUT30Max seconds to drain in-flight work after a shutdown signal before force-exit.

The flag wins over its env var; an invalid env value is a startup error.

Examples:

freshdock run                            # poll live/watch every 5 min; cron modes on schedule
freshdock run --interval 600             # poll every 10 min instead
freshdock run --config /etc/freshdock.toml
RUST_LOG=info freshdock run              # per-container scheduler events

Scheduling and update modes

The freshdock.mode label on an opted-in container decides whether freshdock acts and when. The scheduler (freshdock run) drives all of it.

Modes

ModeWhen it actsWhat it does
liveevery --interval seconds (default 300)Pull and recreate on every new digest.
nightlycron 0 4 * * * (04:00 daily)Recreate if a newer image exists.
weeklycron 0 4 * * 0 (04:00 Sunday)Recreate if a newer image exists.
monthlycron 0 4 1 * * (04:00 on the 1st)Recreate if a newer image exists.
watchevery --interval secondsReport only: emit an available notification, never pull or restart.
offneverIgnored by the scheduler.

A single daemon mixes modes freely. When freshdock.enable=true but no mode is set, the default is watch, or [settings] default_mode if configured (see configuration).

watch de-duplicates: it alerts once per distinct new digest, not every poll, so you aren’t re-notified until you act or the upstream digest changes again.

live and watch containers are checked on the fixed interval set by freshdock run --interval <seconds> (default 300). A container is due when it has never been checked, or when at least --interval seconds have passed since its last check.

Calendar modes (nightly, weekly, monthly)

These fire on a cron schedule. Override any default schedule with a freshdock.schedule label (ignored for live, watch, and off):

labels:
  - "freshdock.enable=true"
  - "freshdock.mode=weekly"
  - "freshdock.schedule=0 2 * * 1"   # 02:00 every Monday

The schedule is evaluated once per scheduler tick (--tick, default 60 s), so a calendar fire can be at most one tick late.

Cron syntax

Standard 5 fields: minute hour day-of-month month day-of-week.

FieldRange
minute0-59
hour0-23
day-of-month1-31
month1-12
day-of-week0-6 (Sunday = 0; names not supported)

Each field accepts:

SyntaxMeaning
*any value
Nan exact value
A-Ban inclusive range
*/n, A-B/n, N/na step
N,M,Oa comma-separated list

Example: */15 9-17 * * 1-5 is every 15 minutes, 09:00 to 17:00, Monday to Friday.

When day-of-month and day-of-week are both restricted (neither is *), a tick matches if either one matches, the Vixie-cron union rule. 0 4 13 * 5 fires at 04:00 on the 13th and every Friday.

Timezone, DST, and missed windows

  • Schedules are evaluated in the host’s system local time, not UTC. Set the container’s TZ/timezone if you want a specific zone.
  • Across a spring-forward, a schedule landing in the skipped hour (e.g. 30 2 * * *) does not fire that day. The 04:00 defaults steer clear of the transition hour.
  • Schedule state is in memory only, so there is no backfill. A window missed while the daemon was down is not caught up; it fires at the next occurrence.

What happens when a container is due

  1. The image digest is probed against its registry.
  2. For watch: if a newer digest appeared, an available notification is dispatched (once per distinct digest). A watch container that is behind its own tag notifies too, so the digest in the notification can be one that is already present locally.
  3. For the updating modes: if a newer digest exists, the container is recreated and health-gated, with rollback on failure.

The digest is compared per container. Docker can record one image under several manifest digests, and the container counts as up to date when upstream’s digest is any of them; otherwise a republished multi-arch index would recreate a healthy container on every run. A container that runs a different image than its tag resolves to locally is behind it and is updated as well, which is what keeps a second container on a shared tag from being hidden by the first one’s update.

A digest that failed the health gate is not retried until upstream publishes a new one; freshdock recreate still forces it. That memory is in-process only: restarting the daemon, or a container leaving the running list for a poll, lets the same digest be tried once more.

Inside a multi-service Compose project, step 3 becomes a project rollout. The one-shot services the project waits on are re-run first, the rest follow in depends_on order, and the whole project is checked once rather than container by container. A project holding only the container that triggered it keeps the ordinary path. See Compose projects.

Docker Compose projects

freshdock’s normal unit of work is one container, which is not enough inside a Compose project.

Take a stack that runs its migrations as a one-shot service:

services:
  migrate:
    image: app:latest
    command: ./bin/run-migrate.sh
    restart: "no"

  web:
    image: app:latest
    labels:
      freshdock.enable: "true"
      freshdock.mode: "live"
    depends_on:
      migrate:
        condition: service_completed_successfully

After a successful run, migrate sits in exited (0), so a per-container updater never sees it: a new app:latest is pulled, web is recreated, and migrate keeps the old code. New application code then runs against an old database schema. Nothing crashes and the health gate is satisfied, so the breakage surfaces later as a missing column or a malformed query.

freshdock treats the whole project as one update unit instead. When an update applies to a container carrying com.docker.compose.project, it:

  1. lists every container in that project, stopped ones included,
  2. picks the ones the moved image applies to,
  3. orders them by depends_on,
  4. re-runs the one-shots the project waits on and blocks until they exit 0,
  5. updates the rest in dependency order, and
  6. restarts the dependents that asked for it.

If a one-shot fails, the rollout stops. Nothing downstream is touched, so web stays up on the image it was already running.

This is on by default. Turn it off with [settings] compose_aware = false or FRESHDOCK_COMPOSE_AWARE=false, and every container is updated on its own again.

No configuration needed

Compose writes the entire dependency graph into the containers’ own labels, so the project is reconstructed from the Docker socket alone: no compose file to locate, no bind mount of the project directory, and no docker compose binary in the freshdock image.

LabelWhat freshdock reads from it
com.docker.compose.projectWhich project a container belongs to.
com.docker.compose.serviceIts service name, the node in the graph.
com.docker.compose.depends_onIts edges: service:condition:restart, comma-separated.
com.docker.compose.oneoffTrue marks a docker compose run leftover, which is never part of a rollout.

What gets updated

The label gate still decides. A container is updated when it is enabled the usual way: its own freshdock.enable=true, or fleet-wide watch_all.

There is one narrow exception. An unlabelled one-shot is re-run anyway when another service waits on it with service_completed_successfully. That condition is the compose file itself saying the service must complete before its dependents start, which is the case above. Requiring a label there would leave the sharp edge in place, since nobody labels their migration service.

The exception covers an absent label, not a quiet one. A service in watch mode is never touched, one-shot or not, because watch means “tell me, never restart me” and re-running a one-shot is a restart.

This matters under watch_all: watch_all enables every container but leaves its mode at watch until [settings] default_mode says otherwise, so on that configuration a rollout has nothing it is allowed to do. Set default_mode to an updating mode, or label the services you want rolled out.

Everything else follows the ordinary rules:

Project memberRolled out?
Labelled freshdock.enable=true (or enabled by watch_all), on the moved imageYes.
Unlabelled, and another service waits on it with service_completed_successfullyYes, re-run as a one-shot.
Unlabelled and long-running, even on the same imageNo. Sharing an image does not opt it in.
In watch modeNo. Watch means detect and report, never restart, and re-running a one-shot is a restart. Such a container is not bumped by a restart: true edge either.
freshdock.enable=false, com.centurylinklabs.watchtower.enable=false, or freshdock.mode=offNo. An explicit opt-out always wins, one-shots included.
Stopped, and not a one-shotNo. If you stopped it, freshdock will not start it.
A one-shot that is currently runningNo. It is mid-run, and stomping it would kill a migration in flight. This holds even for an explicit freshdock recreate of that container.
A <name>-old-<ts> archiveNo. Archives keep the original’s compose labels, so a kept one would otherwise be re-run as a one-shot.
On a different image than the one that movedNo. Members are matched on the image reference, or on the image id when the tag has since moved and the daemon reports a bare id.
freshdock’s own containerNo, same self-guard as watch_all.
A docker compose run one-offNo.

Ordering and conditions

Services are ordered so every dependency is handled before its dependents (a topological sort of depends_on). Independent services are ordered by name, so a rollout is reproducible.

Each depends_on condition maps to what freshdock waits for:

ConditionBehaviour
service_completed_successfullyThe dependency is re-run and the rollout blocks until it exits. A non-zero exit or a timeout aborts the rollout.
service_healthyWhen the dependency is part of the rollout, its own health gate is what the dependents wait on: the next service is not started until it reports healthy. A dependency the rollout does not touch is already up, so there is nothing to wait for.
service_startedOrdering alone, nothing extra to wait for.

A one-shot has 10 minutes to finish before the rollout gives up on it. Raise the budget with [settings] one_shot_timeout or FRESHDOCK_ONE_SHOT_TIMEOUT (seconds) for a slow migration; abandoning one halfway is worse than waiting.

Cyclic depends_on cannot happen through Compose, which rejects it. If a hand-written label produces one anyway, freshdock logs it and rolls the affected services out in name order rather than hanging.

restart: true dependents

Compose’s depends_on.<service>.restart means “restart me when this dependency is recreated”. freshdock honours it, after every update in the rollout has landed:

  web:
    depends_on:
      config-sidecar:
        condition: service_started
        restart: true

This is a restart, not an update: the dependent’s own image never moves, so there is no pull and no rollback surface. The stop honours the container’s own stop_signal and stop_grace_period, as a recreate does.

A dependent is left alone when it explicitly opts out, when its labels cannot be read, or when it is in watch mode: restart: true is written for Compose, while freshdock.mode=watch is written for freshdock and says not to restart this container at all. A stopped dependent is not started, and a restart that fails is logged without failing the rollout.

When a one-shot fails

The rollout stops immediately:

  • The failed container is kept, exited non-zero, and so is the archive of its previous instance. Its logs are the only record of why the migration failed, so neither is cleaned up. docker logs <project>-<service>-1 is the first thing to read.
  • It is not rolled back. The command already ran and may have applied part of its work; restoring the previous container object would change nothing about the database and would throw the evidence away.
  • Every service after it in the order is untouched and still serving its previous image. Your stack keeps working on the old version, and stays that way for the rest of the cycle: the rollout claims the members it never reached, so nothing re-triggers it and re-runs the failed step per member.
  • Services updated before the failure stay updated. There is no safe way to un-apply a migration, so freshdock does not pretend there is.
  • With notifications configured, one failed-trigger event is emitted for the project, naming what was updated and what was not.

Fix the cause, then let the next cycle run, or re-run it by hand:

freshdock recreate web

Reading the logs

A rollout logs per project rather than per container, so the sequence reads as one operation:

INFO rollout: starting compose project rollout project=shop targets=2
INFO rollout: re-running one-shot project=shop container=shop-migrate-1 service=migrate
INFO one-shot: finished container=6a8e89bf0c1a exit_code=0
INFO rollout: updating service project=shop container=shop-web-1 service=web
INFO rollout: restarting dependent (depends_on restart: true) project=shop container=shop-sidecar-1
INFO rollout: complete project=shop steps=3

freshdock recreate prints the same sequence on stdout:

$ freshdock recreate shop-web-1
rollout of compose project shop:
  shop-migrate-1: re-ran to a successful exit
  shop-web-1: updated and healthy (new id 16d07544c3e8)
  shop-sidecar-1: restarted (depends_on restart: true)
rollout complete: 3 step(s)

Limits

  • A rollout is triggered by an enabled container: the scheduler finds an update for one of the project’s enabled containers, or you run freshdock recreate on one. A one-shot on an image that no enabled container shares is never reached; give the one-shot the same image as the service that depends on it (the normal Compose pattern), or label a service that does run that image.
  • A single-service project behaves as it did before. When the plan comes down to the one container that triggered it, with nothing to order and nothing to restart, freshdock uses the ordinary single-container path.
  • docker compose up is not shelled out to. It would need the compose binary and a bind mount of the project directory, neither of which exists in a socket-only deployment. The graph in the labels is enough.
  • Scaled services (--scale) are handled: every replica of a target service is updated, in name order.

See also

Notifications

The scheduler (freshdock run) can notify you when an opted-in container (freshdock.notify=true) reaches one of three events. Configure a target as a [notifications.<name>] table in freshdock.toml, or declare it entirely from the environment with a shoutrrr-style URL.

Events (triggers)

TriggerWhenApplies to modes
availableA newer image exists but was not applied.watch
succeededA recreate passed its health gate.live / nightly / weekly / monthly
failedA recreate failed health and was rolled back, or a Compose rollout aborted on a failed one-shot (one event per project; container then names the project).live / nightly / weekly / monthly

Each target may subscribe to a subset with triggers = [...]; omitting it (or []) subscribes to all three. The failure message includes the reason (health-check timeout, or the container crashed before becoming healthy).

Delivery is best-effort and non-fatal, so a broken notifier never blocks or rolls back an update. A successful send logs notification sent (with the target, trigger, and container) at info; a send that fails logs notification failed; continuing at warn and is skipped. An event that fires with no target subscribed to its trigger is logged at debug, so a missing notification is diagnosable from the logs.

At startup freshdock run logs the targets it loaded (notification targets configured count=2 targets=["ops(discord)[failed,succeeded]", ...]), or no notification targets configured when none were found. A typo’d or unset FRESHDOCK_NOTIFY_* variable is therefore visible at boot rather than only when an update later fails to notify.

Backends

All backends render from the same message (a title and a body); only the wire format differs.

webhook

A minimal, stable JSON object, POSTed so a receiver can route on event or container without parsing prose:

{
  "event": "succeeded",
  "container": "web",
  "title": "...",
  "body": "..."
}
[notifications.ops]
type = "webhook"
url  = "https://example.com/hooks/freshdock"

Discord

A single embed whose left-bar colour encodes severity (amber for available, green for succeeded, red for failed), with the title and body as the embed text.

[notifications.discord]
type        = "discord"
webhook_url = "https://discord.com/api/webhooks/123/abc"
triggers    = ["succeeded", "failed"]

Telegram

A plain-text message via the Bot API (sendMessage).

[notifications.tg]
type      = "telegram"
bot_token = "123456:ABC-DEF"     # or FRESHDOCK_NOTIFY_TG_BOT_TOKEN
chat_id   = "987654321"

SMTP (email)

An email with the message title as the subject. username and password must be set together, or both omitted for an anonymous relay.

tls selects the transport security. A mode and a port that don’t match can never complete a handshake, so port defaults to whatever the mode conventionally uses:

tlsMeaningDefault port
"starttls"upgrade the connection with STARTTLS (the default)587
"implicit"TLS from the first byte (SMTPS)465
"none"no encryption at all; local catchers and development only25

Set port explicitly whenever the relay listens elsewhere (mailpit, for example, takes plaintext SMTP on 1025).

[notifications.email]
type     = "smtp"
host     = "smtp.example.com"
port     = 587                   # optional; 587 is already the starttls default
username = "[email protected]"
password = "s3cr3t"              # or FRESHDOCK_NOTIFY_EMAIL_PASSWORD
from     = "[email protected]"
to       = ["[email protected]"]
tls      = "starttls"            # default
triggers = ["failed"]

starttls = true|false is a legacy alias kept for existing configs:

  • true means tls = "starttls"; false means tls = "implicit", never plaintext.
  • Both keys may be present as long as they agree (tls = "starttls" with starttls = true, or tls = "implicit" with starttls = false), so adding tls to an existing config never breaks it.
  • A contradictory pair is an error and the target is skipped, naming both values. This includes any tls = "none" next to a starttls key, which the boolean cannot express.
  • Prefer tls, and delete the legacy line once it is there.

tls = "none" sends credentials and message content in the clear. freshdock logs a warning for every plaintext target at startup; use it only against a catcher such as mailpit on localhost.

See the SMTP smoke-test playbook to verify delivery against a local catcher.

Declaring targets from the environment

Every backend can be declared without a file, using a shoutrrr-style URL (the same shape Watchtower’s WATCHTOWER_NOTIFICATION_URL uses), so a container deployment needs no freshdock.toml:

Env varValue
FRESHDOCK_NOTIFY_<NAME>_URLthe target URL (schemes below)
FRESHDOCK_NOTIFY_<NAME>_TRIGGERSoptional comma list of available,succeeded,failed; defaults to all

<NAME> is any label you choose; it appears only in logs. The scheme selects the backend:

SchemeExampleBecomes
https / httphttps://example.com/hooks/freshdocka generic webhook
discorddiscord://TOKEN@WEBHOOK_IDa Discord webhook
telegramtelegram://BOT_TOKEN@telegram?chats=CHAT_IDa Telegram message
smtpsmtp://user:pass@host:587/[email protected]&[email protected],[email protected]&tls=starttlsan SMTP email
# one Discord target, succeeded + failed only, no file involved
FRESHDOCK_NOTIFY_OPS_URL=discord://abcdef...@123456789
FRESHDOCK_NOTIFY_OPS_TRIGGERS=succeeded,failed

Notes:

  • A Telegram bot token’s : (and any @/: in an SMTP login) are URL metacharacters. Percent-encode them in the userinfo (%40, %3A) and freshdock decodes them back. A Telegram token written as id:secret is rejoined for you.
  • SMTP ?to= takes a comma list or repeated to=; Telegram uses the first chats= value.
  • SMTP ?tls=starttls|implicit|none mirrors the file key and defaults to starttls; the legacy ?starttls=true|false still works, and the two may both be present as long as they agree. A contradictory pair is an error and the target is skipped.
  • An omitted port in an smtp:// URL follows the same mode-based default as the file key (587 / 465 / 25).
  • An invalid URL or unknown scheme is warned and skipped, never fatal: the same resilience as a malformed file target.
  • If a <NAME> is already declared in the file, the file target wins and the env URL is ignored. Env declaration is purely additive.

Secrets

Webhook URLs, Discord webhook URLs, Telegram bot tokens, and SMTP passwords are treated as secrets and redacted in all logs. A secret on a file-declared target can also come from the environment instead of the file:

  • FRESHDOCK_NOTIFY_<NAME>_BOT_TOKEN: a Telegram target’s bot_token
  • FRESHDOCK_NOTIFY_<NAME>_PASSWORD: an SMTP target’s password

To declare a whole target, secret included, from the environment, use FRESHDOCK_NOTIFY_<NAME>_URL.

See the full environment-variable table.

Health gating & rollback

A container counts as updated only once the new container proves healthy. If it does not, the previous container is restored automatically. That is what makes unattended updates safe.

The recreate lifecycle

Whether triggered by freshdock recreate or by the scheduler, an update runs the same cycle:

  1. inspect the running container (capture its full config).
  2. pull the new image.
  3. stop the old container.
  4. rename it to an archive name <name>-old-<timestamp> (kept as the rollback source).
  5. create the new container from the same config plus the new image.
  6. start it.
  7. health-gate the new container (below).
  8. On success: remove the archive (and optionally clean up the image). On failure: roll back.

The recreated container preserves the original config (networks, volumes, env, caps, user, and so on); a dedicated round-trip test asserts the inspected config comes back byte-identical.

One exception: a hostname Docker assigned itself (the old container’s short id) is not copied. The new container gets its own id as hostname, exactly as docker compose up would give it. A hostname you set explicitly is preserved.

Health verdicts

After start, freshdock polls the container until it reaches one of three verdicts:

VerdictMeaningOutcome
HealthyA declared healthcheck reported healthy, or (no healthcheck declared) the container stayed up for the grace period.Remove the archive; success.
TimeoutA healthcheck was declared but never went healthy within the timeout.Roll back; failure.
CrashedThe container exited before becoming healthy, or before the grace period elapsed.Roll back; failure.

Transient probe errors are tolerated (logged and retried); a persistent probe failure past the timeout resolves to the safe Timeout verdict.

Timings

These are currently hardcoded (not yet label/config/env-configurable):

SettingValueMeaning
health timeout120 sMax wait for a declared healthcheck to report healthy.
grace period10 sHow long a container with no healthcheck must stay running to count as healthy.
poll interval1 sHow often the new container’s state is inspected.

A container without a HEALTHCHECK can only be judged by whether it stayed up, so the grace period is the best signal available. Declare a healthcheck for stronger gating.

Rollback

On Timeout or Crashed, freshdock restores the previous container atomically:

  1. Stop the new (failed) container (best-effort; it may already be dead).
  2. Force-remove the new container.
  3. Rename the archive <name>-old-<timestamp> back to the original name.
  4. Start the restored container.

You are left running what you had before the update. If freshdock.notify=true, a failed notification is sent with the reason and the archive it was restored from.

Image cleanup

A successful, health-passed update always removes the replaced container archive. The superseded image is kept by default; opt into removing it:

  • per container: freshdock.cleanup=true
  • fleet-wide default: [settings] cleanup = true (or FRESHDOCK_CLEANUP=true)
  • plus a daemon-wide dangling-image prune: [settings] prune_dangling = true (or FRESHDOCK_PRUNE_DANGLING=true)

Cleanup is best-effort:

  • The old image is removed by its ID. If another container still references it, the daemon refuses (HTTP 409) and freshdock keeps it. That is the guard against deleting a shared base image, and is not treated as a failure.
  • If no image ID can be resolved (a locally-built image, for example), cleanup is skipped.
  • Any cleanup or prune failure is logged but never fails the update or triggers a rollback.

See the configuration reference for the keys and the cleanup smoke-test playbook to verify it end to end.

Lifecycle hooks

Some apps need a hand around an update: flush a write buffer before going down, or clear a cache once the new version is up (Dolibarr, GLPI, Nextcloud). Lifecycle hooks let a container declare those commands as labels; freshdock runs them inside the container with docker exec at the right moments of the update lifecycle.

services:
  glpi:
    image: glpi/glpi:latest
    labels:
      freshdock.enable: "true"
      freshdock.mode: "nightly"
      freshdock.lifecycle.pre-update: "/opt/glpi/bin/maintenance --enable"
      freshdock.lifecycle.post-update: "php bin/console cache:clear"

Labels

LabelValueDefaultMeaning
freshdock.lifecycle.pre-updateshell command(none)Runs in the old container, after the new image is pulled and before the container is stopped. If it does not succeed, the update is skipped.
freshdock.lifecycle.pre-update-timeoutseconds60Time budget for the pre-update hook. 0 disables the timeout.
freshdock.lifecycle.post-updateshell command(none)Runs in the new container once it has passed the health gate. Best-effort: a failure is logged, the update stands.
freshdock.lifecycle.post-update-timeoutseconds60Time budget for the post-update hook. 0 disables the timeout.

Commands are executed as sh -c "<command>" inside the container, so the image must ship a sh (distroless images cannot use hooks, the same constraint as Watchtower). Hook output is logged at debug level.

Where the hooks sit in the update

inspect -> pull -> PRE-UPDATE HOOK -> stop -> rename -> create -> start
        -> health gate -> remove old container -> POST-UPDATE HOOK -> image cleanup

The pre-update hook runs after the pull so the image is already local and the downtime window stays short, and before the stop so the app is still up to answer the exec.

Pre-update: the veto

The pre-update hook is a gate. The update proceeds only when the hook exits 0. Anything else leaves the container untouched, running its old image:

Hook outcomeWhat happens
Exit code 75 (EX_TEMPFAIL, Watchtower-compatible)The polite “not now”: the app is mid-backup, mid-migration, or has active sessions. freshdock logs the deferral at info and tries again the next time the container is due.
Any other non-zero exitLogged as a warning, update skipped.
TimeoutLogged as a warning, update skipped. Docker has no way to kill an exec, so the command itself keeps running inside the container; only the verdict is decided.
Exec failure (no sh in the image, daemon error)Logged as a warning, update skipped.

The skip is not an error: freshdock recreate reports it and exits 0, and the scheduler retries on the container’s normal cadence (--interval for live, the cron for calendar modes).

A pre-update script that only allows updates outside business hours:

#!/bin/sh
hour=$(date +%H)
if [ "$hour" -ge 8 ] && [ "$hour" -lt 18 ]; then
  exit 75   # EX_TEMPFAIL: defer, try again next cycle
fi
exit 0

Post-update: best-effort maintenance

The post-update hook runs in the new container, after the health gate has passed and the archived old container has been removed. The update has already succeeded at that point, so a failing or timing-out post-update hook is logged as a warning and never fails (or rolls back) the update.

If the update rolls back, the post-update hook does not run: there was no update to follow up on.

Interaction with rollback

Hooks never weaken the safety contract:

  • A pre-update veto happens before anything is stopped, so the container is left alone.
  • Rollback is still driven purely by the health gate; hooks cannot cause or prevent a rollback.

Watchtower equivalents

The Watchtower hook labels are read directly, so a migrated container keeps working without relabelling, and its timeouts are interpreted in Watchtower’s unit (minutes). A freshdock.* label always wins when both are present.

Watchtowerfreshdock
com.centurylinklabs.watchtower.lifecycle.pre-updatefreshdock.lifecycle.pre-update
com.centurylinklabs.watchtower.lifecycle.post-updatefreshdock.lifecycle.post-update
...lifecycle.pre-update-timeout (minutes)freshdock.lifecycle.pre-update-timeout (seconds)
...lifecycle.post-update-timeout (minutes)freshdock.lifecycle.post-update-timeout (seconds)
--enable-lifecycle-hooks / WATCHTOWER_LIFECYCLE_HOOKS(not needed): setting a hook label is the opt-in
...lifecycle.pre-check / ...lifecycle.post-check(no equivalent): freshdock has no per-cycle check hooks, so these are logged and ignored

Note the timeout unit difference when translating to the native labels: Watchtower counts minutes, freshdock counts seconds. A Watchtower pre-update-timeout: "5" becomes freshdock.lifecycle.pre-update-timeout: "300".

Registry authentication

freshdock checks digests against any OCI-compliant registry that uses the Docker registry v2 bearer-token flow: Docker Hub, GHCR, Quay.io, lscr.io, and others. Public images resolve anonymously; private images need credentials, supplied either as FRESHDOCK_REGISTRY_<NAME>_TOKEN environment variables (plus _USERNAME where the registry needs it) or as a [registry.<name>] table in freshdock.toml. The file is the only option for a custom host whose name contains dots.

The exact variables and table keys live in the configuration reference. This page covers the registry-specific guidance: what credentials each registry wants, the alias list, a smoke test, and what is out of scope.

Per-registry notes

Registryusernametoken
Docker Hubthe real account name (required)password or access token
GHCR (ghcr.io)any non-empty valuea PAT with read:packages
Quay.iooptionalrobot-account token / password
lscr.iooptionalas the registry requires
Other OCI + beareras the registry requiresas the registry requires

Quay and lscr authenticate with a token alone. GHCR wants any non-empty username next to the PAT (FRESHDOCK_REGISTRY_GHCR_USERNAME); an omitted username is sent as empty, which GHCR may reject. Docker Hub needs its real account name (FRESHDOCK_REGISTRY_DOCKERHUB_USERNAME).

Anonymous Docker Hub is rate-limited (about 100 requests per 6 h); adding credentials raises the budget. freshdock dedupes to one request per unique image to stay well under it.

A pull hit by a short-lived rate limit (GHCR does this in bursts) is retried after 2, 5 and 10 seconds. Docker Hub’s pull quota is not retried; it resets in hours.

Aliases

A [registry.<name>] table key, and the <NAME> in the env-var form, may be a friendly alias or a literal host; both fold onto the same registry as the matching image reference:

AliasRegistry
dockerhub, docker, docker.io, registry-1.docker.io, index.docker.iodocker.io
ghcrghcr.io
quayquay.io
lscrlscr.io

A literal host ("registry.example.com") works as a table key too. Hosts containing dots can’t be expressed unambiguously as an environment variable name, so configure those in the file. Tokens never appear in logs, even at RUST_LOG=trace.

Manual PAT smoke test

Private-registry auth can’t run in CI (no secrets). To verify a real PAT end to end:

export FRESHDOCK_REGISTRY_GHCR_USERNAME=<your-gh-user>
export FRESHDOCK_REGISTRY_GHCR_TOKEN=<a-PAT-with-read:packages>
# A container whose image is a private ghcr.io/<owner>/<repo> must show a
# digest (not "auth required") in the table:
RUST_LOG=trace cargo run -- check
# Confirm the token never appears in the trace output.

Redaction is also enforced by automated tests (config::tests::token_is_redacted_in_tracing_output and registry::auth::tests::cached_token_debug_redacts_the_token); this manual run is an extra end-to-end check.

Out of scope (v1)

ECR / GCR / ACR / Harbor custom auth schemes; insecure (plain-HTTP) registries, since freshdock always talks HTTPS and a registry reachable only over plain HTTP (the common case for a localhost:5000 dev registry) won’t work; and reusing ~/.docker/config.json. Rate-limit headers are logged but freshdock does not yet throttle proactively.

Deployment

freshdock is a single static binary that talks to the Docker socket. Run it however suits you: as a container alongside the ones it manages, or directly on the host (under systemd, for example). For configuration, see the configuration reference.

Mount the Docker socket and run the run subcommand:

docker run -d \
  --name freshdock \
  --restart unless-stopped \
  -v /var/run/docker.sock:/var/run/docker.sock \
  ghcr.io/turbootzz/freshdock:latest run

Or with compose. The runnable stacks live in examples/compose/:

Socket: read-only vs writable

WorkloadSocket mount
watch / check only (never recreates):ro is enough: -v /var/run/docker.sock:/var/run/docker.sock:ro
Any updating mode (live/nightly/weekly/monthly, or recreate)writable: -v /var/run/docker.sock:/var/run/docker.sock

Configuration: environment first

Most deployments need no config file. Fleet-wide settings, registry credentials, notification targets, and the run flags are all environment variables. Pass them under environment: in compose, or Environment= in a systemd unit:

services:
  freshdock:
    image: ghcr.io/turbootzz/freshdock:latest
    command: ["run"]
    environment:
      FRESHDOCK_DEFAULT_MODE: "nightly"
      FRESHDOCK_REGISTRY_GHCR_USERNAME: "${GHCR_USER:-}"
      FRESHDOCK_REGISTRY_GHCR_TOKEN: "${GHCR_TOKEN:-}"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped

Add FRESHDOCK_WATCH_ALL: "true" to that block if you would rather not label each container: every running container is then included unless it opts out, on the mode set by FRESHDOCK_DEFAULT_MODE. See watching every container.

The full list is the env-var table.

Notifications

A notification target can be declared from the environment with a shoutrrr-style URL, with no file to mount:

services:
  freshdock:
    image: ghcr.io/turbootzz/freshdock:latest
    command: ["run"]
    environment:
      FRESHDOCK_NOTIFY_OPS_URL: "discord://${DISCORD_TOKEN}@${DISCORD_ID}"
      FRESHDOCK_NOTIFY_OPS_TRIGGERS: "succeeded,failed"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    restart: unless-stopped

To use a file instead, declare the target in a [notifications.<name>] table, mount it read-only, and keep its secret in the environment:

services:
  freshdock:
    image: ghcr.io/turbootzz/freshdock:latest
    command: ["run", "--config", "/config/freshdock.toml"]
    environment:
      FRESHDOCK_NOTIFY_EMAIL_PASSWORD: "${SMTP_PASSWORD:-}"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./freshdock.toml:/config/freshdock.toml:ro
    restart: unless-stopped

As a host binary (systemd)

Install the binary (cargo install freshdock, a release binary, or just build), then create a unit. The host needs a CA bundle (the ca-certificates package) or SSL_CERT_FILE pointing at one, or freshdock refuses to start; see Troubleshooting.

# /etc/systemd/system/freshdock.service
[Unit]
Description=freshdock container auto-updater
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/freshdock run
Restart=on-failure
Environment=RUST_LOG=info
# If freshdock.toml is not in the working directory:
# Environment=FRESHDOCK_CONFIG=/etc/freshdock/freshdock.toml
# Run as a user in the docker group rather than root where possible.
# SupplementaryGroups=docker

[Install]
WantedBy=multi-user.target
sudo systemctl daemon-reload
sudo systemctl enable --now freshdock
journalctl -u freshdock -f

Docker socket permissions

freshdock talks to /var/run/docker.sock. permission denied on the socket means the process isn’t allowed to use it:

  • On the host: run as a user in the docker group.
  • In a container: the socket’s group GID inside the container must match the host socket’s owner. On some hosts you must pass --group-add <gid> (find it with stat -c '%g' /var/run/docker.sock).

Access to the Docker socket is effectively root on the host, so grant it with care.

Compatibility

PlatformStatus
Plain DockerPrimary target. The API version is negotiated with your daemon at connect time; CI exercises the Docker engine shipped on GitHub’s ubuntu-latest runners.
Docker Desktop (Linux, macOS, Windows)Supported.
Portainer (CE and Business)Supported via the same Docker socket.
Podman 4+Supported via the Docker-compatible socket (discovery below).
Docker ComposeA multi-service project is updated as one unit, in depends_on order. See Compose projects.
Compose-based UIs (Dockge, Komodo, and similar)Supported; compose files are never read or edited.
Kubernetes / SwarmOut of scope; use platform-native mechanisms.

One daemon-version floor applies: re-creating a container attached to more than one network requires Docker API 1.44 (Docker 25.0), because older daemons cannot attach several networks in a single create call. freshdock checks the negotiated API version before it stops anything, so on an older daemon such a container is refused with an explanatory error rather than being taken down and left unable to come back. Detach the extra networks (re-attaching them after the update) or upgrade the daemon.

Which socket freshdock uses

freshdock connects in this order at startup and logs which family answered:

  1. DOCKER_HOST, if set. unix://, tcp://, http://, https:// and ssh:// are all honoured. Only the scheme is logged; the value may carry credentials.
  2. The local Docker socket: /var/run/docker.sock (named pipe on Windows).
  3. Podman’s sockets: $XDG_RUNTIME_DIR/podman/podman.sock, /run/user/$UID/podman/podman.sock, then /run/podman/podman.sock.

So a rootless or rootful Podman host in the standard location works with no configuration. For anything else, set DOCKER_HOST=unix:///path/to/podman.sock.

A recreate replaces the container with a new ID, so a UI that pinned the old ID may briefly show it “out of sync” until its next refresh. freshdock never edits your compose/stack files, only the running container.

freshdock: A Modern Rust-based Watchtower Successor

Project name: freshdock, verified available on crates.io, GitHub, Docker Hub, npm, PyPI. Status: Shipped (Phases 0-7 complete). Current version: see the changelog. Author: Thijs (Turboot). Date: May 2026 (original plan).

This is the original planning document, kept for design rationale and history. Every phase below has shipped; see the CHANGELOG for the released state.

A fresh dock where containers come to renew themselves. Modern Docker, health-gated rollback, single static binary.


1. Context

Watchtower (containrrr/watchtower) was archived on 17 December 2025 and is no longer maintained. Beyond being abandoned, the codebase ships an outdated embedded Docker SDK (API 1.25), which makes it incompatible with Docker Engine 29+ (which requires API 1.44 or newer). The original maintainers themselves discourage the use of the active forks. There is a clear, current gap in the ecosystem for a maintained, modern, full-cycle (check, pull, restart) container auto-updater.

This project fills that gap, while serving a secondary goal: a substantial real-world project to deepen Rust skills (async, Tokio, error modelling, traits, state machines, packaging).


2. Competitive Landscape

Go-based tools (the majority)

ToolWhat it doesNotes
What’s Up Docker (WUD)Check + optional update + web UI + many notificationsClosest “smart drop-in” replacement; heavy.
DiunNotifications onlyDeliberately read-only.
TugtainerWeb UI, multi-host agents, dependency-aware updates, manual approvalModern, growing user base.
Dockwatch (Notifiarr)Dashboard, fits *arr stacksNiche audience.
dockcheckCLI shell scriptMinimal.
nicholas-fedor/watchtowerActive fork of the originalStop-gap, not a rewrite.

Rust-based tools

ToolWhat it doesGap
Cup (sergi0g/cup)Very fast checker (5.4 MB binary, 58 images in ~3.7s on a Pi 5), CLI + webDeliberately does not pull or restart, checker only.

The gap this project fills

There is no full-cycle (check, pull, recreate, restart, cleanup) Rust-based container auto-updater with active maintenance. Users who want automation today fall back on heavier Go tools that pull 100+ MB images. A Rust tool that combines Cup-level footprint with Watchtower-level capability does not exist yet.


3. Goals and Non-Goals

Goals

  1. Be a true drop-in replacement for Watchtower’s “set and forget” use case in homelabs.
  2. Support modern Docker (API 1.44 or newer) and Podman without hacks.
  3. Multiple update strategies on a per-container basis (live, scheduled, watch-only).
  4. Healthy-by-default: never leave the user with a broken container if a rollback is possible.
  5. Single static binary, small footprint, fast cold start.
  6. Be a sustainable open source project with clear scope, good docs, and tests.

Non-Goals (explicitly out of scope, at least for v1)

  • Kubernetes: Kubernetes has its own update mechanisms; do not compete with them.
  • Docker Swarm orchestration logic.
  • Approval workflows / web UI in v1 (consider for v2).
  • Multi-host agent architecture (consider for v2).
  • Updating compose-managed stacks via the compose file directly (rely on label-based per-container updates instead).
  • Image vulnerability scanning.

4. Differentiators

What this project will do better than the existing landscape:

  1. Modern Docker API. The API version is negotiated with the daemon at connect time, so freshdock speaks whatever that daemon supports.
  2. Health-gated updates. A container is only considered “successfully updated” when the new instance reaches its healthcheck healthy state (or stays running for a configurable grace period if no healthcheck exists). Failed updates trigger automatic rollback to the previous image.
  3. Per-container schedule mixing. A single deployment can have container A on live updates, container B on nightly, container C on weekly, container D in watch-only mode, driven by Docker labels, no global compromise.
  4. Dependency-aware ordering. Containers with depends_on are stopped/started in the correct order (inspired by Tugtainer).
  5. Smaller and faster than Go alternatives while retaining the full update cycle (target: at most 10 MB binary, at most 30 MB resident memory at idle).
  6. OCI-correct. Works with Podman’s API socket without modification.
  7. Honest defaults. Watch-only is the default: opt-in to auto-update per container, not opt-out. This protects users from “Watchtower broke my server overnight” stories.

5. Core Features (MVP)

5.1 Update modes (per container, via labels)

ModeBehaviour
livePoll registry frequently (default 5 min); pull and recreate immediately on new digest.
nightlyCheck at a fixed daily window (default 04:00 local time).
weeklyCheck once per week (configurable day + time).
monthlyCheck on the Nth day of the month (configurable).
watchDetect updates and notify only, never pull or restart.
offIgnore the container entirely.

Global default mode is configurable; per-container labels override.

Example:

labels:
  - "freshdock.enable=true"
  - "freshdock.mode=nightly"
  - "freshdock.notify=true"

5.2 Update lifecycle

For each eligible container, on each scheduled tick:

  1. Resolve current image reference (name + tag, or digest).
  2. Query registry for the digest of that tag.
  3. If digest unchanged, skip.
  4. If digest changed:
    1. Pull new image.
    2. Inspect old container; capture full config (env, mounts, networks, restart policy, healthcheck, labels, command, etc.).
    3. Stop old container gracefully (respect stop signal + timeout).
    4. Rename old container <name>-old-<timestamp> (kept for rollback).
    5. Create new container with captured config + new image.
    6. Start new container.
    7. Wait for healthcheck to become healthy (or grace period if no check).
    8. On success: remove -old- container, optionally prune old image (configurable, off by default). Implemented: [settings] cleanup / per-container freshdock.cleanup removes the replaced image; [settings] prune_dangling adds a daemon-wide dangling prune.
    9. On failure: stop and remove new container, rename -old- back, restart it. Send failure notification.

5.3 Scheduling

  • Single async runtime (Tokio).
  • One scheduler task per mode that picks up containers tagged for that mode.
  • Cron-like expressions accepted for nightly/weekly/monthly (start with 0 4 * * *-style strings; document fields explicitly).

5.4 Notifications (v1 scope)

  • Webhook (generic POST with JSON body).
  • Discord webhook (formatted embed).
  • Telegram bot (broadly used in the homelab community; keeps deployment scope small).
  • Email (SMTP): basic.

Notification triggers: update available (watch mode), update succeeded, update failed (with rollback status).

5.5 Registry support (v1 scope)

  • Docker Hub (anonymous + authenticated).
  • GHCR (PAT or anonymous public).
  • Quay.io.
  • lscr.io (LinuxServer).
  • Generic OCI-compliant registries with bearer-token auth.

ECR, GCR, ACR, Harbor with custom auth: v2.

5.6 Configuration

Two configuration paths:

  1. Container labels (preferred, Watchtower-compatible style).
  2. Single config file (freshdock.toml) for global defaults: poll intervals, notification endpoints, registry credentials, default schedule.

Environment variables override config file for credentials.

5.7 Compatibility targets

PlatformHow it worksNotes
Plain Docker (24.x, 25.x, 27.x, 28.x, 29+)Talks to /var/run/docker.sockPrimary target.
Docker Desktop (Linux/macOS/Windows)Same socketTested manually.
Portainer (CE + BE)Talks to the same Docker socket Portainer usesDocument the “Portainer’s stack view may briefly show out-of-sync state after a recreate” caveat.
Podman 4+Falls back to Podman’s sockets ($XDG_RUNTIME_DIR/podman/podman.sock, /run/user/$UID/podman/podman.sock, /run/podman/podman.sock) when no Docker socket answersRootless and rootful. DOCKER_HOST=unix://... overrides for non-standard paths.
Dockge / Komodo / other compose-based UIsUpdates individual containers via the daemon socketCompose stack files are not edited; users see the new image once they re-run their compose.

6. Technical Architecture

6.1 Stack

ConcernChoice
LanguageRust (stable, edition 2024).
Async runtimeTokio.
Docker clientbollard (mature, supports API 1.53, also handles Podman).
HTTP (registry)reqwest with rustls.
Serializationserde + serde_json + toml.
CLIclap v4 with derive.
Loggingtracing + tracing-subscriber.
Errorsthiserror for libraries, anyhow for the binary entry point.
Schedulingtokio-cron-scheduler or hand-rolled with tokio::time (decide during prototyping).
Configfigment or plain serde over TOML.
Teststokio::test + testcontainers-rs for integration.

6.2 Crate layout

A workspace with separate crates is overkill for v1. Start as a single binary crate with internal modules; promote to a workspace only when a clear library boundary emerges.

src/
  main.rs              // entry, CLI parsing, daemon bootstrap
  config.rs            // TOML + env loading
  labels.rs            // label parsing -> per-container policy
  docker/
    mod.rs             // bollard wrappers
    inspect.rs         // capture full container spec for recreation
    recreate.rs        // recreate-with-same-args logic
  registry/
    mod.rs
    auth.rs            // token negotiation per registry
    digest.rs          // HEAD /manifests/<tag> -> digest
  scheduler.rs         // mode -> tick -> container set
  updater.rs           // the lifecycle state machine
  health.rs            // healthcheck waiting + grace period
  rollback.rs          // -old- container handling
  notify/
    mod.rs
    webhook.rs
    discord.rs
    telegram.rs
    smtp.rs
  errors.rs

6.3 The recreation problem (the hardest part)

Watchtower-style “restart with the same options” is the single most error-prone area. Plan:

  1. Use docker inspect-equivalent (bollard::Docker::inspect_container) to get the full ContainerInspectResponse.
  2. Map that structure into a fresh CreateContainerOptions + Config for the new container.
  3. Re-attach all networks the old container was on (with the same aliases and IP if static).
  4. Re-attach all mounts (binds, volumes, tmpfs).
  5. Preserve restart policy, log driver, capabilities, security opts, sysctls, ulimits, devices, GPU options.
  6. Preserve labels, but strip the lifecycle labels added by this tool itself, then re-add.

Write an integration test that creates a container with a “weird” config (custom network with alias, healthcheck, capabilities, GPU stub, restart policy) and verifies that after a recreate the inspected output is byte-identical except for the image digest and the container ID.

This test is the project’s quality gate. If it passes, the tool is safe to ship. If it fails, the tool is dangerous.


7. Phased Roadmap

✅ All phases (0-7) have shipped. The tool reached v1.0.0 and keeps shipping (see the changelog). The estimates and “cut if pacing slips” caveats below are preserved as the original plan; in the end nothing was cut. Phase-by-phase status is noted inline.

Estimates assume part-time evening/weekend work alongside the dual study programme.

Phase 0: Reserve the name & scaffolding (1 week) ✅ shipped

  • Reserve freshdock everywhere before anything else. Crate names on crates.io are permanent and first-come-first-served. Order: (a) publish a 0.0.1 placeholder crate with a minimal Cargo.toml and a stub main.rs; (b) create the GitHub repo under your account or a freshdock org, which also reserves the GHCR namespace (ghcr.io/<owner>/freshdock) for free, since GHCR uses the GitHub namespace automatically; (c) optional: register freshdock.dev or .io if still available. Docker Hub is intentionally skipped: v1 publishes to GHCR only (revisit post-v1 if discoverability matters).
  • Pick licence: AGPL-3.0 (like Cup, protects against commercial appropriation) or MIT/Apache-2.0 dual (maximum adoption). Decide before first real commit.
  • Set up CI (GitHub Actions: fmt, clippy, test, cross-compile to musl for amd64 + arm64).
  • Set up cargo-deny for license/dependency hygiene.
  • Repo skeleton, README stub with the three differentiators (modern Docker, health-gated rollback, Rust footprint) front and centre.

Phase 1: Read-only spike (2 weeks) ✅ shipped

Goal: prove the concept end-to-end without touching containers.

  • List running containers via Bollard.
  • Parse labels into a policy struct.
  • Implement digest checking against Docker Hub (anonymous).
  • Print a table of “container | current digest | latest digest | update?”.

This becomes the watch-only mode for free.

Phase 2: Single recreate (2 weeks) ✅ shipped

  • Implement the inspect -> stop -> rename -> create -> start cycle for one container.
  • Handle the basic config preservation (env, mounts, networks, restart policy).
  • Manual testing only at this stage.

Phase 3: Health gating + rollback (1-2 weeks) ✅ shipped

  • Wait-for-healthy logic.
  • Grace period for containers without healthchecks.
  • Rollback path on failure.
  • The “weird config” integration test mentioned in §6.3.

Phase 4: Scheduling (1 week) ✅ shipped

  • The five modes (live, nightly, weekly, monthly, watch).
  • Cron expression parsing.
  • Per-container override via labels.

Phase 5: Multi-registry + auth (2 weeks) ✅ shipped

  • GHCR, Quay, lscr.io, generic bearer-token registries.
  • Credentials from config file + env.
  • Rate-limit-aware checking (the Cup approach: HEAD requests, not pulls).

Phase 6: Notifications (1 week) ✅ shipped

  • Webhook, Discord, Telegram, SMTP.
  • Trigger matrix: success / failure / available-only.

Phase 7: Polish and v1.0 release (2 weeks) ✅ shipped

  • Documentation site (mdBook or just a thorough README).
  • Docker image (multi-arch: amd64, arm64, armv7).
  • Sample compose snippets.
  • Migration guide from Watchtower (label translation table).
  • v1.0.0 tag.

Total estimate: ~12 weeks of part-time work to v1.0. Cut Phase 5 and Phase 6 in half if pacing slips. They extend cleanly post-v1.

Post-v1 (no commitment, just ideas)

  • Optional web UI (Leptos or Axum + a simple HTMX frontend).
  • Multi-host agent architecture.
  • Approval workflow (queue updates for manual confirmation).
  • ECR/GCR/ACR registry support.
  • Vulnerability data integration (Trivy / Grype output as a notification field).

8. Risks and Mitigations

RiskMitigation
Recreation loses a non-obvious container setting and silently breaks something.The §6.3 integration test as a hard quality gate; community beta period before tagging v1.0.
Registry auth zoo is bigger than expected.Scope strictly: ship v1 with five well-known registries; document a clear extension point for others.
Burnout from a long parallel project.Phase boundaries are checkpoints; v1 is not “every feature”. It is “watch-only + nightly auto-update reliably”. Ship that.
Crowded space, “yet another Watchtower clone”.Differentiate honestly on three things only: modern Docker, health-gated rollback, Rust footprint. Lead with these in the README.
Portainer users hit confusing UI desync after recreations.Document the behaviour in a dedicated README section; don’t pretend it doesn’t happen.
User expectations from the dead Watchtower carry over and don’t match reality.Provide a “Coming from Watchtower?” page that explicitly maps old labels and flags to the new ones.

9. Success Criteria for v1.0

  • Runs as a single static binary (at most 10 MB) with no runtime dependencies.
  • Successfully auto-updates a fleet of 20+ mixed containers across at least three different image registries on a real homelab for two weeks without intervention.
  • Survives an intentionally bad image push (broken healthcheck) by rolling back cleanly and notifying.
  • Documentation lets a Watchtower user migrate in under 15 minutes.
  • At least one external user has it deployed and gives a thumbs-up.

10. Open Questions

These were left open at Phase 0 kickoff; all four are now resolved:

  1. AGPL-3.0 (like Cup) vs MIT/Apache-2.0 dual licence? Decided: Apache-2.0 (Cargo.toml / LICENSE).
  2. CLI subcommand vs daemon-only? Decided: CLI subcommands shipped: check, recreate, run.
  3. tokio-cron-scheduler or hand-roll? Decided: Hand-rolled (src/cron.rs + src/scheduler.rs); chrono is used only for DST-correct local-time calendar math, not scheduling.
  4. fd.* alias for freshdock.*? Decided: Not adopted. Only the full freshdock.* prefix is parsed (src/labels.rs).

11. References

  • containrrr/watchtower archive announcement (Dec 2025).
  • sergi0g/cup: Rust container update checker.
  • crazy-max/diun: notification-only Go tool.
  • quenary/tugtainer: Go web UI auto-updater with dependency awareness.
  • fjall/bollard: Rust Docker SDK (API 1.52).
  • Watchtower’s original recreation logic (Go), for reference on edge cases worth replicating.

Manual smoke test: freshdock recreate

This walks through the recreate cycle on a real Docker daemon. As of Phase 3 the cycle is health-gated: the new container must reach healthy (or stay running for a grace period if it has no healthcheck) before the run is declared a success and the archived -old- container is removed; a new image that fails health is rolled back to the previous container.

Canonical automated gate. The authoritative “is the tool safe to ship” check is the live round-trip test tests/recreate_roundtrip_live.rs (PLAN §6.3, P3-3). It creates a kitchen-sink container, recreates it, and asserts the inspected config round-trips byte-identical. Run it with a daemon available:

cargo test --test recreate_roundtrip_live -- --ignored

A failure of that test is a release blocker. This manual procedure remains for quick human verification on first install.

Prerequisites

  • A working Docker daemon reachable on the standard socket.
  • freshdock built locally: just build.
  • A checkout where freshdock recreate is wired in (i.e. anything from main post-Phase-2 onward).

Steps

freshdock recreate is a manual admin tool, not the automatic update loop. It exits 1 without touching a container that is not enabled (no freshdock.enable=true, unless watch_all is on) or that opts out with freshdock.enable=false, com.centurylinklabs.watchtower.enable=false, or freshdock.mode=off, and otherwise honours the operator’s explicit invocation regardless of the scheduler mode (live, nightly, weekly, monthly, watch). This is why the test container below uses mode=watch: a watch-mode container is never touched by the automatic loop, but is a fine target for a manual recreate.

# 1. Launch the test container with freshdock labels so the recreate
#    command is willing to act on it.
docker run -d --name fd-smoke \
  --label freshdock.enable=true \
  --label freshdock.mode=watch \
  -p 8081:80 nginx:alpine

# 2. Confirm it's serving.
curl -fsS http://localhost:8081/ > /dev/null && echo "nginx OK"

# 3. Capture Config.Image *before* the recreate so we can assert the
#    round-trip below. This must read `nginx:alpine`, not the resolved
#    `Image` digest, which is a separate field.
docker inspect fd-smoke --format '{{.Config.Image}}'   # -> nginx:alpine

# 4. Run the recreate.
./target/release/freshdock recreate fd-smoke

Expected observations

  • The CLI prints a line starting recreated fd-smoke: healthy that names the removed archive fd-smoke-old-<unix-ts> and the new id.
  • docker ps -a shows a single fd-smoke container, running with a fresh id; the archived fd-smoke-old-<unix-ts> was removed once the new instance passed health gating. (nginx:alpine has no healthcheck, so success is decided by the grace period; add a healthcheck to exercise the healthy path.)
  • The new container has the same port mapping (0.0.0.0:8081->80/tcp), the same freshdock.enable=true / freshdock.mode=watch labels, and the same nginx image.
  • Config.Image round-trip (regression #25). After the recreate, docker inspect fd-smoke --format '{{.Config.Image}}' must read byte-identical to the pre-recreate value (nginx:alpine). Drift to library/nginx:alpine is the original bug. It must not return.
  • curl -fsS http://localhost:8081/ still returns the default nginx page from the new container.

To exercise rollback: recreate a container whose tag has been re-pointed to an image with a failing healthcheck; the CLI prints recreate failed for <name>: ... rolled back to the previous container and the original container is restored under its original name.

Cleanup

docker rm -f fd-smoke fd-smoke-old-*

Weird-config round-trip smoke

Mirrors the dimensions covered by [tests/fixtures/container_inspect_weird.json] so the manual procedure stays in lockstep with the automated round-trip suite in [tests/spec_roundtrip.rs] (the weird_spec_* tests). Run this whenever the recreate orchestrator or ContainerSpec projection changes.

This uses alpine running sleep rather than nginx:alpine, so the container doesn’t fight the config (nginx fails when run as uid 1000 with its cache dirs replaced by tmpfs, orthogonal to recreate). The point of this smoke is to verify spec preservation, not application behaviour.

# Two networks the container will attach to.
docker network create fd-front >/dev/null
docker network create fd-back  >/dev/null

# Bind-mount sources. /tmp keeps these cleanup-friendly and avoids needing
# root for /host/* paths; the container-side destinations match the
# fixture so `weird_spec_preserves_binds_and_tmpfs` exercises the same
# `Binds` shape that this manual procedure does.
mkdir -p /tmp/fd-state /tmp/fd-secrets

# Launch with the kitchen-sink dimensions: non-default user (uid 1500, not
# 1000: `--ulimit nproc` counts every thread of that uid on the host, and on a
# workstation uid 1000 is the desktop user with far more than 512 of them, so
# the container would fail to exec with "resource temporarily unavailable"),
# multi-port
# binding (one with explicit HostIp), bind mounts + tmpfs (HostConfig.Tmpfs
# dict), multiple cap_add / cap_drop, sysctls, restart policy with retry
# count, memory + nano-cpus + pids limit, custom stop signal + timeout,
# multiple ulimits, extra_hosts, and freshdock.* labels alongside user
# labels.
docker run -d --name fd-smoke-weird \
  --label freshdock.enable=true \
  --label freshdock.mode=watch \
  --label freshdock.notify=true \
  --label app=weird --label team=platform --label [email protected] \
  --user 1500:1500 \
  --network fd-front --network-alias weird-front \
  -p 127.0.0.1:18443:8443 -p 19090:9090 \
  -v /tmp/fd-state:/var/lib/state:rw -v /tmp/fd-secrets:/run/secrets:ro \
  --tmpfs /run:rw,size=32m --tmpfs /var/cache:rw,size=64m \
  --cap-add NET_BIND_SERVICE --cap-add SYS_TIME \
  --cap-drop MKNOD --cap-drop AUDIT_WRITE \
  --sysctl net.ipv4.ip_unprivileged_port_start=0 \
  --sysctl net.core.somaxconn=4096 \
  --restart on-failure:5 \
  --memory 128m --memory-reservation 64m --cpus 0.5 --pids-limit 256 \
  --stop-signal SIGUSR1 --stop-timeout 45 \
  --ulimit nofile=8192:16384 --ulimit nproc=512:1024 \
  --add-host db.internal:10.0.0.5 \
  -e APP_MODE=production -e 'APP_TOKEN=base64=padded==' -e EMPTY_VAR= \
  alpine sleep 600

# Attach the second network with its own alias. Mirrors the multi-network
# attachment that `weird_spec_preserves_multi_network_endpoints_with_aliases`
# pins on the fixture side.
docker network connect --alias weird-back fd-back fd-smoke-weird

# Capture every dimension we care about *before* the recreate.
before=$(docker inspect fd-smoke-weird)

# Run the recreate.
./target/release/freshdock recreate fd-smoke-weird

# Spot-check the headline #25 assertion: Config.Image must round-trip
# byte-identical (alpine, NOT library/alpine).
docker inspect fd-smoke-weird --format '{{.Config.Image}}'   # -> alpine

# Diff the recreate-relevant slices of the inspect. Container ID and
# Image (resolved digest) are expected to differ; everything else under
# Config.* and HostConfig.* should match `before`.
after=$(docker inspect fd-smoke-weird)
diff \
  <(echo "$before" | jq '.[0].Config'    ) \
  <(echo "$after"  | jq '.[0].Config'    )    # only Config.MacAddress may
                                              # appear in AFTER (daemon
                                              # quirk: API-created containers
                                              # surface the auto-assigned MAC
                                              # on Config.MacAddress, while
                                              # `docker run` originals expose
                                              # it only on NetworkSettings).
                                              # Not a recreate bug.
diff \
  <(echo "$before" | jq '.[0].HostConfig') \
  <(echo "$after"  | jq '.[0].HostConfig')    # expected: empty diff

# Bind mounts must round-trip: same source/destination/RW. .Mounts entries
# for binds carry only durable fields (the daemon-assigned propagation is
# also stable across recreate).
diff \
  <(echo "$before" | jq '[.[0].Mounts[] | select(.Type == "bind") | {Source, Destination, Mode, RW}] | sort_by(.Destination)') \
  <(echo "$after"  | jq '[.[0].Mounts[] | select(.Type == "bind") | {Source, Destination, Mode, RW}] | sort_by(.Destination)')
                                              # expected: empty diff

# NetworkSettings.Networks: a strict diff would fail because the new
# container gets a fresh IP / MAC / EndpointID. Those are network-driver
# assignments, not part of the spec. Docker also auto-adds the new
# container's own short id as an alias on every attached network, which
# differs per instance. Project to the durable invariants: the set of
# attached networks + the *user-assigned* aliases (filtering out the
# auto-added short id). Then diff those.
sid_before=$(echo "$before" | jq -r '.[0].Id[0:12]')
sid_after=$(echo "$after"  | jq -r '.[0].Id[0:12]')
diff \
  <(echo "$before" | jq --arg sid "$sid_before" '.[0].NetworkSettings.Networks | to_entries | map({net: .key, aliases: ((.value.Aliases // []) | map(select(. != $sid)) | sort)}) | sort_by(.net)') \
  <(echo "$after"  | jq --arg sid "$sid_after"  '.[0].NetworkSettings.Networks | to_entries | map({net: .key, aliases: ((.value.Aliases // []) | map(select(. != $sid)) | sort)}) | sort_by(.net)')
                                              # expected: empty diff

Cleanup:

docker rm -f fd-smoke-weird $(docker ps -a --filter name=fd-smoke-weird-old- -q)
docker network rm fd-front fd-back
rm -rf /tmp/fd-state /tmp/fd-secrets

Current limitations (do not file as bugs)

  • Registry auth is out of scope. Pull works for Docker Hub anonymously and for any image already present in the local cache. GHCR / Quay / lscr.io land in Phase 5.
  • Health timing is not yet configurable from the CLI/config. Phase 3 uses built-in defaults (HealthConfig::default()); sourcing it from freshdock.toml/labels is Phase 4/5.

Manual smoke test: image cleanup

Verifies the opt-in image cleanup that runs after a successful, health-passed update (PLAN §5.2 step 8). Cleanup is off by default; it removes the image the replaced container was running, and is best-effort (a shared image still referenced by another container is kept, and a cleanup failure never fails the update).

There are two knobs:

  • [settings] cleanup = true (or per container freshdock.cleanup=true): remove the replaced image after a healthy update.
  • [settings] prune_dangling = true: additionally run a daemon-wide dangling-image prune after each successful update.

The unit tests in src/docker/recreate.rs (recreate_with_health_removes_old_image_when_cleanup_enabled, recreate_with_health_prunes_dangling_when_enabled, cleanup_failure_does_not_fail_the_update) are the authoritative checks; this procedure is for human verification against a real daemon.

Prerequisites

  • A working Docker daemon on the standard socket.
  • freshdock built locally: just build.

Steps

freshdock recreate always pulls the container’s tag before it recreates, so the tag has to exist upstream: a local-only tag such as fd-cleanup:demo fails the pull with a 404 before cleanup ever runs. The procedure therefore points the real nginx:alpine tag at an older image for the duration of the test. The container keeps that older image id, and the recreate’s own pull moves the tag back to the current upstream image, so the originally-running image is no longer referenced afterwards.

# 1. Point `nginx:alpine` at an older image and run a container on it with
#    cleanup opted in.
docker pull nginx:1.27-alpine
docker tag  nginx:1.27-alpine nginx:alpine

docker run -d --name fd-cleanup \
  --label freshdock.enable=true \
  --label freshdock.mode=watch \
  --label freshdock.cleanup=true \
  nginx:alpine
old_id=$(docker inspect fd-cleanup --format '{{.Image}}')

# 2. Drop the extra tag so nothing but the container references the old image.
#    Skip this step to exercise the shared-image guard instead (see below).
docker rmi nginx:1.27-alpine

# 3. Recreate. The pull restores the upstream `nginx:alpine`; after the new
#    container is healthy the superseded image is removed (best-effort).
./target/release/freshdock recreate fd-cleanup

# 4. The old image id must be gone (no container references it any more).
docker image inspect "$old_id" >/dev/null 2>&1 \
  && echo "FAIL: old image still present" \
  || echo "OK: superseded image removed"

Expected observations

  • The CLI prints a line starting recreated fd-cleanup: healthy that names the removed archive and the new id.
  • Step 4 prints OK: superseded image removed.
  • With freshdock.cleanup=false (or the label omitted and [settings] cleanup unset), step 4 instead prints FAIL, i.e. the old image is kept. That is the correct default-off behaviour; re-run the procedure from step 1 without the cleanup label to confirm.
  • Shared-image guard. Skip step 2 (or keep a second container running on $old_id) and the removal is refused by the daemon (HTTP 409) because the image is still referenced, logged as a warning, and the update still reports success. The image stays until the last reference is gone.

Cleanup

docker rm -f fd-cleanup $(docker ps -a --filter name=fd-cleanup-old- -q) 2>/dev/null
docker rmi nginx:1.27-alpine 2>/dev/null || true
docker pull nginx:alpine   # only needed if you skipped step 3

Manual smoke test: network-namespace dependents

Verifies that containers sharing another container’s network namespace (--network container:X, or compose’s network_mode: service:X) are put back together after freshdock recreates X (issue #68). Recreating X destroys the namespace its dependents were attached to: without repair they keep running but have no network at all, and any container:<old id> reference is now dangling.

freshdock captures the dependents before stopping X and re-creates each one afterwards: on the success path with an id-based reference repointed at the new container id (a literal name reference is left name-based), and on the rollback path with the reference left as-is (the restored container owns its original id again, but the namespace behind it is new either way). The repair runs before X’s -old- archive is removed: Docker refuses to remove a container whose namespace a running container still holds.

Each dependent is re-created from its own reference (Config.Image) as long as that still resolves to the image it runs, so it stays on its tag and freshdock keeps checking it (issue #102). Once the tag points elsewhere, it is re-created from the exact image ID instead, because a repair must never double as an unrequested upgrade; its Config.Image then reads as an image id, it reports pinned (no check), and freshdock logs a warning saying so. (When the daemon reports no image id, which is rare, the reference is used.)

The unit tests in src/docker/recreate.rs (healthy_update_reattaches_id_based_dependent, healthy_update_keeps_name_based_dependent_ref, rollback_reattaches_dependents_without_rewrite, dependent_create_body_drops_what_a_shared_namespace_forbids, dependent_create_failure_restores_the_dependent, dependent_failure_does_not_fail_update, dependent_stays_on_its_tag_while_the_tag_resolves_to_its_image, dependent_is_pinned_to_its_image_id_once_its_tag_moved) and the live tests in tests/contract_live.rs are the authoritative checks; this procedure is for human verification against a real daemon.

Prerequisites

  • A working Docker daemon on the standard socket.
  • freshdock built locally: just build.

Steps

The dependent needs no freshdock labels: it is not being updated, only repaired, so the policy gate that guards recreate does not apply to it. It is skipped only if it explicitly opts out (freshdock.enable=false or freshdock.mode=off), or if it turns out to be the freshdock container itself.

# 1. The namespace owner, the container freshdock will update.
docker run -d --name fd-base \
  --label freshdock.enable=true \
  --label freshdock.mode=live \
  nginx:alpine

# 2. A dependent joined to fd-base's network namespace. `--network
#    container:fd-base` is the name-based form; the id-based form compose
#    produces is covered further down.
docker run -d --name fd-peer --network container:fd-base \
  alpine:3.20 sleep 1d

# 3. Prove the namespace is shared: nginx is reachable on the peer's loopback.
docker exec fd-peer wget -qO- 127.0.0.1 | head -1   # -> <!DOCTYPE html>

# 4. Capture the "before" facts. NOTE: a modern daemon NORMALISES the reference
#    at create time, so this prints `container:<full 64-char id of fd-base>`,
#    not `container:fd-base`.
docker inspect fd-base --format '{{.Id}}'                       # old owner id
docker inspect fd-peer --format '{{.HostConfig.NetworkMode}}'   # container:<old id>
docker inspect fd-peer --format '{{.State.StartedAt}}'
docker inspect fd-peer --format '{{.Image}}'                    # image ID
docker inspect fd-peer --format '{{.Config.Image}}'             # alpine:3.20

# 5. Recreate the owner.
./target/release/freshdock recreate fd-base

Expected observations

  • The CLI prints a line starting recreated fd-base: healthy, and the log carries an re-attaching network-namespace dependent line naming fd-peer.
  • docker inspect fd-peer --format '{{.State.StartedAt}}' is newer than the value from step 4; the dependent was re-created, not left behind.
  • docker inspect fd-peer --format '{{.HostConfig.NetworkMode}}' reads container:<new fd-base id>. Because the daemon normalised container:fd-base to an id at create time, this is the same rewrite as the id-based case below. (freshdock still has a branch that leaves a literal name reference untouched; it only fires on daemons that store the reference verbatim.)
  • docker inspect fd-peer --format '{{.Image}}' is unchanged from step 4, and {{.Config.Image}} still reads alpine:3.20: the tag resolves to the image the dependent runs, so the repair keeps it.
  • docker exec fd-peer wget -qO- 127.0.0.1 | head -1 reaches the new nginx. This is the headline assertion. Before #68 it failed with no route to host.
  • docker ps -a shows no fd-peer-old-<ts> left over: the dependent’s own archive is removed once it is running again, and fd-base-old-<ts> is gone too; it can only be removed after the dependent let go of its namespace.

Id-based reference (what compose, and any modern daemon, actually stores)

docker rm -f fd-peer
base_id=$(docker inspect fd-base --format '{{.Id}}')
docker run -d --name fd-peer --network "container:$base_id" alpine:3.20 sleep 1d

./target/release/freshdock recreate fd-base

# The reference must now point at the NEW owner id, not the dead one.
docker inspect fd-peer --format '{{.HostConfig.NetworkMode}}'
docker inspect fd-base --format '{{.Id}}'   # -> the id inside NetworkMode above
docker exec fd-peer wget -qO- 127.0.0.1 | head -1

Pass criteria: NetworkMode holds container:<new fd-base id> and the wget still succeeds. A container:<old id> here means the rewrite did not happen; the container would refuse to start at all on the next daemon restart.

Moved tag

docker rm -f fd-peer
docker tag alpine:3.20 fd-peer-img:1
docker run -d --name fd-peer --network container:fd-base fd-peer-img:1 sleep 1d
running=$(docker inspect fd-peer --format '{{.Image}}')
docker commit fd-peer fd-peer-img:1   # the tag now points at a new image

./target/release/freshdock recreate fd-base

docker inspect fd-peer --format '{{.Image}} {{.Config.Image}}'   # both $running

Pass criteria: both fields equal $running, and the log carries a warning that fd-peer was re-attached from its image id and is pinned until it is recreated from its tag, with reason="its tag now points to <new id>". Clean up with docker rmi fd-peer-img:1 after the cleanup below.

Rollback path

The failure has to live in the container spec, not in the image: the cycle always pulls the tag first, which undoes any local re-tagging trick. Give the owner a healthcheck that can never pass. It round-trips through ContainerSpec, so the replacement fails the gate no matter what the pull brings in:

docker rm -f fd-base fd-peer 2>/dev/null
docker run -d --name fd-base \
  --label freshdock.enable=true --label freshdock.mode=live \
  --health-cmd 'exit 1' --health-interval 2s \
  nginx:alpine
docker run -d --name fd-peer --network container:fd-base alpine:3.20 sleep 1d

./target/release/freshdock recreate fd-base

The CLI prints recreate failed for fd-base: ... rolled back, and fd-peer is still re-created afterwards (its StartedAt moves) with its NetworkMode unchanged; the restart gave the restored fd-base a fresh namespace, so the dependent has to re-attach even though nothing was updated.

Best-effort contract

If a dependent’s re-creation fails (it was removed by hand mid-run, its name collides, the daemon refuses the create), the run still reports a line starting recreated fd-base: healthy and logs a warning naming the dependent. Re-attachment repairs collateral damage; it never turns a good update into a failure, and one broken dependent never stops the next one from being fixed.

When the failure lands after the dependent was renamed to its -old- form, freshdock renames it back and starts it again, so you are left with a running container on a dead namespace rather than a stopped archive no scheduler will ever look at. If that restore also fails, the warning names the archive to recover by hand.

Opt-out and self-exclusion

docker run -d --name fd-peer-off --network container:fd-base \
  --label freshdock.enable=false alpine:3.20 sleep 1d

freshdock recreate fd-base must leave fd-peer-off alone (its StartedAt does not move) and log a warning that it keeps a dead network namespace until restarted manually. The same applies to freshdock.mode=off. A dependent that turns out to be the freshdock container itself (its id starts with freshdock’s own hostname) is skipped the same way. Stopping it would kill the daemon mid-cycle.

Cleanup

# Scoped to THIS test's archives. A bare `name=-old-` filter would catch
# unrelated containers on the host.
docker rm -f fd-peer fd-base 2>/dev/null
docker ps -aq --filter name=fd-base-old- --filter name=fd-peer-old- | xargs -r docker rm -f

Current limitations (do not file as bugs)

  • Only HostConfig.NetworkMode = container:<ref> is treated as a dependency. Containers attached to a user-defined network keep working across a recreate on their own and need no repair.
  • Id references shorter than Docker’s 12-character short id are ignored; they are not unique enough to attribute to the container being updated.
  • Dependents of dependents are not followed transitively: a chain C -> B -> A re-creates B when A is updated, which in turn breaks C. Nothing repairs C on its own; B is only re-created again when B’s own image updates, and C’s own update cannot fix a dangling container:<old B id> reference. Restart C by hand after such a run.

Manual smoke test: Compose project rollouts

Verifies that a Docker Compose project is updated as one unit (issue #78): the one-shots the project waits on are re-run first, services follow in depends_on order, restart: true dependents are bumped afterwards, and a failed one-shot aborts the rollout without touching anything downstream.

The unit tests in src/rollout.rs and src/compose.rs are the authoritative checks; every rule below is pinned there. This procedure is for human verification against a real daemon, and it is what to re-run after touching the rollout planner.

Prerequisites

  • A working Docker daemon on the standard socket.
  • freshdock built locally: just build (or cargo build for a debug binary).
  • Outbound access to Docker Hub (the stack uses busybox and alpine).

The stack

One project exercising every rule at once. Note which containers carry labels and which deliberately do not.

# docker-compose.yml
services:
  # Unlabelled one-shot the project waits on. MUST be re-run.
  migrate:
    image: busybox:latest
    command: ["sh", "-c", "if [ -f /data/fail ]; then echo failed >> /data/migrations.log; exit 1; fi; echo ran >> /data/migrations.log"]
    volumes: ["./data:/data"]
    restart: "no"

  # Unlabelled one-shot that explicitly opts out. MUST NOT be re-run.
  seed:
    image: busybox:latest
    command: ["sh", "-c", "echo ran >> /data/seed.log; exit 0"]
    volumes: ["./data:/data"]
    restart: "no"
    labels:
      freshdock.enable: "false"

  # The labelled service that triggers the rollout.
  web:
    image: busybox:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]
    labels:
      freshdock.enable: "true"
      freshdock.mode: "live"
    depends_on:
      migrate:
        condition: service_completed_successfully
      seed:
        condition: service_completed_successfully

  # Unlabelled, restart: true on web. MUST be restarted, not recreated.
  sidecar:
    image: alpine:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]
    depends_on:
      web:
        condition: service_started
        restart: true

  # Unlabelled and long-running on the SAME image as web.
  # MUST NOT be updated: sharing an image is not consent.
  bystander:
    image: busybox:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]

  # Labelled but deliberately stopped. MUST NOT be started.
  paused:
    image: busybox:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]
    labels:
      freshdock.enable: "true"
      freshdock.mode: "live"

  # Watch mode on the SAME image as web. MUST NOT be recreated.
  watcher:
    image: busybox:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]
    labels:
      freshdock.enable: "true"
      freshdock.mode: "watch"

  # Watch mode AND restart: true on web. MUST NOT be restarted either.
  watchdep:
    image: alpine:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]
    labels:
      freshdock.enable: "true"
      freshdock.mode: "watch"
    depends_on:
      web:
        condition: service_started
        restart: true
mkdir -p data
docker compose -p fdsmoke up -d
docker stop fdsmoke-paused-1

# Record the baseline: container ids are what the assertions turn on.
docker ps -a --filter label=com.docker.compose.project=fdsmoke \
  --format '{{.Names}}\t{{.Status}}\t{{.ID}}' | sort
# A restart keeps the container id, so those assertions turn on StartedAt.
docker inspect fdsmoke-sidecar-1 fdsmoke-watchdep-1 --format '{{.Name}} {{.State.StartedAt}}'
wc -l data/migrations.log data/seed.log

1. The happy path

freshdock recreate always recreates its target, so it drives a rollout without waiting for an upstream image to move.

freshdock recreate fdsmoke-web-1

Expected output, in this order:

rollout of compose project fdsmoke:
  fdsmoke-migrate-1: re-ran to a successful exit
  fdsmoke-web-1: updated and healthy (new id ...)
  fdsmoke-sidecar-1: restarted (depends_on restart: true)
rollout complete: 3 step(s)

Then check the daemon, not just the log:

CheckExpected
wc -l data/migrations.log2, the unlabelled one-shot was re-run.
wc -l data/seed.log1, freshdock.enable=false was honoured.
fdsmoke-migrate-1 idchanged, recreated on the new image.
fdsmoke-web-1 idchanged.
fdsmoke-sidecar-1 idunchanged, uptime reset: restarted, not recreated.
fdsmoke-watcher-1 idunchanged: watch mode is never recreated, same image or not.
fdsmoke-watchdep-1 State.StartedAtunchanged: watch mode outranks a restart: true edge.
fdsmoke-bystander-1 idunchanged: an unlabelled sibling on the same image is not swept in.
fdsmoke-paused-1still Exited, id unchanged: a stopped container is not started.
docker ps -a | grep -- -old-nothing, archives cleaned up.

Ordering matters as much as the outcome: migrate must appear before web.

2. A failed migration aborts the rollout

This is the case the feature exists for. Trip the marker file and run again:

touch data/fail
freshdock recreate fdsmoke-web-1
rollout ABORTED: fdsmoke-migrate-1 exited with code 1. The services after this
point were not touched and are still running their previous image.
CheckExpected
fdsmoke-web-1 idunchanged: new code never came up against a failed migration.
fdsmoke-sidecar-1not restarted.
fdsmoke-migrate-1present, Exited (1), kept for its logs.
fdsmoke-migrate-1-old-<ts>present, the archive is kept too.

Neither the failed container nor its archive is removed, and there is no rollback: the command already ran, so its logs are the only evidence left.

Clean up before continuing: rm data/fail, then docker compose -p fdsmoke down -v && docker compose -p fdsmoke up -d.

3. The opt-out switch

FRESHDOCK_COMPOSE_AWARE=0 freshdock recreate fdsmoke-web-1

Expected: the pre-#78 single-container output (recreated fdsmoke-web-1: healthy ...), no rollout block, and data/migrations.log still at one line.

4. The scheduler path

The rollout has to fire from the daemon, not just from recreate. Make the local image genuinely stale so the digest probe reports an update:

docker pull busybox:1.36
docker pull busybox:latest

# The whole step only tests anything if the two really are different images.
# busybox:1.36 may already be what :latest points at, and then the scheduler
# finds the project up to date and never rolls anything out.
old=$(docker image inspect busybox:1.36 --format '{{.Id}}')
new=$(docker image inspect busybox:latest --format '{{.Id}}')
[ "$old" != "$new" ] || { echo "busybox:1.36 IS :latest; pick an older tag"; exit 1; }

docker tag busybox:1.36 busybox:latest   # local digest now differs from upstream
docker compose -p fdsmoke down -v && rm -f data/*.log && docker compose -p fdsmoke up -d
docker stop fdsmoke-paused-1

RUST_LOG=freshdock=debug freshdock run --interval 5 --tick 5

Expected on the first tick: one registry rate limit debug line (the image is probed once, for web), then the same three rollout steps. Later ticks probe again but must not re-roll: no further rollout steps.

The single probe is the assertion that matters: it shows the rollout deduplicated its own members instead of processing each one separately.

Restore the tag afterwards with docker pull busybox:latest (the rollout’s own pull normally does this for you).

5. A single-service project is unchanged

mkdir -p solo && cat > solo/docker-compose.yml <<'YAML'
services:
  only:
    image: busybox:latest
    command: ["sh", "-c", "while true; do sleep 5; done"]
    labels:
      freshdock.enable: "true"
      freshdock.mode: "live"
YAML
docker compose --project-directory solo -p fdsolo up -d
freshdock recreate fdsolo-only-1

Expected: the plain recreated fdsolo-only-1: healthy ... line and no rollout block at all. When a plan comes down to the container that triggered it, the ordinary path is used, so a one-service project behaves exactly as it did before this feature existed.

Cleanup

docker compose -p fdsmoke down -v
docker compose --project-directory solo -p fdsolo down -v

Manual smoke test: SMTP notifications

The SMTP backend talks a real mail protocol, so CI can’t exercise the TLS modes without a relay. Message construction is covered automatically by the build_message unit tests in src/notify/smtp.rs, and plaintext delivery by tests/smtp_plaintext.rs against an in-process fake server; this procedure verifies the transport (connection, STARTTLS, auth) against a real catcher.

Prerequisites

  • freshdock built locally: just build.

  • A local SMTP catcher. mailpit is the simplest. It exposes SMTP on :1025 and a web inbox on :8025:

    docker run --rm -p 1025:1025 -p 8025:8025 axllent/mailpit
    

Plain delivery (no TLS, no auth)

  1. Write a freshdock.toml pointing at the catcher. tls = "none" because mailpit’s default listener speaks plaintext SMTP:

    [notifications.email]
    type = "smtp"
    host = "localhost"
    port = 1025
    from = "[email protected]"
    to = ["[email protected]"]
    tls = "none"
    # triggers omitted: subscribes to available, succeeded, and failed
    

    Or skip the file entirely and declare the same target with an env URL:

    export FRESHDOCK_NOTIFY_EMAIL_URL='smtp://localhost:1025/[email protected]&[email protected]&tls=none'
    

    Not starttls = false: that legacy key selects implicit TLS (SMTPS), so the earlier version of these instructions opened a TLS handshake against mailpit’s plaintext listener and could never deliver (#57/#58). Only tls = "none" disables encryption.

    port is spelled out on purpose: an omitted port defaults from the mode, and for tls = "none" that is the classic SMTP port 25. Mailpit listens on 1025.

  2. Trigger a notification. The quickest path is a watch-mode container with a newer image available; or force a failed update (a broken healthcheck) to exercise the failed trigger and its rollback detail. Run the daemon:

    cargo run -- run
    

    Startup logs one WARN per plaintext target (smtp target uses a PLAINTEXT transport ...). That warning is expected here, and it is the reason tls = "none" never belongs in a deployment.

  3. Open the mailpit inbox at http://localhost:8025 and confirm a message arrived with the rendered Subject (Update available: ... / Updated: ... / Update failed: ...) and the matching body.

STARTTLS + auth

STARTTLS and PLAIN/LOGIN auth must be verified against a server that requires them (mailpit’s --smtp-auth modes, a real provider, or smtp4dev with TLS enabled):

  1. Point host/port at the TLS-capable relay, set tls = "starttls", and add credentials. The password may be supplied inline or via the environment override (so it stays out of the file):

    [notifications.email]
    type = "smtp"
    host = "smtp.example.com"
    port = 587
    username = "[email protected]"
    from = "[email protected]"
    to = ["[email protected]"]
    tls = "starttls"
    
    export FRESHDOCK_NOTIFY_EMAIL_PASSWORD='app-password'
    cargo run -- run
    

    Or as a single env URL (percent-encode @/: in the login as %40/%3A):

    export FRESHDOCK_NOTIFY_EMAIL_URL='smtp://freshdock%40example.com:[email protected]:587/[email protected]&[email protected]&tls=starttls'
    
  2. Confirm the message is delivered. A STARTTLS handshake failure surfaces as a smtp send failed: ... WARN line; delivery is non-fatal, so the daemon keeps running regardless.

Pass criteria

  • A message with the correct subject/body lands in the inbox for each trigger.
  • The STARTTLS run authenticates and delivers without falling back to plaintext.
  • The bot token / SMTP password never appears in freshdock’s log output.