Skip to content

Self-hosting

Upgrading and rolling back Beta

Upgrading with versions.json, pinning images, backing up first, the order of steps and one-command rollback.

This page describes a beta feature: it works but may still change.

On this page

Principles#

  • The versions.json manifest at the repository root is the source of truth for each component's version and the compatibility constraints. Services do not read it at run time; it is for operators, the site builder and the Compatibility matrix page.
  • Always pin images. The editing server is pinned by digest through O3O_ONLINE_IMAGE; never use moving tags such as latest in production, because docker compose pull can silently swap the document engine.
  • In v1 the gate and DocBuilder are built locally from the repository. The compose file takes their image tags from O3O_GATE_IMAGE (default o3o/gate:dev) and O3O_DOCBUILDER_IMAGE (default o3o/docbuilder:dev). On every upgrade keep the old image under another tag so a rollback is one command.
  • After checking out the new release, run python tools/release.py env --write online/.env (at the repository root) to write the .env version block from versions.json: O3O_ONLINE_IMAGE, O3O_ONLINE_IMAGE_KIND, O3O_GATE_IMAGE, O3O_DOCBUILDER_IMAGE. The command only replaces that block, keeps the secrets and every other variable, and saves the previous file as .env.bak. Details on Compose file and environment variables.
  • Read the Changelog first: breaking API changes only ship in a new API version, never inside v1.

Before upgrading#

  1. Back up the o3o-gate-data volume: it holds instance-id (tokens may be bound to it), secret.key (signs download URLs), the working copies of embedded documents and the usage log. o3o-docbuilder-data only holds temporary results.
  2. Keep a copy of .env and note the running images.
  3. The DocBuilder queue is in memory: a restart loses queued and running jobs (finished jobs stay downloadable until they expire). Wait until queue.queued and queue.running in GET /v1/status are 0.
  4. Pick a quiet hour for the editing server: check connections.current in GET /o3o/status.
BashBack up the gate data volume
# Volume names are prefixed with the compose project name
docker volume ls | grep -E 'o3o-(gate|docbuilder)-data'

# Archive into the current folder
docker run --rm -v <project name>_o3o-gate-data:/data:ro -v "$PWD":/backup alpine \
  tar czf /backup/o3o-gate-data-$(date +%Y%m%d).tgz -C /data .

Upgrade steps#

BashOne upgrade run
cd online
DC="docker compose --env-file .env -f docker/compose.dev.yml"

# 1. Record the current state and keep the old images
$DC images
cp .env ".env.bak-$(date +%Y%m%d)"
GATE_IMAGE=$(grep -E '^O3O_GATE_IMAGE=' .env | cut -d= -f2-)
DOCBUILDER_IMAGE=$(grep -E '^O3O_DOCBUILDER_IMAGE=' .env | cut -d= -f2-)
docker image tag "${GATE_IMAGE:-o3o/gate:dev}" o3o/gate:rollback
docker image tag "${DOCBUILDER_IMAGE:-o3o/docbuilder:dev}" o3o/docbuilder:rollback

# 2. The DocBuilder queue must be empty; few editing sessions should be open
curl -s http://localhost:8080/v1/status | jq .queue
curl -s http://localhost:8080/o3o/status | jq '{current: .connections.current, open: .embed.open_documents}'

# 3. Get the new release; compare variable lists to add new variables to .env; write the version block (O3O_ONLINE_IMAGE, O3O_GATE_IMAGE, O3O_DOCBUILDER_IMAGE) from versions.json
git fetch --tags
git checkout <release tag>
diff <(grep -oE '^[A-Z0-9_]+=' .env.example | sort) <(grep -oE '^[A-Z0-9_]+=' .env | sort)
python ../tools/release.py env --write .env

# 4. Rebuild the images with the tags from .env (context is the online/ folder)
GATE_IMAGE=$(grep -E '^O3O_GATE_IMAGE=' .env | cut -d= -f2-)
DOCBUILDER_IMAGE=$(grep -E '^O3O_DOCBUILDER_IMAGE=' .env | cut -d= -f2-)
docker build -f gate/Dockerfile -t "${GATE_IMAGE:-o3o/gate:dev}" .
docker build -f docbuilder/Dockerfile -t "${DOCBUILDER_IMAGE:-o3o/docbuilder:dev}" .

# 5. Apply in order: DocBuilder, the gate, then the rest
$DC up -d --force-recreate o3o-docbuilder
$DC up -d --force-recreate o3o-gate
$DC up -d

# 6. Check the versions after the upgrade
curl -s http://localhost:8080/o3o/status | jq '{version, api, edition}'
curl -s http://localhost:8080/v1/status | jq '{version, api, core}'

Rolling back#

BashReturn to the previous release
cd online
DC="docker compose --env-file .env -f docker/compose.dev.yml"

git checkout <previous release tag>
cp .env.bak-YYYYMMDD .env                        # the .env copy saved in step 1
GATE_IMAGE=$(grep -E '^O3O_GATE_IMAGE=' .env | cut -d= -f2-)
DOCBUILDER_IMAGE=$(grep -E '^O3O_DOCBUILDER_IMAGE=' .env | cut -d= -f2-)
docker image tag o3o/gate:rollback "${GATE_IMAGE:-o3o/gate:dev}"
docker image tag o3o/docbuilder:rollback "${DOCBUILDER_IMAGE:-o3o/docbuilder:dev}"
$DC up -d --force-recreate o3o-docbuilder o3o-gate
$DC up -d                                        # recreates o3o-online if its digest changed

Do not run docker image prune while a rollback may still be needed. If the changelog says the gate's data format changed, restore the volume backup as well.