Data registry

Migrate from an old server

Dependencies

Perform the same tasks as for Data support.

Update Salt configuration and halt jobs

1. For the new server:

   1. Check that docker.uid in the server’s Pillar file matches the entry in the /etc/passwd file for the docker.user (deployer).

   2. Change cron.present to cron.absent in the salt/registry/init.sls file.

   3. Comment out the postgres.backup section of the Pillar file.

2. Deploy the old server and the new server.

3. On the old server:

   1. Delete the /etc/cron.d/postgres_backups file.

   2. docker compose down all containers, except the web and static containers of the data-registry service.

4. Check that no crawls are running at https://collect.data.open-contracting.org/jobs.

   If a crawl is running, Django administrators can cancel jobs.

5. Check that no messages are enqueued at https://rabbitmq.data.open-contracting.org.

   If a job is running in Kingfisher Process, job owners can cancel jobs.

Filesystem

Copy these directories from the old server to the new server, using rsync -avz:

• /data/storage/exporter

• /data/storage/spoonbill

• /home/collect/scrapyd/dbs

• /home/collect/scrapyd/eggs

• /home/collect/scrapyd/jobs

• /home/collect/scrapyd/logs

Databases

1. Copy the data_registry and spoonbill databases from the old server to the new server, using pg_dump.

Docker apps

Perform the same tasks as for Data support, skipping pelican-frontend.

Restore Salt configuration and start jobs

1. Change cron.absent to cron.present in the salt/registry/init.sls file.

2. Uncomment the postgres.backup section of the Pillar file.

3. Deploy the new server.