ODK Central Domain Change: A Painless Backup and Restore Tutorial

Introduction

ODK Central is a powerful tool for collecting, managing, and analyzing data using mobile devices. However, there may be situations where you need to move your ODK deployment from one server to another, such as changing hosting providers, upgrading hardware, or switching domains.

This tutorial will walk you through the steps of creating a direct backup of your ODK Central database, copying the backup file and other essential data to the destination server, and restoring the backup file on the new server. The tutorial will also show you how to update the domain names in the Enketo Redis databases and config files to avoid errors when previewing forms on the new server.

By following this tutorial, you will be able to migrate your ODK deployment from one server to another with minimal downtime and data loss.

Prerequisites

Before you begin, ensure that you have the following prerequisites:

A running instance of ODK Central on the source and destination servers.
Both source and destination servers are running the same version of ODK Central.
Access to the command line interface (CLI) of both source and destination servers.
Sufficient disk space on both servers to store the backup files and other essential data.

If you have met these requirements, you will be able to follow this tutorial easily and avoid errors or issues that may arise during the backup and restore process.

Backup ODK Central Main Data on the Source Server

Log in to your ODK Central source server.
From the Terminal window, create a folder odk-backups for the ODK Central backup files in the logged-in user's home folder. You can use a different name that follows your folder naming conventions.
```
    mkdir ~/odk-backups
```
Run the command below to perform a direct backup of your ODK Central database. Replace backup-password, your-odk-username@your-odk-domain-name, your-odk-domain-name, and backup-filename.zip placeholder values with actual values for your ODK Central installation.
```
   curl -X POST -H "Content-Type: application/json" -d '{"passphrase": "backup-password"}' -su your-odk-username@your-odk-domain-name https://your-odk-domain-name/v1/backup --output ~/odk-backups/backup-filename.zip
```
- {"passphrase": "backup-password"}: password that will be used to encrypt the backup file
- your-odk-username@your-odk-domain-name: Username that you use to log into the ODK Central web interface
- your-odk-domain-name: Registered domain name for your ODK Central installation
- backup-filename.zip: The name that will be assigned to the backup file.

You will be asked to provide your user password before the command can run. The password you type will be the password for the user that you use to log into the ODK Central web interface

Wait for the backup process to run to completion.

Below is an example of commands contained in the previous steps.

 root:~#
 root:~# mkdir odk-backups
 root:~#
 root:~# curl -X POST -H "Content-Type: application/json" -d '{"passphrase": "bk202309"}' -k  -u testapp@example.com https://odk-src.kagundajm.codes/v1/backup --output ~/odk-backups/odk-bk20230913-08AM.zip
 Enter host password for user 'testapp@example.com':
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 100  481M    0  481M    0    26  4819k      0 --:--:--  0:01:42 --:--:-- 10.4M
 root:~#
 root:~# cd odk-backups/
 root:~#
 root:~/odk-backups# ls -la
 total 493356
 drwxr-xr-x  2 root root      4096 Sep 12 05:17 .
 drwx------ 10 root root      4096 Sep 12 05:17 ..
 -rw-r--r--  1 root root 505183986 Sep 12 05:19 odk-bk20230913-08AM.zip
 root:~/odk-backups#

List ODK Central Docker Containers

This step is not mandatory, but it helps you understand the source of Docker container names used in this tutorial.

On the source server, change directory to the ODK Central installation folder
```
 cd central
```

List names and state of ODK Central Docker containers using the following command:

 docker ps --format 'table {{.Names}}\t{{.State}}'

--format option will list names and headers of the required fields. \t is a tab character to separate the fields.

Running the command will output a list similar to the following:

 NAMES                          STATE
 central-nginx-1                running
 central-service-1              running
 central-enketo-1               running
 central-pyxform-1              running
 central-enketo_redis_cache-1   running
 central-mail-1                 running
 central-postgres14-1           running
 central-enketo_redis_main-1    running

Backup Enketo Redis Data and Secrets

Redis stores data in memory (RAM) which allows for fast data access and retrieval. The ODK Central installation includes two Redis database instances; central-enketo_redis_main-1 and central-enketo_redis_cache-1. To backup the Redis databases, you run commands to flush the in-memory data to disk.

Log in to your ODK Central source server if you had logged out.
Change directory to the ODK Central installation folder
```
 cd ~/central
```
Flush in-memory data to disk on the main instance of Enketo Redis database by running the following command.
```
 docker exec central-enketo_redis_main-1 redis-cli -p 6379 save
```

The command instructs Docker to execute the Redis Command Line Interface (CLI) tool (redis-cli )within the Docker container named central-enketo_redis_main-1 which is listening on port 6379. save creates a snapshot of the current state of the Redis database and writes it to disk.

The server should display OK after the command completes and a backup file named enketo-main.rdb will be created in the /data/ folder of the central-enketo_redis_main-1 container.

Copy the enketo-main.rdb file from the Docker container to the backup folder you created.

 docker cp  central-enketo_redis_main-1:/data/enketo-main.rdb ~/odk-backups/enketo-main.rdb

Flush in-memory data to disk on the cache instance of Enketo Redis database.
```
 docker exec central-enketo_redis_cache-1 redis-cli -p 6380 save
```
The server should display OK after the command completes and a backup file named enketo-cache.rdb will be created in the /data/ folder of the central-enketo_redis_cache-1 container.

Copy the enketo-cache.rdb backup file from the Docker container to the backup folder you created.

 docker cp  central-enketo_redis_cache-1:/data/enketo-cache.rdb ~/odk-backups/enketo-cache.rdb

The central-service-1 container has a /etc/secrets folder which contains files used to store secret keys for the Enketo server. These keys are used to encrypt and decrypt data. Copy the secrets folder to the odk-backups folder.

 docker cp central-service-1:/etc/secrets ~/odk-backups

The following is an example sequence of the commands and outputs for the above steps.

 root:~/odk-backups# cd ~/central
 root:~/central#
 root:~/central# docker exec central-enketo_redis_main-1 redis-cli -p 6379 save
 OK
 root:~/central# docker cp  central-enketo_redis_main-1:/data/enketo-main.rdb ~/odk-backups/enketo-main.rdb
 Successfully copied 39.9kB to /root/odk-backups/enketo-main.rdb
 root:~/central#
 root:~/central# docker exec central-enketo_redis_cache-1 redis-cli -p 6380 save
 OK
 root:~/central#
 root:~/central# docker cp  central-enketo_redis_cache-1:/data/enketo-cache.rdb ~/odk-backups/enketo-cache.rdb
 Successfully copied 2.05kB to /root/odk-backups/enketo-cache.rdb
 root:~/central#
 root:~/central# docker cp central-service-1:/etc/secrets ~/odk-backups
 Successfully copied 4.61kB to /root/odk-backups
 root:~/central#

Copy Enketo config file from Docker container to a text file in the backup folder.

 docker cp central-enketo-1:/srv/src/enketo_express/config/config.json ~/odk-backups/enketo-config-keys.txt

Open the text file using the nano editor.

 nano  ~/odk-backups/enketo-config-keys.txt

Using Control + K (^K) keys, remove all key/pair values in the text file except "encryption key", "less secure encryption key", and "api key". After you have completed removing all other key/values pairs, your text file should contain entries similar to the following:

 "encryption key": "JAUzy4moYjA284rORjMcd2FIQCEdniSx9rmYZrqZSdxwzmG1WY6Apx6QAphgAFc9",
 "less secure encryption key": "KgyIvGRFtnctP99qGzi0ii1GVg1j453U",
 "api key": "YJXZRqiOirbSijVROqnh3HGCvuhEwhmKFrkyDkwl4HHIYo0Z3YNitfQH1McNp7srlf1RcvCfXX8vzCVoINW7UgZAIT6jbjR8gxaYJR3wzE8dNGZgdEitwl7cs2abkWUA",

Press Control + O (^O) to save the changes.
Press ENTER key to confirm the file name.
Press Control + X (^X) to close the editor.
Log out of the source server

Copy Backups From Source Server To Destination Server

Log in to your ODK Central destination server.
If you are using username/password to connect to the source server, run the command below to copy the ~/odk-backups folder to the destination server. You will be prompted to type the user's password before the rsync command starts the syncing process.
```
 rsync -avz user@remote_server:~/odk-backups  ~
```
Use the below rsync command if you use SSH keys to log in to your source server.
```
 rsync -avz -e "ssh -i ssh-key-file"  user@remote_server:~/odk-backups  ~
```
-a : Archive mode, which preserves file attributes and permissions.

-v : Verbose output, so you can see the progress.

-z : Compresses data during transfer, reducing the network bandwidth used.

-e "ssh -i ssh-key-file" : Tells rsync to use SSH for connecting to the remote server using -i ssh-key-file SSH private key file for authentication.

user@remote_server:~/odk-backups/ : user is the username for the remote server while remote_server is the IP address or registered hostname for the remote server. odk-backups refers to the folder containing the backup filles on the remote server.

Replace user, remote_server, and ssh-key-file with actual values for your environment.

A message and prompt similar to the one below may be displayed if SSH does not have a record of the remote server's public key. Make sure you trust the remote server before typing ‘yes’ to continue connecting.

The authenticity of host 'kagundajm.codes (93.184.216.34)' can't be established.
ED25519 key fingerprint is SHA256:7dvpverJGLBDWqd8uxsLX9kWgBiZnYcXL07B+w3eIds.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes

Restore ODK Central Direct Backup File

The first time ODK Central runs, it creates the /data/transfer folder on the host server and on the central-service-1 Docker container. The purpose of the folder is to allow exchange of data between the host server and the Docker container. Any changes made to /data/transfer in the central-service-1 container will be reflected in /data/transfer on your host server, and vice versa. To restore the Direct Backup file, you require to place the file in this folder.

Log in to your ODK Central destination server if you have logged out.
Move the Direct Backup file to the /data/transfer folder using the command below. Replace {backup-filename.zip} with the actual name that you used during the backup process.
```
 mv ~/odk-backups/{backup-filename.zip} /data/transfer/
```
If you encounter a "cannot create regular file: Permission denied" error, prefix the command with sudo.
```
 sudo mv ~/odk-backups/{backup-filename.zip} /data/transfer/
```
Change directory to ODK Central installation folder by running the following commands.
```
 cd && cd central
```
Before running the command below, be aware that the command will replace all data on the destination server with the backup snapshot data. Once you are certain you want to replace the existing data, restore the backup using the following command:
```
 docker compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/{backup-filename.zip} { backup-password}
```
The command tells Docker to go into the running container named service and execute a Node.js script (restore.js) which expects a backup file (backup-filename.zip) and backup password (backup-password).

Replace backup-filename.zip with the file name you moved to the /data/transfer/ folder and {backup-password} with the password used during the backup process.

If you did not set a backup password during the backup process, press ENTER key immediately after typing the backup filename. Avoid trailing spaces after the filename backup-filename.zip.
```
 docker compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/{backup-filename.zip}
```
If the server displays "no configuration file provided: not found", you are running the command outside of ODK Central installation folder.

Wait for the backup restoration process to complete. The duration of your wait time will depend on the size of your backup file. After the restore process runs to completion, a success message including tasks that you require to revisit and verify will be displayed.

 kagunda:~$
 kagunda:~$ cd && cd central
 kagunda:~/central$ docker compose exec service node /usr/odk/lib/bin/restore.js /data/transfer/odk-bk20230913-08AM.zip bk202309
 Success. You will have to log out of the site and log back in.
     IMPORTANT: EVERYTHING has been restored to the way things were at the time of backup, including:
     * all passwords and email addresses.
     * anything deleted since the backup was made now exists again.
     * your backup settings.
     Please revisit all of these and make sure they are okay.
 '{"success":true}'
 kagunda:~/central$

Restore Enketo Redis Data and Secrets

Make sure you are stilled logged in to the destination server and you are working from the ODK Central installation folder.

Copy the main Enketo Redis database (enketo-main.rdb) to the Docker container using the following commands:

 docker stop central-enketo_redis_main-1
 docker cp ~/odk-backups/enketo-main.rdb central-enketo_redis_main-1:/data/enketo-main.rdb;
 docker start central-enketo_redis_main-1;

Example commands and output:

 kagunda:~/central$
 kagunda:~/central$ docker stop central-enketo_redis_main-1
 central-enketo_redis_main-1
 kagunda:~/central$ docker cp ~/odk-backups/enketo-main.rdb central-enketo_redis_main-1:/data/enketo-main.rdb;
 Successfully copied 39.9kB to central-enketo_redis_main-1:/data/enketo-main.rdb
 kagunda:~/central$
 kagunda:~/central$ docker start central-enketo_redis_main-1;
 central-enketo_redis_main-1
 kagunda:~/central$

Copy the Enketo Redis cache database (enketo-cache.rdb) to the Docker container:

 docker stop central-enketo_redis_cache-1
 docker cp ~/odk-backups/enketo-cache.rdb central-enketo_redis_cache-1:/data/enketo-cache.rdb;
 docker start central-enketo_redis_cache-1;

Copy the secrets folder to the central-service-1 Docker container:

 docker stop central-service-1
 docker cp ~/odk-backups/secrets central-service-1:/etc;
 docker start central-service-1;

Update Destination Server Enketo Config File.

Copy the config keys text file to the config folder in the Enketo Docker container.

 docker cp  ~/odk-backups/enketo-config-keys.txt central-enketo-1:/srv/src/enketo_express/config/

Open a Bash shell in the Enketo Docker container.
```
 docker exec -it central-enketo-1 /bin/bash
```
Install nano editor.
```
 apt-get install nano -y
```
-y option will assume "yes" as the answer to any prompts that may occur during the installation process.
Open the Enketo config file with the nano editor.
```
 nano config/config.json;
```

Prefix "encryption key", "less secure encryption key", and "api key" keys with # (or any other character). This will help you differentiate between the existing keys from the new keys when updating.

 {
     "app name": "Enketo",
     "base path": "-",
     #"encryption key": "contents of enketo-secret",
     "id length": 31,
     #"less secure encryption key": "contents of enketo-less-secret",
     "linked form and data server": {
         #"api key": "contents of enketo-api-key",
         ...
     },
     ...
 }

Move to the beginning of the file and press the ENTER key after the opening curly brace ({) to create a blank line.
Press Control + R (^R) keys. The editor will display a "File to insert [from ./]:" prompt. Type config/enketo-config-keys.txt next to the prompt and press the ENTER key to insert the contents of the file at the cursor location.
Move the cursor to the "encryption key" line of the data that has been inserted to the file.
Press Control + K (^K) keys to cut the line.
Move the cursor to the beginning of "encryption key" prefixed with a # character.
Press Control + U (^U) keys to paste.
Repeat the previous three steps to update "less secure encryption key" and "api key" values.
Remove the lines prefixed with # by moving to each line and pressing Control + K (^K).
Save the changes and exit the nano editor.
Installation of nano editor was temporarily. Therefore, uninstall the editor and exit the Enketo container using the following commands.
```
apt-get remove nano -y;
exit;
```
Restart the Enketo Docker container to effect the changes in the configuration file.
```
docker restart central-enketo-1
```
Optionally show the active status of the docker containers.
```
docker ps --format 'table {{.Names}}\t{{.State}}'
```

Change Domain Names

If the ODK Central domain names on the source and destination servers are different, you will encounter the following error when you attempt to preview restored forms on the destination server.

This form does not exist or you no longer have access to it. Please check the URL for any missing characters. If the form existed previously, it may have been archived, disabled or deleted. If this is unexpected, please contact the person who asked you to fill the form. (Attempted to access form with ID: )

To resolve the error, you must replace the old domain values with the new domain values in the Redis databases.

Create Domain Update Script Files

On the destination server, create ~/main-odk-domains.sh script file.
```
 nano ~/main-odk-domains.sh
```

Update the file with the following:

 #!/bin/sh

 OLD_DOMAIN="old-domain.com"
 NEW_DOMAIN="new-domain.com"

 keys=$(redis-cli -p 6379 KEYS  "*")

 for key in $keys
 do
   if [ "$(redis-cli -p 6379 type "$key")" = "string" ]; then
     if echo "$key" | grep -q "$OLD_DOMAIN"; then
       new_key=$(echo "$key" | sed "s/$OLD_DOMAIN/$NEW_DOMAIN/g")
       redis-cli -p 6379  RENAME "$key" "$new_key"
     fi
   elif [ "$(redis-cli -p 6379 type "$key")" = "hash" ]; then
     existingRosaServer=$(redis-cli -p 6379  HGET "$key" "openRosaServer")
     if echo "$existingRosaServer" | grep -q "$OLD_DOMAIN"; then
       newRosaServer=$(echo "$existingRosaServer" | sed "s/$OLD_DOMAIN/$NEW_DOMAIN/g")
       redis-cli -p 6379 HSET "$key" "openRosaServer" "$newRosaServer"
     fi

     if echo "$key" | grep -q "$OLD_DOMAIN"; then
       new_key=$(echo "$key" | sed "s/$OLD_DOMAIN/$NEW_DOMAIN/g")
       redis-cli -p 6379  RENAME "$key" "$new_key"
     fi
   fi
 done

The script fetches all keys from the Redis database and iterates over each key. For each key, it substitutes any occurrences of old domain values with the new domain values and renames the key in the database. Where the type of key is hash, the value of the openRosaServer field is updated to the new domain.

The sed command is used to substitute old domain values with new domain values. The g at the end of the command tells sed to replace all occurrences of OLD_DOMAIN in the key.

grep -q command searches for old domain value matches within the key and exits with either a success or failure. The -q option suppresses display of the output.

Save the file and exit the nano editor.
Make the script file executable
```
 chmod +x  ~/main-odk-domains.sh
```

Copy main-odk-domains.sh to cache-odk-domains.sh

 cp ~/main-odk-domains.sh ~/cache-odk-domains.sh

Open cache-odk-domains.sh with a text editor and replace any occurrences of redis-cli -p 6379 with redis-cli -p 6380.
Save the changes and exit the editor.

Update Enketo Redis Main Database

Copy the main-odk-domains.sh script file to the main Enketo Redis database Docker container.
```
 docker cp ~/main-odk-domains.sh central-enketo_redis_main-1:/root
```
Open a Bourne shell in the main Enketo Redis database container.
```
 docker exec -it central-enketo_redis_main-1 sh
```
Run the script file
```
 /root/main-odk-domains.sh
```
Optionally, confirm the update has been successful
```
 redis-cli -p 6379 KEYS "*NEW_DOMAIN*"
```
Exit the Docker container
```
 exit
```

Update Enketo Redis Cache Database

Copy the cache-odk-domains.sh script file to the cache Enketo Redis database Docker container.
```
  docker cp ~/cache-odk-domains.sh central-enketo_redis_cache-1:/root
```
Open a Bourne shell in the cache Enketo Redis database container
```
 docker exec -it central-enketo_redis_cache-1 sh
```
Run the script file
```
 /root/cache-odk-domains.sh
```
Exit the Docker container
```
 exit
```

Conclusion

This tutorial has shown you how to backup and restore ODK Central data. You have learned how to create a direct backup of your ODK Central database, copy the backup file and other essential data to the destination server, and restore the backup file on the new server. You have also learned how to update the domain names in the Enketo Redis databases and config files to avoid errors when previewing forms on the new server.

If you found this article useful, please share it with your friends and colleagues who might benefit from it.

Thank you for reading!