11.1. Installing and running the CamCOPS server via Docker

11.1.1. Overview

Docker is a cross-platform system for running applications in “containers”. A computer (or computing cluster) can run lots of containers. They allow applications to be set up in standardized and isolated enviroments, which include their own operating system). The containers then talk to each other, and to their “host” computer, to do useful things.

The core of Docker is called Docker Engine. The Docker Compose tool allows multiple containers to be created, started, and connected together automatically.

CamCOPS provides a Docker setup to make installation easy. This uses Docker Compose to set up several containers, specifically:

• a database system, via MySQL on Linux (internal container name mysql);

• a message queue, via RabbitMQ on Linux (rabbitmq);

• the CamCOPS web server itself, offering SSL directly via CherryPy on Linux (camcops_server);

• the CamCOPS scheduler (camcops_scheduler);

• CamCOPS workers, to perform background tasks (camcops_workers);

• a background task monitor, using Flower (camcops_monitor).

11.1.2. Quick start

1. Ensure you have Docker and Docker Compose installed (see prerequisites).

2. Obtain the CamCOPS source code.

   Todo

   Docker/CamCOPS source: (a) is that the right method? Or should we be using docker-app? (Is that experimental?) (b) Document.

   Todo

   MB 2020-08-19: I wonder if we should follow the approach here for releasing docker images when we create a new release on GitHub – https://docs.github.com/en/actions/language-and-framework-guides/publishing-docker-images – and publish the image on Docker Hub.

3. Set the environment variables required for Docker operation. (You probably want to automate this with a script.)

4. Change to the server/docker/linux directory within the CamCOPS source tree.

   Note

   If you are using a Windows host, change to server/docker/windows instead, and for all the commands below, instead of ./some_command, run some_command.bat.

5. Start the containers with:

   ./start_camcops_docker_interactive
   

   This gives you an interactive view. As this is the first run, it will also create containers, volumes, the database, and so on. It will then encounter errors (e.g. config file not specified properly, or the database doesn’t have the right structure), and will stop.

6. Run this command to create a demonstration config file with the standard name:

   ./print_demo_camcops_config > "${CAMCOPS_DOCKER_CONFIG_HOST_DIR}/camcops.conf"
   

7. Edit that config file. See here for a full description and here for special Docker requirements.

8. Create the database structure (tables):

   ./upgrade_db
   

9. Create a superuser:

   ./camcops_server make_superuser
   

10. Time to test! Restart with

    ./start_camcops_docker_interactive
    

    Everything should now be operational. Using any web browser, you should be able to browse to the CamCOPS site at your chosen host port and protocol, and log in using the account you have just created.

11. When you’re satisfied everything is working well, you can stop interactive mode (CTRL-C) and instead use

    ./start_camcops_docker_detached
    

    which will fire up the containers in the background. To take them down again, use

    ./stop_camcops_docker
    

You should now be operational! If Docker is running as a service on your machine, CamCOPS should also be automatically restarted by Docker on reboot.

11.1.3. Prerequisites

You can run Docker on several operating systems. For example, you can run Docker under Linux (and CamCOPS will run in Linux-under-Docker-under-Linux). You can similarly run Docker under Windows (and CamCOPS will run in Linux-under-Docker-under-Windows).

• You need Docker Engine installed. See https://docs.docker.com/engine/install/.

• You need Docker Compose installed. See https://docs.docker.com/compose/install/.

11.1.4. Environment variables

Docker control files are in the server/docker directory of the CamCOPS source tree. Setup is controlled by the docker-compose application.

Note

Default values are taken from server/docker/.env. Unfortunately, this name is fixed by Docker Compose, and this file is hidden under Linux (as are any files starting with .).

11.1.4.1. CAMCOPS_DOCKER_CONFIG_HOST_DIR

No default. Must be set.

Path to a directory on the host that contains key configuration files. Don’t use a trailing slash.

In this directory, there should be a file called camcops.conf, the config file (or, if you have set CAMCOPS_DOCKER_CAMCOPS_CONFIG_FILENAME, that filename!).

Note

Under Windows, don’t use Windows paths like C:\Users\myuser\my_camcops_dir. Translate this to Docker notation as /host_mnt/c/Users/myuser/my_camcops_dir. As of 2020-07-21, this doesn’t seem easy to find in the Docker docs!

11.1.4.2. CAMCOPS_DOCKER_CAMCOPS_CONFIG_FILENAME

Default: camcops.conf

Base name of the CamCOPS config file (see CAMCOPS_DOCKER_CONFIG_HOST_DIR).

11.1.4.3. CAMCOPS_DOCKER_FLOWER_HOST_PORT

Default: 5555

Host port on which to launch the Flower monitor.

11.1.4.4. CAMCOPS_DOCKER_CAMCOPS_HOST_PORT

Default: 443

The TCP/IP port number on the host computer that CamCOPS should provide an HTTP or HTTPS (SSL) connection on.

It is strongly recommended that you run CamCOPS over HTTPS. The two ways of doing this are:

• Have CamCOPS run plain HTTP, and connect it to another web server (e.g. Apache) that provides the HTTPS component.

  • If you do this, you should not expose this port to the “world”, since it offers insecure HTTP.

  • The motivation for this method is usually that you are running multiple web services, of which CamCOPS is one.

  • We don’t provide Apache within Docker, because the Apache-inside-Docker would only see CamCOPS, so there’s not much point – you might as well use the next option…

• Have CamCOPS run HTTPS directly, by specifying the SSL_CERTIFICATE and SSL_PRIVATE_KEY options.

  • This is simpler if CamCOPS is the only web service you are running on this machine. Use the standard HTTPS port, 443, and expose it to the outside through your server’s firewall. (You are running a firewall, right?)

11.1.4.5. CAMCOPS_DOCKER_CAMCOPS_INTERNAL_PORT

Default: 8000

The TCP/IP port number used by CamCOPS internally. Must match the PORT option in the CamCOPS config file.

11.1.4.6. CAMCOPS_DOCKER_MYSQL_CAMCOPS_DATABASE_NAME

Default: camcops

Name of the MySQL database to be used for CamCOPS data.

11.1.4.7. CAMCOPS_DOCKER_MYSQL_CAMCOPS_USER_PASSWORD

No default. Must be set during MySQL container creation.

MySQL password for the CamCOPS database user (whose name is set by CAMCOPS_DOCKER_MYSQL_CAMCOPS_USER_NAME).

Note

This only needs to be set when Docker Compose is creating the MySQL container for the first time. After that, it doesn’t have to be set (and is probably best not set for security reasons!).

11.1.4.8. CAMCOPS_DOCKER_MYSQL_CAMCOPS_USER_NAME

Default: camcops

MySQL username for the main CamCOPS user. This user is given full control over the database named in CAMCOPS_DOCKER_MYSQL_CAMCOPS_DATABASE_NAME. See also CAMCOPS_DOCKER_MYSQL_CAMCOPS_USER_PASSWORD.

11.1.4.9. CAMCOPS_DOCKER_MYSQL_HOST_PORT

Default: 3306

Port published to the host, giving access to the CamCOPS MySQL installation. You can use this to allow other software to connect to the CamCOPS database directly.

This might include using MySQL tools from the host to perform database backups (though Docker volumes can also be backed up in their own right).

The default MySQL port is 3306. If you run MySQL on your host computer for other reasons, this port will be taken, and you should change it to something else.

You should not expose this port to the “outside”, beyond your host.

11.1.4.10. CAMCOPS_DOCKER_MYSQL_ROOT_PASSWORD

No default. Must be set during MySQL container creation.

MySQL password for the root user.

Note

This only needs to be set when Docker Compose is creating the MySQL container for the first time. After that, it doesn’t have to be set (and is probably best not set for security reasons!).

11.1.4.11. COMPOSE_PROJECT_NAME

Default: camcops

This is the Docker Compose project name. It’s used as a prefix for all the containers in this project.

11.1.5. The CamCOPS configuration file for Docker

The CamCOPS configuration file is described here. There are a few special things to note within the Docker environment.

• CELERY_BROKER_URL. The RabbitMQ (AMQP server) lives in a container named (internally) rabbitmq and uses the default AMQP port of 5672. The CELERY_BROKER_URL variable should therefore be set exactly as follows:

  CELERY_BROKER_URL = amqp://rabbitmq:5672/
                      ^      ^        ^
                      |      |        |
                      |      |        +- port number
                      |      +- internal name of container running RabbitMQ
                      +- "use AMQP protocol"
  

• DB_URL. MySQL runs in a container called (internally) mysql and the mysqlclient drivers for Python are installed for CamCOPS. (These use C-based MySQL drivers for speed). The DB_URL variable should therefore be of the form:

  DB_URL = mysql+mysqldb://camcops:ZZZ_PASSWORD_REPLACE_ME@mysql:3306/camcops?charset=utf8
           ^     ^         ^       ^                       ^     ^    ^      ^
           |     |         |       |                       |     |    |      |
           |     |         |       |                       |     |    |      +- charset options; don't alter
           |     |         |       |                       |     |    +- database name; should match
           |     |         |       |                       |     |       CAMCOPS_DOCKER_MYSQL_CAMCOPS_DATABASE_NAME
           |     |         |       |                       |     +- port; don't alter
           |     |         |       |                       +- container name; don't alter
           |     |         |       +- MySQL password; should match CAMCOPS_DOCKER_MYSQL_CAMCOPS_USER_PASSWORD
           |     |         +- MySQL username; should match CAMCOPS_DOCKER_MYSQL_CAMCOPS_USER_NAME
           |     +- "use mysqldb [mysqlclient] Python driver"
           +- "use MySQL dialect"
  

  It remains possible to point “CamCOPS inside Docker” to “MySQL outside Docker” (rather than the instance of MySQL supplied with CamCOPS via Docker). This would be unusual, but it’s up to you.

• HOST. This should be 0.0.0.0 for operation within Docker 1.

• References to files on disk. CamCOPS mounts a configuration directory from host computer, specified via CAMCOPS_DOCKER_CONFIG_HOST_DIR. From the perspective of the CamCOPS Docker containers, this directory is mounted at /camcops/cfg.

  Accordingly, all user-supplied configuration files should be placed within this directory, and referred to via /camcops/cfg. System-supplied files are also permitted within /camcops/venv (and the demonstration config file will set this up for you).

  For example:

  Host computer:
  
      /etc
          /camcops
              extra_strings/
                  phq9.xml
                  ...
              camcops.conf
              ssl_camcops.cert
              ssl_camcops.key
  
  Environment variables for Docker:
  
      CAMCOPS_DOCKER_CAMCOPS_CONFIG_FILENAME=camcops.conf
      CAMCOPS_DOCKER_CAMCOPS_HOST_PORT=443
      CAMCOPS_DOCKER_CAMCOPS_INTERNAL_PORT=8000
      CAMCOPS_DOCKER_CONFIG_HOST_DIR=/etc/camcops
  
  CamCOPS config file:
  
      [site]
  
      # ...
  
      EXTRA_STRING_FILES =
          /camcops/venv/lib/python3.6/site-packages/camcops_server/extra_strings/*.xml
          /camcops/cfg/extra_strings/*.xml
  
      # ...
  
      [server]
  
      HOST = 0.0.0.0
      PORT = 8000
      SSL_CERTIFICATE = /camcops/cfg/ssl_camcops.cert
      SSL_PRIVATE_KEY = /camcops/cfg/ssl_camcops.key
  
      # ...
  

  CamCOPS will warn you if you are using Docker but your file references are not within the /camcops/cfg mount point.

11.1.6. Using a database outside the Docker environment

CamCOPS creates a MySQL system and database inside Docker, for convenience. However, it’s completely fine to ignore it and point CamCOPS to a database elsewhere on your system. Just set the DB_URL parameter to point where you want.

11.1.7. Tools

All live in the server/docker directory.

11.1.7.1. bash_within_docker

Starts a container with the CamCOPS image and runs a Bash shell within it.

Warning

Running a shell within a container allows you to break things! Be careful.

11.1.7.2. camcops_server

This script runs the camcops_server command within a Docker container. For example:

./camcops_server --help

11.1.7.3. print_demo_camcops_config

Prints a demonstration CamCOPS config file with Docker options set. Save the output as demonstrated above.

11.1.7.4. start_camcops_docker_detached

Shortcut for docker-compose up -d. The -d switch is short for --detach (or daemon mode).

11.1.7.5. start_camcops_docker_interactive

Shortcut for docker-compose up --abort-on-container-exit.

Note

The docker-compose command looks for a Docker Compose configuration file with a default filename; one called docker-compose.yaml is provided.

11.1.7.6. stop_camcops_docker

Shortcut for docker-compose down.

11.1.7.7. upgrade_db

This script upgrades the CamCOPS database to the current version.

• The database is specified by the DB_URL parameter in the CamCOPS config file. See above.

• The config file is found by Docker according to the CAMCOPS_DOCKER_CONFIG_HOST_DIR and CAMCOPS_DOCKER_CAMCOPS_CONFIG_FILENAME environment variables (q.v.).

11.1.7.8. within_docker

This script starts a container with the CamCOPS server image, activates the CamCOPS virtual environment, and runs a command within it. For example, to explore this container, you can do

./within_docker /bin/bash

… which is equivalent to the bash_within_docker script (see above and note the warning).

11.1.8. Troubleshooting

11.1.8.1. Can’t start Docker containers on a Linux host

If you get an error like:

ERROR: Couldn't connect to Docker daemon at http+docker://localunixsocket - is it running?

then check:

1. Is Docker running (ps aux | grep dockerd or a service command, such as service docker status under Ubuntu)? If not, start its service (e.g. under Ubuntu, sudo service docker start).

2. Is your user in the Docker group (grep docker /etc/group)? If not, add your user, then log out and log in again for the changes to be picked up.

11.1.8.2. Explore a running Docker container

The shortcuts above (e.g. bash_within_docker) start a new container (via docker-compose run). To explore a container that is already running, find the container ID via docker container ls and use docker exec, e.g. as docker exec -it CONTAINER /bin/bash.

11.1.8.3. Warnings from Celery under Docker

This warning:

camcops_workers_1    | /camcops/venv/lib/python3.6/site-packages/celery/platforms.py:801: RuntimeWarning: You're running the worker with superuser privileges: this is
camcops_workers_1    | absolutely not recommended!
camcops_workers_1    |
camcops_workers_1    | Please specify a different user using the --uid option.
camcops_workers_1    |
camcops_workers_1    | User information: uid=0 euid=0 gid=0 egid=0
camcops_workers_1    |
camcops_workers_1    |   uid=uid, euid=euid, gid=gid, egid=egid,

… can be ignored.

Todo

Make container apps run as non-root? See https://medium.com/redbubble/running-a-docker-container-as-a-non-root-user-7d2e00f8ee15.

11.1.9. Development notes

• Config information. There are several ways, but mounting a host directory containing a config file is perfectly reasonable. See https://dantehranian.wordpress.com/2015/03/25/how-should-i-get-application-configuration-into-my-docker-containers/.

• Secrets, such as passwords. This is a little tricky. Environment variables and config files are both reasonable options; see e.g. https://stackoverflow.com/questions/22651647/docker-and-securing-passwords. Environment variables are visible externally (e.g. docker exec CONTAINER env) but you have to have Docker privileges (be in the docker group) to do that. Docker “secrets” require Docker Swarm (not just plain Docker Compose). We are using a config file for CamCOPS, and environment variables for the MySQL container.

• Data storage. Should data (e.g. MySQL databases) be stored on the host (via a “bind mount” of a directory), or in Docker volumes? Docker says clearly: volumes. See https://docs.docker.com/storage/volumes/.

• TCP versus UDS. Currently the connection between CamCOPS and MySQL is via TCP/IP. It would be possible to use Unix domain sockets instead. This would be a bit trickier. Ordinarily, it would bring some speed advantages; I’m not sure if that remains the case between Docker containers. The method is to mount a host directory; see https://superuser.com/questions/1411402/how-to-expose-linux-socket-file-from-docker-container-mysql-mariadb-etc-to. It would add complexity. The other advantage of using TCP is that we can expose the MySQL port to the host for administrative use.

• Database creation. It might be nice to upgrade the database a little more automatically, but this is certainly not part of Docker image creation (the image is static and the data is dynamic) and shouldn’t be part of routine container startup, so perhaps it’s as good as is reasonable.

• Scaling up. At present we use a fixed number of containers, some with several processes running within. There are other load distribution mechanisms possible with Docker Compose.

---

Footnotes

1

https://nickjanetakis.com/blog/docker-tip-54-fixing-connection-reset-by-peer-or-similar-errors