13.8. Development command-line tools

If you are developing tasks for CamCOPS, be aware of these additional development tools.

In tablet_qt/tools:

13.8.1. build_qt.py

This program runs on a variety of platforms (including Linux, Windows, macOS) and has the surprisingly tricky job of building the following libraries, from source:

for the following platforms, using a variety of CPUs:

  • Android (32-bit ARM, 64-bit x86 emulator) (compile under Linux)
  • iOS (32-bit ARM, 64-bit ARM, 64-bit x86 simulator) (compile under macOS)
  • Linux (64-bit x86)
  • macOS (64-bit x86)
  • Windows (32-bit x86, 64-bit x86)

It will fetch source code and do all the work. Once built, you should have a Qt environment that you can point Qt Creator to, and you should be able to compile CamCOPS with little extra work. (Though probably not none.)

The --build_all option is generally a good one; this builds for all architectures supported on the host you’re using.

Here’s its help:

usage: build_qt.py [-h] [--show_config_only] [--root_dir ROOT_DIR]
                   [--nparallel NPARALLEL] [--force] [--tee TEE]
                   [--verbose {0,1,2}] [--inherit_os_env]
                   [--no_inherit_os_env] [--build_all]
                   [--build_android_x86_32] [--build_android_arm_v7_32]
                   [--build_android_arm_v8_64] [--build_linux_x86_64]
                   [--build_macos_x86_64] [--build_windows_x86_64]
                   [--build_windows_x86_32] [--build_ios_arm_v7_32]
                   [--build_ios_arm_v8_64] [--build_ios_simulator_x86_32]
                   [--build_ios_simulator_x86_64]
                   [--qt_build_type {debug,release,release_w_symbols}]
                   [--qt_src_dirname QT_SRC_DIRNAME] [--qt_openssl_static]
                   [--qt_openssl_linked] [--android_sdk_root ANDROID_SDK_ROOT]
                   [--android_ndk_root ANDROID_NDK_ROOT]
                   [--android_ndk_host ANDROID_NDK_HOST]
                   [--android_toolchain_version ANDROID_TOOLCHAIN_VERSION]
                   [--java_home JAVA_HOME] [--ios_sdk IOS_SDK]
                   [--jom_executable JOM_EXECUTABLE]

Build Qt and other libraries for CamCOPS

optional arguments:
  -h, --help            show this help message and exit

General:
  General options

  --show_config_only    Show config, then quit (default: False)
  --root_dir ROOT_DIR   Root directory for source and builds (default taken
                        from environment variable CAMCOPS_QT_BASE_DIR if
                        present) (default: /home/rudolf/dev/qt_local_build)
  --nparallel NPARALLEL
                        Number of parallel processes to run (default: 4)
  --force               Force build (default: False)
  --tee TEE             Copy stdout/stderr to this named file (default: None)
  --verbose {0,1,2}, -v {0,1,2}
                        Verbosity level (default: 0)
  --inherit_os_env      Inherit the parent OS environment variables (default:
                        False)
  --no_inherit_os_env   Do not inherit the parent OS environment variables
                        (default: False)

Architecture:
  Choose architecture for which to build

  --build_all           Build for all architectures supported on this host
                        (this host is: Linux/Intel x86 (64-bit)) (default:
                        False)
  --build_android_x86_32
                        An architecture target (Android under an Intel x86
                        32-bit emulator) (default: False)
  --build_android_arm_v7_32
                        An architecture target (Android with a 32-bit ARM
                        processor) (default: False)
  --build_android_arm_v8_64
                        An architecture target (Android with a 64-bit ARM
                        processor) (default: False)
  --build_linux_x86_64  An architecture target (native Linux with a 64-bit
                        Intel/AMD CPU; check with 'lscpu' and 'uname -a'
                        (default: False)
  --build_macos_x86_64  An architecture target (macOS under an Intel 64-bit
                        CPU; check with 'sysctl -a|grep cpu', and see
                        https://support.apple.com/en-gb/HT201948 ) (default:
                        False)
  --build_windows_x86_64
                        An architecture target (Windows with an Intel/AMD
                        64-bit CPU) (default: False)
  --build_windows_x86_32
                        An architecture target (Windows with an Intel/AMD
                        32-bit CPU) (default: False)
  --build_ios_arm_v7_32
                        An architecture target (iOS with a 32-bit ARM
                        processor) (default: False)
  --build_ios_arm_v8_64
                        An architecture target (iOS with a 64-bit ARM
                        processor) (default: False)
  --build_ios_simulator_x86_32
                        An architecture target (iOS with an Intel 32-bit CPU,
                        for the iOS simulator) (default: False)
  --build_ios_simulator_x86_64
                        An architecture target (iOS with an Intel 64-bit CPU,
                        for the iOS simulator) (default: False)

Qt:
  Qt options [Qt must be built from source for SQLite support, and also if
  static OpenSSL linkage is desired; note that static OpenSSL linkage
  requires a Qt rebuild (slow!) if you rebuild OpenSSL]

  --qt_build_type {debug,release,release_w_symbols}
                        Qt build type (release = small and quick) (default:
                        release)
  --qt_src_dirname QT_SRC_DIRNAME
                        Qt source directory (default: qt5)
  --qt_openssl_static   Link OpenSSL statically (ONLY if Qt is statically
                        linked) [True=static, False=dynamic] (default: True)
  --qt_openssl_linked   Link OpenSSL dynamically [True=static, False=dynamic]
                        (default: True)

Android:
  Android options (NB you must install the Android SDK and NDK separately,
  BEFOREHAND)

  --android_sdk_root ANDROID_SDK_ROOT
                        Android SDK root directory (default:
                        /home/rudolf/dev/android-sdk-linux)
  --android_ndk_root ANDROID_NDK_ROOT
                        Android NDK root directory (default:
                        /home/rudolf/dev/android-ndk-r20)
  --android_ndk_host ANDROID_NDK_HOST
                        Android NDK host architecture (default: linux-x86_64)
  --android_toolchain_version ANDROID_TOOLCHAIN_VERSION
                        Android toolchain version (default: 4.9)
  --java_home JAVA_HOME
                        JAVA_HOME directory (default:
                        /usr/lib/jvm/java-11-openjdk-amd64)

iOS:
  iOS options

  --ios_sdk IOS_SDK     iOS SDK to use (leave blank for system default)
                        (default: )

jom:
  'jom' parallel make tool for Windows

  --jom_executable JOM_EXECUTABLE
                        jom executable (typically installed with QtCreator)
                        (default: C:\Qt\Tools\QtCreator\bin\jom.exe)

# Generated at 2019-09-20 11:43:43

13.8.2. chord.py

This generates musical chords as WAV files. It’s not very generic but it generates specific sounds used by the CamCOPS client.

13.8.3. decrypt_sqlcipher.py

This tool requires an installed copy of SQLCipher. It creates a decrypted SQLite database from an encrypted SQLCipher database, given the password.

Here’s its help:

usage: decrypt_sqlcipher.py [-h] [--password PASSWORD] [--sqlcipher SQLCIPHER]
                            [--cipher_compatibility CIPHER_COMPATIBILITY]
                            [--encoding ENCODING]
                            encrypted decrypted

Use SQLCipher to make a decrypted copy of a database

positional arguments:
  encrypted             Filename of the existing encrypted database
  decrypted             Filename of the decrypted database to be created

optional arguments:
  -h, --help            show this help message and exit
  --password PASSWORD   Password (if blank, environment variable
                        DECRYPT_SQLCIPHER_PASSWORD will be used, or you will
                        be prompted) (default: None)
  --sqlcipher SQLCIPHER
                        SQLCipher executable file (if blank, environment
                        variable SQLCIPHER will be used, or the default of
                        'sqlcipher') (default: None)
  --cipher_compatibility CIPHER_COMPATIBILITY
                        Use compatibility settings for this major version of
                        SQLCipher (e.g. 3) (default: None)
  --encoding ENCODING   Encoding to use (default: utf-8)

# Generated at 2019-09-20 11:43:44

13.8.4. encrypt_sqlcipher.py

This tool requires an installed copy of SQLCipher. It creates an encrypted SQLCipher database from a plain SQLite database.

Here’s its help:

usage: encrypt_sqlcipher.py [-h] [--password PASSWORD] [--sqlcipher SQLCIPHER]
                            [--encoding ENCODING]
                            plaintext encrypted

Use SQLCipher to make an encrypted copy of a database

positional arguments:
  plaintext             Filename of the existing plain-text (decrypted)
                        database
  encrypted             Filename of the encrypted database to be created

optional arguments:
  -h, --help            show this help message and exit
  --password PASSWORD   Password (if blank, environment variable
                        ENCRYPT_SQLCIPHER_PASSWORD will be used, or you will
                        be prompted) (default: None)
  --sqlcipher SQLCIPHER
                        SQLCipher executable file (if blank, environment
                        variable SQLCIPHER will be used, or the default of
                        'sqlcipher') (default: None)
  --encoding ENCODING   Encoding to use (default: utf-8)

# Generated at 2019-09-20 11:43:44

13.8.5. open_sqlcipher.py

This tool requires an installed copy of SQLCipher. It opens an encrypted SQLCipher database via the SQLite command line tool, given the password. You can use this to view/edit CamCOPS databases.

Here’s its help:

usage: open_sqlcipher.py [-h] [--password PASSWORD] [--sqlcipher SQLCIPHER]
                         [--cipher_compatibility CIPHER_COMPATIBILITY]
                         [--cipher_migrate] [--encoding ENCODING]
                         encrypted

Open an encrypted database at the SQLCipher command line

positional arguments:
  encrypted             Filename of the existing encrypted database

optional arguments:
  -h, --help            show this help message and exit
  --password PASSWORD   Password (if blank, environment variable
                        DECRYPT_SQLCIPHER_PASSWORD will be used, or you will
                        be prompted) (default: None)
  --sqlcipher SQLCIPHER
                        SQLCipher executable file (if blank, environment
                        variable SQLCIPHER will be used, or the default of
                        'sqlcipher') (default: None)
  --cipher_compatibility CIPHER_COMPATIBILITY
                        Use compatibility settings for this major version of
                        SQLCipher (e.g. 3) (default: None)
  --cipher_migrate      Migrate the database to the latest version of
                        SQLCipher (default: False)
  --encoding ENCODING   Encoding to use (default: utf-8)

# Generated at 2019-09-20 11:43:44

In server/tools:

13.8.6. build_translations.py

Builds translation files for the server. See Internationalization.

13.8.7. create_database_migration.py

Creates a new database migration for the server, in server/camcops_server/alembic/versions/. Here’s its help:

usage: create_database_migration.py [-h] [--verbose] message

Create database revision. Note:

- The config used will be that from the environment variable
  CAMCOPS_CONFIG_FILE (currently '/home/rudolf/Documents/code/camcops/working_l
  ocal_secret/camcops_py3_test.ini').

- Alembic compares (a) the current state of the DATABASE to (b) the state of
  the SQLAlchemy metadata (i.e. the CODE). It creates a migration file to
  change the database to match the code.

- Accordingly, in the rare event of wanting to do a fresh start, you need an
  *empty* database.

- More commonly, you want a database that is synced to a specific Alembic
  version (with the correct structure, and the correct version in the
  alembic_version table). If you have made manual changes, such that the actual
  database structure doesn't match the structure that Alembic expects based on
  that version, there's likely to be trouble.

positional arguments:
  message     Revision message

optional arguments:
  -h, --help  show this help message and exit
  --verbose   Be verbose

# Generated at 2019-09-20 11:43:44

13.8.8. make_xml_skeleton.py

Takes a “secret” XML file (one containing task text that is restricted to a particular site) and makes a generic “skeleton” XML file – the same but with strings replaced by placeholder text – so that others can see the structure required if they too have the permissions to create the full file.

Writes to stdout (so redirect it to save to a file). Here’s its help:

usage: Create an XML string file skeleton, with placeholder text, from a real but secret string file. Writes to stdout.
       [-h] [--replacement REPLACEMENT] filename

positional arguments:
  filename              Input filename

optional arguments:
  -h, --help            show this help message and exit
  --replacement REPLACEMENT
                        Replace string contents with this (default: XXX)

# Generated at 2019-09-20 11:43:44