13.11. Internationalization

13.11.1. String locations in CamCOPS

  • Task-specific XML files.

    These are loaded by the server (see EXTRA_STRING_FILES) and downloaded by the client. They contain string versions in different languages.

    Client calls look like xstring("stringname"), or xstring(STRINGNAME).

    Server calls look like req.xstring("stringname") or req.wxstring("stringname"), etc. The w prefix is for functions that “web-safe” their output (escaping HTML special characters).

  • Client/server shared strings.

    Some strings are shared between the client and the server, but are not specific to a given task. They are in the special camcops.xml string file, loaded as above.

    Client calls typically look like appstring(appstrings::STRINGNAME); the strings are named by constants listed in appstrings.h.

    Server calls typically look like req.wappstring(AS.STRINGNAME) where AS is defined in camcops_server.cc_modules.cc_string.

    If a string is “mission critical” for the client, then it should be built into the client core instead (as below).

  • Client core.

    Some text does not require translation (see below).

    Text visible to the user should be within a Qt tr("some text") call. Qt provides tools to collect these strings into a translation file, translate them via Qt Linguist, and distribute them – see below.

    Client strings that are used only once can live in the source code where they are used.

    Client strings that are used in several places should appear in textconst.h.

  • Server code.

    Some text does not require translation (see below).

    Text visible to the user should look like _("some text"). The use of _() is standard notation that is picked up by internationalization software that scans the source code. The _ function is aliased to an appropriate translation function, usually via _ = req.gettext. A per-request system is used so that different users of the web site can operate simultaneously in different languages.

    Where text is re-used, it is placed in cc_text.py. This is in general preferable because it allows better automatic translation mechanisms than the XML system.

13.11.2. CamCOPS language rules

Hard-coded English is OK for the following:

  • code
  • Qt debugging stream
  • command-line text
  • debugging tests
  • task short names (typically standardized abbreviations)
  • database structure (e.g. table names, field names)
  • config file settings, and things that refer to them

Everything else should be translatable.

13.11.3. Overview of the Qt translation system

  • Begin by editing the Qt project file to add a TRANSLATIONS entry.

    Qt suggests looking up languages via QLocale::system::name() (see https://doc.qt.io/qt-5/internationalization.html). This (https://doc.qt.io/qt-5/qlocale.html#name) returns a string of the form language_country where language is a lower-case two-letter ISO-639 language code (i.e. ISO-639-1), and country is an uppercase, “two- or three-letter” ISO 3166 country code. Thus, for example: en_GB (English, United Kingdom of Great Britain and Northern Ireland); probably da_DK (Danish, Denmark).

    See

    Though the more common format appears to be language-country, Qt Linguist prefers and auto-recognizes the underscore version.

    Moreover, the underscore is the standard format for locale (rather than language tags); the underscore is also used by POSIX and Python

    We will use the underscore notation throughout.

    For example, to add Danish:

    TRANSLATIONS = translations/camcops_da_DK.ts
    

    These files live in the source tree, not the resource file.

  • Run qmake.

  • lupdate scans a Qt project and updates a .ts translation file. In Qt Creator, run Tools ‣ External ‣ Linguist ‣ Update Translations (lupdate)

    • .ts translation files are XML.

    • Pay close attention to error messages from lupdate. In particular, static methods are fine, but classes implementing tr() must both (a) inherit from QObject and (b) use the Q_OBJECT macro.

    • However, there is a Qt bug as of 2019-05-03: C++11 raw strings generate the error “Unterminated C++ string”; https://bugreports.qt.io/browse/QTBUG-42736; fixed in Qt 5.12.2.

    • To delete obsolete strings, use the -no-obsolete option; e.g.

      ~/dev/qt_local_build/qt_linux_x86_64_install/bin/lupdate -no-obsolete camcops.pro
      
  • The Qt Linguist tool edits .ts files.

    • Class-related strings appear helpfully in against their class.

    • Free-standing strings created with QObject::tr() appear against QObject.

    • However, lupdate looks like it reads the C++ in a fairly superficial way; specifically, it will NOT find strings defined this way:

      #define TR(stringname, text) const QString stringname(QObject::tr(text))
      TR(SOMESTRING, "some string contents");
      

      but will find this semantically equivalent version:

      const QString SOMESTRING(QObject::tr("some string contents"));
      

      See also https://doc.qt.io/archives/qq/qq03-swedish-chef.html.

13.11.4. Implementing translatable strings in C++

  • Note also in particular that for obvious initialization order reasons, QObject::tr() doesn’t help with static variables. See e.g. https://stackoverflow.com/questions/3493540/qt-tr-in-static-variable. So, options include:

    • if used in one place in the code, just use tr() from within a Q_OBJECT class, e.g.

      void doSomething()
      {
          // ...
          alert(tr("You need to draw on the banana!"));
      }
      
    • if used multiple in one function, e.g.

      void doSomething()
      {
          const QString mystring(tr("Configure ExpDetThreshold task"));
          // ...
      }
      
    • if used repeatedly from different places, consider a static member function, e.g.

      // something.h
      
      class Something
      {
          Q_OBJECT
          // ...
      private:
          static QString txtAuditory();
      }
      
      
      // something.cpp
      
      QString Something::txtAuditory()
      {
          return tr("Auditory");
      }
      

      … which appears in the right class in Qt Linguist.

  • You are likely to need to re-run qmake before lupdate (or, for example, it can fail to pick up on namespaces).

13.11.5. Setting up the translation system

See code.

13.11.6. Making the binary translation files

  • Run lrelease, e.g. from within Qt Creator as Tools ‣ External ‣ Linguist ‣ Release Translations (lrelease). This converts .ts files to .qm files.
  • You need to add the .qm files to your resources.
  • As always, the :/ prefix in a filename, or qrc:/// for a URL, points to the resources.

13.11.7. Tasks: xstrings

  • Language-specific xstring fetch mechanism implemented.
  • When the client asks for server strings, all languages are sent.

13.11.8. Determining the language

  • Client: the user chooses the language.
  • Server: (a) there is a default server language, which also applies when the user is not logged in; (b) users can choose a language.

13.11.9. Server core

We manage this as follows.

Mako

<% _ = request.gettext %>
## TRANSLATOR: string context described here
<p>${_("Please translate me")}</p>

It would be nice if we could just put the _ definition in base.mako, but that doesn’t come through with <%inherit file=.../>. But we can add it to the system context via camcops_server.cc_pyramid.CamcopsMakoLookupTemplateRenderer. So we do.

Generic Python

_ = request.gettext
# TRANSLATOR: string context described here
mytext = _("Please translate me")

If an appropriate comment tag is used, either in Python or Mako (here, TRANSLATOR:, as defined in build_translations.py), the comment appears in the translation files.

Forms

See cc_forms.py; the forms need to be request-aware. This is quite fiddly.

Efficiency

Try e.g.

inotifywait --monitor server/camcops_server/translations/da_DK/LC_MESSAGES/camcops.mo

… from a single-process CherryPy instance (camcops_server .serve_cherrypy), there’s a single read call only.

13.11.10. Updating server strings

There is a CamCOPS development tool, build_translations.py. Its help is as follows:

usage: build_translations.py [-h] [--verbose] operation

Create translation files for CamCOPS server. CamCOPS knows about the following
locales:

    ['da_DK']

Operations:

    extract
        Extract messages from code that looks like, for example,
            _("please translate me"),
        in Python and Mako files. Write it to this .pot file:
            /home/rudolf/Documents/code/camcops/server/camcops_server/translations/camcops_translations.pot
        
    init_missing
        For any locales that do not have a .po file, create one.

    update
        Updates all .po files from the .pot file.
        
    compile
        Converts each .po file to an equivalent .mo file.
    
    all
        Executes all other operations in sequence.

positional arguments:
  operation   Operation to perform; possibilities are ['extract',
              'init_missing', 'update', 'compile', 'all']

optional arguments:
  -h, --help  show this help message and exit
  --verbose   Be verbose (default: False)

# Generated at 2019-08-08 20:31:44

Todo

There are still some of the more complex Deform widgets that aren’t properly translated on a per-request basis, such as

  • TranslatableOptionalPendulumNode
  • TranslatableDateTimeSelectorNode
  • CheckedPasswordWidget