14.12. Internationalization¶
14.12.1. String locations in CamCOPS¶
Task-specific XML files.
These are loaded by the server (see EXTRA_STRING_FILES) and downloaded by the client. They contain string versions in different languages.
Client calls look like
xstring("stringname")
, orxstring(STRINGNAME)
.Server calls look like
req.xstring("stringname")
orreq.wxstring("stringname")
, etc. Thew
prefix is for functions that “web-safe” their output (escaping HTML special characters).Client/server shared strings.
Some strings are shared between the client and the server, but are not specific to a given task. They are in the special
camcops.xml
string file, loaded as above.Client calls typically look like
appstring(appstrings::STRINGNAME)
; the strings are named by constants listed inappstrings.h
.Server calls typically look like
req.wappstring(AS.STRINGNAME)
whereAS
is defined incamcops_server.cc_modules.cc_string
.If a string is “mission critical” for the client, then it should be built into the client core instead (as below).
Client core.
Some text does not require translation (see below).
Text visible to the user should be within a Qt
tr("some text")
call. Qt provides tools to collect these strings into a translation file, translate them via Qt Linguist, and distribute them – see below.Client strings that are used only once can live in the source code where they are used.
Client strings that are used in several places should appear in
textconst.h
.Server code.
Some text does not require translation (see below).
Text visible to the user should look like
_("some text")
. The use of_()
is standard notation that is picked up by internationalization software that scans the source code. The_
function is aliased to an appropriate translation function, usually via_ = req.gettext
. A per-request system is used so that different users of the web site can operate simultaneously in different languages.Where text is re-used, it is placed in
cc_text.py
. This is in general preferable because it allows better automatic translation mechanisms than the XML system.
14.12.2. CamCOPS language rules¶
Hard-coded English is OK for the following:
code
Qt debugging stream
command-line text
debugging tests
task short names (typically standardized abbreviations)
database structure (e.g. table names, field names)
config file settings, and things that refer to them
Everything else should be translatable.
14.12.3. Overview of the Qt translation system¶
Begin by editing the Qt project file to add a
TRANSLATIONS
entry.Qt suggests looking up languages via
QLocale::system::name()
(see https://doc.qt.io/qt-5/internationalization.html). This (https://doc.qt.io/qt-5/qlocale.html#name) returns a string of the formlanguage_country
wherelanguage
is a lower-case two-letter ISO-639 language code (i.e. ISO-639-1), andcountry
is an uppercase, “two- or three-letter” ISO 3166 country code. Thus, for example:en_GB
(English, United Kingdom of Great Britain and Northern Ireland); probablyda_DK
(Danish, Denmark).See
Though the more common format appears to be
language-country
, Qt Linguist prefers and auto-recognizes the underscore version.Moreover, the underscore is the standard format for locale (rather than language tags); the underscore is also used by POSIX and Python
Python: see
locale.getlocale()
Babel insists on it (rejecting hyphens).
We will use the underscore notation throughout.
For example, to add Danish:
TRANSLATIONS = translations/camcops_da_DK.ts
These files live in the source tree, not the resource file.
Run
qmake
.lupdate
scans a Qt project and updates a.ts
translation file. In Qt Creator, run.ts
translation files are XML.Pay close attention to error messages from lupdate. In particular,
static
methods are fine, but classes implementingtr()
must both (a) inherit fromQObject
and (b) use theQ_OBJECT
macro.However, there is a Qt bug as of 2019-05-03: C++11 raw strings generate the error “Unterminated C++ string”; https://bugreports.qt.io/browse/QTBUG-42736; fixed in Qt 5.12.2.
To delete obsolete strings, use the
-no-obsolete
option; e.g.~/dev/qt_local_build/qt_linux_x86_64_install/bin/lupdate -no-obsolete camcops.pro
The Qt Linguist tool edits
.ts
files.Class-related strings appear helpfully in against their class.
Free-standing strings created with
QObject::tr()
appear againstQObject
.However,
lupdate
looks like it reads the C++ in a fairly superficial way; specifically, it will NOT find strings defined this way:#define TR(stringname, text) const QString stringname(QObject::tr(text)) TR(SOMESTRING, "some string contents");
but will find this semantically equivalent version:
const QString SOMESTRING(QObject::tr("some string contents"));
See also https://doc.qt.io/archives/qq/qq03-swedish-chef.html.
14.12.4. Implementing translatable strings in C++¶
Note also in particular that for obvious initialization order reasons,
QObject::tr()
doesn’t help with static variables. See e.g. https://stackoverflow.com/questions/3493540/qt-tr-in-static-variable. So, options include:if used in one place in the code, just use
tr()
from within aQ_OBJECT
class, e.g.void doSomething() { // ... alert(tr("You need to draw on the banana!")); }
if used multiple in one function, e.g.
void doSomething() { const QString mystring(tr("Configure ExpDetThreshold task")); // ... }
if used repeatedly from different places, consider a static member function, e.g.
// something.h class Something { Q_OBJECT // ... private: static QString txtAuditory(); } // something.cpp QString Something::txtAuditory() { return tr("Auditory"); }
… which appears in the right class in Qt Linguist.
You are likely to need to re-run
qmake
beforelupdate
(or, for example, it can fail to pick up on namespaces).
14.12.5. Setting up the translation system¶
See code.
14.12.6. Making the binary translation files¶
Run
lrelease
, e.g. from within Qt Creator as . This converts.ts
files to.qm
files.You need to add the
.qm
files to your resources.As always, the
:/
prefix in a filename, orqrc:///
for a URL, points to the resources.
14.12.7. Tasks: xstrings¶
Language-specific xstring fetch mechanism implemented.
When the client asks for server strings, all languages are sent.
14.12.8. Determining the language¶
Client: the user chooses the language.
Server: (a) there is a default server language, which also applies when the user is not logged in; (b) users can choose a language.
14.12.9. Server core¶
We need to internationalize (a) Python strings and (b) Mako template strings.
https://docs.pylonsproject.org/projects/pyramid-cookbook/en/latest/templates/mako_i18n.html
https://docs.pylonsproject.org/projects/pylons-webframework/en/latest/i18n.html
http://danilodellaquila.com/en/blog/pyramid-internationalization-howto
https://docs.python.org/3/library/gettext.html#class-based-api; there are editors for
gettext
systems, like Poedit (this is a pretty good open-source one, with a paid professional upgrade; from Ubuntu, install withsudo snap install poedit
)https://stackoverflow.com/questions/37998300/python-gettext-specify-locale-in
Systems like
gettext
and Qt’s work on the basis that you write the actual string in the code, then translate it (rather than having an intermediate “lookup string name”) – clearer for developers.gettext
is very much like Qt’s system:xgettext
or Babel scans your code and extracts message catalogues, producing.po
(Portable Object) files..po
files are compiled to.mo
(Machine Object) filesgettext
loads.mo
files and does the translation.The convention is to use
_("some string")
as the notation for the translation function. (Is that what Babel looks for?) Thus, in Mako, the equivalent is${_("some string")}
.
This looks like a well-established framework. Babel supports Mako.
The difficulty is that many have a monolithic context, rather than a request-specific context, in which they translate.
We manage this as follows.
Mako
<% _ = request.gettext %>
## TRANSLATOR: string context described here
<p>${_("Please translate me")}</p>
It would be nice if we could just put the _
definition in base.mako
,
but that doesn’t come through with <%inherit file=.../>
. But we can add
it to the system context via
camcops_server.cc_pyramid.CamcopsMakoLookupTemplateRenderer
. So we do.
Generic Python
_ = request.gettext
# TRANSLATOR: string context described here
mytext = _("Please translate me")
If an appropriate comment tag is used, either in Python or Mako (here,
TRANSLATOR:
, as defined in build_server_translations.py
), the comment
appears in the translation files.
Forms
See cc_forms.py
; the forms need to be request-aware. This is quite fiddly.
Efficiency
Try e.g.
inotifywait --monitor server/camcops_server/translations/da_DK/LC_MESSAGES/camcops.mo
… from a single-process CherryPy instance (camcops_server
.serve_cherrypy
), there’s a single read call only.
14.12.10. Updating server strings¶
There is a CamCOPS development tool, build_server_translations.py
. Its help
is as follows:
USAGE: build_server_translations.py [-h] [--verbose] [--poedit POEDIT]
operation
Create translation files for CamCOPS server. CamCOPS knows about the following
locales:
['da_DK']
Operations:
extract
Extract strings from code that looks like, for example,
_("please translate me")
in Python and Mako files. Write the strings to this .pot file:
/path/to/camcops/server/translations/camcops_translations.pot
init_missing
For any locales that do not have a .po file, create one.
update
Updates all .po files from the .pot file.
[At this stage, edit the .po files with Poedit or similar.]
poedit
Launch (spawn) Poedit to edit the .po files.
compile
Converts each .po file to an equivalent .mo file.
all
Executes all other operations, except poedit, in sequence.
POSITIONAL ARGUMENTS:
operation Operation to perform; possibilities are ['extract',
'init_missing', 'update', 'poedit', 'compile', 'all']
OPTIONAL ARGUMENTS:
-h, --help show this help message and exit
--verbose Be verbose (default: False)
--poedit POEDIT Path to 'poedit' tool. Default is taken from POEDIT
environment variable or 'which poedit'. (default:
/path/to/poedit)
14.12.11. Updating client strings¶
Similarly, there is a client-side development tool,
build_client_translations.py
. Its help is as follows:
USAGE: build_client_translations.py [-h] [--lconvert LCONVERT]
[--lrelease LRELEASE] [--lupdate LUPDATE]
[--poedit POEDIT] [--trim] [--no_trim]
[--verbose]
operation
Create translation files for CamCOPS client.
Operations:
po2ts
Special. Converts all .po files to .ts files in the translations
directory, if and only if the .po file is newer than the .ts file (or
the .ts file doesn't exist).
update
Updates all .ts files (which are XML, one per language) from the .pro
file and thence the C++ source code.
[At this stage, you could edit the .ts files with Qt Linguist. If you can't
find it, use Qt Creator and look within your project in "Other files" /
"Translations", right-click a .ts file, and then "Open With" / "Qt
Linguist".]
ts2po
Special. Converts all Qt .ts files to .po files in the translations
directory, if and only if the .ts file is newer than the .po file (or
the .po file doesn't exist).
release
Updates all .qm files (which are binary) from the corresponding .ts
files (discovered via the .pro file).
poedit
Launch (spawn) Poedit to edit the .po files.
all
Executes all other operations in sequence, except poedit.
This should be safe, and allow you to use .po editors like Poedit. Run
this script before and after editing.
POSITIONAL ARGUMENTS:
operation Operation to perform; possibilities are ['po2ts',
'update', 'ts2po', 'release', 'poedit', 'all']
OPTIONAL ARGUMENTS:
-h, --help show this help message and exit
--lconvert LCONVERT Path to 'lconvert' tool (part of Qt Linguist). Default
is taken from LCONVERT environment variable or 'which
lconvert'. (default: /path/to/lconvert)
--lrelease LRELEASE Path to 'lrelease' tool (part of Qt Linguist). Default
is taken from LRELEASE environment variable or 'which
lrelease'. (default: /path/to/lrelease)
--lupdate LUPDATE Path to 'lupdate' tool (part of Qt Linguist). Default
is taken from LUPDATE environment variable or 'which
lupdate'. (default: /path/to/lupdate)
--poedit POEDIT Path to 'poedit' tool. Default is taken from POEDIT
environment variable or 'which poedit'. (default:
/path/to/poedit)
--trim Remove redundant strings. (default: True)
--no_trim Do not remove redundant strings. (default: False)
--verbose Be verbose (default: False)
14.12.12. Workflow¶
Write code using the
_("text")
style in Python and thetr("text")
style in C++.Run
./server/tools/build_server_translations.py all ./tablet_qt/tools/build_client_translations.py all
Edit each translation file in turn using Poedit.
Re-run
./server/tools/build_server_translations.py all ./tablet_qt/tools/build_client_translations.py all