`) and downloaded by the client. They contain string
versions in different languages.
Client calls look like ``xstring("stringname")``, or ``xstring(STRINGNAME)``.
Server calls look like ``req.xstring("stringname")`` or
``req.wxstring("stringname")``, etc. The ``w`` prefix is for functions that
"web-safe" their output (escaping HTML special characters).
- Client/server shared strings.
Some strings are shared between the client and the server, but are not
specific to a given task. They are in the special ``camcops.xml`` string
file, loaded as above.
Client calls typically look like ``appstring(appstrings::STRINGNAME)``; the
strings are named by constants listed in ``appstrings.h``.
Server calls typically look like ``req.wappstring(AS.STRINGNAME)`` where
``AS`` is defined in ``camcops_server.cc_modules.cc_string``.
If a string is "mission critical" for the client, then it should be built
into the client core instead (as below).
- Client core.
Some text does not require translation (see below).
Text visible to the user should be within a Qt ``tr("some text")`` call. Qt
provides tools to collect these strings into a translation file, translate
them via Qt Linguist, and distribute them -- see below.
Client strings that are used only once can live in the source code where they
are used.
Client strings that are used in several places should appear in
``textconst.h``.
- Server code.
Some text does not require translation (see below).
Text visible to the user should look like ``_("some text")``. The use of
``_()`` is standard notation that is picked up by internationalization
software that scans the source code. The ``_`` function is aliased to an
appropriate translation function, usually via ``_ = req.gettext``. A
per-request system is used so that different users of the web site can
operate simultaneously in different languages.
Where text is re-used, it is placed in ``cc_text.py``. This is in general
preferable because it allows better automatic translation mechanisms than
the XML system.
CamCOPS language rules
~~~~~~~~~~~~~~~~~~~~~~
Hard-coded English is OK for the following:
- code
- Qt debugging stream
- command-line text
- debugging tests
- task short names (typically standardized abbreviations)
- database structure (e.g. table names, field names)
- config file settings, and things that refer to them
Everything else should be translatable.
Overview of the Qt translation system
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Begin by editing the Qt project file to add a ``TRANSLATIONS`` entry.
Qt suggests looking up languages via ``QLocale::system::name()`` (see
https://doc.qt.io/qt-6.5/internationalization.html). This
(https://doc.qt.io/qt-6.5/qlocale.html#name) returns a string of the form
``language_country`` where ``language`` is a lower-case two-letter ISO-639
language code (i.e. ISO-639-1), and ``country`` is an uppercase, "two- or
three-letter" ISO 3166 country code. Thus, for example: ``en_GB`` (English,
United Kingdom of Great Britain and Northern Ireland); probably ``da_DK``
(Danish, Denmark).
See
- https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes
- https://en.wikipedia.org/wiki/ISO_3166
Though the more common format appears to be ``language-country``, Qt Linguist
prefers and auto-recognizes the underscore version.
- https://en.wikipedia.org/wiki/IETF_language_tag
- https://witekio.com/de/blog/qt-internationalization-arabic-chinese-right-left/
Moreover, the underscore is the standard format for **locale** (rather than
**language** tags); the underscore is also used by POSIX and Python
- https://en.wikipedia.org/wiki/Locale_(computer_software)
- Python: see ``locale.getlocale()``
- Babel insists on it (rejecting hyphens).
**We will use the underscore notation throughout.**
For example, to add Danish:
.. code-block:: none
TRANSLATIONS = translations/camcops_da_DK.ts
These files live in the source tree, not the resource file.
- Run ``qmake``.
- ``lupdate`` scans a Qt project and updates a ``.ts`` translation file.
In Qt Creator, run :menuselection:`Tools --> External --> Linguist --> Update
Translations (lupdate)`
- ``.ts`` translation files are XML.
- **Pay close attention to error messages from lupdate.**
In particular, ``static`` methods are fine, but classes implementing
``tr()`` must both (a) inherit from ``QObject`` and (b) use the
``Q_OBJECT`` macro.
- However, there is a Qt bug as of 2019-05-03: C++11 raw strings generate the
error "Unterminated C++ string";
https://bugreports.qt.io/browse/QTBUG-42736; fixed in Qt 5.12.2.
- To delete obsolete strings, use the ``-no-obsolete`` option; e.g.
.. code-block:: bash
~/dev/qt_local_build/qt_linux_x86_64_install/bin/lupdate -no-obsolete camcops.pro
- The Qt Linguist tool edits ``.ts`` files.
- Class-related strings appear helpfully in against their class.
- Free-standing strings created with ``QObject::tr()`` appear against
``QObject``.
- However, ``lupdate`` looks like it reads the C++ in a fairly superficial
way; specifically, it will NOT find strings defined this way:
.. code-block:: cpp
#define TR(stringname, text) const QString stringname(QObject::tr(text))
TR(SOMESTRING, "some string contents");
but will find this semantically equivalent version:
.. code-block:: cpp
const QString SOMESTRING(QObject::tr("some string contents"));
See also https://doc.qt.io/archives/qq/qq03-swedish-chef.html.
Implementing translatable strings in C++
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Note also in particular that for obvious initialization order reasons,
``QObject::tr()`` doesn't help with static variables. See e.g.
https://stackoverflow.com/questions/3493540/qt-tr-in-static-variable.
So, options include:
- if used in one place in the code, just use ``tr()`` from within a
``Q_OBJECT`` class, e.g.
.. code-block:: cpp
void doSomething()
{
// ...
alert(tr("You need to draw on the banana!"));
}
- if used multiple in one function, e.g.
.. code-block:: cpp
void doSomething()
{
const QString mystring(tr("Configure ExpDetThreshold task"));
// ...
}
- if used repeatedly from different places, consider a static member
function, e.g.
.. code-block:: cpp
// something.h
class Something
{
Q_OBJECT
// ...
private:
static QString txtAuditory();
}
// something.cpp
QString Something::txtAuditory()
{
return tr("Auditory");
}
... which appears in the right class in Qt Linguist.
- You are likely to need to re-run ``qmake`` before ``lupdate`` (or, for
example, it can fail to pick up on namespaces).
Setting up the translation system
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
See code.
Making the binary translation files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- Run ``lrelease``, e.g. from within Qt Creator as :menuselection:`Tools -->
External --> Linguist --> Release Translations (lrelease)`. This converts
``.ts`` files to ``.qm`` files.
- You need to add the ``.qm`` files to your resources.
- As always, the ``:/`` prefix in a filename, or ``qrc:///`` for a URL, points
to the resources.
Tasks: xstrings
~~~~~~~~~~~~~~~
- Language-specific xstring fetch mechanism implemented.
- When the client asks for server strings, all languages are sent.
Determining the language
~~~~~~~~~~~~~~~~~~~~~~~~
- Client: the user chooses the language.
- Server: (a) there is a default server language, which also applies when the
user is not logged in; (b) users can choose a language.
Server core
~~~~~~~~~~~
- We need to internationalize (a) Python strings and (b) Mako template strings.
- https://docs.pylonsproject.org/projects/pyramid-cookbook/en/latest/templates/mako_i18n.html
- https://docs.pylonsproject.org/projects/pylons-webframework/en/latest/i18n.html
- http://danilodellaquila.com/en/blog/pyramid-internationalization-howto
- http://babel.pocoo.org/en/latest/
- https://github.com/wichert/lingua
- https://docs.python.org/3/library/gettext.html#class-based-api; there are
editors for ``gettext`` systems, like Poedit_ (this is a pretty good
open-source one, with a paid professional upgrade; from Ubuntu, install
with ``sudo snap install poedit``)
- https://wiki.gnome.org/Apps/Gtranslator
- https://userbase.kde.org/Lokalize
- https://stackoverflow.com/questions/37998300/python-gettext-specify-locale-in
- Systems like ``gettext`` and Qt's work on the basis that you write the actual
string in the code, then translate it (rather than having an intermediate
"lookup string name") -- clearer for developers.
- ``gettext`` is very much like Qt's system:
- ``xgettext`` or Babel_ scans your code and extracts message catalogues,
producing ``.po`` (Portable Object) files.
- ``.po`` files are compiled to ``.mo`` (Machine Object) files
- ``gettext`` loads ``.mo`` files and does the translation.
- The convention is to use ``_("some string")`` as the notation for the
translation function. (Is that what Babel looks for?) Thus, in Mako, the
equivalent is ``${_("some string")}``.
- This looks like a well-established framework. Babel supports Mako.
- The difficulty is that many have a monolithic context, rather than a
request-specific context, in which they translate.
We manage this as follows.
**Mako**
.. code-block:: none
<% _ = request.gettext %>
## TRANSLATOR: string context described here
${_("Please translate me")}
It would be nice if we could just put the ``_`` definition in ``base.mako``,
but that doesn't come through with ``<%inherit file=.../>``. But we can add
it to the system context via
:class:`camcops_server.cc_pyramid.CamcopsMakoLookupTemplateRenderer`. So we do.
**Generic Python**
.. code-block:: python
_ = request.gettext
# TRANSLATOR: string context described here
mytext = _("Please translate me")
If an appropriate comment tag is used, either in Python or Mako (here,
``TRANSLATOR:``, as defined in ``build_server_translations.py``), the comment
appears in the translation files.
**Forms**
See ``cc_forms.py``; the forms need to be request-aware. This is quite fiddly.
**Efficiency**
Try e.g.
.. code-block:: bash
inotifywait --monitor server/camcops_server/translations/da_DK/LC_MESSAGES/camcops.mo
... from a single-process CherryPy instance (``camcops_server
.serve_cherrypy``), there's a single read call only.
Updating server strings
~~~~~~~~~~~~~~~~~~~~~~~
There is a CamCOPS development tool, ``build_server_translations.py``. Its help
is as follows:
.. literalinclude:: _build_server_translations_help.txt
:language: none
Updating client strings
~~~~~~~~~~~~~~~~~~~~~~~
Similarly, there is a client-side development tool,
``build_client_translations.py``. Its help is as follows:
.. literalinclude:: _build_client_translations_help.txt
:language: none
Workflow
~~~~~~~~
- Write code using the ``_("text")`` style in Python and the ``tr("text")``
style in C++.
- Run
.. code-block:: bash
./server/tools/build_server_translations.py all
./tablet_qt/tools/build_client_translations.py all
- Edit each translation file in turn using Poedit_.
- Re-run
.. code-block:: bash
./server/tools/build_server_translations.py all
./tablet_qt/tools/build_client_translations.py all