15.2.124. camcops_server.cc_modules.cc_request

camcops_server/cc_modules/cc_request.py


Copyright (C) 2012-2020 Rudolf Cardinal (rudolf@pobox.com).

This file is part of CamCOPS.

CamCOPS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CamCOPS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CamCOPS. If not, see <https://www.gnu.org/licenses/>.


Implements a Pyramid Request object customized for CamCOPS.

class camcops_server.cc_modules.cc_request.CamcopsDummyRequest(*args, **kwargs)[source]

Request class that allows manual manipulation of GET/POST parameters for debugging.

Notes:

  • The important base class is webob.request.BaseRequest.

  • self.params is a NestedMultiDict (see webob/multidict.py); these are intrinsically read-only.

  • self.params is also a read-only property. When read, it combines data from self.GET and self.POST.

  • What we do here is to manipulate the underlying GET/POST data.

add_get_params(d: Dict[str, str], set_method_get: bool = True) → None[source]

Add GET parameters.

Parameters
  • d – dictionary of {parameter: value} pairs.

  • set_method_get – also set the request’s method to GET?

clear_get_params() → None[source]

Clear all GET parameters.

fake_request_post_from_dict(d: Dict[str, str], encoding: str = 'utf8', set_method_post: bool = True) → None[source]

Sets the request’s POST body according to a dictionary.

Parameters
  • d – dictionary of {parameter: value} pairs.

  • encoding – character encoding to use

  • set_method_post – also set the request’s method to POST?

set_get_params(d: Dict[str, str], set_method_get: bool = True) → None[source]

Clear any GET parameters, and then set them to new values. See add_get_params().

set_method_get() → None[source]

Sets the fictional request method to GET.

set_method_post() → None[source]

Sets the fictional request method to POST.

set_post_body(body: bytes, set_method_post: bool = True) → None[source]

Sets the fake POST body.

Parameters
  • body – the body to set

  • set_method_post – also set the request’s method to POST?

class camcops_server.cc_modules.cc_request.CamcopsRequest(*args, **kwargs)[source]

The CamcopsRequest is an object central to all HTTP requests. It is the main thing passed all around the server, and embodies what we need to know about the client request – including user information, ways of accessing the database, and so on.

__init__(*args, **kwargs)[source]

This is called as the Pyramid request factory; see config.set_request_factory(CamcopsRequest)

What’s the best way of handling the database client?

  • With Titanium, we were constrained not to use cookies. With Qt, we have the option.

  • But are cookies a good idea? Probably not; they are somewhat overcomplicated for this. See also

  • Let’s continue to avoid cookies.

  • We don’t have to cache any information (we still send username/ password details with each request, and that is RESTful) but it does save authentication time to do so on calls after the first.

  • What we could try to do is:

    • look up a session here, at Request creation time;

    • add a new session if there wasn’t one;

    • but allow the database API code to replace that session (BEFORE it’s saved to the database and gains its PK) with another, determined by the content.

    • This gives one more database hit, but avoids the bcrypt time.

add_export_push_request(recipient_name: str, basetable: str, task_pk: int) → None[source]

Adds a request to push a task to an export recipient.

The reason we use this slightly convoluted approach is because otherwise, it’s very easy to generate a backend request for a new task before it’s actually been committed (so the backend finds no task).

Parameters
  • recipient_name – name of the recipient

  • basetable – name of the task’s base table

  • task_pk – server PK of the task

all_push_recipients()[source]

Cached for speed (will potentially be called for multiple tables in a bulk upload).

property camcops_session

Returns the camcops_server.cc_modules.cc_session.CamcopsSession for this request (q.v.).

Contrast:

ccsession = request.camcops_session  # type: CamcopsSession
pyramid_session = request.session  # type: ISession
complete_request_add_cookies() → None[source]

Finializes the response by adding session cookies. We do this late so that we can hot-swap the session if we’re using the database/tablet API rather than a human web browser.

Response callbacks are called in the order first-to-most-recently-added. See pyramid.request.CallbackMethodsMixin.

That looks like we can add a callback in the process of running a callback. And when we add a cookie to a Pyramid session, that sets a callback. Let’s give it a go…

config()[source]

Return an instance of camcops_server/cc_modules/cc_config.CamcopsConfig for the request.

Access it as request.config, with no brackets.

config_filename()[source]

Gets the CamCOPS config filename in use.

static create_figure(**kwargs) → matplotlib.figure.Figure[source]

Creates and returns a matplotlib.figure.Figure with a canvas. The canvas will be available as fig.canvas.

database_title()[source]

Return the database friendly title for the server.

dbsession()[source]

Return an SQLAlchemy session for the relevant request.

The use of @reify makes this elegant. If and only if a view wants a database, it can say

dbsession = request.dbsession

and if it requests that, the cleanup callbacks (COMMIT or ROLLBACK) get installed.

engine()[source]

Returns the SQLAlchemy Engine for the request.

extrastring_families(sort: bool = True) → List[str][source]

Which sets of extra strings do we have? A “family” here means, for example, “the server itself”, “the PHQ9 task”, etc.

fontdict()[source]

Returns a font dictionary for use with Matplotlib plotting.

matplotlib font handling and fontdict parameter

Linux fonts

Anyway, the main things are (1) that the relevant fonts need to be installed, and (2) that the default is DejaVu Sans.

  • Linux fonts are installed in /usr/share/fonts, and TrueType fonts within /usr/share/fonts/truetype.

  • Use fc-match to see the font mappings being used.

  • Use fc-list to list available fonts.

  • Use fc-cache to rebuild the font cache.

  • Files in /etc/fonts/conf.avail/ do some thinking.

Problems with pixellated fonts in PDFs made via wkhtmltopdf

  • See also https://github.com/wkhtmltopdf/wkhtmltopdf/issues/2193, about pixellated fonts via wkhtmltopdf (which was our problem for a subset of the fonts in trackers, on 2020-06-28, using wkhtmltopd 0.12.5 with patched Qt).

  • When you get pixellated fonts in a PDF, look also at the embedded font list in the PDF (e.g. in Okular: File -> Properties -> Fonts).

  • Matplotlib helpfully puts the text (rendered as lines in SVG) as comments.

  • As a debugging sequence, we can manually trim the “pdfhtml” output down to just the SVG file. Still has problems. Yet there’s no text in it; the text is made of pure SVG lines. And Chrome renders it perfectly. As does Firefox.

  • The rendering bug goes away entirely if you delete the opacity styling throughout the SVG:

    <g style="opacity:0.5;" transform=...>
       ^^^^^^^^^^^^^^^^^^^^
       this
    
  • So, simple fix:

    • rather than opacity (alpha) 0.5 and on top…

    • 50% grey colour and on the bottom.

fontprops()[source]

Return a matplotlib.font_manager.FontProperties object for use with Matplotlib plotting.

get_all_extra_strings() → List[Tuple[str, str, str, str]][source]

Returns all extra strings, as a list of task, name, language, value tuples.

2019-09-16: these are filtered according to the RESTRICTED_TASKS option.

get_bare_dbsession() → sqlalchemy.orm.session.Session[source]

Returns a bare SQLAlchemy session for the request.

See dbsession(), the more commonly used wrapper function.

get_bool_param(key: str, default: bool) → bool[source]

Returns a boolean parameter from the HTTP request.

Parameters
  • key – the parameter’s name

  • default – the value to return if the parameter is not found or is not a valid boolean value

Returns

an integer, or default

Valid “true” and “false” values (case-insensitive): see TRUE_STRINGS_LOWER_CASE, FALSE_STRINGS_LOWER_CASE.

get_date_param(key: str) → Optional[pendulum.date.Date][source]

Returns a date parameter from the HTTP request. If it is missing or looks bad, return None.

Parameters

key – the parameter’s name

Returns

a pendulum.Date, or None

get_datetime_param(key: str) → Optional[pendulum.datetime.DateTime][source]

Returns a datetime parameter from the HTTP request. If it is missing or looks bad, return None.

Parameters

key – the parameter’s name

Returns

a pendulum.DateTime, or None

get_export_recipient(recipient_name: str, save: bool = True) → ExportRecipient[source]

Returns a single validated export recipient, given its name.

Parameters
  • recipient_name – recipient name

  • save – save any freshly created recipient records to the DB?

Returns

of camcops_server.cc_modules.cc_exportrecipient.ExportRecipient

Return type

list

:raises - ValueError if a name is invalid: :raises - camcops_server.cc_modules.cc_exportrecipient.InvalidExportRecipient: if an export recipient configuration is invalid

get_export_recipients(recipient_names: List[str] = None, all_recipients: bool = False, all_push_recipients: bool = False, save: bool = True, database_versions: bool = True) → List[Union[ExportRecipient, ExportRecipientInfo]][source]

Returns a list of export recipients, with some filtering if desired. Validates them against the database.

  • If all_recipients, return all.

  • Otherwise, if all_push_recipients, return all “push” recipients.

  • Otherwise, return all named in recipient_names.

    • If any are invalid, raise an error.

    • If any are duplicate, raise an error.

Parameters
  • all_recipients – use all recipients?

  • all_push_recipients – use all “push” recipients?

  • recipient_names – recipient names

  • save – save any freshly created recipient records to the DB?

  • database_versions – return ExportRecipient objects that are attached to a database session (rather than ExportRecipientInfo objects that aren’t)?

Returns

of camcops_server.cc_modules.cc_exportrecipient.ExportRecipient

Return type

list

:raises - ValueError if a name is invalid: :raises - ValueError if a name is duplicated: :raises - camcops_server.cc_modules.cc_exportrecipient.InvalidExportRecipient: if an export recipient configuration is invalid

get_html_from_pyplot_figure(fig: matplotlib.figure.Figure) → str[source]

Make HTML (as PNG or SVG) from pyplot matplotlib.figure.Figure.

get_id_desc(which_idnum: int, default: str = None) → Optional[str][source]

Get the server’s ID description for the specified which_idnum value.

get_id_shortdesc(which_idnum: int, default: str = None) → Optional[str][source]

Get the server’s short ID description for the specified which_idnum value.

get_idnum_definition(which_idnum: int) → Optional[camcops_server.cc_modules.cc_idnumdef.IdNumDefinition][source]

Retrieves an camcops_server.cc_modules.cc_idnumdef.IdNumDefinition for the specified which_idnum value.

get_int_list_param(key: str) → List[int][source]

Returns a list of integer parameter values from the HTTP request.

Parameters

key – the parameter’s name

Returns

a list of integer values

get_int_param(key: str, default: int = None) → Optional[int][source]

Returns an integer parameter from the HTTP request.

Parameters
  • key – the parameter’s name

  • default – the value to return if the parameter is not found or is not a valid integer

Returns

an integer, or default

get_redirect_url_param(key: str, default: str = None) → Optional[str][source]

Returns a redirection URL parameter from the HTTP request, validating it. (The validation process does not allow all types of URLs!) If it was missing, return default. If it was bad, raise pyramid.httpexceptions.HTTPBadRequest.

Parameters
  • key – the parameter’s name

  • default – the value to return if the parameter is not found, or is invalid

Returns

a URL string, or default

get_str_list_param(key: str, lower: bool = False, upper: bool = False, validator: Callable[[str, Optional[CamcopsRequest]], None] = <function validate_alphanum_underscore>) → List[str][source]

Returns a list of HTTP parameter values from the request. Ensures all have been validated.

Parameters
  • key – the parameter’s name

  • lower – convert to lower case?

  • upper – convert to upper case?

  • validator – validator function

Returns

a list of string values

get_str_param(key: str, default: str = None, lower: bool = False, upper: bool = False, validator: Callable[[str, Optional[CamcopsRequest]], None] = <function validate_alphanum_underscore>) → Optional[str][source]

Returns an HTTP parameter from the request (GET or POST). If it does not exist, or is blank, return default. If it fails the validator, raise pyramid.httpexceptions.HTTPBadRequest.

Parameters
  • key – the parameter’s name

  • default – the value to return if the parameter is not found

  • lower – convert to lower case?

  • upper – convert to upper case?

  • validator – validator function

Returns

the parameter’s (string) contents, or default

gettext(message: str) → str[source]

Returns a version of msg translated into the current language. This is used for server-only strings.

The gettext() function is normally aliased to _() for auto-translation tools to read the souce code.

has_param(key: str) → bool[source]

Is the parameter in the request?

Parameters

key – the parameter’s name

icd10_snomed(code: str) → List[SnomedConcept][source]

Fetches a SNOMED-CT concept for an ICD-10 code

Parameters

code – an ICD-10 code

Returns

a camcops_server.cc_modules.cc_snomed.SnomedConcept

Raises

KeyError – installed)

icd10_snomed_supported()[source]

Is SNOMED-CT supported for ICD-10 codes?

icd9cm_snomed(code: str) → List[SnomedConcept][source]

Fetches a SNOMED-CT concept for an ICD-9-CM code

Parameters

code – an ICD-9-CM code

Returns

a camcops_server.cc_modules.cc_snomed.SnomedConcept

Raises

KeyError – installed)

icd9cm_snomed_supported()[source]

Is SNOMED-CT supported for ICD-9-CM codes?

idnum_definitions()[source]

Returns all camcops_server.cc_modules.cc_idnumdef.IdNumDefinition objects.

is_idnum_valid(which_idnum: int, idnum_value: Optional[int]) → bool[source]

Does the ID number pass any extended validation checks?

Parameters
  • which_idnum – which ID number type is this?

  • idnum_value – ID number value

Returns

valid?

Return type

bool

language()[source]

Returns the language code selected by the current user, or if none is selected (or the user isn’t logged in) the server’s default language.

Returns

a language code of the form en-GB

Return type

str

nonce()[source]

Return a nonce that is generated at random for each request, but remains constant for that request (because we use @reify).

See https://content-security-policy.com/examples/allow-inline-style/.

And for how to make one: https://stackoverflow.com/questions/5590170/what-is-the-standard-method-for-generating-a-nonce-in-python

now()[source]

Returns the time of the request as an Pendulum object.

(Reified, so a request only ever has one time.) Exposed as a property.

now_era_format()[source]

Returns the request time in an ISO-8601 format suitable for use as a CamCOPS era.

now_utc()[source]

Returns the time of the request as a UTC Pendulum.

now_utc_no_tzinfo()[source]

Returns the time of the request as a datetime in UTC with no timezone information attached. For when you want to compare to something similar without getting the error “TypeError: can’t compare offset-naive and offset-aware datetimes”.

prepare_for_html_figures() → None[source]

Switch the server (for this request) to producing figures in a format most suitable for HTML.

prepare_for_pdf_figures() → None[source]

Switch the server (for this request) to producing figures in a format most suitable for PDF.

remote_port()[source]

What port number is the client using?

The remote_port variable is an optional WSGI extra provided by some frameworks, such as mod_wsgi.

The WSGI spec: - https://www.python.org/dev/peps/pep-0333/

The CGI spec: - https://en.wikipedia.org/wiki/Common_Gateway_Interface

The Pyramid Request object: - https://docs.pylonsproject.org/projects/pyramid/en/latest/api/request.html#pyramid.request.Request - … note: that includes remote_addr, but not remote_port.

replace_camcops_session(ccsession: CamcopsSession) → None[source]

Replaces any existing camcops_server.cc_modules.cc_session.CamcopsSession with a new one.

Rationale:

We may have created a new HTTP session because the request had no cookies (added to the DB session but not yet saved), but we might then enter the database/tablet upload API and find session details, not from the cookies, but from the POST data. At that point, we want to replace the session in the Request, without committing the first one to disk.

route_url_params(route_name: str, paramdict: Dict[str, Any]) → str[source]

Provides a simplified interface to Request.route_url() when you have parameters to pass.

It does two things:

  1. convert all params to their str() form;

  2. allow you to pass parameters more easily using a string parameter name.

The normal Pyramid Request use is:

Request.route_url(route_name, param1=value1, param2=value2)

where “param1” is the literal name of the parameter, but here we can do

CamcopsRequest.route_url_params(route_name, {
    PARAM1_NAME: value1_not_necessarily_str,
    PARAM2_NAME: value2
})
server_settings()[source]

Return the camcops_server.cc_modules.cc_serversettings.ServerSettings for the server.

set_database_title(title: str) → None[source]

Sets the database friendly title for the server.

set_figure_font_sizes(ax: Axes, fontdict: Dict[str, Any] = None, x_ticklabels: bool = True, y_ticklabels: bool = True) → None[source]

Sets font sizes for the axes of the specified Matplotlib figure.

Parameters
  • ax – the figure to modify

  • fontdict – the font dictionary to use (if omitted, the default will be used)

  • x_ticklabels – if True, modify the X-axis tick labels

  • y_ticklabels – if True, modify the Y-axis tick labels

snomed(lookup: str) → SnomedConcept[source]

Fetches a SNOMED-CT concept for a CamCOPS task.

Parameters

lookup – a CamCOPS SNOMED lookup string

Returns

a camcops_server.cc_modules.cc_snomed.SnomedConcept

Raises

KeyError – installed)

snomed_supported()[source]

Is SNOMED-CT supported for CamCOPS tasks?

sstring(which_string: camcops_server.cc_modules.cc_text.SS) → str[source]

Returns a translated server string via a lookup mechanism.

Parameters

which_string – which string? A camcops_server.cc_modules.cc_text.SS enumeration value

Returns

the string

Return type

str

switch_output_to_png() → None[source]

Switch server (for this request) to producing figures in PNG format.

switch_output_to_svg(provide_png_fallback: bool = True) → None[source]

Switch server (for this request) to producing figures in SVG format.

Parameters

provide_png_fallback – Offer a PNG fallback option/

tabletsession()[source]

Request a camcops_server.cc_modules.cc_tabletsession.TabletSession, which is an information structure geared to client (tablet) database accesses.

If we’re using this interface, we also want to ensure we’re using the camcops_server.cc_modules.cc_session.CamcopsSession for the information provided by the tablet in the POST request, not anything already loaded/reset via cookies.

task_extrastrings_exist(taskname: str) → bool[source]

Has the server been supplied with any extra strings for a specific task?

property today

Returns today’s date.

property url_camcops_docs

Returns the URL to the CamCOPS documentation.

property url_camcops_favicon

Returns a URL to the favicon (see https://en.wikipedia.org/wiki/Favicon) from within the CamCOPS static files.

Returns a URL to the CamCOPS logo from within our static files. Returns:

property url_local_institution

Returns the local institution’s home URL.

Returns a URL to the local institution’s logo, from somewhere on our server.

property user

Returns the camcops_server.cc_modules.cc_user.User for the current request.

property user_download_bytes_available

Returns the available space for this user in their download area.

property user_download_bytes_permitted

Amount of space the user is permitted.

user_download_bytes_used()[source]

Returns the disk space used by this user.

property user_download_dir

The directory in which this user’s downloads should be/are stored, or a blank string if user downloads are not available. Also ensures it exists.

property user_download_lifetime_duration

Returns the lifetime of user download objects.

property user_id

Returns the integer user ID for the current request.

valid_which_idnums()[source]

Returns the which_idnum values for all ID number definitions.

wappstring(stringname: str, default: str = None, provide_default_if_none: bool = True, language: str = None) → Optional[str][source]

Returns a web-safe version of an appstring (an app-wide extra string). This uses the XML file shared between the client and the server.

wgettext(message: str) → str[source]

A web-safe version of gettext().

why_idnum_invalid(which_idnum: int, idnum_value: Optional[int]) → str[source]

Why does the ID number fail any extended validation checks?

Parameters
  • which_idnum – which ID number type is this?

  • idnum_value – ID number value

Returns

why invalid? (Human-readable string.)

Return type

str

wsstring(which_string: camcops_server.cc_modules.cc_text.SS) → str[source]

Returns a web-safe version of a translated server string via a lookup mechanism.

Parameters

which_string – which string? A camcops_server.cc_modules.cc_text.SS enumeration value

Returns

the string

Return type

str

wxstring(taskname: str, stringname: str, default: str = None, provide_default_if_none: bool = True, language: str = None) → Optional[str][source]

Returns a web-safe version of an xstring() (q.v.).

xstring(taskname: str, stringname: str, default: str = None, provide_default_if_none: bool = True, language: str = None) → Optional[str][source]

Looks up a string from one of the optional extra XML string files.

Parameters
  • taskname – task name (top-level key)

  • stringname – string name within task (second-level key)

  • default – default to return if the string is not found

  • provide_default_if_none – if True and default is None, return a helpful missing-string message in the style “string x.y not found”

  • language – language code to use, e.g. en-GB; if None is passed, the default behaviour is to look up the current language for this request (see language()).

Returns

the “extra string”

camcops_server.cc_modules.cc_request.camcops_pyramid_configurator_context(debug_toolbar: bool = False, static_cache_duration_s: int = 0) → pyramid.config.Configurator[source]

Context manager to create a Pyramid configuration context, for making (for example) a WSGI server or a debugging request. That means setting up things like:

  • the authentication and authorization policies

  • our request and session factories

  • our Mako renderer

  • our routes and views

Parameters
  • debug_toolbar – Add the Pyramid debug toolbar?

  • static_cache_duration_s – Lifetime (in seconds) for the HTTP cache-control setting for static content.

Returns

a Configurator object

Note this includes settings that transcend the config file.

Most things should be in the config file. This enables us to run multiple configs (e.g. multiple CamCOPS databases) through the same process. However, some things we need to know right now, to make the WSGI app. Here, OS environment variables and command-line switches are appropriate.

camcops_server.cc_modules.cc_request.command_line_request_context(user_id: int = None) → Generator[[camcops_server.cc_modules.cc_request.CamcopsRequest, None], None][source]

Request objects are ubiquitous, and allow code to refer to the HTTP request, config, HTTP session, database session, and so on. Here we make a special sort of request for use from the command line, and provide it as a context manager that will COMMIT the database afterwards (because the normal method, via the Pyramid router, is unavailable).

camcops_server.cc_modules.cc_request.complete_request_add_cookies(req: camcops_server.cc_modules.cc_request.CamcopsRequest, response: pyramid.response.Response) → None[source]

Finializes the response by adding session cookies.

See CamcopsRequest.complete_request_add_cookies().

camcops_server.cc_modules.cc_request.get_command_line_request(user_id: int = None)camcops_server.cc_modules.cc_request.CamcopsRequest[source]

Creates a dummy CamcopsRequest for use on the command line. By default, it does so for the system user. Optionally, you can specify a user by their ID number.

  • Presupposes that os.environ[ENVVAR_CONFIG_FILE] has been set, as it is in camcops_server.camcops.main().

WARNING: this does not provide a COMMIT/ROLLBACK context. If you use this directly, you must manage that yourself. Consider using command_line_request_context() instead.

camcops_server.cc_modules.cc_request.get_core_debugging_request()camcops_server.cc_modules.cc_request.CamcopsDummyRequest[source]

Returns a basic CamcopsDummyRequest.

camcops_server.cc_modules.cc_request.get_unittest_request(dbsession: sqlalchemy.orm.session.Session, params: Dict[str, Any] = None)camcops_server.cc_modules.cc_request.CamcopsDummyRequest[source]

Creates a CamcopsDummyRequest for use by unit tests.

  • Points to an existing database (e.g. SQLite in-memory database).

  • Presupposes that os.environ[ENVVAR_CONFIG_FILE] has been set, as it is in camcops_server.camcops.main().

camcops_server.cc_modules.cc_request.make_post_body_from_dict(d: Dict[str, str], encoding: str = 'utf8') → bytes[source]

Makes an HTTP POST body from a dictionary.

For debugging HTTP requests.

It mimics how the tablet operates.