15.2.85. camcops_server.cc_modules.cc_config

camcops_server/cc_modules/cc_config.py


Copyright (C) 2012-2020 Rudolf Cardinal (rudolf@pobox.com).

This file is part of CamCOPS.

CamCOPS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CamCOPS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CamCOPS. If not, see <https://www.gnu.org/licenses/>.


Read and represent a CamCOPS config file.

Also contains various types of demonstration config file (CamCOPS, but also supervisord, Apache, etc.) and demonstration helper scripts (e.g. MySQL).

There are CONDITIONAL AND IN-FUNCTION IMPORTS HERE; see below. This is to minimize the number of modules loaded when this is used in the context of the client-side database script, rather than the webview.

Moreover, it should not use SQLAlchemy objects directly; see celery.py.

In particular, I tried hard to use a “database-unaware” (unbound) SQLAlchemy ExportRecipient object. However, when the backend re-calls the config to get its recipients, we get errors like:

[2018-12-25 00:56:00,118: ERROR/ForkPoolWorker-7] Task camcops_server.cc_modules.celery_tasks.export_to_recipient_backend[ab2e2691-c2fa-4821-b8cd-2cbeb86ddc8f] raised unexpected: DetachedInstanceError('Instance <ExportRecipient at 0x7febbeeea7b8> is not bound to a Session; attribute refresh operation cannot proceed',)
Traceback (most recent call last):
  File "/home/rudolf/dev/venvs/camcops/lib/python3.6/site-packages/celery/app/trace.py", line 382, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/rudolf/dev/venvs/camcops/lib/python3.6/site-packages/celery/app/trace.py", line 641, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/celery_tasks.py", line 103, in export_to_recipient_backend
    schedule_via_backend=False)
  File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_export.py", line 255, in export
    req, recipient_names=recipient_names, all_recipients=all_recipients)
  File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_config.py", line 1460, in get_export_recipients
    valid_names = set(r.recipient_name for r in recipients)
  File "/home/rudolf/Documents/code/camcops/server/camcops_server/cc_modules/cc_config.py", line 1460, in <genexpr>
    valid_names = set(r.recipient_name for r in recipients)
  File "/home/rudolf/dev/venvs/camcops/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line 242, in __get__
    return self.impl.get(instance_state(instance), dict_)
  File "/home/rudolf/dev/venvs/camcops/lib/python3.6/site-packages/sqlalchemy/orm/attributes.py", line 594, in get
    value = state._load_expired(state, passive)
  File "/home/rudolf/dev/venvs/camcops/lib/python3.6/site-packages/sqlalchemy/orm/state.py", line 608, in _load_expired
    self.manager.deferred_scalar_loader(self, toload)
  File "/home/rudolf/dev/venvs/camcops/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 813, in load_scalar_attributes
    (state_str(state)))
sqlalchemy.orm.exc.DetachedInstanceError: Instance <ExportRecipient at 0x7febbeeea7b8> is not bound to a Session; attribute refresh operation cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3)
class camcops_server.cc_modules.cc_config.CamcopsConfig(config_filename: str, config_text: str = None)[source]

Class representing the CamCOPS configuration.

__init__(config_filename: str, config_text: str = None) → None[source]

Initialize by reading the config file.

Parameters
  • config_filename – Filename of the config file (usual method)

  • config_text – Text contents of the config file (alternative method for special circumstances); overrides config_filename

assert_database_ok() → None[source]

Asserts that our database engine is OK and our database structure is correct.

get_all_export_recipient_info() → List[camcops_server.cc_modules.cc_exportrecipientinfo.ExportRecipientInfo][source]

Returns all export recipients (in their “database unaware” form) specified in the config.

Returns

of camcops_server.cc_modules.cc_exportrecipientinfo.ExportRecipientInfo

Return type

list

property get_all_table_names

Returns all table names from the database.

get_celery_beat_pidfilename() → str[source]

Process ID file (pidfile) used by celery beat --pidfile ....

get_dbsession_context() → Generator[[sqlalchemy.orm.session.Session, None], None][source]

Context manager to provide an SQLAlchemy session that will COMMIT once we’ve finished, or perform a ROLLBACK if there was an exception.

get_dbsession_raw() → sqlalchemy.orm.session.Session[source]

Returns a raw SQLAlchemy Session. Avoid this – use get_dbsession_context() instead.

get_export_lockfilename_db(recipient_name: str) → str[source]

Returns a full path to a lockfile suitable for locking for a whole-database export to a particular export recipient.

Parameters

recipient_name – name of the recipient

Returns

a filename

get_export_lockfilename_task(recipient_name: str, basetable: str, pk: int) → str[source]

Returns a full path to a lockfile suitable for locking for a single-task export to a particular export recipient.

Parameters
  • recipient_name – name of the recipient

  • basetable – task base table name

  • pk – server PK of the task

Returns

a filename

get_icd10_snomed_concepts() → Dict[str, List[camcops_server.cc_modules.cc_snomed.SnomedConcept]][source]

Returns all SNOMED-CT concepts for ICD-10-CM codes supported by CamCOPS.

Returns

maps ICD-10 codes to SnomedConcept objects

Return type

dict

get_icd9cm_snomed_concepts() → Dict[str, List[camcops_server.cc_modules.cc_snomed.SnomedConcept]][source]

Returns all SNOMED-CT concepts for ICD-9-CM codes supported by CamCOPS.

Returns

maps ICD-9-CM codes to SnomedConcept objects

Return type

dict

get_master_export_recipient_lockfilename() → str[source]

When we are modifying export recipients, we check “is this information the same as the current version in the database”, and if not, we write fresh information to the database. If lots of processes do that at the same time, we have a problem (usually a database deadlock) – hence this lock.

Returns

a filename

get_sqla_engine() → sqlalchemy.engine.base.Engine[source]

Returns an SQLAlchemy Engine.

I was previously misinterpreting the appropriate scope of an Engine. I thought: create one per request. But the Engine represents the connection pool. So if you create them all the time, you get e.g. a ‘Too many connections’ error.

“The appropriate scope is once per [database] URL per application, at the module level.”

Now, our CamcopsConfig instance is cached, so there should be one of them overall. See get_config() below.

Therefore, making the engine a member of this class should do the trick, whilst avoiding global variables.

get_task_snomed_concepts() → Dict[str, camcops_server.cc_modules.cc_snomed.SnomedConcept][source]

Returns all SNOMED-CT concepts for tasks.

Returns

maps lookup strings to SnomedConcept objects

Return type

dict

class camcops_server.cc_modules.cc_config.CrontabEntry(line: str = None, minute: Union[str, int, List[int]] = '*', hour: Union[str, int, List[int]] = '*', day_of_week: Union[str, int, List[int]] = '*', day_of_month: Union[str, int, List[int]] = '*', month_of_year: Union[str, int, List[int]] = '*', content: str = None)[source]

Class to represent a crontab-style entry.

__init__(line: str = None, minute: Union[str, int, List[int]] = '*', hour: Union[str, int, List[int]] = '*', day_of_week: Union[str, int, List[int]] = '*', day_of_month: Union[str, int, List[int]] = '*', month_of_year: Union[str, int, List[int]] = '*', content: str = None) → None[source]
Parameters
  • line – line of the form m h dow dom moy content content content.

  • minute – crontab “minute” entry

  • hour – crontab “hour” entry

  • day_of_week – crontab “day_of_week” entry

  • day_of_month – crontab “day_of_month” entry

  • month_of_year – crontab “month_of_year” entry

  • content – crontab “thing to run” entry

If line is specified, it is used. Otherwise, the components are used; the default for each of them is "*", meaning “all”. Thus, for example, you can specify minute="*/5" and that is sufficient to mean “every 5 minutes”.

get_celery_schedule() → celery.schedules.crontab[source]

Returns the corresponding Celery schedule.

Returns

a celery.schedules.crontab

Raises

celery.schedules.ParseException

camcops_server.cc_modules.cc_config.get_config(config_filename: str)camcops_server.cc_modules.cc_config.CamcopsConfig[source]

Returns a camcops_server.cc_modules.cc_config.CamcopsConfig from the specified config filename.

Cached.

camcops_server.cc_modules.cc_config.get_config_filename_from_os_env() → str[source]

Returns the config filename to use, from our operating system environment variable.

(We do NOT trust the WSGI environment for this.)

camcops_server.cc_modules.cc_config.get_default_config_from_os_env()camcops_server.cc_modules.cc_config.CamcopsConfig[source]

Returns the camcops_server.cc_modules.cc_config.CamcopsConfig representing the config filename that we read from our operating system environment variable.

camcops_server.cc_modules.cc_config.get_demo_apache_config(rootpath: str = 'camcops', specimen_internal_port: int = None, specimen_socket_file: str = '/run/camcops/camcops.socket') → str[source]

Returns a demo Apache HTTPD config file section applicable to CamCOPS.

camcops_server.cc_modules.cc_config.get_demo_config(for_docker: bool = False) → str[source]

Returns a demonstration config file based on the specified parameters.

Parameters

for_docker – Adjust defaults for the Docker environment.

camcops_server.cc_modules.cc_config.get_demo_supervisor_config() → str[source]

Returns a demonstration supervisord config file based on the specified parameters.

camcops_server.cc_modules.cc_config.warn_if_not_docker_value(param_name: str, actual_value: Any, required_value: Any) → None[source]

Warn the user if a parameter does not match the specific value required when operating under Docker.

Parameters
  • param_name – Name of the parameter in the CamCOPS config file.

  • actual_value – Value in the config file.

  • required_value – Value that should be used.

camcops_server.cc_modules.cc_config.warn_if_not_present(param_name: str, value: Any) → None[source]

Warn the user if a parameter is not set (None, or an empty string), for when operating under Docker.

Parameters
  • param_name – Name of the parameter in the CamCOPS config file.

  • value – Value in the config file.

camcops_server.cc_modules.cc_config.warn_if_not_within_docker_dir(param_name: str, filespec: str, permit_cfg: bool = False, permit_venv: bool = False, permit_tmp: bool = False, param_contains_not_is: bool = False) → None[source]

If the specified filename isn’t within a relevant directory that will be used by CamCOPS when operating within a Docker Compose application, warn the user.

Parameters
  • param_name – Name of the parameter in the CamCOPS config file.

  • filespec – Filename (or filename-like thing) to check.

  • permit_cfg – Permit the file to be in the configuration directory.

  • permit_venv – Permit the file to be in the virtual environment directory.

  • permit_tmp – Permit the file to be in the shared temporary space.

  • param_contains_not_is – The parameter “contains”, not “is”, the filename.