14.2.67. camcops_server.cc_modules.cc_export

camcops_server/cc_modules/cc_export.py


Copyright (C) 2012-2019 Rudolf Cardinal (rudolf@pobox.com).

This file is part of CamCOPS.

CamCOPS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CamCOPS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CamCOPS. If not, see <http://www.gnu.org/licenses/>.


Export and research dump functions.

Export design:

WHICH RECORDS TO SEND?

The most powerful mechanism is not to have a sending queue (which would then require careful multi-instance locking), but to have a “sent” log. That way:

  • A record needs sending if it’s not in the sent log (for an appropriate recipient).
  • You can add a new recipient and the system will know about the (new) backlog automatically.
  • You can specify criteria, e.g. don’t upload records before 1/1/2014, and modify that later, and it would catch up with the backlog.
  • Successes and failures are logged in the same table.
  • Multiple recipients are handled with ease.
  • No need to alter database.pl code that receives from tablets.
  • Can run with a simple cron job.

LOCKING

MESSAGE QUEUE AND BACKEND

Thoughts as of 2018-12-22.

camcops_server.cc_modules.cc_export.export(req: CamcopsRequest, recipient_names: List[str] = None, all_recipients: bool = False, via_index: bool = True, schedule_via_backend: bool = False) → None[source]

Called from the command line.

Exports all relevant tasks (pending incremental exports, or everything if applicable) for specified export recipients.

Obtains a file lock, then iterates through all recipients.

Parameters:
  • req – a camcops_server.cc_modules.cc_request.CamcopsRequest
  • recipient_names – list of export recipient names (as per the config file)
  • all_recipients – use all recipients?
  • via_index – use the task index (faster)?
  • schedule_via_backend – schedule jobs via the backend instead?
camcops_server.cc_modules.cc_export.export_task(req: CamcopsRequest, recipient: camcops_server.cc_modules.cc_exportrecipient.ExportRecipient, task: camcops_server.cc_modules.cc_task.Task) → None[source]

Exports a single task, checking that it remains valid to do so.

Parameters:
camcops_server.cc_modules.cc_export.export_tasks_individually(req: CamcopsRequest, recipient: camcops_server.cc_modules.cc_exportrecipient.ExportRecipient, via_index: bool = True, schedule_via_backend: bool = False) → None[source]

Exports all necessary tasks for a recipient.

Parameters:
camcops_server.cc_modules.cc_export.export_whole_database(req: CamcopsRequest, recipient: camcops_server.cc_modules.cc_exportrecipient.ExportRecipient, via_index: bool = True) → None[source]

Exports to a database.

Holds a recipient-specific file lock in the process.

Parameters:
camcops_server.cc_modules.cc_export.gen_audited_tasks_by_task_class(collection: TaskCollection, audit_descriptions: List[str]) → Generator[[camcops_server.cc_modules.cc_task.Task, NoneType], NoneType][source]

Generates tasks from a collection, across task classes, simultaneously adding to an audit description. Used for user-triggered downloads.

Parameters:
Yields:

camcops_server.cc_modules.cc_task.Task objects

camcops_server.cc_modules.cc_export.gen_audited_tasks_for_task_class(collection: TaskCollection, cls: Type[camcops_server.cc_modules.cc_task.Task], audit_descriptions: List[str]) → Generator[[camcops_server.cc_modules.cc_task.Task, NoneType], NoneType][source]

Generates tasks from a collection, for a given task class, simultaneously adding to an audit description. Used for user-triggered downloads.

Parameters:
Yields:

camcops_server.cc_modules.cc_task.Task objects

camcops_server.cc_modules.cc_export.get_tsv_collection_from_task_collection(req: CamcopsRequest, collection: TaskCollection, sort_by_heading: bool) → Tuple[camcops_server.cc_modules.cc_tsv.TsvCollection, List[str]][source]

Converts a collection of tasks to a collection of spreadsheet-style data.

Parameters:
Returns:

tsv_collection, audit_descriptions where tsv_collection is a camcops_server.cc_modules.cc_tsv.TsvCollection object and audit_descriptions is a list of strings describing the data being fetched.

Return type:

tuple

camcops_server.cc_modules.cc_export.print_export_queue(req: CamcopsRequest, recipient_names: List[str] = None, all_recipients: bool = False, via_index: bool = True, pretty: bool = False) → None[source]

Called from the command line.

Shows tasks that would be exported.

Parameters:
  • req – a camcops_server.cc_modules.cc_request.CamcopsRequest
  • recipient_names – list of export recipient names (as per the config file)
  • all_recipients – use all recipients?
  • via_index – use the task index (faster)?
  • pretty – use str(task) not repr(task) (prettier, slower because it has to query the patient)
camcops_server.cc_modules.cc_export.task_collection_to_ods_response(req: CamcopsRequest, collection: TaskCollection, sort_by_heading: bool) → pyramid.response.Response[source]

Converts a set of tasks to an OpenOffice ODS file.

Parameters:
Returns:

a pyramid.response.Response object

camcops_server.cc_modules.cc_export.task_collection_to_sqlite_response(req: CamcopsRequest, collection: TaskCollection, export_options: camcops_server.cc_modules.cc_simpleobjects.TaskExportOptions, as_sql_not_binary: bool) → pyramid.response.Response[source]

Converts a set of tasks to an SQLite export, either as binary or the SQL text to regenerate it.

Parameters:
Returns:

a pyramid.response.Response object

camcops_server.cc_modules.cc_export.task_collection_to_tsv_zip_response(req: CamcopsRequest, collection: TaskCollection, sort_by_heading: bool) → pyramid.response.Response[source]

Converts a set of tasks to a TSV (tab-separated value) response, as a set of TSV files (one per table) in a ZIP file.

Parameters:
Returns:

a pyramid.response.Response object

camcops_server.cc_modules.cc_export.task_collection_to_xlsx_response(req: CamcopsRequest, collection: TaskCollection, sort_by_heading: bool) → pyramid.response.Response[source]

Converts a set of tasks to an Excel XLSX file.

Parameters:
Returns:

a pyramid.response.Response object