15.2.179. camcops_server.cc_modules.client_api

camcops_server/cc_modules/client_api.py

Copyright (C) 2012, University of Cambridge, Department of Psychiatry. Created by Rudolf Cardinal (rnc1001@cam.ac.uk).

This file is part of CamCOPS.

CamCOPS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CamCOPS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CamCOPS. If not, see <https://www.gnu.org/licenses/>.

Implements the API through which client devices (tablets etc.) upload and download data.

We use primarily SQLAlchemy Core here (in contrast to the ORM used elsewhere).

This code is optimized to a degree for speed over clarity, aiming primarily to reduce the number of database hits.

The overall upload method is as follows

Everything that follows refers to records relating to a specific client device in the “current” era, only.

In the preamble, the client:

Then, in the usual stepwise upload:

  • op_start_upload()

    • Rolls back any previous incomplete changes via rollback_all().

    • Creates an upload batch, via get_batch_details_start_if_needed().

  • If were are in a preserving/finalizing upload: op_start_preservation().

    • Marks all tables as dirty.

    • Marks the upload batch as a “preserving” batch.

  • Then call some or all of:

    • For tables that are empty on the client, op_upload_empty_tables().

      • Current client records are marked as _removal_pending.

      • Any table that had previous client records is marked as dirty.

      • If preserving, any table without current records is marked as clean.

    • For tables that the client wishes to send in one go, op_upload_table().

      • Find current server records.

      • Use upload_record_core() to add new records and modify existing ones, and flag_deleted() to delete ones that weren’t on the client.

      • If any records are new, modified, or deleted, mark the table as dirty.

      • If preserving and there were no server records in this table, mark the table as clean.

    • For tables (e.g. BLOBs) that might be too big to send in one go:

  • In addition, specific records can be marked as _move_off_tablet.

    • upload_record_core() checks this for otherwise “identical” records and applies that flag to the server.

  • When the client’s finished, it calls op_end_upload().

    • Calls commit_all();

    • … which, for all dirty tables, calls commit_table();

    • … which executes the “add”, “remove”, and “preserve” functions for the table;

    • … and triggers the updating of special server indexes on patient ID numbers and tasks, via update_indexes().

    • At the end, commit_all() clears the dirty-table list.

There’s a little bit of special code to handle old tablet software, too.

As of v2.3.0, the function op_upload_entire_database() does this in one step (faster; for use if the network packets are not excessively large).

  • Code relating to this uses batchdetails.onestep.

Setup for the upload code

  • Fire up a CamCOPS client with an empty database, e.g. from the build directory via

    ./camcops --dbdir ~/tmp/camcops_client_test

  • Fire up a web browser showing both (a) the task list via the index, and (b) the task list without using the index. We’ll use this to verify correct indexing. The two versions of the view should never be different.

  • Ensure the test client device has no current records (force-finalize if required).

  • Ensure the server’s index is proper. Run camcops_server reindex if required.

  • If required, fire up MySQL with the server database. You may wish to use pager less -SFX, for better display of large tables.

Testing the upload code

Perform the following steps both (1) with the client forced to the stepwise upload method, and (2) with it forced to one-step upload.

Note that the number of patient ID numbers uploaded (etc.) is ignored below.

Single record

[Checked for one-step and multi-step upload, 2018-11-21.]

  1. Create a blank ReferrerSatisfactionSurvey (table ref_satis_gen). This has the advantage of being an anonymous single-record task.

  2. Upload/copy.

    • The server log should show 1 × ref_satis_gen added.

    • The task lists should show the task as current and incomplete.

  3. Modify it, so it’s complete.

  4. Upload/copy.

    • The server log should show 1 × ref_satis_gen added, 1 × ref_satis_gen modified out.

    • The task lists should show the task as current and complete.

  5. Upload/move.

    • The server log should show 2 × ref_satis_gen preserved.

    • The task lists should show the task as no longer current.

  6. Create another blank one.

  7. Upload/copy.

  8. Modify it so it’s complete.

  9. Specifically flag it for preservation (the chequered flags).

  10. Upload/copy.

    • The server log should show 1 × ref_satis_gen added, 1 × ref_satis_gen modified out, 2 × ref_satis_gen preserved.

    • The task lists should show the task as complete and no longer current.

With a patient

[Checked for one-step and multi-step upload, 2018-11-21.]

  1. Create a dummy patient that the server will accept.

  2. Create a Progress Note with location “loc1” and abort its creation, giving an incomplete task.

  3. Create a second Progress Note with location “loc2” and contents “note2”.

  4. Create a third Progress Note with location “loc3” and contents “note3”.

  5. Upload/copy. Verify. This checks addition.

    • The server log should show 1 × patient added; 3 × progressnote added. (Also however many patientidnum records you chose.)

    • All three tasks should be “current”.

    • The first should be “incomplete”.

  6. Modify the first note by adding contents “note1”.

  7. Delete the second note.

  8. Upload/copy. Verify. This checks modification, deletion, no-change detection, and reindexing.

    • The server log should show 1 × progressnote added, 1 × progressnote modified out, 1 × progressnote deleted.

    • The first note should now appear as complete.

    • The second should have vanished.

    • The third should be unchanged.

    • The two remaining tasks should still be “current”.

  9. Delete the contents from the first note again.

  10. Upload/move (or move-keeping-patients; that’s only different on the client side). Verify. This checks preservation (finalizing) and reindexing.

    • The server log should show 1 × patient preserved; 1 × progressnote added, 1 × progressnote modified out, 5 × progressnote preserved.

    • The two remaining tasks should no longer be “current”.

    • The first should no longer be “complete”.

  11. Create a complete “note 4” and an incomplete “note 5”.

  12. Upload/copy.

  13. Force-finalize from the server. This tests force-finalizing including reindexing.

    • The “tasks to finalize” list should have just two tasks in it.

    • After force-finalizing, the tasks should remain in the index but no longer be marked as current.

  14. Upload/move to get rid of the residual tasks on the client.

    • The server log should show 1 × patient added, 1 × patient preserved; 2 × progressnote added, 2 × progressnote preserved.

With ancillary tables and BLOBs

[Checked for one-step and multi-step upload, 2018-11-21.]

  1. Create a PhotoSequence with text “t1”, one photo named “p1” of you holding up one finger vertically, and another photo named “p2” of you holding up two fingers vertically.

  2. Upload/copy.

    • The server log should show:

      • blobs: 2 × added;

      • patient: 1 × added;

      • photosequence: 1 × added;

      • photosequence_photos: 2 × added.

    • The task lists should look sensible.

  3. Clear the second photo and replace it with a photo of you holding up two fingers horizontally.

  4. Upload/copy.

    • The server log should show:

      • blobs: 1 × added, 1 × modified out;

      • photosequence: 1 × added, 1 × modified out;

      • photosequence_photos: 1 × added, 1 × modified out.

    • The task lists should look sensible.

  5. Back to two fingers vertically. (This is the fourth photo overall.)

  6. Mark that patient for specific finalization.

  7. Upload/copy.

    • The server log should show:

      • blobs: 1 × added, 1 × modified out, 4 × preserved;

      • patient: 1 × preserved;

      • photosequence: 1 × added, 1 × modified out, 3 × preserved;

      • photosequence_photos: 1 × added, 1 × modified out, 4 × preserved.

    • The tasks should no longer be current.

    • A fresh “vertical fingers” photo should be visible.

  8. Create another patient and another PhotoSequence with one photo of three fingers.

  9. Upload-copy.

  10. Force-finalize.

    • Should finalize: 1 × blobs, 1 × patient, 1 × photosequence, 1 × photosequence_photos.

  11. Upload/move.

During any MySQL debugging, remember:

-- For better display:
pager less -SFX;

-- To view relevant parts of the BLOB table without the actual BLOB:

SELECT
    _pk, _group_id, _device_id, _era,
    _current, _predecessor_pk, _successor_pk,
    _addition_pending, _when_added_batch_utc, _adding_user_id,
    _removal_pending, _when_removed_batch_utc, _removing_user_id,
    _move_off_tablet,
    _preserving_user_id, _forcibly_preserved,
    id, tablename, tablepk, fieldname, mimetype, when_last_modified
FROM blobs;
class camcops_server.cc_modules.client_api.Operations[source]

Constants giving the name of operations (commands) accepted by this API.

camcops_server.cc_modules.client_api.all_tables_with_min_client_version() → Dict[str, semantic_version.base.Version][source]

For all tables that the client might upload to, return a mapping from the table name to the corresponding minimum client version.

camcops_server.cc_modules.client_api.audit(req: CamcopsRequest, details: str, patient_server_pk: int = None, tablename: str = None, server_pk: int = None) → None[source]

Audit something.

camcops_server.cc_modules.client_api.audit_upload(req: CamcopsRequest, changes: List[camcops_server.cc_modules.cc_client_api_core.UploadTableChanges]) → None[source]

Writes audit information for an upload.

Parameters
camcops_server.cc_modules.client_api.clear_device_upload_batch(req: CamcopsRequest) → None[source]

Ensures there is nothing pending. Rools back previous changes. Wipes any ongoing batch details.

Parameters

req – the camcops_server.cc_modules.cc_request.CamcopsRequest

camcops_server.cc_modules.client_api.clear_dirty_tables(req: CamcopsRequest) → None[source]

Clears the dirty-table list for a device.

camcops_server.cc_modules.client_api.client_api(req: CamcopsRequest) → pyramid.response.Response[source]

View for client API. All tablet interaction comes through here. Wraps main_client_api(). Handles exceptions.

Internally, replies are managed as dictionaries. For the final reply, the dictionary is converted to text in this format:

k1:v1
k2:v2
k3:v3
...
camcops_server.cc_modules.client_api.client_pks_that_exist(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, clientpk_name: str, clientpk_values: List[int]) → Dict[int, camcops_server.cc_modules.cc_client_api_core.ServerRecord][source]

Searches for client PK values (for this device, current, and ‘now’) matching the input list.

Parameters
Returns

a dictionary mapping client_pk to a ServerRecord objects, for those records that match

camcops_server.cc_modules.client_api.commit_all(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails) → None[source]

Commits additions, removals, and preservations for all tables.

Parameters
camcops_server.cc_modules.client_api.commit_table(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, clear_dirty: bool = True) → camcops_server.cc_modules.cc_client_api_core.UploadTableChanges[source]

Commits additions, removals, and preservations for one table.

Should ONLY be called by commit_all().

Also updates task indexes.

Parameters

  • req – the camcops_server.cc_modules.cc_request.CamcopsRequest

  • batchdetails – the BatchDetails

  • table – SQLAlchemy Table

  • clear_dirty – remove the table from the record of dirty tables for this device? (If called from commit_all(), this should be False, since it’s faster to clear all dirty tables for the device simultaneously than one-by-one.)

Returns

an UploadTableChanges object

camcops_server.cc_modules.client_api.end_device_upload_batch(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails) → None[source]

Ends an upload batch, committing all changes made thus far.

Parameters
camcops_server.cc_modules.client_api.ensure_string(value: Any, allow_none: bool = True) → None[source]

Used when processing JSON information about patients: ensures that a value is a string, or raises.

Parameters

  • value – value to test

  • allow_none – is None allowed (not just an empty string)?

camcops_server.cc_modules.client_api.ensure_valid_field_name(table: sqlalchemy.sql.schema.Table, fieldname: str) → None[source]

Ensures a field name contains only valid characters, and isn’t a reserved fieldname that the user isn’t allowed to access.

Raises UserErrorException upon failure.

  • 2017-10-08: shortcut: it’s OK if it’s a column name for a particular table.

camcops_server.cc_modules.client_api.ensure_valid_patient_json(req: CamcopsRequest, group: camcops_server.cc_modules.cc_group.Group, pt_dict: Dict[str, Any]) → None[source]

Ensures that the JSON dictionary contains valid patient details (valid for the group into which it’s being uploaded), and that (if applicable) this user is allowed to upload this patient.

Parameters
Raises

UserErrorException –

camcops_server.cc_modules.client_api.ensure_valid_table_name(req: CamcopsRequest, tablename: str) → None[source]

Ensures a table name:

  • doesn’t contain bad characters,

  • isn’t a reserved table that the user is prohibited from accessing, and

  • is a valid table name that’s in the database.

Raises UserErrorException upon failure.

  • 2017-10-08: shortcut to all that: it’s OK if it’s listed as a valid client table.

  • 2018-01-16 (v2.2.0): check also that client version is OK

camcops_server.cc_modules.client_api.flag_all_records_deleted(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table) → int[source]

Marks all records in a table as deleted (that are current and in the current era).

Returns the number of rows affected.

camcops_server.cc_modules.client_api.flag_deleted(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, pklist: Iterable[int]) → None[source]

Marks record(s) as deleted, specified by a list of server PKs within a table. (Note: “deleted” means “deleted with no successor”, not “modified and replaced by a successor record”.)

camcops_server.cc_modules.client_api.flag_deleted_where_clientpk_not(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, clientpk_name: str, clientpk_values: Sequence[Any]) → None[source]

Marks for deletion all current/current-era records for a device, within a specific table, defined by a list of client-side PK values (and the name of the client-side PK column).

camcops_server.cc_modules.client_api.flag_modified(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, pk: int, successor_pk: int) → None[source]

Marks a record as old, storing its successor’s details.

Parameters
camcops_server.cc_modules.client_api.flag_multiple_records_for_preservation(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, pks_to_preserve: List[int]) → None[source]

Low-level function to mark records for preservation by server PK. Does not concern itself with the predecessor chain (for which, see flag_record_for_preservation()).

Parameters
camcops_server.cc_modules.client_api.flag_record_for_preservation(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, pk: int) → List[int][source]

Marks a record for preservation (moving off the tablet, changing its era details).

2018-11-18: works back through the predecessor chain too, fixing an old bug.

Parameters
Returns

all PKs being preserved

Return type

list

camcops_server.cc_modules.client_api.get_all_predecessor_pks(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, last_pk: int, include_last: bool = True) → List[int][source]

Retrieves the PKs of all records that are predecessors of the specified one

Parameters
Returns

the PKs

camcops_server.cc_modules.client_api.get_batch_details(req: CamcopsRequest) → camcops_server.cc_modules.cc_client_api_core.BatchDetails[source]

Returns the BatchDetails for the current upload. If none exists, a new batch is created and returned.

SIDE EFFECT: if the username is different from the username that started a previous upload batch for this device, we restart the upload batch (thus rolling back previous pending changes).

Raises
camcops_server.cc_modules.client_api.get_bool_int_var(req: CamcopsRequest, var: str) → bool[source]

Retrieves a Boolean variable (encoded as an integer) from the CamcopsRequest. Zero represents false; nonzero represents true.

Parameters
Returns

value

Raises
camcops_server.cc_modules.client_api.get_dirty_tables(req: CamcopsRequest) → List[sqlalchemy.sql.schema.Table][source]

Returns tables marked as dirty for this device. (See mark_table_dirty().)

camcops_server.cc_modules.client_api.get_fields_and_values(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, fields_var: str, values_var: str, mandatory: bool = True) → Dict[str, Any][source]

Gets fieldnames and matching values from two variables in a request.

See get_fields_from_post_var(), get_values_from_post_var().

Parameters

  • req – the camcops_server.cc_modules.cc_request.CamcopsRequest

  • table – SQLAlchemy Table in which the columns should exist

  • fields_var – name of CSV “column names” variable to retrieve

  • values_var – name of CSV “corresponding values” variable to retrieve

  • mandatory – if True, raise an exception if the variable is missing

Returns

a dictionary mapping column names to decoded values

Raises

  • UserErrorException –

  • no value was provided, or if any field was not valid for the specified –

  • table –

camcops_server.cc_modules.client_api.get_fields_from_post_var(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, var: str, mandatory: bool = True, allowed_nonexistent_fields: List[str] = None) → List[str][source]

Get a comma-separated list of field names from a request and checks that all are acceptable. Returns a list of fieldnames.

Parameters

  • req – the camcops_server.cc_modules.cc_request.CamcopsRequest

  • table – SQLAlchemy Table in which the columns should exist

  • var – name of variable to retrieve

  • mandatory – if True, raise an exception if the variable is missing

  • allowed_nonexistent_fields – fields that are allowed to be in the upload but not in the database (special exemptions!)

Returns

a list of the field (column) names

Raises

  • UserErrorException –

  • no value was provided, or if any field was not valid for the specified –

  • table –

camcops_server.cc_modules.client_api.get_int_var(req: CamcopsRequest, var: str) → int[source]

Retrieves an integer variable from the CamcopsRequest.

Parameters
Returns

value

Raises
camcops_server.cc_modules.client_api.get_json_from_post_var(req: CamcopsRequest, key: str, decoder: json.decoder.JSONDecoder = None, mandatory: bool = True) → Any[source]

Returns a Python object from a JSON-encoded value.

Parameters
Returns

Python object, e.g. a list of values, or None if the object is invalid and not mandatory

Raises
camcops_server.cc_modules.client_api.get_or_create_single_user(req: CamcopsRequest, name: str, patient: camcops_server.cc_modules.cc_patient.Patient) → Tuple[camcops_server.cc_modules.cc_user.User, str][source]

Creates a user for a patient (who’s using single-user mode).

The user is associated (via its name) with the combination of a client device and a patient. (If a device is re-registered to another patient, the username will change.)

If the username already exists, then since we can’t look up the password (it’s irreversibly encrypted), we will set it afresh.

  • Why is a user associated with a patient? So we can enforce that the user can upload only data relating to that patient.

  • Why is a user associated with a device?

    • If it is: then two users (e.g. “Device1-Bob” and “Device2-Bob”) can independently work with the same patient. This will be highly confusing (mainly because it will allow “double” copies of tasks to be created, though only by manually entering things twice).

    • If it isn’t (e.g. user “Bob”): then, because registering the patient on Device2 will reset the password for the user, registering a new device for a patient will “take over” from a previous device. That has some potential for data loss if work was in progress (incomplete tasks won’t be uploadable any more, and re-registering [to fix the password on the first device] would delete data).

    • Since some confusion is better than some data loss, we associate users with a device/patient combination.

Parameters
Returns

camcops_server.cc_modules.cc_user.User, password

Return type

tuple

camcops_server.cc_modules.client_api.get_select_reply(fields: Sequence[str], rows: Sequence[Sequence[Any]]) → Dict[str, str][source]

Formats the result of a SELECT query for the client as a dictionary reply.

Parameters

  • fields – list of field names

  • rows – list of rows, where each row is a list of values in the same order as fields

Returns

{
    "nfields": NUMBER_OF_FIELDS,
    "fields": FIELDNAMES_AS_CSV,
    "nrecords": NUMBER_OF_RECORDS,
    "record0": VALUES_AS_CSV_LIST_OF_ENCODED_SQL_VALUES,
        ...
    "record{nrecords - 1}": VALUES_AS_CSV_LIST_OF_ENCODED_SQL_VALUES
}

Return type

a dictionary of the format

The final reply to the server is then formatted as text as per client_api().

camcops_server.cc_modules.client_api.get_server_id_info(req: CamcopsRequest) → Dict[str, str][source]

Returns a reply for the tablet, as a variable-to-value dictionary, giving details of the server.

camcops_server.cc_modules.client_api.get_single_field_from_post_var(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, var: str, mandatory: bool = True) → str[source]

Retrieves a field (column) name from a the request and checks it’s not a bad fieldname for the specified table.

Parameters
Returns

the field (column) name

Raises

  • UserErrorException –

  • no value was provided, or if the field was not valid for the specified –

  • table –

camcops_server.cc_modules.client_api.get_single_server_patient(req: CamcopsRequest) → camcops_server.cc_modules.cc_patient.Patient[source]

Returns the patient identified by the proquint access key present in this request, or raises.

camcops_server.cc_modules.client_api.get_str_var(req: CamcopsRequest, var: str, mandatory: bool = True, validator: Callable[[str, Optional[CamcopsRequest]], None] = <function validate_anything>) → Optional[str][source]

Retrieves a string variable from the CamcopsRequest.

By default this performs no validation (because, for example, these strings can contain SQL-encoded data or JSON), but there are a number of subsequent operation-specific validation steps.

Parameters
Returns

value

Raises
camcops_server.cc_modules.client_api.get_table_from_req(req: CamcopsRequest, var: str) → sqlalchemy.sql.schema.Table[source]

Retrieves a table name from a HTTP request, checks it’s a valid client table, and returns that table.

Parameters
Returns

a SQLAlchemy Table

Raises

UserErrorException –

camcops_server.cc_modules.client_api.get_tables_from_post_var(req: CamcopsRequest, var: str, mandatory: bool = True) → List[sqlalchemy.sql.schema.Table][source]

Gets a list of tables from an HTTP request variable, and ensures all are valid.

Parameters
Returns

a list of SQLAlchemy Table objects

Raises

  • UserErrorException –

  • no value was provided, or if one or more tables was not valid –

camcops_server.cc_modules.client_api.get_task_schedules(req: CamcopsRequest, patient: camcops_server.cc_modules.cc_patient.Patient) → str[source]

Gets a JSON string representation of the task schedules for a specified patient.

camcops_server.cc_modules.client_api.get_values_from_post_var(req: CamcopsRequest, var: str, mandatory: bool = True) → List[Any][source]

Retrieves a list of values from a CSV-separated list of SQL values stored in a CGI form (including e.g. NULL, numbers, quoted strings, and special handling for base-64/hex-encoded BLOBs.)

Parameters
camcops_server.cc_modules.client_api.insert_record(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, valuedict: Dict[str, Any], predecessor_pk: Optional[int]) → int[source]

Inserts a record, or raises an exception if that fails.

Parameters

  • req – the camcops_server.cc_modules.cc_request.CamcopsRequest

  • batchdetails – the BatchDetails

  • table – an SQLAlchemy Table

  • valuedict – a dictionary of {colname: value} pairs from the client

  • predecessor_pk – an optional server PK of the record’s predecessor

Returns

the server PK of the new record

camcops_server.cc_modules.client_api.json_patient_info(patient: camcops_server.cc_modules.cc_patient.Patient) → str[source]

Converts patient details to a string representation of a JSON list (one patient) containing a single JSON dictionary (detailing that patient), with keys/formats known to the client.

(One item list to be consistent with patients uploaded from the tablet.)

Parameters

patient – camcops_server.cc_modules.cc_patient.Patient

camcops_server.cc_modules.client_api.main_client_api(req: CamcopsRequest) → Dict[str, str][source]

Main HTTP processor.

For success, returns a dictionary to send (will use status ‘200 OK’) For failure, raises an exception.

camcops_server.cc_modules.client_api.make_single_user_mode_username(client_device_name: str, patient_pk: int) → str[source]

Returns the username for single-user mode.

camcops_server.cc_modules.client_api.mark_all_tables_dirty(req: CamcopsRequest) → None[source]

If we are preserving, we assume that all tables are “dirty” (require work when we complete the upload) unless we specifically mark them clean.

camcops_server.cc_modules.client_api.mark_table_clean(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table) → None[source]

Marks a table as being clean: that is,

  • the table has been scanned during the current upload

  • there is nothing to do (either from the current upload, OR A PREVIOUS UPLOAD).

camcops_server.cc_modules.client_api.mark_table_dirty(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table) → None[source]

Marks a table as having been modified during the current upload.

camcops_server.cc_modules.client_api.mark_tables_clean(req: CamcopsRequest, tables: List[sqlalchemy.sql.schema.Table]) → None[source]

Marks multiple tables as clean.

camcops_server.cc_modules.client_api.mark_tables_dirty(req: CamcopsRequest, tables: List[sqlalchemy.sql.schema.Table]) → None[source]

Marks multiple tables as dirty.

camcops_server.cc_modules.client_api.op_check_device_registered(req: CamcopsRequest) → None[source]

Check that a device is registered, or raise UserErrorException.

camcops_server.cc_modules.client_api.op_check_upload_user_and_device(req: CamcopsRequest) → None[source]

Stub function for the operation to check that a user is valid.

To get this far, the user has to be valid, so this function doesn’t actually have to do anything.

camcops_server.cc_modules.client_api.op_delete_where_key_not(req: CamcopsRequest) → str[source]

Marks records for deletion, for a device/table, where the client PK is not in a specified list.

camcops_server.cc_modules.client_api.op_end_upload(req: CamcopsRequest) → None[source]

Ends an upload and commits changes.

camcops_server.cc_modules.client_api.op_get_allowed_tables(req: CamcopsRequest) → Dict[str, str][source]

Returns the names of all possible tables on the server, each paired with the minimum client (tablet) version that will be accepted for each table. (Typically these are all the same as the minimum global tablet version.)

Uses the SELECT-like syntax (see get_select_reply()).

camcops_server.cc_modules.client_api.op_get_extra_strings(req: CamcopsRequest) → Dict[str, str][source]

Fetch all local extra strings from the server.

Returns

a SELECT-style reply (see get_select_reply()) for the extra-string table

camcops_server.cc_modules.client_api.op_get_id_info(req: CamcopsRequest) → Dict[str, Any][source]

Fetch server ID information; see get_server_id_info().

camcops_server.cc_modules.client_api.op_get_task_schedules(req: CamcopsRequest) → Dict[str, str][source]

Return details of the task schedules for the patient associated with this request, for single-user mode. Also returns details of the single patient, in case that’s changed.

camcops_server.cc_modules.client_api.op_register_device(req: CamcopsRequest) → Dict[str, Any][source]

Register a device with the server.

Returns

server information dictionary (from get_server_id_info())

camcops_server.cc_modules.client_api.op_register_patient(req: CamcopsRequest) → Dict[str, Any][source]

Registers a patient. That is, the client provides an access key. If all is well, the server returns details of that patient, as well as key server parameters, plus (if required) the username/password to use.

camcops_server.cc_modules.client_api.op_start_preservation(req: CamcopsRequest) → str[source]

Marks this upload batch as one in which all records will be preserved (i.e. moved from NOW-era to an older era, so they can be deleted safely from the tablet).

Without this, individual records can still be marked for preservation if their MOVE_OFF_TABLET_FIELD field (_move_off_tablet) is set; see upload_record() and the functions it calls.

camcops_server.cc_modules.client_api.op_start_upload(req: CamcopsRequest) → None[source]

Begin an upload.

camcops_server.cc_modules.client_api.op_upload_empty_tables(req: CamcopsRequest) → str[source]

The tablet supplies a list of tables that are empty at its end, and we will ‘wipe’ all appropriate tables; this reduces the number of HTTP requests.

camcops_server.cc_modules.client_api.op_upload_entire_database(req: CamcopsRequest) → str[source]

Perform a one-step upload of the entire database.

  • From v2.3.0.

  • Therefore, we do not have to cope with old-style ID numbers.

camcops_server.cc_modules.client_api.op_upload_record(req: CamcopsRequest) → str[source]

Upload an individual record. (Typically used for BLOBs.) Incoming POST information includes a CSV list of fields and a CSV list of values.

camcops_server.cc_modules.client_api.op_upload_table(req: CamcopsRequest) → str[source]

Upload a table.

Incoming information in the POST request includes a CSV list of fields, a count of the number of records being provided, and a set of variables named record0 … record{nrecords - 1}, each containing a CSV list of SQL-encoded values.

Typically used for smaller tables, i.e. most except for BLOBs.

camcops_server.cc_modules.client_api.op_validate_patients(req: CamcopsRequest) → str[source]

As of v2.3.0, the client can use this command to validate patients against arbitrary server criteria – definitely the upload/finalize ID policies, but potentially also other criteria of the server’s (like matching against a bank of predefined patients).

Compare NetworkManager::getPatientInfoJson() on the client.

There is a slight weakness with respect to “single-patient” users, in that the client asks if the patients are OK (rather than the server enforcing that they are OK, via hooks into op_upload_table(), op_upload_record(), op_upload_entire_database() – made more complex because ID numbers are not uploaded to the same table…). In principle, the weakness is that a user could (a) crack their assigned password and (b) rework the CamCOPS client, in order to upload “bad” patient data into their assigned group.

Todo

address this by having the server require patient validation for all uploads?

camcops_server.cc_modules.client_api.op_which_keys_to_send(req: CamcopsRequest) → str[source]

Intended use: “For my device, and a specified table, here are my client-side PKs (as a CSV list), and the modification dates for each corresponding record (as a CSV list). Please tell me which records have mismatching dates on the server, i.e. those that I need to re-upload.”

Used particularly for BLOBs, to reduce traffic, i.e. so we don’t have to send a lot of BLOBs.

Note new TabletParam.MOVE_OFF_TABLET_VALUES parameter in server v2.3.0, with bugfix for pre-2.3.0 clients that won’t send this; see changelog.

camcops_server.cc_modules.client_api.preserve_all(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table) → None[source]

Preserves all records in a table for a device, including non-current ones.

Parameters
camcops_server.cc_modules.client_api.process_table_for_onestep_upload(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, clientpk_name: str, rows: List[Dict[str, Any]]) → camcops_server.cc_modules.cc_client_api_core.UploadTableChanges[source]

Performs all upload steps for a table.

Note that we arrive here in a specific and safe table order; search for camcops_server.cc_modules.cc_client_api_helpers.upload_commit_order_sorter().

Parameters

  • req – the camcops_server.cc_modules.cc_request.CamcopsRequest

  • batchdetails – the BatchDetails

  • table – an SQLAlchemy Table

  • clientpk_name – the name of the PK field on the client

  • rows – a list of rows, where each row is a dictionary mapping field (column) names to values (those values being encoded as SQL-style literals in our extended syntax)

Returns

an UploadTableChanges object

camcops_server.cc_modules.client_api.process_upload_record_special(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, valuedict: Dict[str, Any]) → None[source]

Special processing function for upload, in which we inspect the data. Called by upload_record_core().

  1. Handles old clients with ID information in the patient table, etc. (Note: this can be IGNORED for any client using op_upload_entire_database(), as these are newer.)

  2. Validates ID numbers.

Parameters
camcops_server.cc_modules.client_api.random_password(length: int = 32) → str[source]

Create a random password.

camcops_server.cc_modules.client_api.record_exists(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table, clientpk_name: str, clientpk_value: Any) → camcops_server.cc_modules.cc_client_api_core.ServerRecord[source]

Checks if a record exists, using the device’s perspective of a table/client PK combination.

Parameters
Returns

a ServerRecord with the required information

camcops_server.cc_modules.client_api.rollback_all(req: CamcopsRequest) → None[source]

Rolls back all pending changes for a device.

camcops_server.cc_modules.client_api.rollback_table(req: CamcopsRequest, table: sqlalchemy.sql.schema.Table) → None[source]

Rolls back changes for an individual table for a device.

camcops_server.cc_modules.client_api.start_device_upload_batch(req: CamcopsRequest) → None[source]

Starts an upload batch for a device.

camcops_server.cc_modules.client_api.start_preserving(req: CamcopsRequest) → None[source]

Starts preservation (the process of moving records from the NOW era to an older era, so they can be removed safely from the tablet).

Called by op_start_preservation().

In this situation, we start by assuming that ALL tables are “dirty”, because they may have live records from a previous upload.

camcops_server.cc_modules.client_api.upload_record_core(req: CamcopsRequest, batchdetails: camcops_server.cc_modules.cc_client_api_core.BatchDetails, table: sqlalchemy.sql.schema.Table, clientpk_name: str, valuedict: Dict[str, Any], server_live_current_records: List[camcops_server.cc_modules.cc_client_api_core.ServerRecord] = None) → camcops_server.cc_modules.cc_client_api_core.UploadRecordResult[source]

Uploads a record. Deals with IDENTICAL, NEW, and MODIFIED records.

Used by upload_table() and upload_record().

Parameters

  • req – the camcops_server.cc_modules.cc_request.CamcopsRequest

  • batchdetails – the BatchDetails

  • table – an SQLAlchemy Table

  • clientpk_name – the column name of the client’s PK

  • valuedict – a dictionary of {colname: value} pairs from the client

  • server_live_current_records – list of ServerRecord objects for the active records on the server for this client, in this table

Returns

a UploadRecordResult object