13.2.85. camcops_server.cc_modules.merge_db

camcops_server/cc_modules/merge_db.py


Copyright (C) 2012-2019 Rudolf Cardinal (rudolf@pobox.com).

This file is part of CamCOPS.

CamCOPS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

CamCOPS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with CamCOPS. If not, see <http://www.gnu.org/licenses/>.


Tool to merge data from one CamCOPS database into another.

Has special code to deal with old databases.

camcops_server.cc_modules.merge_db.ensure_default_group_id(trcon: cardinal_pythonlib.sqlalchemy.merge_db.TranslationContext) → None[source]

Ensure that the TranslationContext has a default_group_id key in its info dictionary. This is the ID, in the destination database, of the group to put records in where those records come from an older, pre-group-based CamCOPS database.

The user may have specified that default_group_id` on the command line. Otherwise, they may have specified a ``default_group_name, so we’ll use the ID of that group (creating it if necessary). If they specified neither, we will raise an AssertionError, because we have come to a situation where we need one or the other.

Parameters:trcon – the TranslationContext
camcops_server.cc_modules.merge_db.ensure_idnumdef(trcon: cardinal_pythonlib.sqlalchemy.merge_db.TranslationContext, which_idnum: int) → camcops_server.cc_modules.cc_idnumdef.IdNumDefinition[source]

Ensure that the destination database contains an ID number definition with the same which_idnum as in the source database, or create one.

If an ID number definition with that which_idnum was present in the source and the destination, ensure they don’t clash (i.e. ensure that they represent the same sort of ID number).

Parameters:
  • trcon – the TranslationContext
  • which_idnum – integer expressing which ID number type to look up
Returns:

the camcops_server.cc_modules.cc_idnumdef.IdNumDefinition, attached to the destination database

camcops_server.cc_modules.merge_db.ensure_no_iddef_clash(src_iddef: camcops_server.cc_modules.cc_idnumdef.IdNumDefinition, dst_iddef: camcops_server.cc_modules.cc_idnumdef.IdNumDefinition) → None[source]

Ensure that a given source and destination pair of ID number definitions, which must match on which_idnum, have the same description and short description, or raise ValueError.

Parameters:
camcops_server.cc_modules.merge_db.fetch_group_id_by_name(group_name: str, dst_session: sqlalchemy.orm.session.Session) → int[source]

Returns the group ID of the group with the specified name, in the destination session.

If there are multiple such groups, that’s a bug, and MultipleResultsFound will be raised.

If there’s no such group in the destination database with that name, one will be created, and its ID returned.

Parameters:
  • group_name – group name
  • dst_session – destination SQLAlchemy Session
Returns:

group ID in the destination database

camcops_server.cc_modules.merge_db.flush_session(dst_session: sqlalchemy.orm.session.Session) → None[source]

Flushes the destination SQLAlchemy session.

camcops_server.cc_modules.merge_db.get_dst_iddef(dst_session: sqlalchemy.orm.session.Session, which_idnum: int) → Optional[camcops_server.cc_modules.cc_idnumdef.IdNumDefinition][source]

Fetches an ID number definition from the destination database.

Parameters:
  • dst_session – destination SQLAlchemy Session
  • which_idnum – integer expressing which ID number type to look up
Returns:

an camcops_server.cc_modules.cc_idnumdef.IdNumDefinition, or None if none was found

camcops_server.cc_modules.merge_db.get_skip_tables(src_tables: List[str]) → List[cardinal_pythonlib.sqlalchemy.table_identity.TableIdentity][source]

From the list of source table names provided, return details of tables in the metadata to skip because they are not in the source database.

Also checks that some core CamCOPS tables are present in the source, or raises ValueError.

Parameters:src_tables – list of all table names in the source database
Returns:list of cardinal_pythonlib.sqlalchemy.table_identity.TableIdentity objects representing tables to skip

Note that other tables to skip are defined in merge_camcops_db().

camcops_server.cc_modules.merge_db.get_src_iddefs(src_engine: sqlalchemy.engine.base.Engine, src_tables: List[str]) → Dict[int, camcops_server.cc_modules.cc_idnumdef.IdNumDefinition][source]

Get information about all the ID number definitions in the source database.

Parameters:
  • src_engine – source SQLAlchemy Engine
  • src_tables – list of all table names in the source database
Returns:

{which_idnum: idnumdef} mappings, where each idnumdef is a camcops_server.cc_modules.cc_idnumdef.IdNumDefinition not attached to any database session

Return type:

dictionary

camcops_server.cc_modules.merge_db.group_exists(group_id: int, dst_session: sqlalchemy.orm.session.Session) → bool[source]

Does a group exist in the destination session with the specified group ID?

Parameters:
  • group_id – integer group ID
  • dst_session – destination SQLAlchemy Session
camcops_server.cc_modules.merge_db.merge_camcops_db(src: str, echo: bool, report_every: int, dummy_run: bool, info_only: bool, skip_export_logs: bool, skip_audit_logs: bool, default_group_id: Optional[int], default_group_name: Optional[str]) → None[source]

Merge an existing database (with a pre-v2 or later structure) into a comtemporary CamCOPS database.

Parameters:
  • src – source database SQLAlchemy URL
  • echo – echo the SQL that is produced?
  • report_every – provide a progress report every n records
  • dummy_run – don’t alter the destination database
  • info_only – show info, then stop
  • skip_export_logs – skip export log tables
  • skip_audit_logs – skip audit log table
  • default_group_id – integer group ID (in the destination database) to use for source records that have no group (because they come from a very old source database) but need one
  • default_group_name – group name (in the destination database) to use for source records that have no group (because they come from a very old source database) but need one
camcops_server.cc_modules.merge_db.postprocess(src_engine: sqlalchemy.engine.base.Engine, dst_session: sqlalchemy.orm.session.Session) → None[source]

Implement any extra processing after merge_db() has been called.

  • Reindexes tasks.
  • Warns you about things that need to be done manually.
Parameters:
  • src_engine – source database SQLAlchemy engine
  • dst_session – destination database SQLAlchemy session
camcops_server.cc_modules.merge_db.translate_fn(trcon: cardinal_pythonlib.sqlalchemy.merge_db.TranslationContext) → None[source]

Function to translate source objects to their destination counterparts, where special processing is required. Called as a callback from cardinal_pythonlib.sqlalchemy.merge_db.merge_db().

Parameters:trcon – the TranslationContext; all the relevant information is in here, and our function modifies its members.

This function does the following things:

  • For any records uploaded from tablets: set _group_id, if it’s blank.
  • For camcops_server.cc_modules.cc_user.User objects: if an identical user is found in the destination database, merge on it rather than creating a new one. Users with matching usernames are considered to be identical.
  • For Device objects: if an identical device is found, merge on it rather than creating a new one. Devices with matching names are considered to be identical.
  • For camcops_server.cc_modules.cc_group.Group objects: if an identical group is found, merge on it rather than creating a new one. Groups with matching names are considered to be identical.
  • For camcops_server.cc_modules.cc_patient.Patient objects: if any have ID numbers in the old format (as columns in the Patient table), convert them to the PatientIdNum system.
  • If we’re inserting a PatientIdNum, make sure there is a corresponding camcops_server.cc_modules.cc_idnumdef.IdNumDefinition, and that it’s valid.
  • If we’re merging from a more modern database with the camcops_server.cc_modules.cc_idnumdef.IdNumDefinition table, check our ID number definitions don’t conflict.
  • Check we’re not creating duplicates for anything uploaded.