.. docs/source/administrator/redcap.rst
.. Copyright (C) 2012, University of Cambridge, Department of Psychiatry.
Created by Rudolf Cardinal (rnc1001@cam.ac.uk).
.
This file is part of CamCOPS.
.
CamCOPS is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
.
CamCOPS is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
.
You should have received a copy of the GNU General Public License
along with CamCOPS. If not, see .
.. _asteval: http://newville.github.io/asteval/index.html
.. _redcap:
REDCap export
=============
.. contents::
:local:
:depth: 3
Overview
--------
REDCap is a secure web application for building and managing online surveys and
databases. For general information about REDCap, see
https://www.project-redcap.org/.
You may wish to run a study from REDCap, designing study-specific
questionnaires and other forms. However, you might want to use CamCOPS for its
tasks and user interface. You can set up CamCOPS to export to REDCap.
For an overview of CamCOPS's export functions, see :ref:`export options
`.
REDCap concepts
---------------
- **Project:** the basic security grouping. Represents a research study.
- **Arms:** not an abbreviation. Groups study events into a sequence (an "arm"
of a study). See
https://labkey.med.ualberta.ca/labkey/wiki/REDCap%20Support/page.view?name=rcarms.
- **Instruments:** what we call tasks in CamCOPS. Data entry forms.
- **Metadata/data dictionary:** you can download all the fields used by the
project.
- **REDCap Shared Library:** a collection of public instruments.
- **Record:** for our purposes, a record corresponds to a subject (person).
REDCap has no internal concept of a subject *per se*. A record can have
multiple instruments, some of which may be **"repeating instruments"**,
allowing the same form to be submitted at different times. Each record has an
ID, which exists on one of the instruments. Record IDs need not be numeric
and can be generated by the client or automatically assigned by the server.
- **Classic projects** don't have the "longitudinal" option enabled (see
below). Typically, this is for when you assess each subject once.
- **Longitudinal projects** have the longitudinal module enabled. Typically,
this is used when you assess each subject a predefined number of times, such
as 5. Required for "repeating events".
- **Repeating instruments:** when an instrument can be performed any number of
times for a given record. For example, if an instrument captures a single
medication that the subject is taking, you could have 0, 1, or 17 different
instances of this instrument for one subject. May be used with classic and
longitudinal projects.
- **Events (repeating events):** when you collect instruments together such as
for a "study visit" -- every visit (event) might include a blood pressure
instrument, a weight instrument, and a questionnaire instrument. Requires a
longitudinal project. Events may be grouped into "arms" (q.v.).
Example field structure
-----------------------
What does this look like in practice?
For a **basic project:**
- You may or may not have authority to create projects yourself. (At some
institutions, only superusers can create projects.) If not, your
administrator will have to create one for you.
- REDCap will always create an initial instrument containing a ``record_id``
field. Its help states that "[t]he very first field in the first data
collection instrument must be used as the record identifier and must be
unique. The record identifiers can be automatically sequentially[ ]generated
or user-entered. The record identifier field cannot be deleted[;] however[,]
you may rename this field by clicking on the pencil icon."
- Suppose you start with a "blank" project containing just the record ID, and
you import a standard instrument from the official REDCap Shared Library.
Let's say you pick "Patient Health Questionnaire 9". (Note that there are
several with similar names.) This will add an instrument with the following
14 fields:
.. code-block:: none
phq9_date_enrolled
phq9_first_name
phq9_last_name
phq9_1
...
phq9_9
phq9_total_score # "Enter Total Score:"
phq9_how_difficult
- If you import the same task again, the field names in the new copy are
altered so as to be unique, e.g. like this:
.. code-block:: none
phq9_date_enrolled_f21b86
phq9_first_name_429d79
phq9_last_name_17b04a
phq9_1_cf6471
...
phq9_9_e40b3f
phq9_total_score_d039bc
phq9_how_difficult_a9487f
As you can see, this gives a flat field structure; one record can be
represented as a single row of a database table.
If you are using **longitudinal data collection** (see above)...
- It is intended for repetition a specific number of times (e.g. you know that
your participants will do a instrument 5 times).
- This is set up via :menuselection:`Project Setup --> Main project settings
--> Use longitudinal data collection with defined events?`.
- It's then configured via :menuselection:`Project Setup --> Define your events
and designate instruments for them`.
- You can assign any of your instruments to any of your events. (For example:
"During Event 1 (Visit 1), I would like everyone to have their blood pressure,
weight, and height recorded, and my questionnaire completed. On Visit 2,
I just want to record their blood pressure and their weight.")
- In reports and data exports, a field called ``redcap_event_name`` records
the event associated with a record.
If you are using **repeating instruments** (see above)...
- This is set up via :menuselection:`Project Setup --> Enable optional modules
and customizations --> Repeatable instruments`.
- You can then choose to repeat entire events (if you are using events) -- so
that all instruments in an event repeat together.
- Alternatively, you can repeat individual instruments within an event (for
example, to record an arbitrary number of medications at a single visit).
- This feature supports unlimited repetition; there may be any number of a
given kind of repeating instrument (including zero) for a given record.
For both...
- In reports and data exports, a field called ``redcap_repeat_instance`` is
created (autonumbered from 1) to number data from repeating instruments or
events. If the data is from a repeating instrument, an extra field called
``redcap_repeat_instrument`` stores the name of the instrument.
So in the most complex sort of setup, with both events and repeating
instruments, an export file might look a little like this:
.. code-block:: none
record_id redcap_event_name redcap_repeat_instrument redcap_repeat_instance patient_id phq9_date_enrolled ...
------------------------------------------------------------------------------------------------------------------
2 event_1_arm_1 ... ... ... ...
Cells are blank if an instrument wasn't used in a particular row.
The field ``record_id`` is not unique. Repeating instruments appear to cluster
within a single ``record_id``, which can have a row with no
``redcap_repeat_instrument`` or ``redcap_repeat_instance``, and some information
filled in, then other rows (with different information) for repeat instance
details.
Prerequisites for exporting CamCOPS tasks to REDCap records
-----------------------------------------------------------
- There must be a REDCap user (with API key) with the following privileges:
- API Import/Update
- API Export
- The CamCOPS configuration file needs to have an export recipient with
``TRANSMISSION_METHOD = redcap`` (see :ref:`The CamCOPS server configuration
file `) and :ref:`REDCap specific settings
` .
- Any instruments in a REDCap record to which CamCOPS exports must be
set up as "repeating" within the REDCap interface (under "Project Setup").
- There must be a separate, non-repeating instrument with a field to store
the patient identifier.
- There must be a fieldmap XML file that tells CamCOPS how the task fields
correspond to the REDCap instrument form fields. Its format is described
below.
- If you are using events, you must specify the name(s) of the event(s) in
the fieldmap XML file. The instruments themselves can repeat within the
event but the entire event cannot be repeating.
.. note::
We don't know of a way that REDCap can tell us dates associated with an
event. So, CamCOPS can't say to REDCap: "Which event was scheduled for
1 June 2020?" and then upload to that one.
Example fieldmap XML file
-------------------------
The fieldmap below describes how three CamCOPS tasks are exported to a REDCap
record for a patient.
There is a tag named ``patient`` that tells CamCOPS where in REDCap to find
or store the patient identifier. In the example, this is in a field named
``patient_id`` in a REDCap instrument named ``patient_record``.
There is a tag named ``record`` that tells CamCOPS where in REDCap to
find or store the record identifier. The field name should be ``record_id``
unless you have changed the name of this field from the default in REDCap.
The instrument should be the one in REDCap where the ``record_id`` field
is defined.
This particular REDCap record has three other instruments, defined within the
``instruments`` tag. Each has a REDCap name (``name`` attribute) and a
corresponding CamCOPS task (``task`` attribute).
If your REDCap project has events, there needs to be a global tag named
``event`` or an ``event`` attribute on the ``instrument`` tag to tell REDCap
which event this record belongs to. If both the global tag and an event
attribute exist, CamCOPS will use the event named in the attribute.
In this example the REDCap instruments are named ``bmi``,
``patient_health_questionnaire_9``, and ``medication_table``, and they
correspond to CamCOPS tasks :ref:`bmi `, :ref:`phq9 ` and
:ref:`khandaker_mojo_medicationtherapy `
respectively.
.. code-block:: xml
Each ``field`` tag describes how the answer for a question will be stored in
REDCap. The ``name`` attribute is the column in the REDCap database table for
the task. The ``formula`` attribute is Python code to generate the REDCap value
before it is exported.
For the formula we use the library `asteval`_, a safer version of Python's
``eval()``. Whilst ``asteval`` tries hard to prevent the Python interpreter
from crashing, it is still possible to write potentially destructive code, so
use this with care and at your own risk!
See the `asteval`_ documentation for supported operations (`numpy
`_ is available). In addition we provide access to the
following symbols:
- ``task`` (the row in the database that contains the answers)
- ``format_datetime`` (:func:`cardinal_pythonlib.datetimefunc.format_datetime`,
a function for date formatting)
- ``DateFormat`` (:class:`camcops_server.cc_modules.cc_constants.DateFormat`)
- ``request`` (the CamCOPS request object, required by a number of functions; a
:class:`camcops_server.cc_modules.cc_request.CamcopsRequest`)
We'll explore each of the three task examples in more detail.
PHQ-9 example
~~~~~~~~~~~~~
In the PHQ-9 example, the nine main questions are simply copied over to REDCap
with no processing:
.. code-block:: xml
The possible answers for the tenth question (known as ``phq9_how_difficult`` on
the REDCap side) have been coded differently in REDCap (1-4 instead of 0-3) so
we need to add one to the answer.
The total score in REDCap is stored (rather than calculated, as in CamCOPS), so
to fill in this field we call the method
:meth:`camcops_server.tasks.Phq9.total_score`.
The REDCap PHQ9 instrument also stores the first and last names of the patient.
We can retrieve these from the patient
(:class:`camcops_server.cc_modules.cc_patient.Patient`) associated with the
task.
Finally, the REDCap PHQ9 instrument has a date field (``phq9_date_enrolled``),
which needs to be in ISO8601 (``yyyy-mm-dd``) format.
BMI example
~~~~~~~~~~~
In the BMI example, the height and weight fields need to be specified to one
decimal place in REDCap so we use the Python built-in ``format()`` function. In
addition, the REDCap instrument records the height in centimetres so we need to
multiply the value in metres by 100.
.. code-block:: xml
Related table/file upload example
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The final instrument in the fieldmap shows one way that a task with a
one-to-many related table can be uploaded to REDCap. The
:ref:`khandaker_mojo_medicationtherapy ` task
has a table of medications (name, dose, frequency, etc.) with multiple entries,
one for each medication. REDCap does not have direct support for this kind of
one-to-many relationship. One workaround is simply to upload a PDF of the task
contents. We achieve this by creating a file upload field in REDCap and
populating this with the output of the method
:meth:`camcops_server.cc_modules.cc_task.Task.get_pdf`.
.. code-block:: xml