12.14. REDCap export¶
12.14.1. Overview¶
REDCap is a secure web application for building and managing online surveys and databases. For general information about REDCap, see https://www.project-redcap.org/.
You may wish to run a study from REDCap, designing study-specific questionnaires and other forms. However, you might want to use CamCOPS for its tasks and user interface. You can set up CamCOPS to export to REDCap.
For an overview of CamCOPS’s export functions, see export options.
12.14.2. REDCap concepts¶
Project: the basic security grouping. Represents a research study.
Arms: not an abbreviation. Groups study events into a sequence (an “arm” of a study). See https://labkey.med.ualberta.ca/labkey/wiki/REDCap%20Support/page.view?name=rcarms.
Instruments: what we call tasks in CamCOPS. Data entry forms.
Metadata/data dictionary: you can download all the fields used by the project.
REDCap Shared Library: a collection of public instruments.
Record: for our purposes, a record corresponds to a subject (person). REDCap has no internal concept of a subject per se. A record can have multiple instruments, some of which may be “repeating instruments”, allowing the same form to be submitted at different times. Each record has an ID, which exists on one of the instruments. Record IDs need not be numeric and can be generated by the client or automatically assigned by the server.
Classic projects don’t have the “longitudinal” option enabled (see below). Typically, this is for when you assess each subject once.
Longitudinal projects have the longitudinal module enabled. Typically, this is used when you assess each subject a predefined number of times, such as 5. Required for “repeating events”.
Repeating instruments: when an instrument can be performed any number of times for a given record. For example, if an instrument captures a single medication that the subject is taking, you could have 0, 1, or 17 different instances of this instrument for one subject. May be used with classic and longitudinal projects.
Events (repeating events): when you collect instruments together such as for a “study visit” – every visit (event) might include a blood pressure instrument, a weight instrument, and a questionnaire instrument. Requires a longitudinal project. Events may be grouped into “arms” (q.v.).
12.14.3. Example field structure¶
What does this look like in practice?
For a basic project:
You may or may not have authority to create projects yourself. (At some institutions, only superusers can create projects.) If not, your administrator will have to create one for you.
REDCap will always create an initial instrument containing a
record_idfield. Its help states that “[t]he very first field in the first data collection instrument must be used as the record identifier and must be unique. The record identifiers can be automatically sequentially[ ]generated or user-entered. The record identifier field cannot be deleted[;] however[,] you may rename this field by clicking on the pencil icon.”Suppose you start with a “blank” project containing just the record ID, and you import a standard instrument from the official REDCap Shared Library. Let’s say you pick “Patient Health Questionnaire 9”. (Note that there are several with similar names.) This will add an instrument with the following 14 fields:
phq9_date_enrolled phq9_first_name phq9_last_name phq9_1 ... phq9_9 phq9_total_score # "Enter Total Score:" phq9_how_difficult
If you import the same task again, the field names in the new copy are altered so as to be unique, e.g. like this:
phq9_date_enrolled_f21b86 phq9_first_name_429d79 phq9_last_name_17b04a phq9_1_cf6471 ... phq9_9_e40b3f phq9_total_score_d039bc phq9_how_difficult_a9487f
As you can see, this gives a flat field structure; one record can be represented as a single row of a database table.
If you are using longitudinal data collection (see above)…
It is intended for repetition a specific number of times (e.g. you know that your participants will do a instrument 5 times).
This is set up via .
It’s then configured via .
You can assign any of your instruments to any of your events. (For example: “During Event 1 (Visit 1), I would like everyone to have their blood pressure, weight, and height recorded, and my questionnaire completed. On Visit 2, I just want to record their blood pressure and their weight.”)
In reports and data exports, a field called
redcap_event_namerecords the event associated with a record.
If you are using repeating instruments (see above)…
This is set up via .
You can then choose to repeat entire events (if you are using events) – so that all instruments in an event repeat together.
Alternatively, you can repeat individual instruments within an event (for example, to record an arbitrary number of medications at a single visit).
This feature supports unlimited repetition; there may be any number of a given kind of repeating instrument (including zero) for a given record.
For both…
In reports and data exports, a field called
redcap_repeat_instanceis created (autonumbered from 1) to number data from repeating instruments or events. If the data is from a repeating instrument, an extra field calledredcap_repeat_instrumentstores the name of the instrument.
So in the most complex sort of setup, with both events and repeating instruments, an export file might look a little like this:
record_id redcap_event_name redcap_repeat_instrument redcap_repeat_instance patient_id phq9_date_enrolled ...
------------------------------------------------------------------------------------------------------------------
2 event_1_arm_1 ... ... ... ...
Cells are blank if an instrument wasn’t used in a particular row.
The field record_id is not unique. Repeating instruments appear to cluster
within a single record_id, which can have a row with no
redcap_repeat_instrument or redcap_repeat_instance, and some information
filled in, then other rows (with different information) for repeat instance
details.
12.14.4. Prerequisites for exporting CamCOPS tasks to REDCap records¶
There must be a REDCap user (with API key) with the following privileges:
API Import/Update
API Export
The CamCOPS configuration file needs to have an export recipient with
TRANSMISSION_METHOD = redcap(see The CamCOPS server configuration file) and REDCap specific settings .Any instruments in a REDCap record to which CamCOPS exports must be set up as “repeating” within the REDCap interface (under “Project Setup”).
There must be a separate, non-repeating instrument with a field to store the patient identifier.
There must be a fieldmap XML file that tells CamCOPS how the task fields correspond to the REDCap instrument form fields. Its format is described below.
If you are using events, you must specify the name(s) of the event(s) in the fieldmap XML file. The instruments themselves can repeat within the event but the entire event cannot be repeating.
Note
We don’t know of a way that REDCap can tell us dates associated with an event. So, CamCOPS can’t say to REDCap: “Which event was scheduled for 1 June 2020?” and then upload to that one.
12.14.5. Example fieldmap XML file¶
The fieldmap below describes how three CamCOPS tasks are exported to a REDCap record for a patient.
There is a tag named patient that tells CamCOPS where in REDCap to find
or store the patient identifier. In the example, this is in a field named
patient_id in a REDCap instrument named patient_record.
There is a tag named record that tells CamCOPS where in REDCap to
find or store the record identifier. The field name should be record_id
unless you have changed the name of this field from the default in REDCap.
The instrument should be the one in REDCap where the record_id field
is defined.
This particular REDCap record has three other instruments, defined within the
instruments tag. Each has a REDCap name (name attribute) and a
corresponding CamCOPS task (task attribute).
If your REDCap project has events, there needs to be a global tag named
event or an event attribute on the instrument tag to tell REDCap
which event this record belongs to. If both the global tag and an event
attribute exist, CamCOPS will use the event named in the attribute.
In this example the REDCap instruments are named bmi,
patient_health_questionnaire_9, and medication_table, and they
correspond to CamCOPS tasks bmi, phq9 and
khandaker_mojo_medicationtherapy
respectively.
<?xml version="1.0" encoding="UTF-8"?>
<fieldmap>
<patient instrument="patient_record" redcap_field="patient_id" />
<record instrument="patient_record" redcap_field="record_id" />
<event name="event_1_arm_1" />
<instruments>
<instrument event="event_2_arm_1" task="phq9" name="patient_health_questionnaire_9">
<fields>
<field name="phq9_1" formula="task.q1" />
<field name="phq9_2" formula="task.q2" />
<field name="phq9_3" formula="task.q3" />
<field name="phq9_4" formula="task.q4" />
<field name="phq9_5" formula="task.q5" />
<field name="phq9_6" formula="task.q6" />
<field name="phq9_7" formula="task.q7" />
<field name="phq9_8" formula="task.q8" />
<field name="phq9_9" formula="task.q9" />
<field name="phq9_how_difficult" formula="task.q10 + 1 if task.q10 is not None else None" />
<field name="phq9_total_score" formula="task.total_score()" />
<field name="phq9_first_name" formula="task.patient.forename" />
<field name="phq9_last_name" formula="task.patient.surname" />
<field name="phq9_date_enrolled" formula="format_datetime(task.when_created,DateFormat.ISO8601_DATE_ONLY)" />
</fields>
</instrument>
<instrument task="bmi" name="bmi">
<fields>
<field name="pa_height" formula="format(task.height_m * 100.0, '.1f')" />
<field name="pa_weight" formula="format(task.mass_kg, '.1f')" />
<field name="bmi_date" formula="format_datetime(task.when_created, DateFormat.ISO8601_DATE_ONLY)" />
</fields>
</instrument>
<instrument task="khandaker_mojo_medicationtherapy" name="medication_table">
<files>
<field name="medtbl_medication_items" formula="task.get_pdf(request)" />
</files>
</instrument>
</instruments>
</fieldmap>
Each field tag describes how the answer for a question will be stored in
REDCap. The name attribute is the column in the REDCap database table for
the task. The formula attribute is Python code to generate the REDCap value
before it is exported.
For the formula we use the library asteval, a safer version of Python’s
eval(). Whilst asteval tries hard to prevent the Python interpreter
from crashing, it is still possible to write potentially destructive code, so
use this with care and at your own risk!
See the asteval documentation for supported operations (numpy is available). In addition we provide access to the following symbols:
task(the row in the database that contains the answers)format_datetime(cardinal_pythonlib.datetimefunc.format_datetime(), a function for date formatting)DateFormat(camcops_server.cc_modules.cc_constants.DateFormat)request(the CamCOPS request object, required by a number of functions; acamcops_server.cc_modules.cc_request.CamcopsRequest)
We’ll explore each of the three task examples in more detail.
12.14.5.1. PHQ-9 example¶
In the PHQ-9 example, the nine main questions are simply copied over to REDCap with no processing:
<instrument event="event_2_arm_1" task="phq9" name="patient_health_questionnaire_9">
<fields>
<field name="phq9_1" formula="task.q1" />
<field name="phq9_2" formula="task.q2" />
<field name="phq9_3" formula="task.q3" />
<field name="phq9_4" formula="task.q4" />
<field name="phq9_5" formula="task.q5" />
<field name="phq9_6" formula="task.q6" />
<field name="phq9_7" formula="task.q7" />
<field name="phq9_8" formula="task.q8" />
<field name="phq9_9" formula="task.q9" />
<field name="phq9_how_difficult" formula="task.q10 + 1" />
<field name="phq9_total_score" formula="task.total_score()" />
<field name="phq9_first_name" formula="task.patient.forename" />
<field name="phq9_last_name" formula="task.patient.surname" />
<field name="phq9_date_enrolled" formula="format_datetime(task.when_created,DateFormat.ISO8601_DATE_ONLY)" />
</fields>
</instrument>
The possible answers for the tenth question (known as phq9_how_difficult on
the REDCap side) have been coded differently in REDCap (1-4 instead of 0-3) so
we need to add one to the answer.
The total score in REDCap is stored (rather than calculated, as in CamCOPS), so
to fill in this field we call the method
camcops_server.tasks.Phq9.total_score().
The REDCap PHQ9 instrument also stores the first and last names of the patient.
We can retrieve these from the patient
(camcops_server.cc_modules.cc_patient.Patient) associated with the
task.
Finally, the REDCap PHQ9 instrument has a date field (phq9_date_enrolled),
which needs to be in ISO8601 (yyyy-mm-dd) format.
12.14.5.2. BMI example¶
In the BMI example, the height and weight fields need to be specified to one
decimal place in REDCap so we use the Python built-in format() function. In
addition, the REDCap instrument records the height in centimetres so we need to
multiply the value in metres by 100.
<instrument task="bmi" name="bmi">
<fields>
<field name="pa_height" formula="format(task.height_m * 100.0, '.1f')" />
<field name="pa_weight" formula="format(task.mass_kg, '.1f')" />
<field name="bmi_date" formula="format_datetime(task.when_created, DateFormat.ISO8601_DATE_ONLY)" />
</fields>
</instrument>