11.13. REDCap export

11.13.1. Overview

REDCap is a secure web application for building and managing online surveys and databases. For general information about REDCap, see https://www.project-redcap.org/.

You may wish to run a study from REDCap, designing study-specific questionnaires and other forms. However, you might want to use CamCOPS for its tasks and user interface. You can set up CamCOPS to export to REDCap.

For an overview of CamCOPS’s export functions, see export options.

11.13.2. REDCap concepts

  • Project: the basic security grouping. Represents a research study.

  • Arms: not an abbreviation. Groups study events into a sequence (an “arm” of a study). See https://labkey.med.ualberta.ca/labkey/wiki/REDCap%20Support/page.view?name=rcarms.

  • Instruments: what we call tasks in CamCOPS. Data entry forms.

  • Metadata/data dictionary: you can download all the fields used by the project.

  • REDCap Shared Library: a collection of public instruments.

  • Record: for our purposes, a record corresponds to a subject (person). REDCap has no internal concept of a subject per se. A record can have multiple instruments, some of which may be “repeating instruments”, allowing the same form to be submitted at different times. Each record has an ID, which exists on one of the instruments. Record IDs need not be numeric and can be generated by the client or automatically assigned by the server.

  • Classic projects don’t have the “longitudinal” option enabled (see below). Typically, this is for when you assess each subject once.

  • Longitudinal projects have the longitudinal module enabled. Typically, this is used when you assess each subject a predefined number of times, such as 5. Required for “repeating events”.

  • Repeating instruments: when an instrument can be performed any number of times for a given record. For example, if an instrument captures a single medication that the subject is taking, you could have 0, 1, or 17 different instances of this instrument for one subject. May be used with classic and longitudinal projects.

  • Events (repeating events): when you collect instruments together such as for a “study visit” – every visit (event) might include a blood pressure instrument, a weight instrument, and a questionnaire instrument. Requires a longitudinal project. Events may be grouped into “arms” (q.v.).

11.13.3. Example field structure

What does this look like in practice?

For a basic project:

  • You may or may not have authority to create projects yourself. (At some institutions, only superusers can create projects.) If not, your administrator will have to create one for you.

  • REDCap will always create an initial instrument containing a record_id field. Its help states that “[t]he very first field in the first data collection instrument must be used as the record identifier and must be unique. The record identifiers can be automatically sequentially[ ]generated or user-entered. The record identifier field cannot be deleted[;] however[,] you may rename this field by clicking on the pencil icon.”

  • Suppose you start with a “blank” project containing just the record ID, and you import a standard instrument from the official REDCap Shared Library. Let’s say you pick “Patient Health Questionnaire 9”. (Note that there are several with similar names.) This will add an instrument with the following 14 fields:

    phq9_date_enrolled
    phq9_first_name
    phq9_last_name
    phq9_1
    ...
    phq9_9
    phq9_total_score  # "Enter Total Score:"
    phq9_how_difficult
    
  • If you import the same task again, the field names in the new copy are altered so as to be unique, e.g. like this:

    phq9_date_enrolled_f21b86
    phq9_first_name_429d79
    phq9_last_name_17b04a
    phq9_1_cf6471
    ...
    phq9_9_e40b3f
    phq9_total_score_d039bc
    phq9_how_difficult_a9487f
    

As you can see, this gives a flat field structure; one record can be represented as a single row of a database table.

If you are using longitudinal data collection (see above)…

  • It is intended for repetition a specific number of times (e.g. you know that your participants will do a instrument 5 times).

  • This is set up via Project Setup ‣ Main project settings ‣ Use longitudinal data collection with defined events?.

  • It’s then configured via Project Setup ‣ Define your events and designate instruments for them.

  • You can assign any of your instruments to any of your events. (For example: “During Event 1 (Visit 1), I would like everyone to have their blood pressure, weight, and height recorded, and my questionnaire completed. On Visit 2, I just want to record their blood pressure and their weight.”)

  • In reports and data exports, a field called redcap_event_name records the event associated with a record.

If you are using repeating instruments (see above)…

  • This is set up via Project Setup ‣ Enable optional modules and customizations ‣ Repeatable instruments.

  • You can then choose to repeat entire events (if you are using events) – so that all instruments in an event repeat together.

  • Alternatively, you can repeat individual instruments within an event (for example, to record an arbitrary number of medications at a single visit).

  • This feature supports unlimited repetition; there may be any number of a given kind of repeating instrument (including zero) for a given record.

For both…

  • In reports and data exports, a field called redcap_repeat_instance is created (autonumbered from 1) to number data from repeating instruments or events. If the data is from a repeating instrument, an extra field called redcap_repeat_instrument stores the name of the instrument.

So in the most complex sort of setup, with both events and repeating instruments, an export file might look a little like this:

record_id  redcap_event_name  redcap_repeat_instrument  redcap_repeat_instance  patient_id  phq9_date_enrolled ...
------------------------------------------------------------------------------------------------------------------
2          event_1_arm_1      ...                       ...                     ...         ...

Cells are blank if an instrument wasn’t used in a particular row.

The field record_id is not unique. Repeating instruments appear to cluster within a single record_id, which can have a row with no redcap_repeat_instrument or ``redcap_repeat_instance`, and some information filled in, then other rows (with different information) for repeat instance details.

11.13.4. Prerequisites for exporting CamCOPS tasks to REDCap records

  • There must be a REDCap user (with API key) with the following privileges:

    • API Import/Update

    • API Export

  • The CamCOPS configuration file needs to have an export recipient with TRANSMISSION_METHOD = redcap (see The CamCOPS server configuration file) and REDCap specific settings .

  • Any instruments in a REDCap record to which CamCOPS exports must be set up as “repeating” within the REDCap interface (under “Project Setup”).

  • There must be a separate, non-repeating instrument with a field to store the patient identifier.

  • There must be a fieldmap XML file that tells CamCOPS how the task fields correspond to the REDCap instrument form fields. Its format is described below.

  • If you are using events, you must specify the name(s) of the event(s) in the fieldmap XML file. The instruments themselves can repeat within the event but the entire event cannot be repeating.

    Note

    We don’t know of a way that REDCap can tell us dates associated with an event. So, CamCOPS can’t say to REDCap: “Which event was scheduled for 1 June 2020?” and then upload to that one.

11.13.5. Example fieldmap XML file

The fieldmap below describes how three CamCOPS tasks are exported to a REDCap record for a patient.

There is a tag named patient that tells CamCOPS where in REDCap to find or store the patient identifier. In the example, this is in a field named patient_id in a REDCap instrument named patient_record.

There is a tag named record that tells CamCOPS where in REDCap to find or store the record identifier. The field name should be record_id unless you have changed the name of this field from the default in REDCap. The instrument should be the one in REDCap where the record_id field is defined.

This particular REDCap record has three other instruments, defined within the instruments tag. Each has a REDCap name (name attribute) and a corresponding CamCOPS task (task attribute).

If your REDCap project has events, there needs to be a global tag named event or an event attribute on the instrument tag to tell REDCap which event this record belongs to. If both the global tag and an event attribute exist, CamCOPS will use the event named in the attribute.

In this example the REDCap instruments are named bmi, patient_health_questionnaire_9, and medication_table, and they correspond to CamCOPS tasks bmi, phq9 and khandaker_mojo_medicationtherapy respectively.

<?xml version="1.0" encoding="UTF-8"?>
<fieldmap>
    <patient instrument="patient_record" redcap_field="patient_id" />
    <record instrument="patient_record" redcap_field="record_id" />
    <event name="event_1_arm_1" />
    <instruments>
        <instrument event="event_2_arm_1" task="phq9" name="patient_health_questionnaire_9">
            <fields>
                <field name="phq9_1" formula="task.q1" />
                <field name="phq9_2" formula="task.q2" />
                <field name="phq9_3" formula="task.q3" />
                <field name="phq9_4" formula="task.q4" />
                <field name="phq9_5" formula="task.q5" />
                <field name="phq9_6" formula="task.q6" />
                <field name="phq9_7" formula="task.q7" />
                <field name="phq9_8" formula="task.q8" />
                <field name="phq9_9" formula="task.q9" />
                <field name="phq9_how_difficult" formula="task.q10 + 1" />
                <field name="phq9_total_score" formula="task.total_score()" />
                <field name="phq9_first_name" formula="task.patient.forename" />
                <field name="phq9_last_name" formula="task.patient.surname" />
                <field name="phq9_date_enrolled" formula="format_datetime(task.when_created,DateFormat.ISO8601_DATE_ONLY)" />
            </fields>
        </instrument>
        <instrument task="bmi" name="bmi">
            <fields>
                <field name="pa_height" formula="format(task.height_m * 100.0, '.1f')" />
                <field name="pa_weight" formula="format(task.mass_kg, '.1f')" />
                <field name="bmi_date" formula="format_datetime(task.when_created, DateFormat.ISO8601_DATE_ONLY)" />
            </fields>
        </instrument>
        <instrument task="khandaker_mojo_medicationtherapy" name="medication_table">
            <files>
                <field name="medtbl_medication_items" formula="task.get_pdf(request)" />
            </files>
        </instrument>
    </instruments>
</fieldmap>

Each field tag describes how the answer for a question will be stored in REDCap. The name attribute is the column in the REDCap database table for the task. The formula attribute is Python code to generate the REDCap value before it is exported.

For the formula we use the library asteval, a safer version of Python’s eval(). Whilst asteval tries hard to prevent the Python interpreter from crashing, it is still possible to write potentially destructive code, so use this with care and at your own risk!

See the asteval documentation for supported operations (numpy is available). In addition we provide access to the following symbols:

We’ll explore each of the three task examples in more detail.

11.13.5.1. PHQ-9 example

In the PHQ-9 example, the nine main questions are simply copied over to REDCap with no processing:

<instrument event="event_2_arm_1" task="phq9" name="patient_health_questionnaire_9">
    <fields>
        <field name="phq9_1" formula="task.q1" />
        <field name="phq9_2" formula="task.q2" />
        <field name="phq9_3" formula="task.q3" />
        <field name="phq9_4" formula="task.q4" />
        <field name="phq9_5" formula="task.q5" />
        <field name="phq9_6" formula="task.q6" />
        <field name="phq9_7" formula="task.q7" />
        <field name="phq9_8" formula="task.q8" />
        <field name="phq9_9" formula="task.q9" />
        <field name="phq9_how_difficult" formula="task.q10 + 1" />
        <field name="phq9_total_score" formula="task.total_score()" />
        <field name="phq9_first_name" formula="task.patient.forename" />
        <field name="phq9_last_name" formula="task.patient.surname" />
        <field name="phq9_date_enrolled" formula="format_datetime(task.when_created,DateFormat.ISO8601_DATE_ONLY)" />
    </fields>
</instrument>

The possible answers for the tenth question (known as phq9_how_difficult on the REDCap side) have been coded differently in REDCap (1-4 instead of 0-3) so we need to add one to the answer.

The total score in REDCap is stored (rather than calculated, as in CamCOPS), so to fill in this field we call the method camcops_server.tasks.Phq9.total_score().

The REDCap PHQ9 instrument also stores the first and last names of the patient. We can retrieve these from the patient (camcops_server.cc_modules.cc_patient.Patient) associated with the task.

Finally, the REDCap PHQ9 instrument has a date field (phq9_date_enrolled), which needs to be in ISO8601 (yyyy-mm-dd) format.

11.13.5.2. BMI example

In the BMI example, the height and weight fields need to be specified to one decimal place in REDCap so we use the Python built-in format() function. In addition, the REDCap instrument records the height in centimetres so we need to multiply the value in metres by 100.

<instrument task="bmi" name="bmi">
    <fields>
        <field name="pa_height" formula="format(task.height_m * 100.0, '.1f')" />
        <field name="pa_weight" formula="format(task.mass_kg, '.1f')" />
        <field name="bmi_date" formula="format_datetime(task.when_created, DateFormat.ISO8601_DATE_ONLY)" />
    </fields>
</instrument>