Commentary on SPIRIT-PRO Guidance on Mode of Administration JAMA Feb 2018
“SPIRIT-18a(ii)-PRO Extension: Include a data collection plan outlining the permitted mode(s) of administration (e.g., paper, telephone, electronic, other) and setting (e.g., clinic, home, other).
Explanation: It is important that both research personnel and trial participants understand how, when, and where PRO data will be collected in the study. Increasingly, electronic PRO assessment is undertaken in trials, so evidence of equivalence between different modes of administration should be considered. If electronic PRO measures contain only minor modifications with respect to the paper-based versions, usability testing and cognitive debriefing may provide sufficient evidence of equivalence. The setting for PRO data collection should be described and standardized across trial intervention groups and sites.”
In the most recent SPIRIT-PRO Guidance extension the importance of implementing a data collection plan is highlighted. While this is certainly positive, it’s worth going a step further to state that the protocol should accurately reflect the planned mode of data collection, rather than, for example, simply replacing “paper” with “electronic”. There are unique considerations for the planning of a study using electronic means of data capture, like if devices are provided to patients or returned to sites, and these intricacies should be reflected in the protocol.
Decisions around mode of administration are too often left until the last minute and as a result, protocols may simply include reference to the fact that questionnaires will be collected electronically without making any updates to the planned methodology with the mode of administration in mind.
Additionally, the broad statement around the need for equivalence testing outlines that “usability testing and cognitive debriefing may provide sufficient evidence of equivalence”. While accurately rooted in the Coons et al (2009) paper, it does not reflect how far the industry has come since that publication. There are now three meta-analyses1,2,3 summarizing the multitude of studies that have been conducted showing minimal difference in how patients respond across modalities (patient bias shift), with countless usability testing and cognitive debriefing studies showing the same. These recommendations should be treated on a case-by-case basis, dependant on a number of factors including, but not limited to, the questionnaires place in the endpoint hierarchy and previous electronic use of the questionnaire. Implying that this work always needs to be done is inaccurate and slows the continued uptake of eCOA.
Another aspect of the SPIRIT-PRO Guidance extension that could be investigated more deeply is the cultural adaptation element. This challenge is addressed by the 2018 ISPOR SIG paper Emerging good practices for Translatability Assessment (TA) of Patient-Reported Outcome (PRO) measures. Too often this element is loosely addressed. Consider that the cultural population of the target audience—patients—is often overlooked while creating a measure. There are numerous measures that do not conceptually translate for all patient demographics, and adaptation must take place. Guidelines outlining a more homogenous, universal approach would lessen the need for changes and adaptation in the translation process. Performing a linguistic validation translation is already fraught with potential errors, notably with either the selection of unqualified translators, or the awareness of deviating from the original verbiage of the instrument. Back translations are subjective, so importance must be placed on staying with the essence of the PRO.
