To be able to determine whether a healthcare intervention is effective, we need randomized trials that are appropriately designed and conducted. But, appropriate design and conduct of a trial in and of itself is not enough.
The trial’s methods, conduct and results need to be comprehensively reported to convince others of the validity of the results, allow replication of the trial methodology, and the incorporation of the results in synthesis products such as meta-analysis.
Benefits and pitfalls of the cluster randomized cross-over trial design
The cluster randomized cross-over (CRXO) design provides an exciting opportunity to evaluate interventions in settings where cluster randomization is required, but a parallel group cluster randomized design is not feasible…
The cluster randomized cross-over (CRXO) design (see here and here) provides an exciting opportunity to evaluate interventions in settings where cluster randomization is required, but a parallel group cluster randomized design is not feasible because the number of required clusters is prohibitively large.
In a CRXO trial, schools, hospitals or other groups of individuals (“clusters”) are assigned to a sequence of interventions. This design differs to a parallel group cluster randomized design where each cluster is assigned to just one intervention. The CRXO design incorporates a cross-over of interventions at the level of the cluster.
The cross-over aspect means that the interventions are compared within cluster, and hence the between cluster variation is removed from the estimate of the difference between the interventions. As a result, the design generally requires fewer participants than the parallel group cluster randomised design.
While the CRXO design has the potential benefit of requiring fewer participants, the design is not always appropriate for evaluating interventions (e.g. where it is not possible to ‘remove’ an intervention, such as an educational intervention targeted at clinicians within a hospital); has greater potential for bias than other designs (e.g. through carry-over effects, identification/recruitment of participants into the trial); and, has more complexity in the sample size calculations and statistical methods.
But there’s huge room for improvement when it comes to reporting
Our previous research found that in a cohort of CRXO trials, appropriate statistical methods for sample size calculations and statistical analyses were rarely used.
In our paper published today in Trials, we examined the completeness of reporting of CRXO trials, and found that they are not well reported. Specifically:
The justification for using the design was infrequently stated.
- Inconsistent language was used in the title and abstract to describe the trial design. As a result, many CRXO trials may not be indexed appropriately in electronic databases.
- The justification for using the design was infrequently stated. This information is essential to determine if the CRXO design is best suited to address the research question.
- The potential for carryover was infrequently discussed. Clear reporting of the method used to reduce the risk of carryover is required to allow readers to assess the risk of bias to the intervention effect arising from the potential carryover.
- A justification for the sample size was presented in only half of the trials, and incomplete reporting of the sample size calculation within those trials was common. Clear sample size reporting is important for replicability, transparency and ethical reasons.
- Reporting of which researchers and participants were aware of the allocated interventions was incomplete. The trial is at risk of selection bias (a systematic difference in the characteristics of the groups of individuals that are compared between interventions) if: the person assigning the intervention sequences to clusters is aware of the intervention sequence; the person responsible for recruiting participants knew which intervention sequence had been assigned to the cluster; or individuals within a cluster are aware of the cluster allocation before giving consent.
A justification for the sample size was presented in only half of the trials, and incomplete reporting of the sample size calculation within those trials was common.
Our findings are not unsurprising; they are aligned with a large body of evidence that has consistently demonstrated sub-optimal reporting of randomized trials (see here, here and here).
However, there is evidence to suggest that reporting guidelines improve the completeness of reporting.
What next for cluster randomized cross-over trial reporting?
A potential explanation for the incomplete reporting in the case of CRXO trials is that there are no reporting guidelines that address the unique features of this design.
As part of our research, we therefore proposed reporting items for the CRXO trial design. We based our proposed items on the 2012 cluster randomized trials extension to the CONSORT statement and relevant items that have been proposed for reporting a related design, the stepped wedge design.
A potential explanation for the incomplete reporting in the case of CRXO trials is that there are no reporting guidelines that address the unique features of this design.
The results of our study highlight the need for a CONSORT extension for CRXO trials, with the items proposed in our paper providing the starting point for such an extension. However, even with such an extension, there is no guarantee that these guidelines will be used.
This raises the broader question, how do we get researchers to use reporting guidelines? While there is some evidence to suggest that journal endorsement of the CONSORT Statement improves adherence to reporting guidelines, reporting is still below acceptable levels. Research on the barriers and enablers to appropriate reporting would be valuable, as would evaluations of interventions to improve design, conduct and reporting of randomized trials.
Comments