Hawthorne effect reporting in orthodontic randomized controlled trials: truth or myth? Blessing or curse?

Affiliations


Abstract

Objective: To investigate in 10 orthodontic journals how many randomized controlled trials (RCTs) considered the Hawthorne effect, and if considered, to determine whether it was related to the patients or the therapists involved in the trial and, finally, to discuss the Hawthorne effect in an educational way.

Materials and methods: A search was performed on the Medline database, via PubMed, for publication type 'randomized controlled trial' published for each journal between 1 August 2007 and 31 July 2017. The American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, Australian Orthodontic Journal, Dental Press Journal of Orthodontics, European Journal of Orthodontics, Journal of Orthodontics, Journal of Orofacial Orthopedics, Korean Journal of Orthodontics, Orthodontics and Craniofacial Research and Progress in Orthodontics were assessed. Two independent reviewers extracted the data and identified whether the Hawthorne effect was considered or discussed in the articles and whether the Hawthorne effect was related to the behaviour of the patients, the therapists, or both.

Results: The initial search generated 502 possible trials. After applying the inclusion and exclusion criteria, 290 RCTs were included and assessed. The Hawthorne effect was considered or discussed in 10 of 290 RCTs (3.4%), and all were related to the patients' and none to the therapists' behaviour.

Conclusions: The Hawthorne effect reported in orthodontic RCTs was suboptimal. The researchers' lack of knowledge about this phenomenon is evident, despite evidence that the Hawthorne effect may cause over-optimistic results or false-positive bias.


Similar articles

Intention to treat (ITT) analysis as reported in orthodontic randomized controlled trials-evaluations of methodology and recommendations for the accurate use of ITT analysis and handling dropouts.

Bondemark L, Abdulraheem S.Eur J Orthod. 2018 Jul 27;40(4):409-413. doi: 10.1093/ejo/cjx084.PMID: 29069355

What's in a title? An assessment of whether randomized controlled trial in a title means that it is one.

Koletsi D, Pandis N, Polychronopoulou A, Eliades T.Am J Orthod Dentofacial Orthop. 2012 Jun;141(6):679-85. doi: 10.1016/j.ajodo.2011.12.020.PMID: 22640669

Quantity and quality assessment of randomized controlled trials on orthodontic practice in PubMed.

Shimada T, Takayama H, Nakamura Y.Angle Orthod. 2010 Jul;80(4):525-30. doi: 10.2319/090809-507.1.PMID: 20482358 Free PMC article.

The reporting of blinding in orthodontic randomized controlled trials: where do we stand?

Abdulraheem S, Bondemark L.Eur J Orthod. 2019 Jan 23;41(1):54-58. doi: 10.1093/ejo/cjy021.PMID: 29697755 Review.

Clinical trials in orthodontics I: demographic details of clinical trials published in three orthodontic journals between 1989 and 1998.

Harrison JE.J Orthod. 2003 Mar;30(1):25-30; discussion 21. doi: 10.1093/ortho/30.1.25.PMID: 12644604 Review.


Cited by

Evaluation of the Flash-Free Adhesive System for a 6-month Period: A Split-Mouth Trial.

Baker D, Elekdağ Türk S.Turk J Orthod. 2023 Jun 22;36(2):118-125. doi: 10.4274/TurkJOrthod.2022.2021.0249.PMID: 37346169 Free PMC article.

Manual vs. interactive power toothbrush on plaque removal and salivary Streptococcus mutans and Lactobacillus casei levels : Single-center, examiner-blinded, randomized clinical trial in orthodontic patients.

Erden T, Camcı H.J Orofac Orthop. 2023 Jun 2. doi: 10.1007/s00056-023-00470-6. Online ahead of print.PMID: 37266910 English.

Compliance with retainer wear using audiovisual integration and reminder: a randomized clinical trial.

Wafaie K, Mohammed H, Xinrui W, Zhou J, El Sergani AM, Yiqiang Q.Sci Rep. 2023 May 26;13(1):8543. doi: 10.1038/s41598-023-35686-4.PMID: 37237095 Free PMC article. Clinical Trial.

The efficacy of mouthwashes on oral microorganisms and gingivitis in patients undergoing orthodontic treatment: a systematic review and meta-analysis.

Ren X, Zhang Y, Xiang Y, Hu T, Cheng R, Cai H.BMC Oral Health. 2023 Apr 6;23(1):204. doi: 10.1186/s12903-023-02920-4.PMID: 37024817 Free PMC article.

Patient compliance with Twin Block appliance during treatment of Class II malocclusion: a randomized controlled trial on two check-up prescriptions.

Frilund E, Sonesson M, Magnusson A.Eur J Orthod. 2023 Mar 31;45(2):142-149. doi: 10.1093/ejo/cjac046.PMID: 35968672 Free PMC article. Clinical Trial.


KMEL References