2012 Annual Conference

Edward P. Mulligan

UT Southwestern - Dallas, TX

The DFW Sports Medicine Journal Club meets on a monthly basis and represents a diverse group of health care clinicians including multiple medical specialties, residents, interns, chiropractors, physical therapists, and athletic trainers.  During a journal club meeting in 2011 the group reviewed an article from the American Journal of Sports Medicine titled “Performance on the Single-Leg Squat Task Indicates Hip Abductor Muscle Function” by Crossley K, et al. The purpose of this report is to describe the impact of active participation and replication of the study methods on the understanding and integration of the content reviewed from this article during an evidence-based journal club meeting.

7 physicians, 9 physical therapists, 2 athletic trainers, and one chiropractor participated in the journal club review of this article.  47% reported assessing the performance of a unilateral step-down on a regular basis while 53% reported they rarely performed this test. 67% agreed they were confident in their ability to analyze and discriminate step-down performance. 

Following a discussion of the article and its grading scheme, attendees rated the performance of 7 different video-based step-down events. After viewing 3-4 repetitions, each rater rendered an independent decision of the performance by assigning a grade of good, fair, or poor.  The only demographic identifier collected for each rater was their healthcare discipline.  One week later, an on-line survey was distributed to each of the 19 journal club attendees to assess their experience and confidence with assessing a unilateral step-down performance and solicit their opinion on the value of this journal club learning methodology.

Multi-rater kappa, κ, statistics were used to assess intertester agreement of quality of the step-down performance. 

The overall κ = 0.42, with a κ of 0.55 between physical therapists and 0.38 for physician participants. 92 of the 133 (69%) performance judgments agreed with the expert opinion rendered by the article authors. 2 of the 19 (11%) individual journal club raters agreed 100% of the time with the reference standard as provided by the article authors.  Individual ratings ranged from 43% (3 of 7) to 100% (7 of 7) correct for the seven unique assessments of step-down performance.  There was not one instance in which a rater graded a test as poor when in fact the expert panel rated the performance as good or a good performance rated as poor (76 out of 76 instances).  The largest discrepancy in judgment was differentiating a fair and poor performance. In only 10% of ratings was a discrepancy seen between fair and good performance; however, there were disagreements 50% of the time between a fair and poor performance in those video subjects who did not have a good performance. 

Our journal club members could not match the substantial level of agreement reported by the authors of the study (κ = 0.60-80).  It may be that additional training, additional categories of performance classification, and/or similar discipline backgrounds are necessary to elevate the level of agreement to this reported degree of consistency. In a post-journal club survey of participants, 100% of respondents agreed that the presentation and discussion about the article was beneficial in their understanding and integration of the content.  Additionally, 100% of the journal club survey respondents reported that viewing and rating the video examples were beneficial in the interpretation and integration of the content into their clinical practice.

Journal clubs should consider replicating elements of published studies to enhance learning and personalize the external validity of published investigation results. Presentation, demonstration, and practice of the study methods may improve the understanding of the article’s content and the clinical application of the involved skill.