PHYS THER
Vol. 86, No. 7, July 2006, pp. 1036-1037
Author Response
Dorcas E Beaton
Mobility Program Clinical Research Unit St Michael's Hospital Institute for Work and Health Toronto, Ontario, Canada
Carol A Kennedy
Institute for Work and Health Toronto, Ontario, Canada
beatond{at}smh.toronto.on.ca
 |
Introduction
|
|---|
We thank Irrgang for his thoughtful reflections on our article. His comments supported the fact that studies of prognosis are being recognized for their clinical importance and that—as we move toward the application of their results to clinical practice—their methods are undergoing the same scrutiny that randomized controlled trials received several years ago. There has been an exponential rise in the number of systematic reviews of prognosis over the past few years, and, with that, a rise in the need for standardization of the ways we conduct, report, and interpret studies of prognosis.2 Prognostic studies are vulnerable to biases in selection, prognostic factor assessment, study attrition, analytic approach, and outcome determination2 and require careful thought and description of the decisions made.
We would like to respond to 3 of Irrgang's comments. First, the issues of format of the outcome; second, the issues of modeling; and finally, issues of moving forward from here—what was missed, and what lies ahead. In many ways, we do not dispute Irrgang's comments, and we appreciate the opportunity to respond to them.
|
Issues of the Format of the Outcome
|
|---|
Consumers of the prognosis literature often are happy to find multiple studies that, for example, look at disability or at pain as an outcome, and they may neglect to consider the impact of how the outcome was formatted for analysis. The authors, or systematic reviewers, may well describe whether the study looked at a final state of disability or at a magnitude of change, without realizing that this is now an important part of the way we interpret prognostic findings. We were surprised by the results of our study, which continued when we also modeled the predictors of the "course" of disability over time.3 The results suggest one more layer to consider vis à vis Hayden and colleagues' guidelines for systematic reviews2: the form of the outcome measure as part of the "adequate measurement of outcome." In our situation, each approach to formatting the Disabilities of the Arm, Shoulder, and Hand (DASH) outcome measure (change, final state, or course) would likely have been considered adequate measurement of outcome, but the differences in predictors attributed to the format would be overlooked. We hope that our findings will help consumers of the literature realize that a different array of predictive factors across studies, all of which seem to predict "disability," could simply be due to the way the outcome was formatted in each study.
|
Issues of the Modeling
|
|---|
Irrgang raised 3 important issues regarding our decisions related to statistical modeling, all of which are choices that need to be made when conducting an analysis with integrity. First, Irrgang raised the issue of the number of variables we could consider. We were unable to include likely important psychosocial variables such as fear-avoidance. In deciding with our clinical collaborators which variables we would include, we were faced with issues of power. We wanted to follow the guideline of 10 observations for each parameter considered (variables plus any dummy variables needed) and were therefore limited even with a large sample size. Following an even larger cohort to gain the additional power was not feasible, and our team members were more rehabilitation oriented and were not trained in clinical psychology, so we ethically needed to decide what we were qualified to manage in 81 clinics across the province and to interpret with our skills. We therefore decided to focus on what our main stakeholders—physical therapists who manage these problems—and the literature suggested as clinically important and potentially useful predictors. We did not focus our analysis on the R2 values (proportion of variance explained) but rather on the significance, magnitude, and direction of the beta coefficients. We also were able to use standardized coefficients to allow some comparison across variables in a single model. The lower, although not insignificant, R2 values do suggest that there is more to be learned about factors that are predictive of how people will do in physical therapy.
This leads to Irrgang's second point—the way in which we measured range of motion and other clinical variables. We held training sessions with each of the clinicians involved in the study and provided guidance as to how to measure various predictors. Our pool of assessors was large. We decided to measure some of the clinical variables as "normal/abnormal" simply to reduce measurement error due to interobserver variability. Although this does reduce the power of an analysis, it also improves the interpretability of the factor in a predictive model. Our model was testing whether restrictions in various planes of motion were associated with outcome. A model with a continuous measure of range of motion would be saying that a difference of 1 degree is predictive of outcome—a level that is neither clinically important nor useful for the individual client.
The final issue of modeling raised by Irrgang is what to do with baseline disability in a model of change. We thought extensively about this issue, which we raised in the discussion section of our article. We sought the advice of several epidemiologists as well as statisticians. From a mathematical perspective, baseline state is part of the change in state (follow-up-baseline); therefore, it cannot be on both sides of the equation. From a clinical perspective, the importance of baseline status in a predictive model is critical. There was no consensus, and we were not able to find a source in the literature to give us a definitive answer. We did the analysis both ways and found a shift in the factors that were statistically important.
We re-ran a new model for the outcome of change in DASH score when the baseline DASH score was included in the model. In this new model for change in DASH score, the Physical Component Score (PCS) and pain intensity were no longer retained in the model. Age (by decade), duration of current problem, and surgery remained in the model. In addition, baseline DASH score and having a workers' compensation claim were significantly associated with change in DASH score. Therefore, the baseline DASH score replaced PCS and pain intensity (which were moderately correlated with baseline DASH score) in the new model.
|
Issues of Moving Forward: What Was Missed, and What Lies Ahead
|
|---|
Our final response echoes Irrgang's comments regarding the next steps with our study. Our study, which followed a large number of people across many clinical settings, was challenging to carry out. Yet, it still did not fulfill enough of the criteria for a clinical prediction rule to be adopted into a study. We did move up Altman's hierarchy of studies of prognosis4 by basing our variable selection on the existing literature. We also attempted to address many of the quality issues in our study design and reporting by following existing appraisal guidelines.5,6
Further work is needed to run this model in a similar sample and validate the results. We would be encouraged if others were interested in doing so—it is a critical step in the Childs and Cleland guidelines for developing a clinical prediction rule.1 We agree with Irrgang's analogy to the classification systems in low back pain aiding the treatment of these individuals; however, it might be premature to apply our model in this way until it has been validated and tested for its clinical impact.
We thank Dr Irrgang for his thorough and insightful comments on our article and in particular for his emphasis on the importance that the format of the outcome seems to have had in our models. We look forward to others picking up the challenge to test our model and our findings about the format of the outcome so that we can continue to learn about the factors affecting studies of prognosis and their eventual clinical application.
|
References
|
|---|
- Childs JD, Cleland JA. Development and application of clinical prediction rules to improve decision making in physical therapist practice. Phys Ther. 2006;86:122-131.[Free Full Text]
- Hayden JA, Cote P, Bombardier C. Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med. 2006;144:427-437.[Abstract/Free Full Text]
- Kennedy CA, Haines T, Beaton DE. Predictive factors associated with response patterns during physiotherapy for soft tissue disorders were identified. J Clin Epidemiol. In press.
- Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat. 1998;52:289-303.[Web of Science][Medline]
- Cole DC, Hudak PL. Prognosis of nonspecific work-related musculoskeletal disorders of the neck and upper extremity. Am J Ind Med. 1996;29:657-668.[Web of Science][Medline]
- Kuijpers T, van der Windt DA, van der Heijden GJ, Bouter LM. Systematic review of prognostic cohort studies on shoulder disorders. Pain. 2004;109:420-431.[Web of Science][Medline]
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
Copyright © 2006 by the American Physical Therapy Association.
Top
Issues of the Format...
Issues of the Modeling
