To the editor
I enjoyed reading the update by Childs and Cleland titled “Development and Application of Clinical Prediction Rules to Improve Decision Making in Physical Therapist Practice” (January 2006) and would like to offer a comment on the use of clinical prediction rules to select treatment approaches for individual patients.
The premise is that there are patient characteristics that moderate the effect of one intervention compared with another intervention (or compared with no treatment). The authors cite a range of studies that have tested clinical prediction rules, but many of the cited studies were uncontrolled. For example, in the study by Hicks et al,1 all subjects received a stabilization exercise program. Studies with only one treatment group cannot provide evidence that one intervention is preferable to another (or no treatment) overall or for a particular subgroup of patients. To do this, we need studies where one intervention is compared with another intervention (or no treatment). The most that can be inferred from these uncontrolled studies is that the rule predicts the magnitude of improvement over time, not the effects of intervention. This patient-specific information on prognosis is useful, but it does not help select one treatment approach over another for an individual patient.
In contrast, the cited study by Childs et al2 was a randomized controlled trial comparing manipulation and exercise with exercise alone. This study design does provide information on the effects of manipulative treatment. By including a clinical prediction rule x treatment group x time interaction term in the analysis, the study can test whether the effect of manipulative treatment is moderated by patient characteristics. This study does provide high-quality evidence of a treatment effect modifier and is the design of choice for future work in this area.
- Physical Therapy
We agree with Maher’s assertion that clinical trials with comparison groups are necessary to test the validity of data obtained with a prediction rule before it can be recommended for widespread implementation in clinical practice. We readily discuss the importance of this concept throughout our article when we elaborate on the requisite steps involved in the development and validation of prediction rules and the hierarchy of evidence for clinical prediction rules.
Although clinical prediction rules in the early stages of development should not be defended as an endpoint of the research process, the fact that some of the studies we cited were uncontrolled is merely a reflection of the evolutionary stage of the rule’s development. For example, Maher refers to our previous work related to the manipulation clinical prediction rule.1 The precursor to this study also was an uncontrolled study.2 At that time, no evidence was available suggesting which factors were associated with a positive response to spinal manipulation in patients with low back pain.
Given the absence of preliminary work and the cost and effort necessary to conduct high-quality clinical trials, a randomized trial with a comparison group was not the most sensible design at this early stage in the rule’s development. The fact that it would be impossible to definitively conclude that the rule predicted a specific response to spinal manipulation versus any other treatment or the passage of time was known a priori as a question that would have to be answered in subsequent follow-up studies. As Maher points out, a more definitive randomized clinical trial testing the rule’s validity against a comparison group has been completed1; thus, the rule is ready for widespread implementation in clinical practice.
Other studies are still necessary to determine the rule’s accuracy in different practice settings and its effect on outcomes of care and costs. Impact analysis studies will provide immense insight far beyond what can be concluded from derivation and initial validation studies alone. However, without initial development studies, validation and impact studies are more difficult (if not impossible) to design and much more likely to yield negative results, attesting to the value of viewing research as a process rather than an “end game.’
As Maher points out, in the study by Hicks et al,3 all patients received a lumbar stabilization exercise program. Without a comparison group, one cannot exclude the explanation that the predictors could be predicting merely the passage of time rather than a specific response to lumbar stabilization exercise. At this stage, the rule corresponds to level IV in the evidence hierarchy4 and cannot yet be recommended for widespread implementation in clinical practice. However, an interesting question is whether the absence of validation studies implies that the results are clinically useless. Sackett et al5 defined evidence-based practice as “conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” We would suggest that, given the virtually nonexistent risks associated with lumbar stabilization exercise and the lack of higher levels of evidence to guide decision making in determining which patients are most likely to benefit from lumbar stabilization exercise, application of the rule in the care of patients with low back pain may not be altogether inappropriate. The nature of the predictors (eg, presence of aberrant movements, positive prone instability test) in this study also appears to offer at least some face validity that the predictors are associated with response to lumbar stabilization exercise. The caveat is that clinicians cannot be confident in the accuracy of its use until more definitive validation studies are completed.
In conclusion, we agree with the issues raised by Maher. We would simply offer that the best research design depends on the question being asked, the evolution of research related to the research question, and the available resources with which to answer the question. We appreciate Maher’s comments and the opportunity to respond.