HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

Rapid Responses to:

Perspectives: Mark Hancock, Robert D. Herbert, and Christopher G. Maher A Guide to Interpretation of Studies Investigating Subgroups of Responders to Physical Therapy Interventions PHYS THER 2009; 89: 698-704 [Abstract] [Full text] [PDF]

Rapid Responses: Submit a response to this article

Rapid Responses published:

Author Response

Mark Hancock, Rob Herbert and Christopher G. Maher   (24 August 2009)

Evidence Suggests Single-arm Trials Useful for CPR Derivation

Stephen C. Allison   (12 August 2009)

Author Response 24 August 2009
Mark Hancock, Lecturer Back Pain Research Group, The University of Sydney, Sydney, New South Wales, Australia, Rob Herbert and Christopher G. Maher Send rapid response to journal: Re: Author Response M.Hancock{at}usyd.edu.au Mark Hancock, et al.	Allison writes to defend the use of single-arm trials to investigate treatment effect modification. This position, in our view, is not defensible. The effect of treatment is the difference between outcomes with and without treatment. A single-arm trial cannot provide information about outcomes without treatment, so it cannot quantify the effect of treatment. Therefore, single-arm trials cannot quantify treatment effect modification. In our view, much of the confusion with regard to clinical prediction rules (CPRs) for physical therapy treatment has arisen because investigators have ignored these basic methodological considerations. In contrast to Allison, we believe the distinction between prognostic factors and treatment effect modifiers is important. Allison provides an example in which a patient is told, “Given your clinical presentation, your prognosis is a 95% probability of success with this proposed treatment.” This statement implies that success is due to the treatment. If the trial is a single-arm trial, then it is not possible to determine whether a successful outcome is due in any way to the treatment, so the information provided to the patient is potentially misleading. In the second example, the patient is told he or she “will likely respond well to spinal manipulation”; again, this statement is potentially misleading if it is based on a trial with no control group. In this case, treatment effects cannot be established. It could be that patients with those characteristics may recover equally quickly without any treatment or with an alternative treatment. Patients have the right to know if their outcome is due to treatment (treatment effects) or equally likely without treatment (prognosis). Allison suggests we categorically dismiss any use of single-arm trials in CPR derivation. This was not our intent, nor do we believe the article states this. On this point, our article states, “Where researchers believe there is a rationale for why a CPR derived in a single-arm study also may predict response to an intervention, the CPR should be investigated in a controlled trial before any suggestion about the role in predicting response to treatment is made.”1(p700) Some prognostic factors may later be shown to be effect modifiers, but many will not. Until tested in a controlled trial, there is no evidence that the predictor can identify people with a favorable response to treatment, and it is misleading to readers of the literature and patients to suggest otherwise. Allison refers to the trials of Flynn et al2 and Childs et al3 as support for the notion that single-arm trials can be used to identify responders to intervention. He does not refer to the far more numerous examples of secondary analyses of clinical trials clearly demonstrating that certain factors associated with prognosis are not associated with response to a specific treatment. His position also ignores the most fundamental principles of clinical research. The need for controlled trials to identify the effect of specific interventions is universally accepted. Julie Fritz, who was a senior author on the 2 trials referred to by Allison, participated in a recent PTJ podcast on the topic of effect modification. We encourage readers to listen to the podcast. Fritz’s comments on this issue broadly align with ours. Mark Hancock, Rob Herbert, Christopher G. Maher M. Hancock, PT, PhD, is Lecturer, Back Pain Research Group, The University of Sydney, Sydney, New South Wales, Australia (M.Hancock{at}usyd.edu.au) R. Herbert PT, PhD, is Senior Research Fellow, The George Institute for International Health, The University of Sydney. C.G. Maher, PT, PhD, is Director, The George Institute for International Health, The University of Sydney. References 1 Hancock MJ, Herbert RD, Maher CG. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Phys Ther. 2009;89:698–704. 2 Flynn T, Fritz J, Whitman J, et al. A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation. Spine. 2002;27:2835–2843. 3 Childs JD, Fritz JM, Flynn TW, et al. A clinical prediction rule to identify patients with low back pain most likely to benefit from spinal manipulation: a validation study. Ann Intern Med. 2004;141:920–928.
Evidence Suggests Single-arm Trials Useful for CPR Derivation 12 August 2009
Stephen C. Allison, Physical Therapist Rocky Mountain University of Health Professions, Baylor University Send rapid response to journal: Re: Evidence Suggests Single-arm Trials Useful for CPR Derivation Stephen_Allison{at}Baylor.edu Stephen C. Allison	I read with interest and some puzzlement the article titled “A Guide to Interpretation of Studies Investigating Subgroups of Responders to Physical Therapy Interventions” by Hancock et al1 in the July 2009 issue of PTJ. These authors expressed in their article the opinion that treatment-related clinical prediction rules (CPRs) are flawed when derived using single-arm studies. The single-arm research design has been used for virtually all published CPRs in this category, so the implication seems to be that all currently published CPR derivations of this type are inherently flawed. These authors drew distinctions among treatment effect modifiers, prognostic factors, and diagnostic tests. They asserted that although a single-arm research design is appropriate for prognostic and diagnostic studies, this design is inappropriate for determining treatment effect modifiers, which are best determined with randomized controlled trials (RCTs). As an author who has participated in several CPR derivation studies,2–4 I believe it is unnecessarily limiting to rigidly compartmentalize patient attributes quantitatively associated with treatment success or nonsuccess as treatment effect modifiers, prognostic factors, or diagnostic tests. Any patient attribute or combination of attributes measured at an initial assessment with power to predict treatment success or nonsuccess is potentially clinically useful regardless of what label we place on it. Using a CPR to quantitatively predict a probability of treatment success can be conceived of as prognosis: “Given your clinical presentation, your prognosis is a 95% probability of success with this proposed treatment.” Using a CPR as part of a treatment-based classification scheme also can be conceived of as diagnosis: “Given your clinical presentation, my diagnosis is that you have the kind of low back pain that will likely respond well to spinal manipulation.” If predictive attributes derived quantitatively are later validated in independent studies, the research design used to first identify those attributes is, by any pragmatic consideration, adequate. Hancock et al stated their opinion that “there is no reason to expect that factors found in single-arm trials to be predictive of outcome will subsequently be found in 2-arm trials to be predictive of response to treatment.”1(p700) This statement unaccountably follows immediately after their description of 2 such related studies that contradict that premise. Flynn et al2 used a single-arm trial to derive a CPR to suggest which patients with low back pain might be the best candidates for spinal manipulation. Childs et al5 later used that derived CPR in an RCT and found that the attributes identified by Flynn et al indeed were predictive of treatment success. The existence of these 2 studies, combined with subsequent reports of successful implementation of this CPR in clinical settings6,7 seems to undermine the basic premise presented by Hancock et al,1 yet there was no apparent attempt in their perspective article to reconcile this contradiction. An alternate interpretation of our efforts to develop treatment-related CPRs in physical therapy might be that the single-arm trial design has been our most successful method to date for this purpose: 100% of CPRs derived from single-arm trials subsequently subjected to RCT validation (1 of 1) demonstrated a clinically useful result. We have only begun to explore methods and results to achieve the highly desirable outcome of clinically useful CPRs for helping to determine which patients are best candidates for specific physical therapy treatments. I believe the categorical dismissal by Hancock et al1 of the single-arm trial method as a data-driven approach to CPR derivation is unwarranted. As an alternative to data-driven methods used with single-arm trials, Hancock et al1 suggested instead that researchers rely upon intuitive use of “plausible rationale” to define patient attributes possibly predictive of treatment success that then can be used to define subgroups in RCTs that would hopefully show significant interaction effects. Although I respect the need to combine clinical judgment with results of data analysis in order to derive CPRs that are both statistically sound and clinically useful, I am surprised by the suggestion that we revert to notions of biologic plausibility for our initial estimates of treatment effect modifiers to study. Overreliance on notions of biologic plausibility for treatment decisions is thought to be partly responsible for our current problem of unwarranted variation in patient care offered by physical therapists and others.8,9 I hope that discussion of methods and uses of CPRs intended to help inform clinical treatment decisions will continue. However, I believe it is unhelpful to categorically dismiss an approach using a specific research design that has been shown to successfully identify a cluster of patient attributes later validated to help identify which patients are the best candidates for a specific treatment. As others have noted, it is now time for more research to determine whether derived CPRs can be validated and thereafter used to provide better patient outcomes when applied in the clinic. Stephen C. Allison S.C. Allison, PT, PhD, is Professor, Rocky Mountain University of Health Professions, and Associate Professor, Baylor University (Stephen_Allison{at}Baylor.edu) References 1 Hancock M, Herbert RD, Maher CG. A guide to interpretation of studies investigating subgroups of responders to physical therapy interventions. Phys Ther. 2009;89:698–704. 2 Flynn T, Fritz J, Whitman J, et al. A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation. Spine. 2002;27:2835–2843. 3 Wainner RS, Fritz JM, Irrgang JJ, et al. Reliability and diagnostic accuracy of the clinical examination and patient self-report measures for cervical radiculopathy. Spine. 2003;28:52–62. 4 Wainner RS, Fritz JM, Irrgang JJ, et al. Development of a clinical prediction rule for the diagnosis of carpal tunnel syndrome. Arch Phys Med Rehabil. 2005;86:609–618. 5 Childs JD, Fritz JM, Flynn TW, et al. A clinical prediction rule to identify patients with low back pain most likely to benefit from spinal manipulation: a validation study. Ann Intern Med. 2004;141:920–928. 6 Childs JD, Fritz JM, Piva SR, Erhard RE. Clinical decision making in the identification of patients likely to benefit from spinal manipulation: a traditional versus an evidence-based approach. J Orthop Sports Phys Ther. 2003;33:259–272. 7 Cleland JA, Fritz JM, Whitman JM, et al. The use of a lumbar spine manipulation technique by physical therapists in patients who satisfy a clinical prediction rule: a case series. J Orthop Sports Phys Ther. 2006;36:209–214. 8 Fritz J, Flynn TW. Autonomy in physical therapy: less is more. J Orthop Sports Phys Ther. 2005;35:696–698. 9 Wennberg JE. Unwarranted variations in healthcare delivery: implications for academic medical centres. BMJ. 2002;325(7370):961–964.