| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Research Reports |
DU Jette, PT, DSc, is Professor and Chair, Department of Rehabilitation and Movement Science, University of Vermont, Rowell 305, 106 Carrigan Dr, Burlington, VT 05405 (USA).
J Halbert, PT, BS; C Iverson, PT, BS; E Miceli, PT, BS; and P Shah, PT, MS, are students at the University of Vermont.
Address all correspondence to Dr Jette at: diane.jette{at}uvm.edu
Submitted August 3, 2008;
Accepted October 30, 2008
 | Abstract |
|---|
Objective: The primary purpose of this study was to determine: (1) the extent of the use of standardized outcome measures and (2) perceptions regarding their benefits and barriers to their use. A secondary purpose was to examine factors associated with their use among physical therapists in clinical practice.
Design: The study used an observational design.
Methods: A survey questionnaire comprising items regarding the use and perceived benefits and barriers of standardized outcome measures was sent to 1,000 randomly selected members of the American Physical Therapy Association (APTA).
Results: Forty-eight percent of participants used standardized outcome measures. The majority of participants (>90%) who used such measures believed that they enhanced communication with patients and helped direct the plan of care. The most frequently reported reasons for not using such measures included length of time for patients to complete them, length of time for clinicians to analyze the data, and difficulty for patients in completing them independently. Use of standardized outcome measures was related to specialty certification status, practice setting, and the age of the majority of patients treated.
Limitations: The limitations included an unvalidated survey for data collection and a sample limited to APTA members.
Conclusions: Despite more than a decade of development and testing of standardized outcome measures appropriate for various conditions and practice settings, physical therapists have some distance to go in implementing their use routinely in most clinical settings. Based on the perceived barriers, alterations in practice management strategies and the instruments themselves may be necessary to increase their use.
| Introduction |
|---|
 TopAbstract Introduction MethodResultsDiscussionConclusionReferences |
|---|
The drive for use of standardized outcome measures in practice has been motivated to some extent by the recognition that goals for patients' improvement not only must consider the traditionally measured impairments in body function (eg, range of motion, strength [force-generating capacity]) but also should consider patients' points of view and preferences for daily activities and life participation.10 Although we do not know of any clinical trials that have demonstrated the direct effects of using standardized outcome measures, suggested benefits include identifying patients who are at risk for poor or adverse outcomes,4 facilitating improved continuity of care for patients transitioning from one health care setting to another,11 determining the most cost-effective settings for patients to receive rehabilitation services,11 assessing practitioner and organizational performance,4 and determining the most-effective interventions for particular conditions.4
The need for physical therapists to use standardized outcome measures has been recognized at the national level in the United States. The Centers for Medicare & Medicaid Services sponsored a report in 2006 to determine the possibility of a uniform rehabilitation outcomes assessment method for patients leaving acute care.11 The authors proposed several purposes for this type of assessment, including provider decision making, patient safety, and ability to determine patients' health and function longitudinally.11 On a smaller scale, the Commission on Accreditation in Physical Therapy Education12 supports the use of standardized outcome measures in practice by requiring all education programs to demonstrate that their graduates have some experience in using and interpreting them during their professional (entry-level) education.
The literature provides relatively few reports of the overall use of standardized outcome measures by physical therapists. Physical therapists in 5 academically affiliated institutions in Toronto were surveyed in 19929 and again in 19988 to determine their use of standardized outcome measures and the perceived obstacles to their use. A second part of the latter study used qualitative methods to explicate the findings.7 The studies included questions about use of a variety of types of outcomes measures; however, the authors included manual muscle testing and goniometric measurements in their definition of outcomes measures. In the 1998 study, a high proportion of respondents used manual muscle testing (88%) and goniometry (90%), whereas relatively low proportions used measures such as the Functional Independence Measure (FIM) (18%) or the Impairment Inventory scale of the Chedoke-McMaster Stroke Assessment (35%).
In 1997, a study examining the use of outcome measures in rehabilitation centers in the United Kingdom showed that 77% of the centers used at least one tool; of those centers, 28% used some measures of general motor function, and 88% used at least one measure of disability.13 In 2001, 2 studies were published that examined the use of outcome measures in Europe.6,14 Haigh et al6 found that a few rehabilitation centers used a large number of tools on a small proportion of patients. For patients with orthopedic conditions, the outcomes measured were largely at the body function level. For patients with neurological conditions, disease-specific measures of disability were used more frequently. There was minimal use of generic measurement tools that can be used regardless of condition. Although specific data were not reported, Torenbeek et al14 noted low overall satisfaction with outcome measurement for patients with stroke and low back pain among rehabilitation professionals in 5 European countries. In addition, there was little consensus about which outcome measures to use. In a study of physical therapists in outpatient clinics in the United States, Russek et al15 found that only 50% of the respondents used the outcome tools they had been provided by their clinics' corporate owner.
A few studies7,8,13,15,16 have examined perceptions of the benefits of and barriers to using standardized outcome measures among rehabilitation professionals, and many of the reported barriers were similar across studies. Perceptions about barriers include lack of time and inconvenience; lack of familiarity, know-how, and training; and lack of resources such as staffing and automation. Attitudes and perceptions related to use of outcome measures among other health care providers, including mental health practitioners, oncologists, general practitioners (GPs), and nurses, also have been reported. Garland et al3 found variability in attitudes across mental health practitioners, but noted that, in general, the responses reflected ambivalence. All of the practitioners interviewed had participated in mandated outcome assessments, yet they reported being more likely to use their own intuition than standardized measures to evaluate clients' progress. Similarly, Taylor et al17 reported that many oncologists they interviewed relied on their own impressions and informal assessments of patients' quality of life to inform their decisions. Most respondents argued that the use of standardized measures made decision making more difficult rather than facilitating it. As in the previously mentioned studies, approximately one half of GPs and nurses interviewed in a study by Meadows et al18 said that they preferred relying on their own clinical judgment in the management of their patients.
Because of the lack of recent information about the use of standardized outcome measures among physical therapists in the United States and the professional and governmental emphasis on the collection and application of data from such instruments, this study was conducted to determine the extent of their use, their clinical applications, perceptions of their value, and barriers to their use. Secondarily, we examined the relationships between practice setting and therapist characteristics and the use of standardized outcome measures.
| Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
The letter sent to potential participants noted that the instruments we were asking about were "referred to by various names and often include information that is related to patients'/clients' social, physical, or psychological status as they relate to daily activities or role participation. Examples include Oswestry Low Back Pain Questionnaire, Functional Independence Measure (FIM), Arthritis Impact Questionnaire (AIM), and SF-36 [Medical Outcome Study 36-Item Short-Form Health Survey]. This study asks you to think broadly about the measures." The questionnaire indicated that in thinking broadly, respondents should consider instruments "described with terms such as ‘health status,’ ‘quality of life,’ ‘disability,’ ‘functional status,’ or ‘outcomes measures.’" In the survey questionnaire, we referred to the instruments as "health status questionnaires." In an attempt to be consistent with terms used in the most recent rehabilitation literature, we use the term "standardized outcome measures" throughout this article, recognizing the various terms used to identify these measures.
Approximately 3 weeks after the initial mailing, those therapists who did not respond and who had e-mail addresses listed in the APTA Web site directory were sent a reminder e-mail, with the survey questionnaire and letter as attachments. After an additional week, another survey questionnaire was mailed to those who had not responded to the initial mailing or e-mail.
Instrument
The survey instrument (eAppendix 1) was designed by the investigators. The initial draft was sent to 14 clinician colleagues for input. Eight clinicians in various types of practice, including acute care, outpatient hospital-based care, and private practice, responded. They had between 15 and 30 years of practice as physical therapists. They were asked to assess the face and content validity of the items in the survey instrument, to indicate whether there were important gaps, and to indicate whether any items were unclear or confusing. Changes to the survey instrument were made based on their feedback. We also used the previous literature (cited in the introduction of this report) related to health care practitioners' attitudes toward, and use of, standardized outcome measures to support the content validity of the instrument. Construct validity of the parts of the instrument that assessed beliefs about the usefulness of and barriers to using instruments in practice was assessed through factor analysis. A principal components factor analysis with varimax rotation resulted in 5 factors that explained 57% of the variance in item responses. Cronbach alpha was determined for each of the factors to provide evidence for internal consistency. We interpreted the 5 factors to support the framework for attitudes and beliefs provided by the literature. The factors represented benefits for the management of the patient (7 items, 
=.85), problems or limitations for the physical therapist (6 items, =.77), problems or limitations for the patient (6 items, =.77), benefits for external communication (3 items, =.67), and limitations due to culture or language (2 items, =.59). Taken all together, the internal consistency of the items related to beliefs about the benefits of using standardized outcome measures was good (=.84). The internal consistency of all items related to beliefs about problems of or barriers to the use of standardized outcome measures was similarly good (=.83).
Data Analysis
Data were analyzed using SPSS statistical software, version 15.0.* Response frequencies and means or medians for the survey items were determined and displayed in tabular and graphic formats. After examining the response frequencies, and before examining the associations among variables, some variable categories were collapsed in order to allow further analysis and derive stable models.
Logistic regression analyses were conducted to examine the association of participant and practice characteristics with the use of standardized outcome measures. We used a forward selection process to derive models, requiring P<.05 to enter and P<.10 to delete. Odds ratios and their 95% confidence intervals were recorded for each level of the independent variables that were significant. We chose one level of each variable as a reference group to allow the most salient interpretation of results.
| Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
Sixty-eight percent of the participants were female, and 32% were male. The majority (61%) worked in an outpatient setting. A slim majority (53.4%) of participants had postbaccalaureate professional degrees. Thirty-two percent were certified clinical specialists. Although not formally tested, the sample seemed to reflect the demographics of APTA members reported in 2006 and 2007 fairly well.20 Our sample had a slightly greater proportion of those with postbaccalaureate degrees and less time in practice. Our sample also appears to have had slightly more therapists working in outpatient and acute care settings. It is difficult to determine whether these differences were due to the different time frames in which the data were collected or to bias in the sample. Participant and practice characteristics of the sample are shown in Tables 1 and 2, respectively.
|
|
|
|
Fifty-two percent of participants indicated they did not use standardized outcome measures in practice, and 49% of them indicated that they did not plan to implement their use in future. The 3 most common reasons for not using standardized outcome measures were: they are too time consuming for patients to complete (43%); they are too time consuming for clinicians to analyze, calculate, and score (30%); and they are too difficult for patients to complete independently (29.1%) (Tab. 5).
|
|
| Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
In our study, the finding that 3 of the most frequently used measures are useful in orthopedic conditions is not surprising given the fact that a majority of the participants practiced in outpatient settings and approximately 11% had orthopedic clinical specialty certification. Among Australian physical therapists who managed mostly patients with orthopedic conditions, Abrams et al16 found a relatively high use of the ODI; approximately 50% of the therapists indicated that they used the ODI frequently or always. Measures specific to other body regions were used less frequently, but the authors did not indicate the specific names of most of those measures. Haigh et al6 reported that the ODI was used in only approximately 4% of assessments done for patients with low back pain across 418 rehabilitation centers in Europe in 1998. Torenbeek et al14 indicated that the ODI was used in rehabilitation facilities in 4 out of 5 European countries; they reported the highest use in Ireland (12.5% of facilities). The date of their survey was not reported. The ODI is available in the public domain, and the ISI Web of Science citation index22 identifies 1,035 citations of the article in which it was originally reported in 1980.23 The ISI Web of Science citation index also indicates that the articles in which the LEFS and DASH were originally reported have been cited 74 and 431 times since their original publications in 1999 and 1996, respectively.24,25 These data suggest that the measures are fairly well known, at least among those publishing articles in scientific journals. Many standardized outcome measures have been developed within the last decade or so, and this timing may explain why the participants who had been practicing for more than 20 years were much more likely than their younger colleagues to learn about them from continuing education workshops and other therapists than from formal, professional education.
One surprise is the relatively high use (22%) of "home-grown" measures. Similarly, Kay et al8 reported that 18% of the physical therapists surveyed in their study used departmentally developed instruments. This practice seems unnecessary given the large number of existing measures that cover all body regions and many specific conditions. The finding also is somewhat contradictory, given that 68% of those who used standardized outcome measures indicated that one reason for choosing an instrument was its documented validity and reliability. We also found that participants defined outcome measures broadly to include not only measures of activity and participation but also some measures of body function such as the BBS. This finding is reflected in the literature in that previous reports of use of outcome measures by physical therapists have included references to measures of body function.6–8
The problems perceived by physical therapists who used standardized outcomes measures and the reasons given for not using them among those who did not use them were fairly similar and included issues that have been discussed in the literature for more than a decade.13,15,16 Even in the most recent study,16 the majority of participants indicated lack of familiarity with, lack of training in, and lack of access to measures were barriers. Practitioners in other health care specialties have reported the same types of barriers as those reported by physical therapists. Meadows et al18 reported that 39% of GPs and 28% of nurses indicated having insufficient time to discuss health outcome data with their patients. Logistical problems such as time, additional paperwork, and costs of personnel were cited as the most important reason for not using the measures among psychologists.19 Based on our results, it appears that many physical therapist practices may not yet have determined how best to address these barriers.
Twenty-seven percent of the participants in our study who did not use standardized outcome measures cited the lack of a support system in terms of technology and staffing as a reason, and only 11% of those who used the measures indicated that office staff helped patients to complete them. Similarly, more than 10 years ago, Russek et al15 reported that the physical therapists in their study identified lack of personnel to assist in data management as a barrier to implementation of these measures. Kay et al8 reported that approximately 42% of the physical therapists they surveyed in 1998 thought that lack of resources was an obstacle. The study of nurses and GPs indicated that they, too, would be more willing to use standardized measures if the data were collected and analyzed by someone else.18
In our study, approximately 7% of the participants indicated that computers, and not paper, were used for completion and analysis of measures, and slightly fewer than 10% of participants indicated that they chose measures based on their ability to analyze data electronically. Recent literature has suggested that implementation of computerized systems is critical to clinical practice in terms of evaluating both individual patients and overall practice performance. For example, in 1994, Shields et al26 described the development of a computer-based clinical database in the acute care setting and urged its implementation to better measure outcomes of physical therapy interventions. More recently, Jette et al,27 reporting on a new standardized outcome measure that uses a computerized adaptive testing format, suggested that challenges for implementation included assisting clinicians in carrying out the testing as well as understanding and interpreting the data derived from such measures. They stressed the need for training, technical support, and access to software.
In our study, 18% of the participants who did not use standardized outcome measures cited the lack of relevance to the plan of care as a reason. Kay et al8 found that 39% of physical therapists surveyed in 1998 thought that outcome measures did not meet the needs of their patients. Researchers reporting the perceptions of nurses, GPs, psychologists, and oncologists also cite lack of clinical relevance as a barrier to use of standardized outcome measures. For example, Hatfield and Ogles19 reported that a substantial number of psychologists felt that standardized outcome measures could "distort" the effects of treatment. General practitioners and nurses stated that they were more likely to use standardized outcomes measures if they helped in the care of the individual patient,18 and oncologists indicated that informal collection of data seemed a better way to understand individual patient needs than using standardized outcome measures.17 Among the physical therapists in our study who used standardized outcome measures, however, the majority believed that these measures could aid in directing the plan of care and enhancing the thoroughness of their examinations. Similarly, previous studies7,14 have shown that physical therapists perceived planning of care and monitoring the effects of treatment as benefits of standardized outcome measures. Although it is likely that many physical therapists are similar to other health care practitioners in valuing and applying the qualitative information gathered from patients, differences in perceptions regarding the usefulness of standardized outcome measures may be due to the fact that physical therapists have better tools for measuring the constructs that provide a basis for evaluating the effectiveness of their care.
Limitations
One limitation of our study is that our data reflect what has been reported by physical therapists rather than what has been observed, and although we provided our participants with a definition of standardized outcome measures, they may have thought about the measures they used in different ways. Additionally, the validity and test-retest reliability of our survey data were not tested. We attempted, however, to demonstrate content validity through use of previous literature on the topic and construct validity through factor analysis. There was good internal consistency within the items assessing the perceived benefits and barriers to using outcome measures. Another limitation was that we sent survey questionnaires only to members of APTA. Therefore, the results of this study may be biased and not representative of the entire profession of physical therapy. Given that APTA members may be more likely than nonmembers to attend national meetings, they may be more likely to have been exposed to issues related to measuring outcomes. Therefore, we might speculate that those who are members would be more likely than nonmembers to use standardized outcome measures. We considered our response rate to be adequate in that it was comparable to that reported in similar studies; however, there is the possibility that the sample was biased.
Implications
Despite more than a decade of development and testing of measures appropriate for various conditions and practice settings, the physical therapy profession appears to have some distance to go in implementing standardized outcome measurement routinely in most clinical settings. The development of such measures for acute care settings may need to be a particular focus. Regardless of setting, practices will need to help clinicians to manage time so that collection of data can become routine despite productivity expectations. Given the perceived time-consuming nature of standardized outcome measurement, investment in computerized systems for quick data entry and analysis may be warranted.
Although the content, properties, and applicability of many standardized outcome measures have been reported in the literature for more than a decade, clinicians continue to report that the measures are not used because they are not applicable to their patients or that they cannot interpret the scores. It appears, therefore, that disseminating information through the professional literature may not be an efficient or effective mechanism. Further instruction and enculturation through continuing education as well as professional and graduate professional education may increase the use of standardized outcome measures. Education should include the use of hardware and software to facilitate their usage. In addition, software should be made readily available to provide analyses that assist in the interpretation of scores. Interpretation could include comparing patients' scores with norms; using scores to qualify severity of condition or predict outcome or duration of an episode of care; or categorizing changes in scores as worse, stable, or improved. Such data could assist physical therapists in making decisions about change in management strategies, referral, or discharge from services. As noted by Jette et al,27 the essential strategies to improve use of standardized outcome measures may well require new funding mechanisms.
Given that many of our participants believed that standardized outcome measures are confusing and difficult for patients to complete, efforts should be made to ensure readability and interpretability by patients. Reading level, font size, and general appearance of measurement tools need to be considered. Language and cultural concerns were cited by relatively few of our participants; however, given the changing nature of the US population, these concerns may become magnified and necessitate adaptations to the commonly used instruments.
| Conclusion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
| Footnotes |
|---|
The study was approved by the Institutional Review Board of the University of Vermont.
* SPSS Inc, 233 S Wacker Dr, Chicago, IL 60606. 
| References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  D. U. Jette, R. Brown, N. Collette, W. Friant, and L. Graves Physical Therapists' Management of Patients in the Acute Care Setting: An Observational Study Physical Therapy, November 1, 2009; 89(11): 1158 - 1181. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
