| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Research Reports |
AM Jette, PT, PhD, FAPTA, is Director, Health & Disability Institute, and Professor of Health Policy & Management, School of Public Health, Boston University, 580 Harrison Ave, 4th Floor, Boston, MA 02118 (USA).
Address all correspondence to Dr Jette at: ajette{at}bu.edu
In 2001, the World Health Organization (WHO) released the International Classification of Functioning, Disability and Health (ICF), which provided a comprehensive framework of health states that encompassed a biological, personal, and social perspective.1 Since the May 2001 World Health Assembly endorsement as a member of the WHO family of international classifications, all member states were asked to implement the ICF in their respective health sectors.2 Toward that goal, several international efforts have been launched to develop classification approaches for the assessment and reporting of ICF functioning and health concepts in clinical studies or clinical encounters.3 The Extended ICF Core Set for Stroke represents one such effort, and the excellent article by Starrost et al4 reports on the agreement between 2 physical therapists when rating patients’ functioning using the Extended ICF Core Set for Stroke.
As I have written elsewhere,5 I believe the development of the ICF framework is an important milestone that can contribute importantly to the field of physical therapy and rehabilitation. One of the most exciting aspects of the ICF framework is its potential to provide a universal, standardized disablement framework that, if widely adopted, will promote a common, international language that will facilitate communication and scholarly discourse across disciplines as well as across national boundaries, stimulate interdisciplinary research, improve clinical care, and ultimately better inform health policy. Recently, the ICF was endorsed by an Institute of Medicine (IOM) report, The Future of Disability in America.6 Nonetheless, as pointed out in the IOM report, challenges around the operationalization of the ICF's core concepts need to be resolved if the ICF framework is to become an international standard. In this regard, Starrost et al are to be commended for publishing the first study of the interrater reliability of applying the ICF classification qualifiers in the Extended ICF Core Set for Stroke.
The Extended ICF Core Set for Stroke uses the ICF classification approach to capturing information about functioning and disability within each of its core domains. Unlike the overall ICF classification system, which includes more than 1,400 discrete categories within health and health-related domains, the ICF Core Sets comprise more manageable subsets of ICF categories that attempt to comprehensively describe a subset of health and health-related states within a specific health condition. The Extended ICF Core Set for Stroke, for example, includes 166 categories drawn from the ICF categories. For each category within a Core Set, users select appropriate qualifiers, which are codes that specify the extent or the magnitude of the functioning or disability in that category, or the extent to which an environmental factor is a facilitator or barrier. The qualifiers identify the presence and severity of a decrement in functioning for each ICF component (ie, body functions and structures, activities and participation, and environmental factors).
A 5-point ordinal scale is used to record the severity of impairment for each category as: no, mild, moderate, severe, or complete impairment. To illustrate using a case example, one could use the ICF coding approach to classify the walking ability of Mrs Jones, an 80-year-old woman who had a stroke, as ICF code d450.32. This particular ICF code denotes that walking is in the Activities and Participation domain (alphabetical code d), the mobility domain (chapter 4), the walking category (50), and severe performance difficulty (3) and moderate difficulty with capacity (2).
The ICF Core Set coding approach to capturing ICF function and disability information offers several attractive advantages. ICF coding establishes a common language for describing health-related states and provides a systematic classification scheme for health information systems. The comprehensiveness of ICF's classification system facilitates the widespread and systematic recording of functional status data in medical and rehabilitation patient records and could influence clinical care.7–9 ICF coding also could permit comparison of health and health-related states across patients, studies, and countries, as well as across clinical services. ICF codes can be used to compare the distribution and determinants of health-related states of different populations, to predict health-system usage and costs, and to provide evidence for social policies and laws.
To be operationally useful, however, the ICF classification system must meet key methodological standards, one of which is acceptable reliability of the ratings when performed by different raters. In this regard, I question Starrost and colleagues’ conclusion that the findings from their study may be regarded as a "promising starting point." I think their study raises serious questions about the methodological adequacy of the ICF Core Set ratings.
In their study, the authors used a cross-sectional reliability design to examine the consistency of ICF ratings when applied by 2 therapists for 30 outpatients who had a stroke. The therapists were well trained and received information from several different sources prior to making their ratings. Their training consisted of workshops, a pilot phase with discussion of a case before starting the study, and the supervision of an expert from the ICF Research Branch, Munich, Germany. The therapists applied the ICF ratings using their clinical judgment based on information gleaned from a patient interview and observation, proxy interview, information provided by different professionals, and data from the medical records containing the results of standardized examinations. However, despite all this careful preparation and attending the same patient interviews, the overall result for inter-therapist agreement was a kappa value of .41, which they interpreted as poor to moderate. Even more troubling, almost half (48%) of the kappa coefficients for the therapists’ ratings of these patients did not achieve agreement levels beyond chance. Despite this extensive level of therapist preparation and access to several different sources of information about the patient, the therapists had considerable disagreement as to the rating for individual ICF categories. These findings raise serious concerns as to whether the ICF classification approach is methodologically adequate. As similar findings have been reported for investigations of other ICF Core Sets, these methodological concerns appear inherent in the approach and not limited to the Extended ICF Core Set for Stroke.10–12
Are there feasible alternatives to the ICF Core Set approach that might be considered for operationalizing the ICF core concepts? One proposed alternative to categorical ratings like the ICF coding approach is to construct quantitative scales from multiple items used to assess each ICF core concept, possibly using items generated within various ICF Core Sets.13 By pooling information that functional items have in common, quantitative scales could be constructed to assess various ICF core concepts. Methodological techniques such as item response theory (IRT) methods are well suited to the construction of interval level scales designed to measure targeted ICF core concepts.14 Some examples already exist in the published literature.15,16
In contrast to the ICF's approach to applying severity ratings to each discrete function and disability category, the generation of quantitative scale scores based on standardized instruments would provide summary quantitative scores made up of multiple items. For example, one can use the Activity Measure for Post-Acute Care (AM-PAC) to illustrate how a quantitative scale could be used to measure the walking ability of Mrs Jones, our 80-year-old woman who had a stroke. The AM-PAC is an IRT-derived quantitative scale that consists of 101 functional activity items guided by the ICF activity subdomains.15 Its basic mobility scale includes an assessment of walking ability along with numerous other basic mobility activities. As an interval measure, if Mrs Jones reported she had great difficulty walking indoors, was unable to walk outside, and had some difficulty getting out of an armless chair, she would receive a score of 42 on the AM-PAC mobility scale, where the mean mobility score for patients receiving post–acute care is 50, with a standard deviation of ±10 points on this quantitative scale.
Scaling techniques, such as those used in constructing the AM-PAC, increase score reliability beyond the level that can be achieved for individual items. Scaling techniques enhance measurement validity by providing a more representative sample of information about an underlying functional domain. Scaling techniques also help decrease score variability and increase measurement precision and responsiveness, which consequently increases the ability to detect meaningful change over time.17 The precision of a multi-item measurement can be estimated, for example, by calculating useful statistics such as the standard error of measure (SEM), which, in turn, can be used to obtain confidence intervals to help interpret the meaning of functional change scores.18
These features of quantitative scales permit their use in discriminating between groups and tracking change in patient function and disability over time. Unlike discrete ICF Core Set ratings, change scores between 2 administrations of a quantitative assessment at different time points can be calculated in order to examine change in function or an effect of treatment. For example, with the SEM, we can examine whether score changes are statistically significant by comparing 2 confidence intervals. Also in contrast to the ICF coding approach, a quantitative scale increases the ease of assessment and reporting of results of measurement by reducing the sheer number of final scores or data elements required.
Problems identified with the ICF Core Set approach such as those reported by Starrost et al underscore the need to explore a range of approaches to assessing the ICF core concepts. As an alternative to the ICF Core Sets approach, I recommend that more research be undertaken to expand and evaluate the development and interpretation of quantitative approaches to operationalizing key ICF concepts. Future measurement research initiatives may yield important new information that may influence future revisions of the ICF framework.
 | References |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  S. Geyh, G. Stucki, and A. Cieza Author Response Physical Therapy, July 1, 2008; 88(7): 854 - 856. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
