| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Research Reports |
DH Kothari, PT, MS, is Research Associate, VA Rehabilitation Research and Development Center (Building 51), 3801 Miranda Ave, Palo Alto, CA 94304 (USA) (dhara{at}rrdmail.stanford.edu).
SM Haley, PT, PhD, is Director, Center of Rehabilitation Effectiveness, and Associate Professor of Physical Therapy, Sargent College of Health and Rehabilitation Sciences, Boston University, Boston, Mass
KM Gill-Body, PT, MS, DPT, NCS, is Clinical Associate Professor, MGH Institute of Health Professions, and Clinical Associate, Massachusetts General Hospital, Physical Therapy Services, Boston, Mass
HM Dumas, PT, MS, PCS, is Manager, The Research Center for Children With Special Health Care Needs, Franciscan Children's Hospital and Rehabilitation Center, Boston, Mass
Address all correspondence to Ms Kothari
Submitted December 6, 2002;
Accepted May 5, 2003
 | Abstract |
|---|
Key Words: Item response theory • Outcome measurement • Pediatric brain injury • Pediatric Evaluation of Disability Inventory • Rasch measurement
| Introduction |
|---|
 TopAbstract Introduction MethodResultsDiscussionConclusionReferences |
|---|
The PEDI1,2 scale structure is depicted in Figure 1. The PEDI consists of 3 parts: (1) Functional Skills (197 items), (2) Caregiver Assistance (20 items), and (3) Modifications (20 items). Capability of a child is assessed by the identification of skills for which the child has demonstrated mastery and competence using the PEDI Functional Skills part. Level of performance is measured by the amount of caregiver assistance needed by the child to accomplish major functional activities included in the Caregiver Assistance part. Each part consists of items that focus on 3 content areas or domains: (1) Self-care, (2) Mobility, and (3) Social Function. Our study focuses only on the Self-care and Mobility domains. The Self-care domain includes activities such as eating, grooming, dressing, bathing and toileting. The Mobility domain includes activities such as floor mobility, ambulation, transfers, and mobility in different environments with or without adaptive equipment. Each domain contains a Functional Skills subscale, a Caregiver Assistance subscale, and a Modifications subscale. The subscales included in our study are shown in Figure 1. On the Functional Skills subscales, using a dichotomous (not capable/capable) scoring system, examiners indicate the child's capability to perform (without physical assistance) the 73 self-care tasks and the 59 mobility tasks. The Caregiver Assistance subscales consist of complex functional activities, such as lower-body dressing or car transfers, in which the amount of assistance provided is assessed on a 6-point ordinal scale (polytomous) ranging from "total assistance" (0) to "independent" (5).
|
Data are used to construct these item positions. Items that are easier for most children (eg, crawling, sitting, indoor walking) are placed at the less difficult end of the measurement subscale, and items that are harder for most children (eg, walking outdoors, use of stairs) are placed at the more difficult end of the measurement subscale. Figure 2 is a schematic representation of the Functional skills Mobility subscale. The Rasch measurement scales enable an estimation of item positions and person scores on the same scale as depicted in Figure 2. Other instruments used commonly in pediatric physical therapy also have utilized a one-parameter Rasch model for scale validation and positioning items along constructs such as gross motor skills (Gross Motor Function Measure).6
|
In practice, item positions and the order of items within functional instruments across samples can vary slightly.10–13 For instance, Haley et al previously found that the physical functioning items of the SF-36 varied slightly across people with different chronic conditions,11 and the subsequent variation in sensitivity of these measurements was tolerable, given the purpose of measuring change in physical functioning in large group analyses.12 One major unanswered question with the generic PEDI scale is whether its subscales are truly applicable to all samples. For example, are the PEDI item positions and order (hierarchy) applicable for measuring function in children with various conditions? If differences exist in the order of items across samples, this leads to the question of whether variability in item positions or item order across samples has an impact on the calculation of summary and change scores on the PEDI. To date, there is only one report of an examination of the stability of the PEDI item positions. Haley and colleagues14 reported only minor differences in item positions of the Caregiver Assistance Mobility and Self-care subscales of the PEDI when measurements from home and school settings were compared. This finding, we believe, indicates that the Caregiver Assistance subscales of the PEDI can be used to monitor progress over time in children when the initial and follow-up assessments are performed in different settings.
In our study, a sample of children with ABI was used to examine these 2 questions. We chose to study children with ABI because ABI is a leading cause of disability in childhood15,16 and accounts for a large number of pediatric rehabilitation hospital admissions per year in the United States.17 This cohort of children with disabilities would represent a large percentage of the caseload for rehabilitation specialists such as physical therapists and occupational therapists. A second reason we chose to study children with ABI was because of the observed variability in the course and outcome of recovery in this population. We would expect that this population would differ from the typically developing population in the order of skill attainment during recovery.
We believe there are major practical advantages in using a generic functional assessment instrument such as the PEDI that can be applied to children with a variety of conditions if PEDI measurements can be shown to be sample free and the order of items does not influence test sensitivity. We proposed to test the applicability of the generic PEDI in children with ABI. The purpose of our study was to test the congruence of functional item positions along Rasch measurement scales from a sample of children without functional disabilities and an ABI-specific sample. Further, if item positions among the measurement scales differ, we predicted that these item locations and order disparities would have little or no effect on the sensitivity of the PEDI summary scores to detect changes in children with ABI during inpatient rehabilitation hospitalization.
| Method |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
We selected this particular cohort of children with ABI because we had complete item-level data available in the hospital records. The diagnostic classifications of ABI included traumatic brain injury (58.6%), seizures (4.6%), brain tumors (13.8%), hemorrhage (9.2%), anoxia (3.4%), and cerebral infarct (2.3%). Of the potentially available 94 records, 87 records were complete with admission and discharge PEDI administrations, although a few PEDI scales had missing or unusable data. Demographic data are presented in Table 1. We did not stratify the sample based on diagnostic classification within the broad category of ABI because of the effect of we believe such stratification would have in reducing the sample size needed for the creation of a reliable ABI measurement scale.
|
Wright and Boschen19 evaluated the intrarater and interrater reliability of scores for multiple respondents (physical therapists, occupational therapists, parent, and teacher) when using the PEDI Functional Skills part for all 3 domains with 40 children with varying severities of cerebral palsy aged 3 to 7 years.20 The Functional Skills part of the PEDI yielded ICCs greater than .95 for all the total scores and ICCs greater than .80 for all 3 domains within each respondent group. Between the respondent groups, the Functional Skills Self-care subscale yielded ICCs of .87 and the Functional Skills Mobility subscale yielded ICCs of .81.
Haley at al20 examined the content validity of PEDI scores using a panel of 31 experts in the field of pediatric rehabilitation. The experts were requested to complete a questionnaire to determine whether the PEDI measured function and the extent to which the items on the PEDI sampled pediatric functional activities. The results of this study, we believe, served to validate and confirm the functional content of the PEDI. Feldman et al21 examined what they considered construct validity of the PEDI by testing whether it could be used to discriminate between children without functional disabilities (n=20) and children with functional disabilities (n=20). Their results showed that PEDI Mobility and Self-care domain scores were different between the children with disabilities and the children without disabilities.
PEDI Subscales Based on Data From a Sample of Children Without Functional Disabilties (Generic PEDI Scale)
During the initial standardization of the PEDI, data were collected from 412 children without functional who ranged in age from 6 months to 7.5 years. These data were used to define the "standard measurement scale" for newly acquired functional items. Details of the sampling approach are given in the PEDI manual.1 The Rasch model transforms raw scores into interval measures by using log-odd units (logits). Logits run from positive infinity to negative infinity. To make the scores more easily interpretable, a linear transformation is applied,4 which calibrates the scale from 0 to 100 (as shown in Fig. 2) instead of logit units. Thus, scores on each scale could range from 0 to 100, with scores approaching 0 indicating limited functional ability and scores approaching 100 indicating problem-free ability. Larger ability scores indicate an increase in the level of capability in performing a functional skill and an increase in the level of independence to perform complex functional activities. Using a simplified example with selected items from the Functional Skills Mobility subscale, the number of "capable" items is converted to a transformed score on a 0 to 100 scale. Using the example in Figure 2, if the child is capable of successfully completing only one item, it is most likely to be the item that is at the bottom of the hierarchy; in this case, the item would be "crawls, creeps," and the child would be assigned a score of approximately 6. If a child is capable of successfully completing 6 items, the child's performance is converted to a score of approximately 50 (based on the expected performance of all items up to and including "walks outdoors"). Scores on the PEDI are directly related to the positions of the items along the measurement subscale.
The transformed scores do not take into account the age of the child, but they provide an estimate of the level of capability in each domain. For this reason, transformed scores theoretically can be used for children of all ages. The values of the transformed scores, derived from a sample without functional disabilities, are organized in the form of tables in the PEDI manual,1 such that each possible raw summary score has a corresponding transformed score value based on the item locations within that scale.
PEDI Scales Based on Data From ABI Sample (ABI-Specific PEDI)
Individual item scores of the Functional Skills and Caregiver Assistance Self-care and Mobility sub–scales of the PEDI obtained shortly after hospital admission and then at the time of discharge were analyzed using WINSTEPS,22,23 a Rasch computer program. One approach is to fix the calibrations of the discharge data.24 This means that the discharge data were used to create the ABI-specific item ruler. This was done because the variability of the items was greater at discharge, as some subjects improved small amounts, whereas others improved much more. Anchoring locations to the discharge data allowed construction of hierarchical scales based on order of item attainment following recovery. At the time of admission, many of the items were truncated due to the subjects' limited functional performance, and therefore admission scores could not be used to develop the hierarchical scales. Thus, Rasch analysis was used to estimate item locations based on discharge scores. The summary score estimates at admission and discharge were then computed using the measurement scale transformations based on the discharge item locations.
Data Analysis
To estimate the degree of correspondence of the item locations between the generic and ABI-specific scales, Pearson product moment correlation coefficients were used to evaluate the strength of the relationship between average logit item locations obtained from use of the 2 scales. A series of z tests4 were then performed on the respective pairs of item positions to identify those items that had different positions between the 2 scales (P<.05). A critical value of 1.96 was used to identify those items that had item positions that were different (P<.05) between the 2 scales.4 These individual tests were not corrected for a Type I error because their purpose is to locate any potential discrepancy between conditions. This strategy does not protect against making a Type I error but rather the commission of a Type II error, that is, overlooking an item pair with potential differences.4 We conducted separate analyses for each of the subscales within Mobility and Self-care domains.
We used paired t tests to evaluate whether the mean change scores between the admission and discharge data of our study sample were different from zero. The responsiveness of the 2 scales (ABI-specific and generic), to assess change, was then determined on the basis of relative precision (RP)12 estimates and effect size. Relative precision is a ratio of pair-wise F statistics (Fclinical/Fnormative). F is equal to t2; the value of t was obtained from the paired t tests. In our study, the RP estimates indicated, in proportional terms, how much more or less precise the ABI-specific PEDI scales were than the generic PEDI scales in detecting change in functional status in children with ABI.
Relative precision depends on 2 elements: the magnitude of difference between the 2 test occasions being compared and within-group variance. We hypothesized that the RP estimate would be 1; that is, both scales would be equally sensitive to change, or the magnitude of change measured by both scales would be equal. An RP estimate of greater than 1 would indicate that the ABI-specific scale is more sensitive to change in the functional status of children with brain injury than the generic scale. The level of significance was set at P<.05. The effect size for each scale was calculated as the ratio of the mean change score to the standard deviation of the score at admission. To evaluate the responsiveness of the 2 scales, data sets of only those children with complete admission and discharge scores were used for analysis.
| Results |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
Differences in Item Locations
Our results showed differences in item position and order between the generic and ABI-specific scales. The estimated item locations along a continuum for either the Self-care or Mobility domain suggest a relative placement of activity difficulties within a sample. In the convention we have adopted, item locations with small values mean that most children can accomplish the activity, whereas item locations with larger values indicate that the items are more difficult to accomplish. Overall, a greater number of the Mobility domain items differed in their item locations between the 2 scales as compared with the Self-care domain items (Tab. 2). In general, with the ABI-specific scale, items common to both the Functional Skills and Caregiver Assistance parts that yielded larger item location estimates included the locomotor items in the Mobility domain and eating items in the Self-care domain. This finding indicates that these items were more difficult to accomplish for most of the children. Items with smaller item location estimates included the transfer items and stairs in the Mobility domain and grooming in the Self-care domain, which indicates to us that these items were fairly simple to accomplish for most of the children. Items specific to the Functional Skills part with a smaller item location estimate included bathing and with a larger item location estimate included certain lower-body dressing skills for the Self-care domain. (For a full list of the specific item calibrations and differences across the 2 scales for each part or subscale, contact the first author [DHK].)
|
|
|
| Discussion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
Scoring Implications
Our findings, we contend, have important implications for using the PEDI in clinical outcome studies where the magnitude of the change is examined. Our results suggest that the PEDI scores obtained by examining children without disabilities are acceptable for detecting change in children with ABI. Although a substantial number of items had different paired item locations across samples, the only major difference in RP estimates of the 2 scales was only on one subscale (Caregiver Assistance Self-care subscale). We maintained an alpha level of .05 when identifying item pairs that had calibrations that were different. This means that the Type I family-wise error rate most likely exceeded .05 because of the multiple tests, and consequently some of the differences we observed in item pairs may have been due to chance. This strategy, however, enabled us to identify any potential item that might be influenced by the difference in samples. One of the major influences on the differences in paired calibrations was more limited variability in the ABI-specific scale in comparison with the generic scale. Even though we used the discharge item calibrations (greater variability than the calibrations at admission) as an anchor for the ABI scale, yet the variability of the ABI calibrations approached only 70% to 80% of that of the sample without functional disabilities.
The scoring impact of the ABI-specific scale can be understood by examining the distribution of the rating scale categories for the Caregiver Assistance Self-care and Mobility subscales (Figs. 3 and 4, respectively). We have highlighted in the figures the positions of the scale categories for the item "bathing" on the Caregiver Assistance Self-care subscale and the item "stairs" on the Caregiver Assistance Mobility subscale. The relative distances between scale categories, however, is the same for all Self-care and Mobility domain items within each subscale. Each item on the Caregiver Assistance Self-care and Mobility subscales is scored on a 6-point scale, from "total assistance" (0) to "independent" (5). In Figures 3 and 4, we have identified scale locations for "maximum assistance=1," "moderate assistance=2," "minimal assistance=3," "supervision=4," and "independent=5."
|
|
For the item "stairs" on the Caregiver Assistance scale of the Mobility domain (Fig. 4), there was a wide spread in the location of the scale categories for the ABI-specific scale as compared with the generic scale, but this difference was not sufficient to have an impact on scale sensitivity (RP estimate=1.04). Although the ABI-specific scale was not found to be more sensitive than the generic scale, the relative distances between its scale points may reflect the patterns of behavior specific to children with ABI. As shown in Figure 4, the location of the scale category "stairs-independent" was relatively higher on the ABI-specific scale than on the generic scale. This finding may reflect the need for more close supervision in children with ABI on stairs than in children without functional disabilities who may not exhibit the same degree of variability in safety behaviors around stairs. Future work with ABI-specific scale may improve the scale so that physical therapists can better describe individual patterns of functional recovery.
Clinical Implications
Our study indicates that early stages of recovery in children with ABI do not necessarily follow the same pattern of re-acquiring competence and independence in performance of functional activities as occur with children who are typically developing. This is indicated by the differences found in the order and spacing of the item locations on the Functional Skills and Caregiver Assistance Mobility and Self-care subscales between the ABI-specific and generic models. Our findings are consistent with those of several other researchers16,25–27 who examined the short- and long-term manifestations in children with brain injury and found disruption of normal development secondary to multiple deficits in motor, communicative, cognitive, sensory, behavioral, and emotional systems.
Paired items with different calibrations between samples were closely inspected to identify the possible underlying themes. A group of items occurred relatively high in the hierarchic sequence for children with ABI as compared with children without functional disabilities on the PEDI Functional Skills part. This group included items such as indoor locomotion, outdoor locomotion, and tub transfers in the Mobility domain and items such as lower-body dressing and manipulation in the Self-care domain. Capability on these Mobility domain items is believed to require balance, appropriate reaction times to prevent falls, endurance, and peer-matched speed and muscle force, whereas success on the Self-care domain items is believed to require grasping strength, precision, eye-hand coordination, and bilateral coordination, which are common deficit areas after brain injury.28
A second group of items occurred relatively earlier in the sequence of items for the children with ABI than for the children without functional disabilities. These items included basic transfer skills, stairs, and bathing and grooming. We believe there are a couple of reasons for this pattern. Transfer skills pose contextual challenges related to height, such as the chair or surface height, size of the toilet, and height of the bed or crib. The sample with ABI consisted of children who were older (age range=1–20 years; 54% of the children were above 7.5 years of age) than the children without disabilities (age range=6 months to 7.5 years). The pattern seen in grooming and bathing skills also may be age-related, as hygiene tasks are expected of older children and are especially important to adolescents. The use of adaptive equipment such as sliding boards or tub benches may facilitate the relearning of transfer skills in children with ABI.
We noted similar patterns of item sequence differences for the Caregiver Assistance part. Items higher in the hierarchic sequence in children with ABI included indoor and outdoor locomotion and eating, all potentially related to concerns with safety and continued need for supervision. Coster and Haley29 have presented a conceptual model of pediatric disability that underscores the influence of "context" in the ability of a child to perform a functional task. Functional independence is highly dependent on opportunities presented by the social and physical environments that a child experiences.30 The differences in the item calibrations between the 2 scales also may be partially due to the different settings in which the sample of children with ABI and the sample of children without functional disabilities were assessed (hospital versus community, respectively).
The relatively small sample size of children with ABI limits the generalizability of our conclusions. We considered traditional Rasch model issues of item and person misfit, item and person separation, rating scale threshold estimates, and diagnostic analysis of the residuals to be irrelevant for the purpose of our study. As in the original development of the PEDI, misfit issues and residual analyses are crucial for establishing the soundness of the psychometric characteristics of a new measurement instrument. In such an application, we would argue that the fit of the data to the model should be aggressively undertaken from many different perspectives. In our analysis, however, no intent was presumed. Rather, the analyses were conducted to establish whether a difference in the ABI-specific and generic scales would have an impact on summary scores.
Despite the limitations of a relatively small sample, we believe our analyses have laid important groundwork for understanding the impact of condition-specific scales on summary scores. The statistical effect of the small sample was to yield standard errors of item estimates that were larger than would have been the case if more clinical data were available from more subjects. Furthermore, we recognize that a small sample size cannot adequately represent the diversity of functional patterns seen in the recovery of children with ABI. Researchers in the future should seek to verify the item hierarchy in a well-controlled prospective study with a larger group of children recovering from ABI and to establish how that knowledge can be translated to improve physical therapists' management of patients or clients.
| Conclusion |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
| Footnotes |
|---|
Ms Kothari was a graduate student at the MGH Institute of Health Professions during this study, which was undertaken in partial fulfillment of the requirements for the post-professional degree of Master of Science in Physical Therapy.
The Spaulding Rehabilitation Hospital Institutional Review Board and the Human Subjects Committee at Franciscan Children's Hospital and Rehabilitation Center approved this study.
| References |
|---|
|
TopAbstractIntroductionMethodResultsDiscussionConclusionReferences |
|---|
CiteULike 
Complore 
Connotea 
Del.icio.us 
Digg 
Reddit 
Technorati What's this?
This article has been cited by other articles:
|
|
 |
|
  Y.-P. Chiu, S. L Fritz, K. E Light, and C. A Velozo Use of Item Response Analysis to Investigate Measurement Properties and Clinical Validity of Data for the Dynamic Gait Index Physical Therapy, June 1, 2006; 86(6): 778 - 787. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. M. Haley, R. J. Graham, and H. M. Dumas Outcome Rating Scales for Pediatric Head Injury J Intensive Care Med, July 1, 2004; 19(4): 205 - 219. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
