This perspective article provides an example of a study planned using guidelines for comprehensive rehabilitation outcomes research, an approach that is believed to give service providers meaningful evidence to support practice. This line of investigation has been guided by the World Health Organization's International Classification of Functioning, Disability and Health. The short title of a study under way is Move & PLAY (Movement and Participation in Life Activities of Young Children). The article briefly describes the conceptual model, provides guidelines on how indicators and measures are selected, alludes to the details of selected measures, and describes processes of preparing for data collection, including obtaining ethics approval, preparing data collection booklets, training assessors and interviewers, and sampling. The aim of this investigation is to gain a better understanding of the multiple child, family, and service factors associated with changes in mobility, self-care, and play of preschool children with cerebral palsy as a result of using this research method. Comprehensive rehabilitation outcomes research holds promise in providing evidence that supports the complexities of planning rehabilitation services with clients with chronic conditions, such as children with cerebral palsy.
Rehabilitation service providers work with clients with a range of conditions whose outcomes are associated with many factors, including, but not limited to, the services they receive. Traditionally, outcomes of health care have been attributed solely to 2 aspects of services provided: the structure of practice (ie, the training of care providers and the nature of equipment and the facility) and the process of delivering care (ie, the selection of the appropriate intervention and skillful administration).1 Kane2 suggested that outcomes also are a result of patient-specific factors such as baseline status and clinical, demographic, and psychological characteristics, as well as the treatment and the setting in which the treatment was conducted. More recently, emerging conceptualizations of “disability” or “rehabilitation” outcomes research3,4 recognize that outcomes are attributable to a complex array of factors, both within the individuals involved and within the environments in which they live.
This complexity poses a challenge for clinicians who strive to be evidence-based. To be useful, research evidence to support clinical practice must reflect this complexity; increasingly, there is a call for research to meet this need.5–7 In contrast to using the standard reductionist approaches to answering clinical questions in rehabilitation, Collins proposed that “inclusion of complex systems approaches to theory, study design, methods, conduct and analysis may…enhance the clinical relevance of research.”6(p1394) We concur with other authors8–10 who believe that research needs to make use of, rather than eliminate, sources of variation in outcomes and that phenomena, such as early motor development of children with cerebral palsy (CP), need to be studied holistically. By “holistically,” we mean that key multiple determinants relating to the outcome of interest need to be considered. These determinants can be within individuals with a specific health condition or within their contexts, or they can be an interaction between personal and environmental factors. In short, rehabilitation service providers need research evidence that reflects the “messiness” of their clinical realities.11 We have embraced “comprehensive rehabilitation outcomes research”5 as a method that acknowledges complexity, is consistent with client-centered care, brings clinicians and researchers together in collaboration, and bridges the gap between conducting research and incorporating it into practice. The goal of comprehensive rehabilitation outcomes research is to provide an understanding of the complex influences on health status and outcomes, including both direct and indirect effects, for clients with chronic conditions. Comprehensive rehabilitation outcomes research is described in detail in an article published in 20045; a brief overview is provided next.
Conducting Comprehensive Rehabilitation Outcomes Research
A first step is to work collaboratively with a team to develop a conceptual model using personal perspectives, theory, and research evidence to inform model development. At a minimum, team members consist of clients, clinicians, and researchers who work together to identify relevant outcomes and determinants of those outcomes and to propose relationships among the identified constructs for a target clinical population. These constructs are identified as “latent” because they are not directly measured. Accordingly, the next step is to identify indicators of the latent constructs. Again, these constructs are identified collaboratively using personal perspectives, theory, and research evidence. Following the identification of key indicators for each construct, it is primarily the researchers' responsibility to identify measures that provide reliable and valid estimates of these indicators and that are acceptable to clients and clinically feasible to administer. A challenge of this type of work is a lack of existing measures with psychometrically sound properties; thus, there is frequently a need to develop and test some measures in the context of a study. Prior to collecting data, ethical approval and funding or in-kind support must be obtained.
A second challenge in conducting this type of research is that the required sample sizes are large and significant resources are needed to recruit and retain participants and implement data collection at multiple sites. Traditionally, granting agencies have not been enthusiastic about supporting this type of research, in part because “outcomes research” is considered a relatively low form of evidence (ie, level 2c in the Levels of Evidence Table produced by the Oxford Centre for Evidence-Based Medicine12). Appropriate research, however, should be used to answer the posed research questions. Our group believes that level 2c is the best design option for comprehensive rehabilitation outcomes research.13
Once the data have been collected, analytical approaches including structural equation modeling (SEM) have been identified as ideally suited to complex rehabilitation research.14,15 Using SEM, one is able to test the fit between data collected on the indicators of constructs and the conceptual model, as specified by the research team. Importantly, one is able to test direct and indirect effects among constructs simultaneously. Analysis poses a third challenge to this type of research; typically a statistical consultant with a high level of skill is required, adding to the costs of conducting this type of work. Once the data have been analyzed, the team is assembled to discuss interpretation and prepare a range of dissemination materials, including both peer-reviewed and non–peer-reviewed products. Because clients and clinicians have been collaborators throughout, meaningful knowledge translation is facilitated.5
Adoption of the biopsychosocial model of disability described in the World Health Organization's International Classification of Functioning, Disability and Health (ICF)16 has facilitated the era of “disability” or “rehabilitation” outcomes research.3,4,17 The ICF and the recently published ICF for children and youth18 consider “body structure and function” (or impairments), “activity” (what people can do), and “participation” (what people actually do) as being modified by “personal factors” (things that are associated with people, but not their health conditions) and physical, social, and attitudinal aspects of the “environment.” We distinguish between primary impairments (present at the outset of a condition) and secondary impairments (arising over time as a result of a primary impairment)19 because of the important clinical focus on tertiary prevention for rehabilitation service providers working with clients with chronic conditions.20,21 Using the ICF, “functioning” is the product of the complex interaction among all of these components, which need to be considered when planning both comprehensive rehabilitation outcomes research and rehabilitation services for individual clients.
Pros and Cons of This Type of Research
We started this perspective by highlighting that services are one of many determinants of an outcome. We need to be clear that we are indeed interested in the impact of services, in the context of other determinants, on outcomes. A limitation of comprehensive rehabilitation outcomes research is that it cannot establish causality of services definitively; it can only illuminate associations among a range of determinants and outcomes. The only research design that can establish causality definitively is the randomized controlled trial (RCT). The RCT was designed for drug trials in which a single intervention is provided and compared with a placebo. The RCT provides evidence of effectiveness of an intervention that is obtained under standardized and controlled conditions. The challenge in applying the RCT to physical therapy is that patients/clients receive multiple interventions (communication and coordination, patient/client-related instruction, and procedural interventions) and standardized conditions may not be feasible in routine practice. The RCT is used to determine group responses. The assumption is that individual differences are either controlled or eliminated through randomization22,23 and that there is little or no interaction of the intervention with personal or environmental factors.
If, however, one is interested in studying a rehabilitation intervention that is multidimensional, complex and interactive, and individualized, observational designs might provide the best options, assuming the sample size is sufficiently large.8 As indicated earlier, like other authors, we are interested in using a design that makes use of, rather than eliminates, the natural inter-individual variability that is commonly associated with the people we see in practice.9,24–26 Figure 1 illustrates the spectrum of factors that contribute to the decision to do an RCT (a study of difference) or a study to understand the natural variation of a phenomenon and its determinants (a study of association). Comprehensive rehabilitation outcomes research is a study of association. Structural equation modeling, the analytical method used in this approach, relies on variance and covariance matrices as data input; it capitalizes on the relationships among factors.
Although this approach cannot establish a causal link between an aspect of intervention and a specific outcome, standard epidemiological principles are used to increase the probability or likelihood that at least part of the outcome is associated with the intervention. These principles include temporality, reversibility, strength, dose-response, consistency, and plausibility, as well as ruling out alternative explanations.27 In short, detailed critical appraisal of results is required, in the context of existing literature, to assist with interpretation of study results. Observational designs do not eliminate potential confounding factors. In this design, it is of interest to ascertain the contribution of all possible determinants, including confounding factors, together on the outcome of interest. In reality, this type of research reflects actual practice; therefore, findings of association should add rich information to the evidence base for practice. On balance, and despite some inherent limitations, we feel that comprehensive rehabilitation outcomes research is best-suited to our research agenda.
The purposes of this article are: (1) to describe an example of comprehensive rehabilitation outcomes research that is guided by the ICF, (2) to provide an overview of the measurement model of a study in progress, (3) to describe the preparation for data collection, and (4) to outline the next steps of our work, as well as to discuss the clinical implications of conducting this type of research. Communicating the results of this type of work is difficult because of the complexity of the approach. Thus, we have elected to prepare separate articles to describe in detail our updated conceptual model (unpublished research) and our measurement model (in this article). We plan to prepare future articles that report in detail the sample and analytic results and specific implications for rehabilitation practice.
Our Context: The Move & PLAY Study
A dilemma facing rehabilitation practitioners working with children with CP and their families is that the evidence base supporting current practice from a comprehensive perspective is very weak. We are conducting a multi-site study to test a multivariate conceptual model of determinants of basic motor abilities, self-care, and play that is consistent with systems theory,28 theories of human ecology,29 and a philosophical approach incorporating family-centered care30 in the context of the framework of the ICF16,18 of functionally distinct groups of young children with CP classified using the Gross Motor Function Classification System (GMFCS).31 The short title of our study is Move & PLAY (referring to “Movement and Participation in Life Activities of Young children). In a related article (unpublished research), we will describe the development of the conceptual model in detail (illustrated in Fig. 2).
Our outcomes of interest include the acquisition of basic motor abilities (an activity-level variable) and engagement in self-care and play (participation-level variables). Determinants of these outcomes include the primary and secondary impairments and associated and comorbid health problems related to the CP diagnosis and personal child factors that are unrelated to having a diagnosis of CP. In addition, 2 aspects of the environment are considered: (1) supports provided by families and (2) supports provided by rehabilitation and community services. The ICF offers a framework for understanding the biopsychosocial nature of disability that is compatible with systems theory and theories of human ecology. These all suggest that development occurs in a specific context as a result of the interaction of many elements of both the person and environment, with family environment being a key aspect for children. In accordance with family-centered care, rehabilitation and community services are considered in the family context.
The Measurement Model for the Move & PLAY Study
Selection of Indicators
Guided by the original conceptual model,19 and in the spirit of parsimony, physical therapists in the province of Ontario assisted in the development of the measurement model by coming to a consensus about the most important indicators of the specified constructs that are associated with the acquisition of basic motor abilities among young children with CP.32 The indicators that they selected were consistent with the available research literature at that time,32 and these indicators provided a starting point for measurement selection. The indicators that we used in the Move & PLAY study were refined by study investigators in the process of grant preparation and are listed in Tables 1 and 2.
Selection of Measures
As recommended by Andresen,50 measures have been selected with the following criteria in mind: (1) the instrument is a valid measure of an indicator of a specific construct in our model; (2) the instrument can be reliably administered to young children with CP and their parents; (3) the instrument is feasible to administer in a community setting; (4) the length and content of each instrument are acceptable to children and their parents; (5) the instrument is easy to administer, score, and interpret; (6) the instrument (if an indicator of a determinant) has properties of a discriminative tool, is sensitive to variations among children and families, and is not subject to either ceiling or floor effects; and (7) the instrument (if an indicator of an outcome) has properties of an evaluative tool (ie, is sensitive to change). In addition, compatible with the requirements of SEM,51 the instruments selected to measure the latent constructs should: (1) be able to detect intersubject variability, (2) provide relatively normally distributed data, and (3) have a linear relationship to the dependent variable or construct. The latter 2 points will be dealt with at the analysis stage; transformations of variables that are not normally distributed or have a nonlinear relationship to the dependent construct will be considered. In considering the criterion of “functionality,” we opted for instruments that measure behavior in context. For those measures that either did not have psychometric information or were developed for the purpose of this study, we built measurement substudies into our current work (see “Psychometric Properties” in Tabs. 1 and 2) for future reporting.
Two parent consultants reviewed the measures that were intended for parent completion, either through self-completion or through a structured interview. We asked them to pay particular attention to the measures that we had developed for this study. The parent consultants recommended revision of terminology to be more family-centered, provision of additional examples to clarify questions on several measures, additional content on some of the measures we developed for the study, and revisions to the instructions of one standardized measure that was perceived to be difficult to complete. A summary of the constructs, indicators, and measures is contained in Tables 1 and 2 for the therapist- and parent-completed measures, respectively, as well as the psychometric properties of these instruments and the time to complete them.
Brief Description of Measures
Outcomes of interest.
Basic motor abilities were measured using the Gross Motor Function Measure (GMFM),37 a reliable and valid evaluative instrument for measuring change in motor function of children with CP. Each item is measured by observation and scored on a 4-point ordinal scale from 0 (“does not initiate”) to 3 (“completes”). We used a reliable and valid abbreviated version of this measure, as well as the Gross Motor Ability Estimator (GMAE), to calculate a total GMFM-66 score.52
The construct “engagement in self-care and play” reflects the participation component of the ICF (ie, involvement in life situations).16,18 This construct was measured using 2 instruments, of which the first is a questionnaire (Tab. 1) developed by the research team. The questionnaire consists of 2 parts: (1) interaction with others and play (11 items), and (2) self-care (7 items). The first part (interaction with others and play) measures the participation and degree of enjoyment of the child in family and community life and in leisure and recreational activities. The 2 dimensions are scored on a 5-point Likert scale: how often a child participates (“very often” to “never”) and the degree of enjoyment (“a great deal” to “not at all”). The second part (self-care) measures the degree to which the child is able to participate in daily self-care activities (feeding, dressing, bathing, and toileting). The ratings for daily self-care activities (“yes, initiates and performs consistently” to “no, unable”) distinguish the need for physical assistance of an adult and, for children who do not require adult assistance, whether the child is able to perform the activity consistently. Average dimension scores for participation, enjoyment, and self-care will be used for analysis.
The second instrument, the Test of Playfulness (version 4),38,39 is an observational assessment of the process and playfulness of play of children aged 6 months to adolescence (Tab. 2). The measure consists of 30 items rated on a 4-point ordinal scale that reflects extent, intensity, or skillfulness of specific behaviors. We collected data from direct observation of parents playing with their children. A total score on the Test of Playfulness is determined using Rasch analysis to account for individual rater leniency in scoring and item-difficulty order.39 The score represents the relative playfulness of the child. Higher scores indicate that the child is more playful.
Spasticity (or hypertonicity) was measured using the Modified Ashworth Scale (MAS).42 Three repetitions of tests to measure spasticity of the elbow flexor and hamstring muscles were performed by moving a joint passively through its range of motion at a standard speed and rating the resistance of stretched muscles on a 6-point scale (from 0, meaning “no resistance,” through 1, +1, 2, 3, and 4, meaning the affected part is “rigid in flexion or extension”). The scoring was rescaled from 0 through 5 for analysis. Analyses are planned based on the average values in the 4 body regions.
Quality of movement was measured by portions of an adapted version of the Gross Motor Performance Measure (GMPM).44 We selected 2 attributes (“dissociated movement” and “coordination”) and aimed for 3 trials on 2 items for each attribute, with the item determined individually based on the child's functional abilities. Attributes were scored on a 4-point ordinal scale from 0 (“severely abnormal”) to 3 (“normal, without cues”) (personal communication, Virginia Wright, Bloorview Kids Rehab, Spring 2007). Analysis will be done on average scores of available data.
Balance was measured using the Pediatric Balance Scale (PBS)45 for children in GMFCS levels I, II, and III. This scale has 14 items, each of which is scored on a 5-point ordinal scale from 0 (“needs assistance to perform”) to 5 (“able to do task independently”). The maximum score is 56, which is determined by summing the item scores. For children in GMFCS levels III, IV, and V, balance was measured using selected items from the “Automatic Reactions” section of the Movement Assessment of Infants (MAI).46 Each item is measured on a 4-point ordinal scale. For our purposes, we have reversed the response options from 1, indicating “no response,” to 4, indicating “complete and consistent response,” so that scaling is consistent among all indicators of the primary impairments. We have plans to amalgamate data across these 2 measures so that we have a continuous measure for children of all GMFCS levels.
Distribution of involvement was as follows: (1) monoplegia—involvement of 1 extremity, (2) hemiplegia—involvement predominantly on one side of the body, (3) diplegia—involvement in the lower extremities greater than the upper extremities, (4) triplegia—involvement of 3 extremities, and (5) quadriplegia—involvement in all 4 limbs, with the upper extremities involved as much as or more than the lower extremities.
Range of motion was measured using the 26-item Spinal Alignment and Range of Motion Measure (SAROMM).49 Each item is scored on a 5-point Likert scale, with 0 indicating “normal alignment and range with or without active correction,” 1 indicating “normal alignment and range with passive correction,” and 2, 3, and 4 indicating fixed deformities or contractures that are “mild,” “moderate,” or “severe,” respectively, based on prespecified cut-points and supported by photographs in the training manual (available at: http://www.canchild.ca/en/measures/saromm.asp). We will use average item scores for analysis.
We approached examination of muscle strength with an emphasis on obtaining an estimate from major muscle groups only. We examined neck and trunk flexors and extensors, hip and knee extensors, ankle plantar flexors, and shoulder flexors bilaterally. Each item was evaluated on a 5-point ordinal scale from 0 (“no initiation of movement against gravity”) to 4 (“full available range against gravity and some or strong resistance”). We plan to use the average item scores for the analysis.
Endurance was measured using a newly constructed, parent-rated questionnaire. This measure contains 10 questions related to activity level and fatigue, 8 of which are answered “never,” “rarely,” “sometimes,” “often,” or “always.” Two additional questions ask how many minutes, on average, the child moves actively, either indoors or outdoors, before needing a rest (with response options of 0, 1–5, 6–10, 11–20, and longer than 20 minutes). For analysis, we will use the average of items reflecting indoor activity, outdoor activity, and energy levels. High scores indicate high levels of endurance.
Associated conditions and comorbidities.
We developed a scale to measure the extent to which health problems influence children's activities based on the associated conditions and comorbidities included in the new international consensus definition of CP32 and categories contained in the ICF.16,18 The content of the health questions includes seeing, hearing, learning and understanding, speaking or communicating, controlling emotions or behavior, seizures or epilepsy, oral motor, teeth and gums, digestion, growth, sleeping, repeated infections, breathing, skin, heart, and pain. Parents responded either “yes” or “no” to each question (eg, “Does your child have problems seeing?”). If parents answered “yes,” they also were asked, “To what extent does this problem affect your child's daily activities?” (with response options of 1 [“not at all”] to 7 [“to a very great extent”]). Descriptive analysis will be conducted based on the number of health problems and the average effect on activity level.
Child adaptive behavior.
Child adaptive behavior was measured using the Early Coping Inventory,34 which is designed for infants and toddlers 4 to 36 months of age; however, we used it for all of the children in the study. It consists of 48 items; for each statement, parents rate their child's behavior from 1 (“not effective”) to 5 (“consistently effective across situations”). We will use the total score in the analysis.
Family functioning was measured using the 90-item Family Environment Scale.35 Items represent the following 3 dimensions (and 10 subscales of family functioning): (1) relationships (cohesion, expressiveness, and conflict), (2) personal growth (independence, achievement orientation, intellectual-cultural orientation, active-recreational orientation, and moral-religious orientation), and (3) system maintenance (organization and control). Each item is scored “true” or “false.” Preliminary analysis will inform which dimensions and subscales are used in the analysis.
Family's Expectations of Child and Family Support to Child are 5- and 6-item measures scaled from 0 (“not at all”) to 7 (“to a very great extent”) that were developed through a consensus process by 9 parents of young children with CP. We will use the average score for each measure for analysis.
The Family Support Scale36 has 18 items listing people and groups who are often helpful to parents raising a young child. Parents are asked to rate the support to family from 1 (“not at all helpful”) to 5 (“extremely helpful”). For the purpose of model testing, we will use the average scores from the following sources: immediate family members; informal supports (friends, other parents, and coworkers); and formal supports (group and professional supports).
Rehabilitation services are being measured by a questionnaire developed by the research team. It consists of 5 sections to reflect the following 5 components of services hypothesized to influence child outcome: (1) types and intensities of the programs and services that a child receives, (2) availability of and access to services, (3) coordination of services, (4) the extent to which services are meeting the child's needs, and (5) the focus of therapy services. We are in the process of determining how to aggregate these data.
Confirming Acceptability of the Measures
Prior to submitting the application for funding consideration, pilot testing was conducted with 6 children with CP across GMFCS levels and their parents. This pilot testing was successful in obtaining better time estimates for completion of all of the measures. For the most part, parents and children found the measures to be acceptable, and we judged them to be feasible to administer in a home, community, or clinical setting. These pilot tests were videotaped; portions of these tapes were used for the development of training and criterion testing materials.
Preparing for Data Collection
Prior to collecting data, 21 ethics approvals across multiple Canadian sites (n=8) and sites in the regions around Philadelphia (n=4), Atlanta (n=2), Oklahoma (n=3), and Seattle (n=4) were obtained. All of these ethics approvals were renewed annually. Clearly, this is an important, but remarkably time-consuming, part of conducting multi-site research.
Preparation of Data Collection Materials
The timing of data collection is reported in Table 3. Two data collection booklets were prepared for time 1: one for the parent for self-completion and one for a therapist assessor for administration and scoring. One data collection booklet was prepared for interviewers at time 2 (aiming for 6 months after time 1). Two data collection booklets were prepared for time 3 (similar to time 1, to be done 1 year later). All of the instruments intended for completion by the parents (either through self-completion or interview) were translated into French and Spanish (if not already available) and were back-translated and checked for accuracy of meaning and modified, as necessary, before they were used. Databases were prepared concurrently with the data collection booklets.
Training and Testing of Data Collectors
All therapist assessors who collected data at times 1 and 3 participated in a regional 1-day training session, having come prepared by reviewing 2 binders containing the procedures and the measures of the study prior to the training day. Prior to collecting data for the study, each assessing therapist was required to complete a calibration procedure with the Test of Playfulness. This procedure involved scoring 12 videotapes of children who had previously been scored by other expert raters. Using standard procedures established by the developer of the test,39 each participant's scores were entered into a normative data set to check his or her responses for conformance to the Rasch model expectations. Participants whose responses did not conform required additional training and repeat testing until their responses more closely matched the responses of others.
In addition, prior to collecting data, all assessors were required to obtain greater than 80% item agreement on videotaped clips of selected items on a variety of measures with the gold standard (determined via consensus among the investigators). The average item percentage agreement is provided in parentheses after each of the following measures: the GMFM37 (91.4%), the PBS45 (91.4%), selected items from the MAI46 (90.8%), the SAROMM49 (88.9%), and the GMPM44 (92.1%). The results of this criterion testing will be useful when deciding on the magnitude of measurement error53 to build into the structural equation model for final analysis. Also, we are prepared to consider greater error for physical assessments that did not undergo criterion testing (ie, the MAS and the strength measure, which are impossible to score from a video clip). In addition, therapists attained acceptable classification of children using the GMFCS on the training workshop day or shortly thereafter. Fidelity checking was done in all regions to ensure that data were being collected as planned.
Interviewers were recruited from each region to collect data through telephone interviews at time 2. Each interviewer participated in 1 of 2 scheduled training teleconferences. Also, prior to the teleconference, they received a binder that contained an overview of the study and detailed information pertinent to the time 2 data collection point. Our parent consultants also participated in these teleconferences to share parents' perspectives on participating in research or to share recommendations on interviewing approaches that would be well received by families. Prior to collecting data, each interviewer practiced the interview with 2 parents who were not involved in the study. We did have interviewers who were fluent in Spanish and French Canadian available to families for whom either of these was their primary language.
Children were eligible for this study if they had, or were suspected to have, a diagnosis of CP and were between the ages of 18 months and 5 years at recruitment. Eighteen months was selected as the minimum age because a diagnosis of CP is more certain at 18 months than 12 months.54 The age interval was selected because this is when children demonstrate the greatest rate of change on the Ontario Motor Growth curves.55 Although reliability of the GMFCS scores is greater after 2 years,31 the GMFCS level, as well as a firm diagnosis of CP, was confirmed at the end of the year-long study for those children under 2 years of age at study onset. Five years was selected as the oldest age at recruitment in this longitudinal study over a 1-year period because there is still some room for “growth” in motor development, which is less apparent by 5½ to 6 years of age.55 Families needed to be able to speak English, French-Canadian, or Spanish to participate in this study.
We aimed for a total sample size of 900 children, 300 each in GMFCS levels I/II, III, and IV/V. We built in an anticipated 10% attrition rate; therefore, we planned to recruit 1,000 children and their families. We planned a quota convenience sample,56 aiming for a relatively even distribution across the age and GMFCS levels, with consideration of regional variation in population density. The planned strategy would yield approximately an equal number of children in each of the 3 GMFCS groups and in each of 3 specified age categories. We planned to sample sites across both Canada and the United States to: (1) increase the generalizability of our results, (2) capture the known practice variation that exists, and (3) ensure that we would have representation from urban, suburban, and rural areas. Collectively, we had excellent contact with clinicians at our selected sites, thus enhancing the probability to recruit this targeted rate.
To arrive at the optimal sample size for our study, we relied on the commonly recommended sample requirement for SEM models. The literature suggests that a sample of about 200 to 300 participants (per group) provides sufficient statistical power needed to conduct the most rigorous test of the data using SEM techniques.57 Despite our well-intentioned plan, we were not successful in obtaining the target sample size; issues associated with sampling will be described in a related article (unpublished research). We did recruit 430 young children with CP and their families, and we will be able to test the conceptual model for children in GMFCS levels I/II and III/IV/V separately, with more than 200 children in each of the 2 groups.
Prior to testing the fit between our conceptual model and the data we collect using SEM, we will conduct a thorough confirmatory factor analysis or testing of internal consistency to analyze in detail the relationships between the constructs and their indicators.58 Items in each of the measures that do not hold up in these analyses will be eliminated. Thus, we will be able to assess the validity of the proposed measurement model described in this article. Based on the results from that analysis, we might refine the original measurement model, if there is strong theoretical support for making such refinements.57,59 Once the item composition for each measure is confirmed, we will evaluate the nature of the distribution of each variable and its relationship to change in basic motor abilities, self-care, and play to determine whether transformations are necessary prior to model testing. To facilitate this process, we have engaged in data checking and cleaning strategies as data are entered.
In terms of dissemination, we have already posted some preliminary conference presentations (ie, workshops, presentations, and posters) on the CanChild Web site (http://www.canchild.ca/en/; search “Move & PLAY). We will work closely with our parent consultants, therapist assessors, and interviewers to interpret the full study results in meaningful ways for children and families. Finally, in addition to disseminating the results of our work in peer-reviewed publications, we are committed to preparing a variety of non–peer-reviewed resources. These dissemination products include lay summaries for parents, brief summaries for service providers, presentations to families and service providers, brief articles in journals for families with children with disabilities, curriculum recommendations for professional education programs, practice guidelines and competencies, reports and presentations to provincial or state interagency coordinating councils and technical assistance programs, a position statement on policy implications, and possibly an online continuing education course. Furthermore, our research-in-progress has generated great interest internationally. One of us (S.W.M.) has already presented our plans to rehabilitation researchers and service providers in Taiwan, and 2 of us (D.J.B. and L.A.C.) presented early results of our time 1 data to physical therapists in Norway at their annual pediatric conference.
Implications for Rehabilitation Practice
We propose 3 fundamental goals of early rehabilitation for children with CP: (1) to optimize motor function within each child's prognostic potential, (2) to prevent the development of secondary conditions that affect life-long health, and (3) to promote children's development and participation in their daily lives in the context of their families. From the results of model testing, significant determinants that are amenable to change will be targets for future intervention, thus contributing to more-effective service delivery. In contrast, but equally important, significant determinants that are not amenable to change will assist with realistic goal setting, thus enhancing efficiency of services.13 Importantly, results of the confirmatory factor analysis and tests for internal consistency might lead to shorter measures containing the key elements of each of the indicators in the model. We plan to post all of the instruments we have developed for the Move & PLAY study on the CanChild Web site. We will negotiate with the developers of standardized measures to determine the best way to disseminate information about reduced item sets of those measures. Reducing the number of items alone might contribute to reduced respondent burden in the future, in both research and practice contexts, and thus will affect efficiency of service delivery and research in a very tangible way.
We believe that decisions about intervention options and methods of service delivery for children with CP logically should occur in the context of knowledge about the relationships among primary and secondary impairments, associated conditions and comorbidities, child adaptive behaviors, and aspects of the children's environment related to their families and available services with the outcomes of basic motor abilities, self-care, and play. This view is consistent with the approaches advocated in complex systems approaches,6 comprehensive rehabilitation outcomes research,5 and the interactionist perspective of rehabilitation research supported by an adaptation of the ICF.13
Half a century ago, Crothers and Paine60 advocated for open and honest communication with parents about their primary concerns relating to the motor and developmental prognosis of their young children with CP. They emphasized that over-optimism is dangerous and that unjustified pessimism is worse. To date, rehabilitation service providers have not had evidence to support these complex discussions with parents from a holistic, biopsychosocial perspective. We believe that the results of this study will provide information for rehabilitation service providers to assist with prognostic discussions with clients and their families, establishment of realistic and attainable goals, and selection of effective interventions to enhance the acquisition of basic motor abilities, self-care, and play of young children with CP. Thus, we expect that the results will contribute to more efficient and effective services for young children at a critical point in their lives to optimize their longer-term outcomes related to education, employment, leisure, and social roles.
Service providers might benefit from applying the generic framework of comprehensive rehabilitation outcomes research5 to develop conceptual models to support clinical decision making in the absence of evidence. Collaborators, whether they are clients, clinicians, researchers, or policy makers, can work together to propose outcomes of interest to specific client groups in specific contexts. They also can propose determinants of those outcomes, differentiating determinants that are amenable to change from those that are not. This reasoning process can enhance effectiveness and efficiency of service provision by assisting with decisions about targets for intervention and realistic goal setting. Discussing the thought processes behind developing a conceptual model can assist novice clinicians in understanding the intuitive, implicit decision making of service providers with higher levels of expertise, thus contributing to their professional development. It is equally important for parents to understand these perspectives toward service planning and interventions.
In this article, we described in detail the measurement model and implementation of the Move & PLAY study as an example of comprehensive rehabilitation outcomes research. We believe that this approach will provide research evidence to assist rehabilitation service providers and families who need to make complex decisions about individual children. Furthermore, we believe that thinking about services in the context of a holistic model will contribute to the development of clinical expertise, recognizing that the validity of developed conceptual models will need to be determined through research. Although this research-in-progress is presented in the context of children with CP and their families, we suggest that comprehensive rehabilitation outcomes research will be of interest to rehabilitation service providers and researchers who work with clients with chronic conditions in other diagnostic groups.
Dr Bartlett, Dr Chiarello, Dr McCoy, Dr Palisano, Dr Rosenbaum, and Dr Jeffries provided concept/idea/project design. Dr Bartlett, Dr Chiarello, Dr McCoy, Dr Palisano, Dr Rosenbaum, Dr Jeffries, and Ms Stoskopf provided writing. Dr Bartlett, Dr Chiarello, Dr McCoy, Dr Rosenbaum, Dr Fiss, and Ms Stoskopf provided project management. Dr Bartlett, Dr Chiarello, Dr McCoy, Dr Palisano, and Dr Rosenbaum provided fund procurement. Dr Chiarello, Dr McCoy, Dr Rosenbaum, Dr Jeffries, Dr Fiss, and Ms Stoskopf provided participants. Dr Bartlett, Dr Rosenbaum, Dr Jeffries, and Ms Stoskopf provided facilities/equipment. Dr Bartlett, Dr Chiarello, Dr Rosenbaum, Dr Jeffries, Dr Fiss, and Ms Stoskopf provided institutional liaisons. Dr Bartlett, Dr Chiarello, Dr Palisano, Dr Rosenbaum, Dr Jeffries, Dr Fiss, and Ms Stoskopf provided consultation (including review of manuscript before submission).
The authors acknowledge other members of the investigative team: Beth Tieman, initial study collaborator in Atlanta; the parent consultants, Barbara Sieck Taylor and Tina Hjorngaard; their statistical consultant, Piotr Wilk; Samantha Doralp, research assistant at The University of Western Ontario (for development of electronic training materials and for coordinating the criterion testing); and Audrey Wood and Allison Yocum, coordinators in Philadelphia and Seattle, for all of their contributions to date.
The Move & PLAY study is supported through funds from the Canadian Institutes of Health Research (MOP 81107) and the US Department of Education, National Institutes of Disability and Rehabilitation Research (H133G060254).
- Received December 22, 2009.
- Accepted May 31, 2010.
- © 2010 American Physical Therapy Association