Advertisement

Factor Analysis as a Tool for Survey Analysis Using a Professional Role Orientation Inventory as an Example

Laura L Swisher, Jason W Beckstead, Muriel J Bebeau

Abstract

Background and Purpose. The purpose of this article is to illustrate how confirmatory factor analysis can be used to extend and clarify a researcher's insight into a survey instrument beyond that afforded through the typical exploratory factor analytic approach. The authors use as an example a survey instrument developed to measure individual differences in professional role orientation among physical therapists, the Professional Role Orientation Inventory for Physical Therapists (PROI-PT). Sample. Five hundred three physical therapists responded to a mail survey instrument that was sent to a random sample of 2,000 American Physical Therapy Association members. Methods. An adapted version of the Professional Role Orientation Inventory, a 40-item Likert-scale instrument developed to assess professional role orientation on 4 dimensions (authority, responsibility, agency, and autonomy), was used. Exploratory and confirmatory factor analyses were used to examine the factorial validity of the PROI-PT. Results. Exploratory factor analysis served as a starting point for examining the factor structure of the instrument. Confirmatory factor analysis then was used to test the hypothesized factor structure and to suggest refinements to the PROI-PT that would improve a psychometric property (internal consistency). Discussion and Conclusion. Although further refinement of the PROI-PT is needed, an instrument that yields valid and reliable measurements of individual differences in professionalism among physical therapists could further our understanding of the psychosocial aspects of physical therapist practice. Exploratory and confirmatory factor analyses can be used by researchers who study various psychosocial constructs in physical therapy.

Over the last several decades, members of the physical therapy profession have intensified their efforts to enhance the profession's body of knowledge through research.14 Many of these research efforts have focused on examination of the effectiveness of clinical interventions. A number of psychosocial constructs that we believe are essential to the practice of physical therapy have not been subjected to similar research. For example, “professionalism,”1,57 “autonomous practice,”79 and “accountability”10 have occupied a central place in professional discussion but have generated relatively little research. As physical therapists continue to work toward practice without referral and increased levels of professionalism, we contend that it will be important to subject professional practice concepts to the same rigorous scholarship that is being focused on clinical issues. Scholarship of this type, in our view, requires the development of appropriate measurement tools through the use of social scientific methods. The purpose of this article is to illustrate exploratory and confirmatory factorial analyses as tools to analyze data collected by instruments that measure psychosocial constructs. We use the Professional Role Orientation Inventory for Physical Therapists (PROI-PT) as an example of an instrument designed to measure professionalism among physical therapists.

Background–Professionalism

The terms “profession,” “professional,” and “professionalism” feature prominently in physical therapists' discussions. Speaking in 1965, Worthingham11 referred to physical therapy as an “emerging profession.” By 2001, Massey could say, “As a profession, we have arrived. We have defined our scope of practice. We have developed a unique body of knowledge. We are documenting the effectiveness of our outcomes. We adhere to a code of ethics. And we take responsibility for the well-being of patients and clients. True autonomy is the destination.”12(p1831)

Despite the frequent use of the terms “profession” and “professionalism,” their meaning may not be entirely clear.13,14 Larson15 attributed this ambiguity to the fact that social scientists have basically used the term “profession” in a casual fashion. He observed, “Given that this is not a concept but rather a notion taken from social practice which connotes more than it denotes, it is not surprising to find the term used in contradictory or inconsistent ways.”15(p607)

The ambiguity of the term “professional” is further complicated by the sheer volume and diversity of related literature spanning several centuries and by the changing focus of scholarship about the professions. Although early research focused on the common characteristics or attributes that define the professions,16,17 more recent work has examined the sociopolitical process of “professionalization.”1821 The comments of Massey would seem to emphasize the attributes of the profession, whereas Worthingham's observations about “steps” of the emerging profession address the process of professionalization.

In this article, professionalism is defined as the conceptualization of obligations, attributes, interactions, attitudes, and role behaviors required of professionals in relationship to individual patients or clients and to society as a whole. This definition embraces the idea that professionalism reflects an implied contract among the profession, the individual professional, and society.22,23 The implicit social contract of professionalism serves both normative and descriptive functions by describing and prescribing roles, relationships, obligations, and behaviors.

Although much of the sociological literature focuses on the characteristics common to all professions, Bellner24 noted that there are differences between professions and individual professionals regarding the concepts of professionalism. Concepts of professionalism, therefore, may refer to the profession as a whole or to individual members of a profession.13,14 Individual professionalism, or professional role concept, refers to the internalized beliefs of an individual professional regarding professional obligations, attributes, interactions, attitudes, and role behaviors.

The multiple levels and meanings attached to the term “professionalism” make it especially difficult to measure professionalism. Instruments used to measure professionalism have been based on the early sociological concept of professionalism, with particular emphasis on autonomy as the defining mark of a professional. For example, Hall25,26 developed a widely used 50-item Likert-scale instrument to assess 5 “attitudinal attributes” of professionals: use of professional organization as reference, belief in public service, belief in self-regulation, sense of calling, and perceived autonomy in work.

One line of scholarship on physical therapy professionalism also has emphasized attributes and autonomy. Some studies of physical therapy27,28 were based on the ideas of Moore29 and Pavalko30 that professionalism should be evaluated as a continuum of attributes. Moore's continuum ranked professional attributes in ascending order of importance: motivation, established professional organization, specialized body of knowledge, evaluative skills, and autonomy of judgment.29(p5)

Another line of research focused on the professional behaviors of physical therapists. Lopopolo31,32 developed a 26-item instrument to evaluate changes in role behaviors following hospital restructuring. May and associates33 developed the Generic Abilities Assessment to self-assess professional behaviors necessary for practice, with professionalism and responsibility as 2 of the 10 generic abilities. Jette and Portney's34 subsequent research to determine the construct validity of data obtained with this instrument generated 7 factors. The American Physical Therapy Association's (APTA) Department of Education has recently delineated the 7 core values of professionalism based on consensus and a review of research in the field of medicine: accountability, altruism, compassion/caring, excellence, integrity, professional duty, and social responsibility.6

There was a sociological or behavioral emphasis in many instruments used to evaluate professionalism. Bebeau et al,35 however, developed an instrument based on philosophers' or ethicists' descriptions of models of professionalism3639 that appeared to guide professional relationships and decision making. Working with dentists, they created the Professional Role Orientation Inventory (PROI), a 40-item Likert-scale instrument, to measure 4 dimensions of professionalism: authority, responsibility, autonomy, and agency. These constructs are defined in Figure 1.

Figure 1.

Theoretical dimensions of professionalism in the Professional Role Orientation Inventory (PROI)35 and the Professional Role Orientation Inventory for Physical Therapists (PROI-PT).

Exploratory factor analysis of their results confirmed the existence of the 4 anticipated subscales, plus a fifth factor that appeared to be a combination of agency, autonomy, and authority. The PROI was sensitive to individual differences on each of the dimensions, detected change over time (first- to fourth-year students), and showed differences between groups of dentistry professionals who would be expected to differ in professional role concepts.35,40,41

We used an adaptation of the PROI for use with physical therapists—the PROI-PT (Appendix). This instrument was selected because it is supposed to examine professionalism as a relational concept and because of its ability to provide information about the professional's concept of the “implied contract” with patients and clients and with society. In contrast to many of the existing instruments for evaluating professionalism, the PROI is not based exclusively on autonomy or the professional image or status of the physical therapist or on the stage of professionalization of the profession as a whole.

Background–Exploratory and Confirmatory Factor Analyses

Development of a survey instrument such as the PROI-PT is commonly done in social science to examine multidimensional constructs.42,43 The researcher generates a series of questions, each of which is supposed to address a dimension of the construct under investigation. As in the PROI, each item of the PROI-PT is intended to address 1 of 4 dimensions of professionalism: responsibility, agency, authority, or autonomy.

To evaluate an instrument that purports to measure psychosocial concepts, such as professionalism, patterns of variation (variance) and correlation (covariance) among responses to the items representing each dimension of the construct are often examined.44,45 Factor analysis refers to a group of statistical techniques that are often used to pursue this line of examination. Several methods (eg, principal axis factoring, principal components analysis, image factor extraction, alpha factoring) are known collectively as exploratory factor analysis (EFA). Confirmatory factor analysis (CFA) is another class of methods for examining the results of these types of instruments. The following section briefly discusses these 2 approaches commonly used in the evaluation of multiple-item instruments such as the PROI-PT.

In EFA, the researcher begins with a set of variables in which the analytical focus is to discover relatively independent, coherent subsets of variables.46(p582) That is, EFA allows the researcher to examine numerous variables and, under certain conditions, “to reduce them to a smaller, more manageable set of underlying concepts.”43(p607) The question addressed by researchers using EFA is: What are the underlying or latent constructs that could have produced the observed pattern of variances and covariances among the variables? The relationships among the latent constructs and the observed variables (test items) are modeled using a set of equations that contain factor loading coefficients, which are analogous to standardized regression coefficients. Typically, there is no underlying theory in EFA about which variables should be quantitatively associated with which factors; they are simply empirically associated. The ideal or expected outcome in EFA is that the pattern of factor loadings show “simple structure,” that is, that each item loads strongly on (that is, it correlates with or regresses on) only one factor and has near-zero loadings on all other factors. Exploratory factor analysis methods rely on various rules of thumb, with factor loading cutoff criteria ranging from .30 to .55, for establishing what is considered to be a strong factor loading coefficient.

A focal step in EFA often involves deducing names for the factors based on the content (ie, wording) of the items that load heavily upon them.43(p613) Two popular EFA techniques are principal components analysis, where the goal is to account for variance in the set of items, and principal axis factoring, where the goal is to reproduce the matrix of correlations between all pairs of items. Unfortunately, these 2 approaches are often simply referred to as “factor analysis,” which leads to confusion when applying and interpreting them. Both forms of EFA are considered data reduction strategies, but they are used with different goals and criteria for success in mind. The success of a principal components analysis is expressed in terms of the amount of variance in a large set of test items that may be explained by modeling them using equations based on a few latent components. The greater the proportion of variance explained, the better the principal components solution is said to be.46 Principal axis factoring methods quantify their success in terms of the amount of discrepancy between the matrix of observed correlations and those reproduced from the factor equations. The smaller these discrepancies, the better the factor solution is said to be.46

In contrast to EFA, the aim of which is simply to identify the factor structure present in a set of variables, the aim of CFA is to test a hypothesized factor structure or model and to assess its fit to the data.44,45 Confirmatory factor analysis may be viewed as a submodel of the more general structural equation modeling (SEM) approach to analysis. Specifically, CFA is a measurement model of the relationships of indicators (observed variables) to factors (latent variables) as well as the correlations among the latter. Confirmatory factor analysis is generally based on a strong theoretical or observational foundation that allows the analyst to specify an exact factor structure in advance. The CFA approach usually restricts which variables will load on which factors, as well as which factors will be correlated. This approach also provides significance tests on each factor loading coefficient, in contrast to relying on rules of thumb (eg, factor loading cutoff criteria of .30 or .40). With CFA, each observed variable has an error term, or residual, associated with it that expresses the proportion of variance in the variable that is not explained by the factors. These error terms also contain measurement error due to any lack of reliability in data for the observed variables. The typical research question with CFA is: Are the covariances (or correlations) among variables consistent with a hypothesized factor structure? As such, CFA is quite useful for studying the factorial validity of data obtained with multiple-item, multiple-subscale instruments such as the PROI-PT.

An articulate comparison of EFA with CFA methods used in our study has been provided by Jöreskog and Sörbom: It is important to distinguish between exploratory and confirmatory analysis. In an exploratory analysis, one wants to explore the empirical data to discover and detect characteristic features and interesting relationships without imposing any definite model on the data. An exploratory analysis may be structure generating, model generating, or hypothesis generating. In confirmatory analysis, on the other hand, one builds a model assumed to describe, explain, or account for the empirical data in terms of relatively few parameters. The model is based on a priori information about the data structure in the form of a specified theory or hypothesis, a given classificatory design for items or subtests according to objective features of content and format, known experimental conditions, or knowledge from previous studies based on extensive data. Most studies are to some extent both exploratory and confirmatory since they involve some variables of known and other variables of unknown composition. The former should be chosen with great care in order that as much information as possible about the latter may be extracted. It is highly desirable that a hypothesis that has been suggested by mainly exploratory procedures should subsequently be confirmed, or disproved, by obtaining new data and subjecting these to more rigorous statistical techniques.47

Pedhazur and Schmelkin48(pp631-632) also contrasted the 2 approaches. With EFA, all variables (items) have loadings (not necessarily meaningful ones) on all of the factors, whereas a major feature of CFA is that the researcher can specify which variables load on which factors. Whether or not factors are correlated is an all-or-nothing decision with EFA. That is, with EFA, it is not possible to specify that only some factors are intercorrelated. In contrast, with CFA, it is possible to specify that only some of the factors are intercorrelated. With EFA, it is assumed that residual, or error, terms within variables are not correlated. With CFA, such correlated errors may be tested as part of the model. The EFA and CFA methods are compared in Table 1.

Table 1.

Features of Exploratory Factor Analysis and Confirmatory Factor Analysis

Method

Sample

A list of 2,000 physical therapist members of APTA who were randomly selected from the membership mailing list of the organization served as the sample for this cross-sectional study. Because we were interested in the concept of professionalism of practicing physical therapists, subjects who were retired from or not practicing in one of the roles of the physical therapist were not included in the study.

Instrumentation

The PROI-PT contains 10 items in each of 4 subscales, with the subscales designed to assess 4 dimensions of professionalism on a Likert scale (1=“strongly disagree” to 6=“strongly agree”). Adding together Likert scores for the 10 items produces a maximum possible score of 60 for each of the 4 subscales (responsibility, authority, agency, autonomy).

Because professionals share common characteristics, 2 of the authors (LLS in consultation with MJB) changed only those PROI items that were thought to be inconsistent with physical therapist practice. The criteria for changing an item was whether we believed that the item referred to experiences and personnel typically encountered by physical therapists. Item 13 of the PROI (“With respect to hygienists, I believe the public should decide who is given the right to independent access.”) was adapted on the PROI-PT to read “With respect to other health care providers (ATCs [certified athletic trainers], massage therapists, OTs [occupational therapists]), I believe the public should decide who is given the right to direct access.” Item 31, referring to the burden of mandatory reporting to the National Practitioner Databank, was adapted to read “Medicare documentation and reporting requirements are an unnecessary infringement and a gross burden on my profession.” The term “auxiliary utilization” in the original PROI was adapted by using the term “use of support personnel” in the PROI-PT.

Procedure

The PROI-PT was pilot tested by one of the authors (LLS) in a small sample of physical therapists (n=15) in order to determine clarity of the items and directions. A second pilot test was conducted with a small group of physical therapists (n=14) we thought to be experts in professional role evaluation to establish some level of face validity. Criteria for inclusion in the expert group were service, publication, or presentation in a national forum on professional roles or ethics. A list of physical therapist experts was compiled by one of the authors (LLS) based on a search of literature related to professionalism through electronic databases and review of relevant presentations made at the 2 annual meetings of APTA. Addresses were obtained through journals of publication or by a search of the member directory of APTA. As anticipated based on results of studies with the original PROI,35 the experts scored higher on the responsibility subscale than on the other 3 dimensions.

Following these pilot tests, and subsequent revisions to clarify the instructions, we mailed a short information form and the PROI-PT to each person in the sample. A cover letter and demographic questionnaire informed subjects of the voluntary nature of participation, their rights as research participants, and our intention to publish the results of the research. In order to increase the response rate, we mailed prenotification and follow-up postcards to each subject to encourage participation. Although the mail packet also contained an instrument to measure moral reasoning, this article reports only on the professional role portion of the research.

Data Analysis

Initially, the PROI-PT data were subjected to EFA using a maximum likelihood extraction of 4 factors with oblique rotation (which allows factors to be correlated) available in SPSS.* Maximum likelihood extraction estimates population values for factor loadings by calculating loadings that maximize the probability of sampling the observed correlation matrix from a population.46 This serves as a point of departure for applying the CFA approach.

The factorial validity of data obtained with the PROI-PT was then subjected to CFA using structural equation modeling. The models below were tested with LISREL 8.54.,47 Evaluation of each model was based on considering a variety of fit measures, and model comparisons are based on incremental differences in fit. These measures are now briefly discussed. The chi-square minimum fit function test is an inferential test of the plausibility of a model explaining the data. It is calculated from the discrepancies between the original and reproduced correlations among the items. As such, smaller values indicate a better fit of the model to the data. The root mean square error of approximation (RMSEA) expresses the lack of fit due to reliability and model specification or misspecification.49 The RMSEA expresses fit per degree of freedom of the model and should be less than .1 for acceptable fit, with .05 or lower indicating a very good-fitting model. The goodness-of-fit index (GFI) and adjusted goodness-of-fit index (AGFI), which adjust for the number of parameters estimated, range from 0 to 1, with values of .9 or greater indicating a good-fitting model.47 These 2 indexes are analogous to R2 in multiple regression. In structural equation models, such as the comparative fit index (CFI), each item is modeled using a set of parameters (ie, factor loading coefficient[s] and an error or residual variance term representing variance in the item that is not associated with any factors). The CFI assesses fit relative to a null model using noncentrality parameters.50 The CFI also ranges from 0 to 1, with values of .9 or greater indicative of good-fitting models. The standardized root mean square residual (RMR) is the average of differences between the sample correlations and the estimated population correlations. The RMR has a range from 0 to 1; values of .08 or less are desired.51

Results

A total of 503 subjects responded by returning completed, usable survey instruments, a response rate of approximately 25%. This relatively low degree of participation may reflect the length and time-consuming nature of the survey, as well as survey instruments that were undeliverable due to wrong address. Given the complexity of the instrument and relatively long completion time for both instruments in the mailing (about 1 hour), however, this response rate appeared to us to be acceptable because the resulting sample of 503 was greater than the 361 recommended by some standard sample size formulas for a 95% confidence interval and a 5% degree of accuracy in a population of 50,000 to 61,000.52

Our sample was composed predominantly of white (96.2%) female (73.8%) subjects, whose professional degree was the baccalaureate degree (54.7%). A total of 52.1% of the sample had a master's degree, and 3.6% had earned a doctoral degree. The average age of respondents was 38.29 years (SD=10.03, range=23-63), with an average of 13.03 years (SD=10.21, range=<1–38) of experience in physical therapy. Compared with the APTA membership,53 our sample had more female subjects (73.8% versus 67.2% of APTA members), fewer minority subjects (3.8% versus 9.2% of APTA members), and a greater number of respondents who had earned a master's degree (52.1% versus 45.2% of APTA members). In addition, 7.9% more physical therapists in our sample worked in an outpatient setting (57.6% versus 49.7% of APTA members).53

Prior to conducting multivariate analyses, the data were screened for univariate and multivariate outliers. Based on inspections of frequency distributions, no univariate outliers (values more than 3 standard deviations from the mean)46 were found. A multivariate outlier is a case that has such an extreme pattern of response values across all 40 items that it distorts statistics. Screening for multivariate outliers was done by calculating Mahalanobis distance scores for all cases. Mahalanobis distance is the distance of a case from the centroid of the remaining cases, where the centroid is the point created at the intersection of the means of all 40 PROI-PT items. Using a critical value of χ2=73.402, α=.001, and df=40, 25 multivariate outliers (less than 5% of the total sample) were identified and removed from subsequent factor analyses (final n=478). The demographic profile of these outlying cases was not systematically different from that of the total sample. The means and standard deviations for the 4 subscales are shown in Table 2. Estimates of internal consistency (ie, Cronbach alpha coefficients) for the 4 subscales also are shown in Table 2 and will be discussed.

Table 2.

Means, Standard Deviations, and Cronbach Alpha Values for the Professional Role Orientation Inventory for Physical Therapists (PROI-PT)a

Results of Exploratory Analysis

The Kaiser-Meyer-Olkin measure of sampling adequacy was .764, and the Bartlett test of sphericity was significant (χ2=3579.78, df=780, P<.001). Both of these measures indicated that the data were appropriate for factor analysis.48,54 Initial inspection of a scree plot of the eigenvalues evidenced a major departure from linearity coinciding with a 4-factor solution (Fig. 2). This finding, coupled with the expectation that the PROI-PT reflects 4 dimensions of professionalism, prompted us to request and evaluate a 4-factor solution wherein each item was allowed to load onto all 4 factors.

Figure 2.

Scree plot for exploratory factor analysis.

To assess the quality of the solution, we examined the discrepancies between the 820 unique elements of the 40 × 40 matrix of observed correlation coefficients and their corresponding elements in the matrix of correlation coefficients estimated from the factor loadings. The 4-factor solution resulted in 177 discrepancies (22%) with absolute values greater than .05. The resulting factor loadings are shown in Table 3.

Table 3.

Results of Exploratory Factor Analysis Specifying 4-Factor Solutiona

Typical of EFA, the factor loading coefficients were examined for their resemblance to simple structure, the ideal case where each item has a relatively large loading on one factor and near-zero loadings on all other factors. It appears that some items showed this pattern. However, using EFA, it is not possible to determine whether there is “enough” simple structure to support the hypothesized 4-factor structure of the PROI-PT.

An important consideration in EFA is the determination of an appropriate value to determine whether an item has “loaded” on a factor. Loadings of .3042(p374),43(p614) or .4043(p614) and above are typically considered the “rule of thumb” threshold for this determination. An examination of Table 3 reveals that selection of the appropriate cutoff point could produce different assessments of the instrument. A value of .30 would produce many more items that load on at least one factor, but more items that cross-load. Using a value of .40 would leave few items that loaded on factors. A factor loading cutoff of .40 would leave only 14 items in the PROI-PT.

In addition to low factor loadings and the criteria for eliminating items, EFA also revealed concerns about the pattern of loadings. Given the goals of “simple structure,” it was especially troubling to us that factor 1 contained so many items from both autonomy and agency and that so many of the autonomy and agency items cross-loaded on other factors. These issues and the fact that the 4 factors accounted for only 21.8% of the total variance raised doubts about how well the foundational model of the PROI-PT had performed. To gain greater insight into the factor structure of the PROI-PT, a series of CFA models were tested.

Results of Confirmatory Factor Analysis

Confirmatory factor analysis was performed on the covariance matrix of the PROI-PT items. The model parameters were estimated using maximum likelihood. A series of 6 models was tested. The sequence of modeling decisions made, and their resulting summary statistics, are reviewed. The factor loadings for the final model of CFA also are reported for comparison with those of EFA in Table 3. One advantage of CFA is that the researcher can specify the simple structure that he or she is looking for and obtain feedback on the extent to which this structure is supported by the data. That is, LISREL can estimate one factor loading for each item while setting or “fixing” its loadings on all other factors to equal zero. LISREL then provides tests of significance for each loading, as well as the various global indexes of how well the hypothesized factor structure fits the data.

The initial model tested (PROI-PT40A) was one in which each item loaded on only 1 of 4 factors corresponding to its composite subscale. This hypothesized 4-factor model did not fit the data well from either a statistical perspective (χ2=1707.04, df=734, P=.001) or a practical perspective (GFI=.838, AGFI=.819, CFI=.666, RMSEA=.056, and RMR=.068). This model, therefore, was rejected. These results and those for each of the 6 models tested are summarized in Table 4.

Table 4.

Summary of Model Fit Statistics (Confirmatory Factor Analysis)a

In addition to the summary statistics discussed, LISREL also provides modification indexes (MIs) for each parameter (eg, factor loading) that has been fixed to equal zero. An MI suggests by how much the chi-square test of fit is expected to decrease if a given fixed parameter is freed to be estimated. Thus, MIs can be useful for making decisions about revising hypotheses about factor structure. However, as Pedhazur and Schmelkin48(pp673–674) cautioned, researchers should not blindly rely on MI to improve model fit while ignoring the substantive meaning of freeing a parameter. A review of the MIs revealed some abnormally large values representing stress or misfit associated with items 2 and 6. A second model (PROI-PT40B) was tested that explicitly allowed these 2 items to “cross-load” on both the authority and responsibility factors. Model fit statistics are shown in Table 4. A comparative test of this model against the previous more restrictive model, achieved by contrasting the difference in their chi-square values relative to the difference in their degrees of freedom, confirmed that freeing these 2 parameters made an improvement in the fit of the model to the data (χ2=86.88, df=2, P<.05). This test is known as the likelihood ratio test and is used in testing nested structural equation models.55 Nonetheless, inspection of the fit statistics (Tab. 4) indicated that some degree of model misfit still remained.

Our inspection of the factor loading coefficients from the second confirmatory model revealed that 7 items (items 13, 14, 15, 17, 18, 19, and 20) did not load on any of the 4 hypothesized factors. Therefore, a third model (PROI-PT33) that eliminated these items was tested next. Model fit statistics for each of the 6 models tested are shown in Table 4. The likelihood ratio test confirmed that dropping these items improved the fit of the model to the data (χ2=552.89, df=245, P<.05). There was still room, however, for improvement. Our inspection of the factor loadings from this model showed that item 7 loaded on all 4 factors. Items with such a pattern of loadings violate the tenet of simple structure in factor-analytic theory and are therefore poor candidates for inclusion in multiple-factor inventories. Consequently, model 4 (PROI-PT32A) that excluded this item was tested next; summary statistics are shown in Table 4. The likelihood ratio test showed that dropping this item improved the fit of the model to the data (χ2=90.96, df=31, P<.05).

Extending the application of simple structure to the developing model, we tested whether the dual loadings of items 2 and 6 were necessary by deleting their loadings on their original hypothesized factors. Thus, model 5 (PROI-PT32B) included 32 items, each loading on only one factor, but with item 2 loading on authority and item 6 loading on responsibility. Model fit statistics are shown in Table 4. The likelihood ratio test showed that fixing these original loadings to zero, while allowing these items to load on their “unintended” factors confirmed in model 2, did not detract from the fit of the model to the data (χ2=3.71, df=2, P<.05).

Nonetheless, inspection of the fit statistics in Table 4 indicated that some degree of model misfitting still remained. Indeed, a review of the MIs revealed some abnormally large values associated with error covariances among various items. Typically, the error terms for any pair of items are assumed to be uncorrelated. Despite common findings of correlated error variance terms, there remains considerable controversy in the CFA literature regarding their interpretability and cause. Bentler and Chou56 remarked that model specification that forces all error terms to be uncorrelated is rarely appropriate with real data. Incorporation of these correlated error terms into CFA does not otherwise undermine the factorial validity of data obtained with the PROI-PT, but rather it provides a more realistic factorial representation of the observed data structure.

Based on inspection of MIs associated with the correlated error variances, specific error covariance terms were freed sequentially. That is, one parameter was freed and then the likelihood ratio test was used to assess the significance of improvement in the fit of the model. This process continued until freeing additional parameters did not produce an improvement in model fit. The resulting model (model 6, PROI-PT32C) showed improvement in fit over model 5 (PROI-PT32B). The likelihood ratio test showed that including 9 correlated error variances (between items 9 and 29, 9 and 31, 11 and 16, 21 and 23, 21 and 25, 24 and 25, 24 and 33, 32 and 33, and 38 and 39) improved the fit of the model to the data (χ2=223.93, df=9, P<.05). Fit statistics for this final model are given at the bottom of Table 4. Given the large sample size (n=478), it is not surprising that the chi-square minimum fit function test was significant; however, a good-fitting model may be indicated when the ratio of the chi-square value to the degrees of freedom is less than 2.0.46 In the case of model 6, df × 2=948 and χ2=756.09, thus satisfying this criterion. Based on this, and the other indexes of fit, this model was consistent with the observed data. The factor loadings from model 6 (PROI-PT32C) are shown in Table 5.

Table 5.

Confirmatory Factor Analysis Factor Loadings for the Professional Role Orientation Inventory for Physical Therapists (PROI-PT) (Model 6, PROI-PT32C)a

Based on the CFAs reported, the authority and responsibility subscales were shortened by dropping items 13, 14, 15, 17, 18, 19, and 20 and exchanging items 2 and 6 on these subscales. The revised authority subscale with 7 items had internal consistency of .50, and the revised responsibility subscale with 5 items had internal consistency of .62. The kappa coefficients calculated on these 2 revised subscales were higher (P<.05) when compared with the coefficients calculated on the original subscales (Tab. 2) using the method proposed by Alsawalmeh and Feldt.57 Table 6 illustrates the resulting items and internal consistency of subscales for the final models of CFA and EFA.

Table 6.

Items and Internal Consistency (Cronbach Alpha) for Final Models in Confirmatory Factor Analysis (CFA) and Exploratory Factor Analysis (EFA)a

The correlations among the 4 factors in confirmatory model 6, also calculated by LISREL, are shown in Table 7. The autonomy and agency factors appeared to be quite strongly correlated (r=.98). The responsibility factor was not correlated with either the agency factor or the autonomy factor. The authority and responsibility factors were slightly, although inversely, related.

Table 7.

Correlations Among Factors of the Professional Role Orientation Inventory for Physical Therapists (PROI-PT) After Confirmatory Factor Analysisa

Discussion

Our analyses focused on evaluating the PROI-PT, an instrument adapted from the PROI, for measuring individual differences in professional role orientation among physical therapists. We set out to show how CFA could be used to assess the factorial validity of data obtained with the PROI-PT by testing the hypothesized relationships among the items that comprise it. In an effort to better illustrate the utility of CFA, we included an analysis of the data using EFA for continuity. Our goal was not to compare the 2 methods of analysis, but to illustrate how CFA can be used to extend and clarify a researcher's insight into a survey instrument beyond that afforded through the EFA approach. We consider these insights useful when considering future use and development of the PROI-PT as well as revision of theory of professionalism that underlies the instrument.

Utility and Interpretation of Factor Analytic Approaches

Our analyses illustrate the different types of information that may be obtained from the CFA and EFA approaches to assessing the factorial validity of data obtained with survey instruments such as the PROI-PT. Beginning with EFA, the scree plot of the eigenvalues (Fig. 2) suggested that the point of diminishing return was 4 factors; adding additional factors to the solution would not improve the ratio of factors to variance accounted for. However, the low proportion of variance in the data (21.8%) that was accounted for by the 4-factor solution raises questions about how well the PROI-PT measures the 4-dimension (authority, responsibility, autonomy, and agency) theoretical model of professionalism. In EFA, the decision to group items together into subscales involves rules of thumb about cutoff values. If the liberal cutoff value of .30 had been applied to exploratory factor loadings in Table 3, only 26 items would be retained, and 4 of these items would be considered as loading on more than one factor. A more conservative cutoff value of .40 would have resulted in retaining only 14 items, with one of them loading on 2 factors (Tab. 6).

Confirmatory factor analysis offered further, and more specific, insight into the factor structure of the PROI-PT by providing tests of significance on each factor loading and modification indexes that suggested where the structural equations that represent the factor structure could be improved. Eliminating items 7, 13, 14, 15, 17, 18, 19, 20 and incorporating item 2 into the authority subscale and item 6 into the responsibility subscale improved the fit of the data to the 4-factor model. These changes also produced increases in internal consistency of these subscales as assessed by Cronbach coefficients.

A number of items did not “behave” as they should have (in either CFA or EFA) based on the 4-dimension theory of professionalism. Several items did not load where they were hypothesized to in CFA. Some items cross-loaded in EFA on the autonomy, agency, and authority factors. In the CFA, there was a strong correlation (.98) between the agency and autonomy factors. These findings raise the question of whether these theoretical dimensions of professionalism are independent and simply measured poorly by the PROI-PT or whether these components could be replaced by a single, broader theoretical component.

Implications for Research on Professionalism

Our results indicate that the psychometric properties of the PROI-PT could be further improved. Future validation studies could address rewording of the problematic items identified and perhaps develop additional (and replacement) items that more adequately reflect the concept of the professional role among physical therapists. We believe our findings are germane to the debate surrounding the appropriate strategy for improving an instrument such as the PROI-PT. Snizek26 recommended that future users of Hall's professionalism scale should reduce the number of items from 50 to 25. Snizek based this recommendation on an EFA in which many of the items had low factor loadings, some cross-loaded, some failed to load, and some seemed not to “fit.” Fox and Vonk,58 however, were critical of Snizek's “lack of an explicit criterion for 'acceptable' factor loadings,”58(p393) and voiced concerns about the overall effects of deleting items on the internal consistency and validity of data obtained with the instrument.

With further refinement, the PROI-PT could provide valuable information about how physical therapists view their professional roles. For example, Bebeau59 used the original instrument with dental students to assess changes in professionalism as students progress through the curriculum. The PROI also has been useful in quantifying the impact of ethics training among dental professionals.35,59 Similar research could be undertaken with physical therapists to address questions of professionalism and direct access. Do physical therapists who practice with more direct access or in primary care contexts hold different views of autonomy and responsibility? For example, it is possible that physical therapists in military settings who practice with few practice restraints may score higher on the autonomy scale than other physical therapists. Do physical therapists who practice in acute care settings differ in professional role orientation from those who practice in skilled nursing facilities?

Individual differences in professional role orientation, theoretically, also may be linked to professional behaviors and clinical outcomes. Do physical therapists who exhibit desirable professional behaviors or who obtain greater therapeutic outcomes hold different views of professionalism than other, less successful, therapists? Valid and reliable data on individual differences in professionalism could be used to quantify and analyze trends and changes in professionalism, to assess changes in individual therapists over the course of their practice, and to assess effects of educational interventions among physical therapists and physical therapist students.

The theoretical model of professionalism underlying the PROI-PT may have particular relevance in the current health care environment. Although many physical therapists continue to rely on models of professionalism that emphasize autonomy, this approach is regarded by some sociologists as outdated. The current health care environment has markedly decreased the autonomy of health care providers. If a person accepts the premise that professional autonomy is a “litmus test” for professionals, then physicians and other health care providers may be forced to accept the fact that they have been “deprofessionalized.”60 Alternatively, professionalism may be viewed as a reflection of professionals' contract with, and responsibility to, society. From this perspective, changes in the health care environment present opportunities for professionals to renegotiate their contract with society and re-evaluate their own sense of professionalism. For example, in the medical profession, Stevens emphasized the need to turn away from the past in “reinventing professionalism,”61(p357) and Sullivan called for “civic professionalism.”62(p11) From this point of view, the PROI-PT, and its underlying theory, may prove useful for understanding differences among physical therapists as we conceptualize and renegotiate our role in the current health care environment.

Limitations

Some limitations must be kept in mind concerning our results. First, the low response rate and demographic differences from the general membership of APTA warrant caution in generalizing our results to the larger population of physical therapists. We sampled only APTA members, and many physical therapists are not members of the Association. It is possible that the same study performed on a sample of physical therapists who are not members of APTA might have had different results. The sample size was sufficient to analyze the instrument. If we had wanted to establish norms for the profession or to compare groups, however, it would have been necessary to ensure that this sample was representative of the APTA membership and the profession in general, because non-APTA members are very likely to have different views than APTA members of professional role models. Second, the data analyzed were obtained from self-reports and consequently may reflect bias in reporting feelings or opinions. Third, obtaining a “good-fitting” model when conducting structural equation modeling, as in CFA, does not confirm that the resulting model is the only acceptable model or the best model, merely that it is consistent with the observed data.48(p667) The fact that our CFA efforts produced an acceptable model (model 6-PROI-PT32C) should not deter other researchers from critiquing and revising theoretical models of professionalism.

Conclusions

This article has illustrated the use of CFA and EFA for improving researchers' insight into survey instruments such as the PROI-PT. The results of our survey of only APTA members suggest that the PROI-PT would benefit from further refinement of its underlying theoretical model and by revising the set of survey items that comprise it. We believe that an instrument that yields valid and reliable measurements of individual differences in professionalism among physical therapists could further our understanding of the psychosocial aspects of physical therapist practice. Exploratory factor analysis and CFA, we contend, are powerful tools for researchers who study various psychosocial constructs in physical therapy.

Appendix

Appendix.

The Professional Role Orientation Inventory (Physical Therapist Version)a

Footnotes

  • All authors provided concept/idea/research design and writing. Dr Swisher provided data collection, and Dr Swisher and Dr Beckstead provided data analysis. Dr Swisher provided project management and fund procurement. Dr Beckstead and Dr Bebeau provided consultation (including review of manuscript before submission). The authors thank the members of the American Physical Therapy Association who generously contributed their time and effort to complete questionnaires. They also thank Celinda Evitt, Marjory Lattin, Kelly Lattin, Brian Lattin, and Travis Lattin, who assisted in the preparation of the survey and processing of the data.

    Preliminary results of this study were presented at Physical Therapy 2002: Annual Conference and Exposition of the American Physical Therapy Association; Cincinnati, Ohio; June 8, 2002.

    This study was approved by the University of South Florida Institutional Review Board under exempt category 2.

    This work was supported, in part, by the University of South Florida Research and Creative Scholarship Grant Program under Grant No. 804.

  • * SPSS Inc, 233 S Wacker Dr, Chicago, IL 60606.

  • Scientific Software International Inc, 7383 N Lincoln Ave, Suite 100, Chicago, IL 60646-1704.

  • Received September 10, 2003.
  • Accepted April 7, 2004.

References

View Abstract