Content Research

Location:

Denver, CO

Posted:

February 14, 2013

Contact this candidate

Resume:

Journal of Applied Psychology Copyright **** by the American Psychological Association

****, ***. **, **. *, 1062 108*-****-****/08/$12.00 DOI: 10.1037/0021-9010.93.5.1062

Research in Industrial and Organizational Psychology From 1963 to 2007:

Changes, Choices, and Trends

Wayne F. Cascio and Herman Aguinis

University of Colorado Denver

The authors conducted a content analysis of all articles published in the Journal of Applied Psychology

and Personnel Psychology from January 1963 to May 2007 (N 5,780) to identify the relative attention

devoted to each of 15 broad topical areas and 50 more specific subareas in the field of industrial and

organizational (I O) psychology. Results revealed that (a) some areas have become more (or less)

popular over time, whereas others have not changed much, and (b) there are some lagged relationships

between important societal issues that involve people and work settings (i.e., human-capital trends) and

I O psychology research that addresses them. Also, much I O psychology research does not address

human-capital trends. Extrapolating results from the past 45 years to the next decade suggests that the

field of I O psychology is not likely to become more visible or more relevant to society at large or to

achieve the lofty goals it has set for itself unless researchers, practitioners, universities, and professional

organizations implement significant changes. In the aggregate, the changes address the broad challenge

of how to narrow the academic practitioner divide.

Keywords: I O psychology, research in, future of, human-capital trends

Significant scientific advances in the field of and industrial and Cascio, 2007b, 2008; Gelade, 2006; Hodgkinson, 2006; Markides,

organizational (I O) psychology are documented in detail in sev- 2007; Rynes, 2007; Rynes, Bartunek, & Daft, 2001), despite

eral available reviews (e.g., Anderson, Ones, Sinangil, & Viswes- efforts to reduce it.

varan, 2001; Borman, Ilgen, & Klimoski, 2003; Dunnette & In light of these concerns, it seems reasonable to address ques-

Hough, 1990, 1991, 1992; Rogelberg, 2007; Triandis, Dunnette, & tions such as the following: What type of knowledge, in terms of

Hough, 1994). Although scientific advances are important in areas content, has I O psychology produced and what is it currently

central to the field, such as assessment, validation, motivation, producing? Does the knowledge produced by I O psychology

leadership, performance appraisal, training, data-analytic tech- address important societal issues that involve people and work

niques, and research methodology, one constant in the field has settings (i.e., human-capital trends)? Does I O psychology pro-

been its support of the scientist practitioner model (Bass, 1974; duce research that is relevant to employees, their managers,

Dunnette, 1990; Lapointe, 1990; McHenry, 2007; Murphy & Saal, broader stakeholders, and society at large and that informs human

1990; Rupp & Beal, 2007). resource management (HRM) practitioners and other organiza-

Unfortunately, there are concerns with the effective implemen- tional decision makers?

tation of the scientist practitioner model because there is a serious For purposes of this article, we adopt Gelade s (2006) definition

disconnect between the knowledge that academics are producing of practitioners, namely, those who make recommendations about

and the knowledge that practitioners are consuming (McHenry, the management or development of people in organizational set-

2007; Rynes, Colbert, & Brown, 2002). As a result, many orga- tings or advise those who do. Research is relevant to the extent that

nizations fail to adopt the practices that I O psychology research it generates insights that practitioners find useful for understanding

suggests are most effective (Dipboye, 2007), and the scientist their own organizations and situations better than before (Ver-

practitioner gap persists (Aguinis & Pierce, 2008; Anderson, 2007; meulen, 2007). In today s age of accountability and inadequate

funding for research and universities in general, I O psychology

surely would benefit from producing knowledge that is seen as

relevant and important by stakeholders outside of academic I O

Wayne F. Cascio and Herman Aguinis, Business School, University of

psychology circles and even outside of the I O psychology field in

Colorado Denver.

general (e.g., high-level university administrators, public policy-

We thank Richard Klimoski, Frank Landy, Chuck Pierce, Steven Rogel-

berg, and Neal Schmitt for their extremely helpful comments on an earlier makers, members of the community at large).

version of this article. This research was conducted, in part, while Herman At one level, it seems ironic even to be asking questions related

Aguinis was on sabbatical leave from the University of Colorado Denver and to the broader relevance of I O psychology research because the

holding visiting appointments at the University of Salamanca, Salamanca,

field, from its inception, was created to address, inform, and help

Spain, and the University of Puerto Rico Ricinto de R o Piedras.

solve important human-capital challenges (e.g., selection of sol-

Correspondence concerning this article should be addressed to

diers to fill a wide variety of jobs in World Wars I and II). For

Wayne F. Cascio, Business School, University of Colorado Denver,

example, consider the recent account of some of the classic appli-

Campus Box 165, P.O. Box 173364, Denver, CO 80217-3364. E-mail:

cations of organizational psychology, along with rich detail about

*****.******@********.***

1062

INDUSTRIAL ORGANIZATIONAL PSYCHOLOGY: 196*-****-****

for narrowing the academic practitioner divide, along with

the difficulties of practical implementation provided by Highhouse

changes that can be made by researchers, universities, and profes-

(2007). That same sense of mission is true today, as Rogelberg

sional organizations. Our hope is that such changes might improve

(2007, p. xxxv) noted, In general the goals of I/O psychology are

human welfare in the workplace; inform debates over human-

to better understand and optimize the effectiveness, health, and

capital issues that are critical to employees, their managers,

well-being of both individuals and organizations. Likewise, the

broader stakeholders, and society at large; and lead the field of I O

mission statement of the Society for Industrial and Organizational

psychology closer to achieving the lofty goals it has set for itself.

Psychology (SIOP) is to enhance human well-being and perfor-

mance in organizational and work settings by promoting the sci-

ence, practice, and teaching of I O psychology (SIOP, 2007).

Method

In the present article, we offer a 45-year (1963 2007) content

analysis and review of published research in I O psychology in the Overview and Background

two leading journals in the field: Journal of Applied Psychology

(JAP) and Personnel Psychology (PPsych). Such an analysis and We used content analysis, which is a method that converts

review affords us the opportunity to assess the extent to which qualitative material into quantitative material that subsequently

the field of I O psychology is producing research that addresses can be analyzed subject to the limitations of nominal-level mea-

the questions mentioned above and, therefore, allows us to surement (cf. Aguinis, Henle, & Ostroff, 2001; G. Shapiro &

inform the important discussion of the extent to which the field Markoff, 1997). More specifically, our data collection procedure

is achieving its goals and fulfilling its mission. consisted of manually coding every article published in every issue

Another purpose of this article is to determine whether the between January 1963 and May 2007 in JAP and PPsych on the

research literature in the field of I O psychology, as well as two basis of its content. In total, we coded 5,780 articles in the two

characteristics of those who generate it geographic location (i.e., journals (n 4,329 for JAP and n 1,451 for PPsych).

U.S. based vs. non-U.S. based) and affiliation (i.e., academic vs. Why did we begin with the year 1963? Initially, the rationale

nonacademic) reflect patterns of stability and change. An in- was to cover a 30-year period, given that preliminary results of this

crease in non-U.S.-based authors over time would suggest a trend research were presented by Wayne F. Cascio as part of his presi-

toward the globalization of I O psychology and also that the field dential address to SIOP in 1993. Although any starting point is

is responding to the broader societal trend toward globalization. In arbitrary, our decision to code all articles over a 45-year period

addition, an increase in nonacademic authors would suggest a was based on the rationale that this is certainly long enough to

trend toward research that is more amenable to implementation, identify trends reliably. On a related note, our 45-year time span

given that nonacademics often make decisions and plan interven- included a sufficiently large number of editors. Specifically, our

tions related to the topics their research addresses. review includes the following editorial terms for JAP (based on the

Previous reviews of the I O psychology literature have been journal s masthead): Kenneth E. Clark (1961 1970), Edwin A.

selective and have not addressed general content issues directly Fleishman (1971 1976), John P. Campbell (1977 1982), Robert

(Austin, Scherbaum, & Mahlman, 2002; Muchinsky, 1979; M. Guion (1983 1988), Neal Schmitt (1989 1994), Philip Bobko

Strasser & Bateman, 1984). As a further step to locate any addi- (1995 1996), Kevin R. Murphy (1997 2002), and Sheldon Zedeck

tional content analyses of the I O psychology literature, we man- (2003 2007). It also includes the following editorial terms for

ually searched each of the 153 issues of JAP and each of the 109 PPsych (based on the journal s masthead): G. Frederic Kuder

issues PPsych from January 1980 to May 2007 for articles that (1959 1963), John A. Hornaday (1964 1969), Frederic Kuder

described analyses of the content of published literature in the and John A. Hornaday (1970 1971), Frederic Kuder (1971 1974),

field. With the exceptions of the articles just cited, we found none. Milton D. Hakel (1974 1984), Paul R. Sackett (1985 1990),

To our knowledge, therefore, our study is the first systematic Michael A. Campion (1991 1996), John R. Hollenbeck (1997

assessment of the general content of published research articles in 2002), and Ann Marie Ryan (2003 2007). Finally, there are also

the field of I O psychology covering more than four and one half parallels at the individual level. The time period from 1963 to 2007

decades, as reflected in the two leading journals in the field. The spans more than four decades, or roughly an entire professional

present article differs from previous reviews in that our focus is on career. Assuming a 5-year graduate training period, 45 years spans

topics, content areas, geographic location, author affiliations, and nine cohorts of new scholars in the field who contributed to the

the correspondence between published research and human-capital published literature and thus it is important to assess the aggregate

trends rather than on methodology or other narrower aspects of the nature of changes in that literature.

I O psychology literature. Content analysis is primarily a qualitative methodology, but, as

In sum, to answer the questions included above, we examined noted above, it also includes a quantitative component, which

the topical areas investigated by each of the 5,780 articles pub- provides an advantage over other more purely qualitative methods,

lished in JAP and PPsych from January 1963 to May 2007. This is such as literary interpretation and hermeneutics (Duriau, Reger, &

the most extensive content analysis of the I O psychology litera- Pfarrer, 2007). Content analysis methodology is especially suited

ture conducted to date. Our purposes are to (a) provide a descrip- for rigorous exploration of many important issues that are often

tion of topical areas that have received the most and least, as well difficult to study (Duriau et al., 2007). Also, it is particularly suited

as increased or decreased, attention over the past 45 years; (b) to answer the questions we posed in the introduction because, as

describe these results in light of important human-capital trends noted by Duriau et al. (2007), content analysis (a) provides a

within each decade; (c) examine lagged relationships between replicable methodology to access deep collective values (i.e., top-

published research and human-capital trends; and (d) discuss im- ical areas that are of more [or less] interest in the field of I O

plications for the field of I O psychology and propose strategies psychology), (b) is highly flexible (i.e., it allows us to categorize

CASCIO AND AGUINIS

1064

articles into various topical areas in the field of I O psychology), tions. Despite their limitations, we examined these data as addi-

and (c) is nonintrusive (i.e., authors of the articles we coded did not tional independent forms of evidence to answer the questions

participate actively in our research study). posed in the introduction.

Content Analysis Taxonomy Coding Process and Agreement Checks

To provide a systematic basis for content coding each article, we Wayne F. Cascio coded articles in JAP and PPsych from 1963

developed a taxonomy that included 15 broad categories, which to 1987, and a graduate student whom he trained, and who was

subsume 50 more specific ones. The final version of the taxonomy, naive to the purposes of the study, coded data from 1987 to 1992.

which was used to code all the articles, is contained in the Ap- As a check on interrater agreement, both individuals independently

pendix. Wayne F. Cascio developed the taxonomy in an iterative coded data from the year 1987. The Pearson correlation between

fashion. He began by identifying key themes, as reflected in I O the frequency counts tallied by the graduate student and Wayne F.

psychology textbooks that described the field. He then pilot tested Cascio for each of the 15 categories included in the Appendix was

several early versions of the taxonomy by content coding 4 years .88. Wayne F. Cascio also coded all articles in JAP from 2001 to

of journal articles published in JAP and PPsych, two early years 2003, all articles in PPsych from 2004 to 2006, and the first and

(1965 and 1969) and two later years (1988 and 1991). The pilot third issues of JAP in 2007. A second I O psychology graduate

test revealed the need for a more refined classification scheme to student, naive to the purposes of the study, coded the articles in

cover the broad range of topics that appeared in these two journals. JAP and PPsych from 1993 to 1997. Herman Aguinis coded all

The addition of the full range of 50 subcategories shown in the articles in PPsych for 1993 1994 and 1996 1997, all articles in

Appendix resulted in a content-coding scheme that provided com- JAP and PPsych for 1998 2000, all articles in PPsych for 2001

plete coverage of all articles in the 4 years that comprised the pilot 2003 and 2007 (first issue only, given that our review goes to May

test. The taxonomy then was applied to all other years of published 2007), all articles in JAP from 2004 to 2006, and the second issue

articles. of JAP for 2007. We purposefully divided up the coding task this

Like any other taxonomy that is created inductively, we cannot way, with different coders coding different journals over time, to

say unequivocally that this is the only possible taxonomy. We can minimize any potential systematic coding bias.

say, however, that it is sufficiently comprehensive to classify each In cases where an article had a good fit with more than one

of the 5,780 articles included in our review. Also, it is important to category, we chose the category with the perceived best fit, which

note that the comparison of publication trends with broader may have led to the potential underrepresentation of some areas.

human-capital trends involves the inevitable task of trying to map However, there is no reason to believe that this potential under-

categories from one area onto the other. This can be challenging, representation was systematic across articles or that it affected

especially when the terms used by academics and practitioners do specific areas adversely. In other words, there is no evidence that

not overlap, as we illustrate in the Discussion section with the the resulting relative rank order of areas is biased.

particular case of talent management. Given the use of different As an additional check on the replicability of the coding process,

terminology, and sometimes different meanings attached to the we computed correlation coefficients between the frequency

same terms, the exercise of trying to map one set of categories onto counts tallied by the second graduate student and Herman Aguinis

the other is challenging, regardless of the categories included in the for each of the 15 categories included in the Appendix. Resulting

taxonomies. correlations were .94 for 1993, .98 for 1994, .93 for 1996, and .92

for 1997. This high level of agreement was expected because the

graduate student had completed all coursework toward a master s

Geographic Location and Affiliation

degree in I O psychology and met several times with Wayne F.

We used the authors bylines on the first page of each article to Cascio to discuss the nature of the coding task (i.e., participated in

determine geographic location and affiliation. We defined aca- several coding training sessions) and Herman Aguinis is an expe-

demic as a university affiliation. We coded all other affiliations rienced researcher in I O psychology who has coauthored a widely

(e.g., government, corporation, consulting organization, applied used graduate-level I O psychology textbook with Wayne F. Cas-

research organization) as nonacademic. The rationale for including cio (Cascio & Aguinis, 2005). Thus, each of the four coders had a

geography is that greater participation of non-U.S.-based authors common, shared understanding (i.e., a shared mental model) of the

would be an indicator of a trend toward globalization (note that we field of I O psychology and how to categorize each article, which

coded authors geographic location irrespective of data collection explains why the agreement among coders was so high.

location). The rationale for coding for affiliation is that, in our

opinion, authors not affiliated with universities are, on average, Time Blocks

more likely to conduct research that is amenable to implementation

because they usually make decisions and plan interventions related Although the coding of the articles was done year by year, we

to the topics of their research on a regular basis. At the same time, aggregated the results on the basis of 5-year time blocks, resulting in

some internal I O psychologists may have narrow views and see the following nine time periods: (a) 1963 1967, (b) 1968 1972, (c)

only what is of interest to their organizations. Geographic location 1973 1977, (d) 1978 1982, (e) 1983 1987, (f) 1988 1992, (g)

and affiliation are less than perfect proxies, however, because there 1993 1997, (h) 1998 2002, and (i) 2003 May 2007 (note that at the

are U.S.-based authors who are aware of and concerned about time of the writing of this article, only the first of the four issues of

global issues and academic authors who are aware of and con- 2007 PPsych and the first three of the six issues of 2007 JAP were

cerned about the implementation of research findings in organiza- available and included in the analyses). Five-year time blocks are

INDUSTRIAL ORGANIZATIONAL PSYCHOLOGY: 196*-****-****

arbitrary, but they represent a smoothing technique that is more likely

Row total

298

284

161

179

103

1,451

to reveal underlying trends. As we noted earlier, they also generally

represent a complete period of doctoral training. Examination of

content categories on a yearly basis creates noise, or illusory trends,

that tend to be clarified over longer time periods.

2003 2007

111

Results

We organize results into the following sections. First, we pro-

1998 2002

vide an analysis of trends regarding the 15 broad areas, including

146

areas for which there is an upward trend, a downward trend, and no

change. Second, we provide an analysis of trends regarding geo-

graphic location and affiliation. Third, we analyze trends regarding

1993 1997

the 50 subcategories. Finally, we analyze the correspondence

between published research in I O psychology and major human-

150

capital trends identified by decade.

1988 1992

Trends Regarding the 15 Broad Topical Areas

140

Tables 1 and 2 include results for the entire 45-year period for

PPsych and JAP, respectively, for each of the 15 broad topical

areas. Table 1 reveals that the top five topical areas (ranked from

1983 1987

1 to 5) published in PPsych from 1963 to 2007 are as follows (the

163

first number in parentheses indicates the total number of articles

for each topic, and the second number indicates the percentage of

the 1,451 articles published on this topic over the entire 45-year

period): 1978 1982

Frequency Counts of Articles Based On Topical Areas For Personnel Psychology (1963 2007)

205

1. Methodology psychometric issues (298; 20.54%),

2. Predictors of performance (284; 19.57%),

1973 1977

3. Work motivation and attitudes (179; 12.34%),

183

4. Performance measurement work outcomes (161;

11.10%), and

1968 1972

206

5. Leader influences (103; 7.10%).

Table 2 reveals that the top five topical areas (ranked from 1 to

5) published in JAP from 1963 to 2007 differ only slightly from

1963 1967

those above (the first number in parentheses indicates the total

147

number of articles for each topic, and the second number indicates

the percentage of the 4,329 articles published on this topic over the

entire 45-year period):

Human factors applied experimental psychology

1. Methodology psychometric issues (940; 21.71%),

Research methodology psychometric issues

Performance measurement work outcomes

2. Work motivation and attitudes (688; 15.89%),

3. Predictors of performance (544; 12.57%),

Topical area

Work motivation and attitudes

4. Performance measurement work outcomes (425;

Training and development

Predictors of performance

9.82%), and

Work groups teams

Contact this candidate