Insufficient Effort Responding as a Potential Confound between Survey Measures and Objective Tests
- PDF / 702,203 Bytes
- 22 Pages / 595.276 x 790.866 pts Page_size
- 39 Downloads / 188 Views
ORIGINAL PAPER
Insufficient Effort Responding as a Potential Confound between Survey Measures and Objective Tests Jason L. Huang 1
&
Justin A. DeSimone 2
# Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Following research that demonstrates insufficient effort responding (IER) may confound survey measures and inflate observed correlations (Huang, Liu, & Bowling, 2015c), a question emerges as to whether and when IER can act as a confound between objective tests and surveys. Using data (N = 243) originally designed to examine training and transfer, study 1 demonstrates that (a) IER is negatively related to performance on tests, and (b) IER’s influence on surveys depends on the sample means of these measures. As a result, IER could inflate a test’s association with other tests and surveys. Study 2 investigates the impact of two parameters—within-person consistency of IER and percentage of IER cases in the sample—by randomly replacing bootstrapped attentive responses (10,000 bootstrapped samples of 200 cases identified from study 1). When predicting the confounding effects of IER, within-person consistency has positive linear and quadratic effects, percentage of IER cases has a positive linear effect, and consistency and percentage have a positive interactive effect. Research and practical implications for the design and evaluation of surveys and tests are discussed. Keywords Insufficient effort responding . Careless responding . Random responding . Response effort . Measurement
Research in management and organizational psychology often relies on respondents to provide data, be it reports of internal states, perceptions, and experiences in surveys or performance on objective tests and tasks. When utilizing surveys, researchers have emphasized the need to screen for insufficient effort responding (IER; see DeSimone, Harms & DeSimone, 2015; Johnson, 2005; Kam & Meyer, 2015; Meade & Craig, 2012; Maniaci & Rogge, 2014), which occurs when respondents provide careless or random responses to survey items due to low motivation to comply with survey instructions (Huang, Curran, Keeney, Poposki, & DeShon, 2012). This increased awareness has led researchers to propose methods to indicate whether a participant engaged in IER on a survey
* Jason L. Huang [email protected] Justin A. DeSimone [email protected] 1
School of Human Resources and Labor Relations, Michigan State University, 368 Farm Lane, 437 SKH, East Lansing, MI 48824, USA
2
Culverhouse College of Business, University of Alabama, Tuscaloosa, AL, USA
(e.g., Curran, 2016; Huang, Bowling, Liu, & Li, 2015b; Oppenheimer, Meyvis, & Davidenko, 2009; Wood, Harms, Lowman, & DeSimone, 2017). Researchers long believed that IER has an attenuating effect on observed associations (McGrath, Mitchell, Kim, & Hough, 2010). However, recent research identified conditions under which IER can inflate correlations between survey measures, confounding estimates of association. Specifically, when attentive respondents score very high or very low on a sub
Data Loading...
