Quantifying the benefits of using decision models with response time and accuracy data
- PDF / 841,678 Bytes
- 14 Pages / 595.224 x 790.955 pts Page_size
- 65 Downloads / 167 Views
Quantifying the benefits of using decision models with response time and accuracy data Tom Stafford1
· Angelo Pirrone2 · Mike Croucher3 · Anna Krystalli4
Received: 26 August 2019 / Revised: 25 January 2020 / Accepted: 28 January 2020 © The Author(s) 2020
Abstract Response time and accuracy are fundamental measures of behavioral science, but discerning participants’ underlying abilities can be masked by speed–accuracy trade-offs (SATOs). SATOs are often inadequately addressed in experiment analyses which focus on a single variable or which involve a suboptimal analytic correction. Models of decision-making, such as the drift diffusion model (DDM), provide a principled account of the decision-making process, allowing the recovery of SATO-unconfounded decision parameters from observed behavioral variables. For plausible parameters of a typical between-groups experiment, we simulate experimental data, for both real and null group differences in participants’ ability to discriminate stimuli (represented by differences in the drift rate parameter of the DDM used to generate the simulated data), for both systematic and null SATOs. We then use the DDM to fit the generated data. This allows the direct comparison of the specificity and sensitivity for testing of group differences of different measures (accuracy, reaction time, and the drift rate from the model fitting). Our purpose here is not to make a theoretical innovation in decision modeling, but to use established decision models to demonstrate and quantify the benefits of decision modeling for experimentalists. We show, in terms of reduction of required sample size, how decision modeling can allow dramatically more efficient data collection for set statistical power; we confirm and depict the non-linear speed–accuracy relation; and we show how accuracy can be a more sensitive measure than response time given decision parameters which reasonably reflect a typical experiment. Keywords Speed-accuracy trade-off · Drift-diffusion model · Statistical power · Response time · Accuracy Abbreviations DDM - Drift diffusion model SATO - Speed–accuracy trade-off
Introduction Speed–accuracy trade-offs Speed and accuracy of responding are fundamental measures of performance, collected by behavioral scientists across diverse domains in an attempt to track participants’ Tom Stafford
[email protected] 1
Department of Psychology, University of Sheffield, 1 Vicar Lane, Sheffield, S1 2LT, UK
2
Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science, London, UK
3
Numerical Algorithms Group Ltd, Oxford, UK
4
Research Software Engineering, University of Sheffield, Sheffield, UK
underlying capacities. As well as being affected by the capacity of participants to respond quickly and accurately, the two measures are also related by participants’ strategic choices of a speed–accuracy trade-off (SATO; for reviews see Heitz, 2014; Wickelgren, 1977). The SATO confounds measurement of participant capacity—which means that we cannot d
Data Loading...
