Rethinking Risk Elicitation Tasks – RETRISK

Risk is an important component of decision making. Risk attitudes play a crucial role in a variety of settings, from insurance, financial decision making, education and career choices, to development and public policy. The accurate measure of risk preferences is of large importance in both theoretical and applied work.
Increasing experimental evidence points to the fact that the behavioral measures we use to elicit risk attitudes fail us. Risk elicitation tasks (RETs), at least in the way they are typically implemented in behavioral economics and psychology, 1. correlate poorly with self-reported risk attitudes, real-world risk behaviors, and among themselves; 2. introduce distinct measurement errors and behavioral biases, and 3. are not robust to sit-resit exercises. This project is an ambitious and comprehensive attempt at rethinking risk elicitation.
The project addresses the question of the low predictive power of RETs by means of a meta-analysis and two large experiments.
The meta-analysis will focus on the predictive validity of RETs, and will both document exhaustively the extent of the problems faced by RETs and give a detailed map of which features of a RET are more conducive to predictive validity.
Experiment 1 will focus on the role of noise in explaining the disappointing results of RETs. Firmly rooted in utility theory, the experiment will make use of standard pairwise lottery choices and structural estimation techniques. The experiment will be a conscious effort aimed directly at maximising the external validity of the estimates. We will use a mixture of meta-analysis, simulations, on-line and lab experiments, machine learning techniques, to find the set of lotteries that can obtain the largest possible external validity while keeping high internal validity. The estimates obtained will then be correlated with a risk factor extracted from self-reported measures. This experiment will allow us to assess the predictive gain (if any) obtained by ‘going big’ and taking into account noise.
Experiment 2 will be a step outside of utility theory and will focus on risk perception. Econmists usually (tacitly) assume that the riskiness of a lottery is an objective metrics. In Experiment 2 we will allow subjects to have differing perceptions of risk. We will ask subjects to directly assess their risk perception of lotteries before we ask them to choose. This Experiment will a) develop an empirical way to measure the risk perception of different lotteries; b) use this to further optimize the set of lottery choices to be given to subjects to best identify their risk attitudes and c) check if behavior, once cleaned from the effects of risk perception, is more consistent than previously thought across RETs and between RETs and self-reported measures.
The project will bring about several improvements in the field of risk elicitation: a deeper understanding of the state of the art in the external validity of risk elicitation measures (via the meta-analysis); a map of the features a RET needs to have predictive power (via preparatory work and results of Experiment 1); and an enhanced understanding of the risk perception of RETs (via methods and results of Experiment 2).
If successful, the project will most importantly deliver a new, stronger, more predictive RET robust to noise and incorporating risk perceptions, that could replace the RETs used in the field and provide more reliable and predictive risk data. This is something the whole field of experimental economics has been awaiting for years. This is not just an interesting academic exercise. Fixing the low predictive power, reliability and robustness of RETs would have direct and valuable consequences in all fields involving uncertain options – as in insurance, education, medicine.