Document Type

Article

Publication Date

4-2023

Publication Source

Nature Human Behaviour

Abstract

How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists' forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data

Inclusive pages

484-501

ISBN/ISSN

2397-3374

Document Version

Published Version

Comments

This open-access article is provided for download in compliance with the publisher’s policy on self-archiving. To view the version of record, use the DOI: https://doi.org/10.1038/s41562-022-01517-1

Department of Psychology, University of Waterloo, Waterloo, Ontario, Canada: Igor Grossmann, Amanda Rotella, Konstantyn Sharpinskyi, Dillon T. Browne & Geoffrey T. Fong

Department of Psychology, Northumbria University, Northumbria, UK: Amanda Rotella

Department of Psychology, University of Toronto Scarborough, Toronto, Ontario, Canada: Cendri A. Hutcherson

Department of Psychology, Arizona State University, Tempe, AZ, USA: Michael E. W. Varnum

Institute of Management Accounting and Simulation, Hamburg University of Technology, Hamburg, Germany: Sebastian Achter

Department of Psychology, Middlesex University London, London, UK: Mandeep K. Dhami

Department of Experimental Psychology, University of California, San Diego, San Diego, CA, USA: Xinqi Evie Guo

Department of Psychology, Toronto Metropolitan University, Toronto, Ontario, Canada: Mane Kara-Yakoubian

Defence Research and Development Canada, Toronto, Ontario, Canada: David R. Mandel & Robert N. Collins

Department of Psychology, York University, Toronto, Ontario, Canada: David R. Mandel

Department of Economics, Tilburg University, Tilburg, the Netherlands: Louis Raes

Department of Psychological Sciences, Purdue University, West Lafayette, IN, USA: Louis Tay

Mathematical Institute, University of Oxford, Oxford, UK: Aymeric Vie

Institute of New Economic Thinking, University of Oxford, Oxford, UK: Aymeric Vie

Jacobs Center for Productive Youth Development, University of Zurich, Zurich, Switzerland: Lisa Wagner

Institute of Psychology, University of Prešov, Prešov, Slovakia: Matus Adamkovic & Gabriel Baník

Institute of Social Sciences, CSPS, Slovak Academy of Sciences, Bratislava, Slovakia: Matus Adamkovic

Department of Mechanical and Mechatronics Engineering, University of Waterloo, Waterloo, Ontario, Canada: Arash Arami

Toronto Rehabilitation Institute (KITE), University Health Network, Toronto, Canada: Arash Arami

Iscte-University Institute of Lisbon, CIS, Lisbon, Portugal: Patrícia Arriaga

Melbourne Centre for Data Science, University of Melbourne, Melbourne, Victoria, Australia: Kasun Bandara

Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, the Netherlands: František Bartoš & Shiva Pauer

Department of Food Marketing, Haub School of Business, Saint Joseph’s University, Philadelphia, PA, USA: Ernest Baskin

Department of Data Science and Artificial Intelligence, Monash University, Melbourne, Victoria, Australia: Christoph Bergmeir

Institute of Psychology, University of Wrocław, Wrocław, Poland: Michał Białek & Michal Misiak

Department of Management, Aarhus University, Aarhus, Denmark: Caroline K. Børsting & Christian T. Elbaek

Anderson School of Management, University of California, Los Angeles, Los Angeles, CA, USA: Eugene M. Caruso

Department of Psychology, Dominican University of California, San Rafael, CA, USA: Rong Chen

Department of Industrial Economics, Tamkang University, New Taipei City, Taiwan: Bin-Tzong Chie

Department of Psychology, Michigan State University, East Lansing, MI, USA: William J. Chopik

Independent Researcher, Penang, Malaysia: Chin Wen Cong

Psychology Department, Grove City College, Grove City, PA, USA: Lucian G. Conway

Department of Economics, Siena College, Loudonville, NY, USA: Matthew Davis

Department of Psychology, Memorial University of Newfoundland, St. John’s, Newfoundland, Canada: Martin V. Day

UBC Sauder School of Business, University of British Columbia, Vancouver, British Columbia, Canada: Nathan A. Dhaliwal

Department of Psychology, University of Oklahoma, Norman, OK, USA: Justin D. Durham & Scott D. Gronlund

Faculty of Psychology and Cognitive Science, Adam Mickiewicz University, Poznań, Poland: Martyna Dziekan

Department of Psychology, University of Groningen, Groningen, the Netherlands: Eric Shuman

Laboratory for Comparative Studies in Mass Consciousness, Expert Institute, HSE University, Moscow, Russia: Marharyta Fabrykant

Faculty of Philosophy and Social Sciences, Belarusian State University, Minsk, Belarus: Marharyta Fabrykant

Department of Sociology, Radboud University, Nijmegen, the Netherlands: Mustafa Firat

Ontario Institute for Cancer Research, Toronto, Ontario, Canada: Geoffrey T. Fong

Department of Psychology, University of Winnipeg, Winnipeg, Manitoba, Canada: Jeremy A. Frimer

Department of Psychology, University of Washington, Seattle, WA, USA: Jonathan M. Gallegos, Cheryl R. Kaiser & Adam H. Smiley

Department of Counseling Psychology, University of Wisconsin–Madison, Madison, WI, USA: Simon B. Goldberg

Department of Leadership and Organizational Behaviour, BI Norwegian Business School, Oslo, Norway: Anton Gollwitzer

Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany: Anton Gollwitzer

School of Public Health Sciences, University of Waterloo, Waterloo, Ontario, Canada: Julia Goyal

TU Dortmund University, Dortmund, Germany: Lorenz Graf-Vlachy

ESCP Business School, Paris, France: Lorenz Graf-Vlachy

IESE Business School, Barcelona, Spain: Sebastian Hafenbrädl

School of Social Sciences, Singapore Management University, Singapore, Singapore: Andree Hartanto & Verity Y. Q. Lua

Center for Healthy Minds, University of Wisconsin–Madison, Madison, WI, USA: Matthew J. Hirshberg

University of Queensland Business School, Brisbane, Queensland, Australia: Matthew J. Hornsey

Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Victoria, Australia: Piers D. L. Howe & Marcellin F. R. C. Martinie

Department of Marketing, University of Massachusetts Dartmouth, Dartmouth, MA, USA: Anoosha Izadi

Department of Experimental and Applied Psychology, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands: Bastian Jaeger, Joshua M. Tybur & Alberto López Ortega

Department of Psychology, Faculty of Arts, Pavol Jozef Šafárik University in Košice, Košice, Slovakia: Pavol Kačmár

Cambridge Judge Business School, University of Cambridge, Cambridge, UK: Yeun Joon Kim

Hermes Germany GmbH, Hamburg, Germany: Ruslan Krenzler

University of Hamburg, Hamburg, Germany: Ruslan Krenzler

Department of Psychology, Illinois State University, Normal, IL, USA: Daniel G. Lannin

Department of Business Administration, National Pingtung University, Pingtung City, Taiwan: Hung-Wen Lin

Department of Psychology, University of Victoria, Victoria, British Columbia, Canada: Nigel Mantou Lou

Centre for Youth and Society, University of Victoria, Victoria, British Columbia, Canada: Nigel Mantou Lou

Department of Psychology, California State University, Fullerton, Fullerton, CA, USA: Aaron W. Lukaszewski

Center for the Study of Human Nature, California State University, Fullerton, Fullerton, CA, USA: Aaron W. Lukaszewski

Department of Psychology, Loma Linda University, Loma Linda, CA, USA: Albert L. Ly & Jaime M. Napan

University of Nottingham, Nottingham, UK: Christopher R. Madan

Department of Experimental Psychology, University College London, London, UK: Maximilian Maier

Singapore Management University, Singapore, Singapore: Nadyanna M. Majeed

Department of Psychology, Florida State University, Tallahassee, FL, USA: David S. March

Department of Psychology, Georgetown University, Washington, DC, USA: Abigail A. Marsh

School of Anthropology & Museum Ethnography, University of Oxford, Oxford, UK: Michal Misiak

School for Business and Society, University of York, York, UK: Kristian Ove R. Myrseth

Department of Psychology, Columbia University, New York, NY, USA: Jonathan Nicholas, Paul A. Bloom & Margaux N. A. Wienk

IHRR Forecasting Laboratory, Durham University, Durham, UK: Konstantinos Nikolopoulos

Department of Psychology, Aberystwyth University, Aberystwyth, UK: Jiaqing O

School of Business and Law, Department of Management, University of Agder, Kristiansand, Norway: Tobias Otterbring

Institute of Retail Economics, Stockholm, Sweden: Tobias Otterbring

Institute of Psychology, University of Silesia in Katowice, Katowice, Poland: Mariola Paruzel-Czachura

Department of Neurology, Penn Center for Neuroaesthetics, University of Pennsylvania, Philadelphia, PA, USA: Mariola Paruzel-Czachura

Central Connecticut State University, New Britain, CT, USA: John Protzko

Department of Psychology, University of Arizona, Tucson, AZ, USA: Quentin Raffaelli & Jessica R. Andrews-Hanna

Faculty of Education, Institute for Research and Development of Education, Charles University, Prague, Czech Republic: Ivan Ropovik

Faculty of Education, University of Prešov, Prešov, Slovakia: Ivan Ropovik

School of Psychology, Macquarie University, Sydney, New South Wales, Australia: Robert M. Ross

Department of Human Service, University of Haifa, Haifa, Israel: Yefim Roth

Promenta Center, Department of Psychology, University of Oslo, Oslo, Norway: Espen Røysamb

Department of Sociology, Cornell University, Ithaca, NY, USA: Landon Schnabel

Institute of Psychology, University of Bamberg, Bamberg, Germany: Astrid Schütz

IE Business School, IE University, Madrid, Spain: Matthias Seifert

Faculty of Psychology and Human Movement Science, University of Hamburg, Hamburg, Germany: T. Sevincer

Publisher

Nature Portfolio

Volume

7

Issue

4

Peer Reviewed

yes

Download

Share

COinS