The Limits of Predicting Individual-Level Longevity

Luca Badolato, The Ohio State University
Ari Decter-Frain, Cornell University
Nicholas Irons, University Of Washington
Maria Miranda, Max Planck Institute for Demographic Research
Erin Walk, Massachusetts Institute of Technology
Elnura Zhalieva, Mohamed Bin Zayed University of Artificial Intelligence
Monica Alexander, University of Toronto
Ugofilippo Basellini, Max Planck Institute for demographic Research
Emilio Zagheni , Max Planck Institute for demographic Research

Individual-level mortality prediction is a fundamental challenge with implications for people and societies, including improving life planning, targeting high-risk individuals, and organizing social policies and public spending. However, demographers and actuaries have been primarily concerned with mortality prediction at a macro level, overlooking individual-level mortality predictions outside of clinical settings. We model and predict individual-level lifespan using data from the U.S. Health and Retirement Study, a nationally representative longitudinal survey of people over 50 years of age. We estimate 12 statistical and machine learning survival analysis models using over 150 predictors measuring behavioral, biological, demographic, health, and social indicators. Extending previous research, we investigate inequalities in individual mortality prediction by gender, race and ethnicity, and education. Machine learning and statistical models report comparable accuracy and relatively high discriminative performance, notably when including time-varying information (best mean Area Under the Curve = 0.87). However, the models and predictors fail to account for most lifespan heterogeneity at the individual level. We observe consistent inequalities in mortality predictability and risk discrimination, with lower prediction accuracy for men, non-Hispanic Blacks, and low-educated individuals. Additionally, people in these groups show lower accuracy in their subjective predictions of their own lifespan. Finally, top features across groups are similar, with variables related to habits, health history, and finances being relevant predictors. We conclude by highlighting the limits of predicting mortality from representative surveys and the implications of inequalities in predictability across social groups, providing baselines and guidance for future research and public policies.

See paper

 Presented in Session 74. Mortality Modelling