Development of tools and frameworks for sensitivity analysis in agent-based models
The potential of Agent Based Models (ABMs) as an important tool in population health lies in their unique ability to explore explicitly how behaviours and interactions at the individual level drive and are driven by aggregate or population-level interactions and outcomes.
Although ABMs are typically tested against real-world data, sometimes this is not feasible e.g. due to data being unavailable or sparse. An alternative strategy is to use sensitivity analysis – enabling the modeller to identify which parameters are most sensitive to model assumptions, and to which parameters the system is most sensitive. Ultimately this helps identify the most important targets for interventions and/or key targets for further data collection.
The aim of this core funded project is to develop and compare methods for uncertainty quantification in ABMs. The most widespread technique for uncertainty quantification is currently Gaussian Process Emulation, in which a simulation is conceptualised as a function of parameter values to output values, and a statistical model of the simulation is then constructed using a set of training data that captures simulation behaviour across a wide range of input parameter values. This simplified statistical model of the original model is called an emulator and allows the modeller to conduct detailed analyses of the simulation much more quickly by using the emulator as a surrogate. This method is highly effective but has displayed some limitations in complex agent-based models where the simulation may have sudden and significant changes in output values even at similar values of the input parameters.
In this project we will investigate alternative machine-learning methods for uncertainty quantification to Gaussian Process Emulation and develop tools to facilitate the use of these methods in agent-based modelling projects. Early work has indicated that deep neural networks are highly effective for generating emulators even when the model's behaviour is complex; however, generating deep-learning-based emulators demands significantly more processing time than GP emulation. Other ML-based methods such as gradient-boosted trees and support vector machines also performed well, suggesting that these methods may be viable alternatives when processing power is at a premium.
Key outputs of this project will include the development of accessible emulation tools for modellers, and the production of documentation and guidance to encourage the wider uptake of these methods.
