Bayesian variable selection
Stage 4 project, 2026/27
- Supervisor: Darren Wilkinson
- Project research area: Statistics
Project outline
Bayesian variable selection (BVS) is concerned with the commonly encountered problem of deciding which variables to include in a statistical model based on whether or not they are useful, in a Bayesian manner. Since every subset of variables corresponds to a different model, this is a model selection problem. However, if there are p variables, there are 2^p models to choose from, and so explicit evaluation of the posterior probability of every possible model may not be practical. Nevertheless, Bayesian analysis proceeds by placing a prior distribution over models and model parameters, and computationally intensive methods are typically employed to explore the resulting posterior distribution. There are several quite different approaches that can be taken to address this problem, and the potential applications are many and varied.
Potential areas for more in-depth study
- Discrete slab-and-spike regression in the style of Kuo and Mallick
- Gibbs variable selection (GVS)
- Reversible jump MCMC for BVS
- Continuous shrinkage priors
- The Bayesian LASSO
- Global-local shrinkage
- Regularised horseshoe priors
- Non-local priors
- Mixing of MCMC algorithms for various different shrinkage/selection priors
- BVS in probabilistic programming languages
- Modelling sparsity
- Applications to tractable and intractable regression models
- Graphical model selection
- Applications to multivariate time series and Granger causality
Mode of operation and evidence of learning
The project will involve learning through reading and discussion, and programming in R. Students will demonstrate their understanding by implementing different BVS methods and testing them on real and simulated data. Students will clearly communicate the material in both written and oral formats.
Pre-requisites
You must have taken Bayesian Computation and Modelling III (MATH3421) at Stage 3. You must also be comfortable with programming in R.
Some relevant resources
Books
Papers
- Dellaportas, P., Forster, J. J., Ntzoufras, I. (2002) On Bayesian model and variable selection using MCMC, Statistics and Computing, 12:27-36.
- George, E. I., McCulloch, R. E. (1993) Variable Selection via Gibbs Sampling, JASA, 88(423):881-889.
- Griffin, J. E., Brown, P. J. (2013) Some Priors for Sparse Regression Modelling, Bayesian Analysis, 8(3): 691-702.
- Kuo, L., Mallick, B. (1998) Variable Selection for Regression Models, Sankhya, 60(B1): 65-81.
- O’Hara, R. B., Sillanpää, M. J. (2009) A review of Bayesian variable selection methods: what, how and which, Bayesian Analysis, 4(1): 85-117.
- Piironen, J., Vehtari, A. (2017) Sparsity information and regularization in the horseshoe and other shrinkage priors, Electronic Journal of Statistics, 11(2): 5018-5051.
- Polson, N. G., Scott, J. G. (2011) Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction, in Bayesian Statistics 9, 501-538.
- Rossell, D., Telesca, D. (2017) Non-local Priors for High-dimensional Estimation, JASA, 112(517):254-265.
The review paper by O’Hara and Sillanpää (2009) provides a nice introductory overview of the area.