🗓️ Week 10 - Neural Networks and Bayesian Neural Networks

Author

Affiliation

Dr. Simone Rossi

EURECOM

This week we introduce neural networks as flexible parametric models for supervised learning. We cover the probabilistic interpretation of standard training objectives, the connection between weight decay and Gaussian priors, and the extension to Bayesian neural networks (BNNs), which place distributions over the weights. We also explore approximate inference strategies for BNNs (MC dropout, deep ensembles) and the theoretical link between infinitely-wide neural networks and Gaussian processes. Finally, we discuss applications of Bayesian machine learning including uncertainty decomposition, Bayesian continual learning, and Bayesian active learning.

🖥️ Lecture Slides

• Neural Networks and Deep Learning
• Applications of Bayesian Machine Learning

📑 Lecture Notes (PDF)

• Neural Networks and Deep Learning
• Applications of Bayesian Machine Learning

📚 Recommended Reading

• Murphy (2023): Chapter 16 (Neural Networks); 17 (Bayesian Neural Networks)
• Bishop (2006): Chapter 5 (Neural Networks)

📙 References

Bishop, C. M. (2006). Pattern recognition and machine learning (1st ed. 2006. Corr. 2nd printing 2011). Hardcover; Springer. http://www.worldcat.org/isbn/0387310738

Murphy, K. P. (2023). Probabilistic machine learning: Advanced topics. MIT Press. http://probml.github.io/book2