HCI Logo
Heidelberg University Logo
IWR Logo

Machine Learning and Physics

Machine learning is becoming a transformational force in our society, and will profoundly impact humanity in ways both good and bad. On the good side, recent scientific breakthroughs such as having solved the protein folding problem will dramatically accelerate the development of new drugs and vaccines; on the bad side, we are facing a future in which deep fakes are used for manipulation, autonomous weapons roam our skies and machine learning supports surveillance in totalitarian states.

On the scientific side, machine learning is making its weight felt across all disciplines. Some go so far as to postulate a fifth paradigm of scientific discovery, fueled by machine learning.

In view of the role that machine learning is starting to play also in physics, the faculty of Physics and Astronomy has decided to establish a new MSc core course: Machine Learning and Physics.

Contents

Physics of Machine Learning: Highlight core physics concepts that drive ML

Machine Learning for Physics: Equip you with tools to help conduct, and interpret, future experiments

The course introduces some of the most important techniques for inference, and for regression, classification, dimension reduction and density estimation; and it emphasizes the physical ideas and laws needed to make these work. See below for a more detailed curriculum.

Taking part, and admin stuff

  1. If, after browsing the FAQ below, you believe this course is for you: Then please register here.
  2. The course starts with a python refresher in the tutorial on Oct 17th or 18th (identical content). Unless you are familiar with python and it's basic scientific stack (jupyter, numpy, matplotlib, scipy), please take part to help you solve the computational exercises.
  3. The main lectures are on Tuesdays and Thursdays from 09h15-11h00 in Großer Hörsaal, Philosophenweg 12.
  4. Identical plenary tutorials are offered on Mondays and Tuesdays, from 16h15-18h00. Pick whichever day suits you best. Tutorials are held in KIP, INF 227 HS2.

FAQ

  • Q: Do I need prior knowledge in machine learning?

  • A: No.

  • Q: I just want to learn the basics. Is this the right course?

  • A: This course will have a steep learning curve; if you only want to cover the basics, you probably find easier alternatives.

  • Q: Is this course about deep learning?

  • A: Neural networks will play an important role; but this course is more about principles. For sure we will not discuss details of the latest architectures.

  • Q: Will this course be repeated next year?

  • A: Yes, like every MSc core course. In winter semester 2023, the course will be given by Jan Pawlowski and Tilman Plehn.

  • Q: I do not find the course listed as MSc core course in the MSc module handbook. Why?

  • A: Because the version on the departmental's website has not been updated yet; this will happen before start of the teaching term.

  • Q: Is there a text book?

  • A: The book with the biggest overlap is the soon-to-be-published Murphy, Probabilistic Machine Learning: Advanced Topics. Full pdf available here.

  • Q: Exam modalities?
  • A: To be admitted to the written exam at the end of the semester, you need to gain 50% of the points in the exercise sheets.

Preliminary curriculum

  1. Introduction & linear dimension reduction
  2. Nonlinear dimension reduction: connection to stat. mechanics
  3. Nonparametric density estimation
  4. Basic clustering techniques, review of information theory
  5. Classification, take 1: discriminative
  6. Review: Multivariate distributions, Bayes theorem, conjugate priors
  7. Classification, take 2: parametric / generative
  8. Regression
  9. Regularization, linear differential operators
  10. Gaussian processes
  11. Classification, take 3: logistic regression, generalized linear models
  12. Multi-layer perceptrons
  13. Multi-layer perceptrons: capacity
  14. Deep neural networks
  15. Architectures
  16. Directed Probabilistic Graphical Models
  17. Hidden Markov Models
  18. Kalman filter
  19. Markov decision processes, Reinforcement learning
  20. Gaussian Mixture Model (GMM), variational methods, mean field
  21. Variational auto-encoders
  22. Markov Chain Monte Carlo, Hamiltonian Monte Carlo
  23. Geometric Machine Learning: symmetries, groups, graph neural networks
  24. Attention / transformers
  25. Diffusion models, normalizing flow
  26. Nonlinear association measures
  27. Optimal transport
  28. Graph partitioning and network analysis
  29. Ethics of ML
  30. Q&A