Machine learning provides an invaluable toolbox for the natural sciences, but it also comes with many open questions that the theoretical branches of the natural sciences can investigate. In this Presidential Lecture, Lenka Zdeborová will describe recent trends and progress in exploring questions surrounding machine learning. She will discuss how diffusion or flow-based generative models sample (or fail to sample) challenging probability distributions. She will present a toy model of dot-product attention that presents a phase transition between positional and semantic learning. She will also revisit some classical methods for estimating uncertainty and their status in the context of modern overparameterized neural networks. More details: https://www.simonsfoundation.org/event/statistical-physics-of-machine-learning/