Machine Learning and Data Analysis for Nuclear Physics, a Nuclear TALENT Course at the ECT*, Trento, Italy, July 19 to July 30 2021.
Probability theory and statistical methods play a central role in science. Nowadays we are surrounded by huge amounts of data. For example, there are about one trillion web pages; more than one hour of video is uploaded to YouTube every second, amounting to 10 years of content every day; the genomes of 1000s of people, each of which has a length of more than a billion base pairs, have been sequenced by various labs and so on. This deluge of data calls for automated methods of data analysis, which is exactly what machine learning provides. The purpose of this Nuclear Talent course is to provide an introduction to the core concepts and tools of machine learning in a manner easily understood and intuitive to physicists and nuclear physicists in particular. We will start with some of the basic methods from supervised learning and statistical data analysis, such as various regression methods before we move into deep learning methods for both supervised and unsupervised learning, with an emphasis on the analysis of nuclear physics experiments and theoretical nuclear physics. The students will work on hands-on daily examples as well as projects than can result in final credits. Exercises and projects will be provided and the aim is to give the participants an overview on how machine learning can be used to analyze and study nuclear physics problems (experiment and theory). The major scope is to give the participants a deeper understanding on what Machine learning and Data Analysis are and how they can be used to analyze data from nuclear physics experiments and perform theoretical calculations of nuclear many-body systems.
The goals of the Nuclear Talent course on Machine Learning and Data Analysis are to give the participants a deeper understanding and critical view of several widely popular Machine Learning algorithms, covering both supervised and unsupervised learning. The learning outcomes involve an understanding of the following central methods:
- Basic concepts of machine learning and data analysis and statistical concepts like expectation values, variance, covariance, correlation functions and errors;
- Estimation of errors using cross-validation, blocking, bootstrapping and jackknife methods;
- Optimization of functions
- Linear Regression and Logistic Regression;
- Dimensionality reductions, from PCA to clustering
- Boltzmann machines;
- Neural networks and deep learning;
- Convolutional Neural Networks
- Recurrent Neureal Networks and Autoencoders
- Decisions trees and random forests
- Support vector machines and kernel transformations
We are targeting an audience of graduate students (both Master of Science and PhD) as well as post-doctoral researchers in nuclear experiment and theory.
The teaching teams consists of both theorists and experimentalists. We believe such a mix is important as it gives the participants a better understanding on how data are obtained, and what are the limitations and possibilities in understanding and interpreting the experimental information.
A recently established initiative, Training in Advanced Low Energy Nuclear Theory, aims at providing an advanced and comprehensive training to graduate students and young researchers in low-energy nuclear theory. The initiative is a multinational network between several European and Northern American institutions and aims at developing a broad curriculum that will provide the platform for a cutting-edge theory for understanding nuclei and nuclear reactions. These objectives will be met by offering series of lectures, commissioned from experienced teachers in nuclear theory. The educational material generated under this program will be collected in the form of WEB-based courses, textbooks, and a variety of modern educational resources. No such all-encompassing material is available at present; its development will allow dispersed university groups to profit from the best expertise available.
This two-week online TALENT course on nuclear theory will focus on Machine Learning and Data Analysis algorithms for nuclear physics and to use such methods in the interpretation of data on the structure of nuclear systems.
We propose approximately twenty hours of lectures over two weeks and a comparable amount of practical computer and exercise sessions, including the setting of individual problems and the organization of various individual projects.
The mornings will consist of lectures and the afternoons will be devoted to exercises meant to shed light on the exposed theory, the computational projects and individual student projects. These components will be coordinated to foster student engagement, maximize learning and create lasting value for the students. For the benefit of the TALENT series and of the community, material (courses, slides, problems and solutions, reports on students' projects) will be made publicly available using version control software like git and posted electronically on github (this site).
At the end of the course the students should have a basic understanding of
- Statistical data analysis, theory and tools to handle large data sets.
- A solid understanding of central machine learning algorithms for supervised and unsupervised learning, involving linear and logistic regression, support vector machines, decision trees and random forests, neural networks and deep learning (convolutional neural networks, recursive neural networks etc)
- Be able to write codes for linear regression, logistic regression and use modern libraries like Tensorflow, Pytorch, Scikit-Learn in order to analyze data from nuclear physics experiments and perform theoretical calculations
- A deeper understanding of the statistical properties of the various methods, from the bias-variance tradeoff to resampling techniques.
The tentative (prone to revisions) lecture plan is as follows
-
Monday, 2pm-4pm CET Linear Regression and intro to statistical data analysis (Morten Hjorth-Jensen MHJ). Learning slides at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Introduction/html/Introduction.html and https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day1/html/Day1.html. Welcome by the director of the ECT* Prof. Gert Aarts at 4pm.
-
Tuesday, 2pm-4pm CET Logistic Regression and classification problems, intro to gradient methods (MHJ). Learning slides at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day2/html/Day2.html
-
Wednesday, 2pm-4pm CET Decision Trees, Random Forests and Boosting methods (MHJ). Learning slides at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day3/html/Day3.html
-
Thursday, 2pm-4pm CET Basics of Neural Networks and writing your own Neural Network code (MHJ). Learning slides at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day4/html/Day4.html
-
Friday, 2pm-4pm CET Beta-decay experiments, how to analyze various events, with hands-on examples . (Sean Liddick) Videos and teaching material https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day5/html/Day5.html.
Exercise sessions and project work follow from 4pm to 5pm CET every day.
- Monday, 2pm-4pm CET Neural Networks and Deep Learning (Raghu Ramanujan, RR). PDF file of the presented slides at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day6/pdf/Day6.pdf. Jupter-Notebook at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day6/ipynb/Day6.ipynb.
- Tuesday, 2pm-4pm CET From Neural Networks to Convolutional Neural Networks and how to analyze experiment (classification of events and real data) (Michelle Kuchera, MK). Jupyter-notebook of lecture at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day7/ipynb/Day7.ipynb.
- Wednesday, 2pm-4pm CET Discussion of nuclear experiments and how to analyze data, presentation of simulated data from Active-Target Time-Projection Chamber (AT-TPC) (Daniel Bazin). Slides of lectures (PDF) at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day8/pdf/Day8.pdf.
- Thursday, 2pm-4pm CET Generative models (MK). Slides of lectures (PDF) at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day9/pdf/Day9.pdf. Jupyter-notebook at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day9/ipynb/Day9.ipynb.
- Friday, 2pm-4pm CET Reinforcement Learning (RR). Slides of lectures (PDF) at https://nucleartalent.github.io/MachineLearningECT2021/doc/pub/Day10/pdf/Day10.pdf. Future directions in machine learning and summary of course.
Exercise sessions and project work follow from 4pm to 5pm CET every day.
The course will be taught as an online intensive course of duration of two weeks, with a total time of 20 h of lectures and 10 h of exercises, questions and answers. Videos and digital learning material is available from the above links. It is possible to work on a final assignment of 2 weeks of work. The total load will be approximately 80 hours, corresponding to 5 ECTS in Europe. The final assignment will be graded with marks A, B, C, D, E and failed for Master students and passed/not passed for PhD students. A course certificate will be issued for students requiring it from the University of Trento.
The organization of a typical course day is as follows:
Time and Activity
- 2pm-4pm (Central European time=CET) Lectures, project relevant information and directed exercises
- 4pm-5pm (CET) Questions and answers, Computational projects, exercises and hands-on sessions
The teachers and organizers are
- Daniel Bazin at Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University, East Lansing, Michigan, USA (DB)
- Morten Hjorth-Jensen at Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University, East Lansing, Michigan, USA (MHJ)
- Michelle Kuchera at Physics Department, Davidson College, Davidson, North Carolina, USA (MK)
- Sean Liddick at Department of Chemistry and National Superconducting Cyclotron Laboratory, Michigan State University, East Lansing, Michigan, USA (SL)
- Raghuram Ramanujan at Department of Mathematics and Computer Science, Davidson College, Davidson, North Carolina, USA (RR)
Morten Hjorth-Jensen will also function as student advisor and coordinator.
Students and post-doctoral fellows interested in nuclear physics, experiment and theory alike interested in data analysis and machine learning applied to nuclear physics.
The students are expected to have operating programming skills in programming, and in particular on interpreted languages like Python. Preparatory modules on Python programing will be given, idem for a review on linear algebra and statistical data analysis.
Students who have not studied the above topics are expected to gain this knowledge prior to attendance. Additional modules for self-teaching will be provided in good time before the course begins
For more information, please go to https://github.com/NuclearTalent/MachineLearningECT2021 or go to http://nucleartalent.github.io/MachineLearningECT2021/doc/web/course.html for better display of course material and topics to be covered. Admission The target group is Master of Science students, PhD students and early post-doctoral fellows. Also senior staff can attend but they have to be self-supported.
Recommended textbooks: In addition to the lecture notes, we recommend the books of Bishop and Goodfellow et al.
- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, https://www.springer.com/gp/book/9780387310732. Chapters 1-7, 11 and 12 are the most relevant. If you login to your University Library or access this site via a University IP number, you can download for free the textbook in PDF format or epub format.
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. The different chapters are available for free at https://www.deeplearningbook.org/. Chapters 2-14 are highly recommended. Additional textbooks:
- Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer, https://www.springer.com/gp/book/9780387848570. This is a well-known text and serves as additional literature.
- Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly, https://www.oreilly.com/library/view/hands-on-machine-learning/9781492032632/. This text is very useful since it contains many code examples and hands-on applications of all algorithms discussed in this course.
General learning book on statistical analysis:
- Christian Robert and George Casella, Monte Carlo Statistical Methods, Springer
- Peter Hoff, A first course in Bayesian statistical models, Springer
General Machine Learning Books:
- Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press
- David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press
- David Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press