Chloé-Agathe Azencott

PSL week Spring Course 2022

MINES28 Large-Scale Machine Learning

March 7-11, 2022

Mines Paris, 60 boulevard Saint-Michel, 75006 Paris Room L.106

This course is co-organized by Chloé-Agathe Azencott (MINES ParisTech & Institut Curie) and Fabien Moutarde (MINES ParisTech).

outline | schedule | registration | grading | textbook | practical sessions

Outline

Machine learning is a fast-growing field at the interface of mathematics, computer science and engineering, which provides computers with the ability to learn without being explicitly programmed, in order to make predictions or take rational actions. From cancer research to finance, natural language processing, marketing or self-driving cars, many fields are nowadays impacted by recent progress in machine learning algorithms that benefit from the ability to collect huge amounts of data and "learn" from them.

The goal of this intensive 5-day advanced course is to present the theoretical foundations and practical algorithms to implement and solve large-scale machine learning and data mining problems, and to expose the students to current applications and challenges of "big data" in science and industry.

Prerequisites:

• Numerical Python (ie familiarity with programming in Python and the numpy, scipy, matplotlib librairies).
• Basics of machine learning (such as the content of the Apprentissage Artificiel course for MINES ParisTech students).

Schedule

Practical sessions are only open to officially enrolled PSL students taking the course for credit.

Monday, March 7, 2022

• 09:00 ­– 12:15 Lecture: Introduction to large-scale ML & optimization (C.-A. Azencott)
• 13:45 – 17:00 Practical session: ML on large data with scikit-learn; this session will also contain an introduction to scikit-learn for those who have not used the library before.

Tuesday, March 8, 2022

• 09:00 ­– 12:15 Lecture: Deep learning, convolutional neural networks, and generative models (F. Moutarde)
• 13:45 – 17:00 Practical session: Deep learning with Python

Wednesday, March 9, 2022

• 09:00 ­– 12:15 Lecture: Systems for large-scale ML: focus on MapReduce (C.-A. Azencott)
• 13:45 – 17:00 Practical session: Stochastic Gradient Descent

Thursday, March 10, 2022

• 09:00 ­– 12:15 Lecture: Deep reinforcement learning (F. Moutarde)
• 13:45 – 17:00 Practical session: Deep reinforcement learning with Python

Friday, March 11, 2022

• 09:00 ­– 12:15 Deep learning at scale (G. Synnaeve & A. Défossez, Facebook AI Research)
• 13:45 – 16:15 Exam.

Registration

PSL students must enroll officially through their institutions.

Mines ParisTech students and staff are welcome to attend the lectures remotely by connecting to room L.106.

PhD students who want to participate may email C.-A. Azencott to register and receive a certificate of attendance. These students will also be allowed to attend the practical sessions, although priority will be given to assisting engineering students who are officially enrolled.

All course materials will be in English but some lectures will be given in French.

Grade

If you are taking this class for credit, you will be ask to turn in the notebooks of all your practical sessions.
There will also be a written exam.
Total credits: 2 ECTS.

Practical sessions

Practical sessions will take the form of Jupyter notebooks on the course github repo.

Please follow the instructions there to install Python3 and all the relevant packages. An alternative (sometimes preferable for deep learning notebooks) is to use Google Colab, for which you will need a Google account.

TAs: Jesus Bujalance Martin, Joseph Gesnouin, Matthieu Najm.

Textbook

There is no single textbook for this course, but the following resources are relevant:

• Mining of massive datasets by Leskovec, Rajaraman and Ullman;
• Deep learning by Goodfellow, Bengio and Courville;
• Large-Scale Optimization: Beyond Stochastic Gradient Descent and Convexity by Sra and Bach.

This course is not an introductory course to machine learning! If you want to learn the basics, or need a refresher, we recommend:

• In French, the lectures of the Parcours Data Scientist sur OpenClassrooms (vidéos et textes en accès libre);
• In French, Introduction au Machine Learning. Chloé-Agathe Azencott, Collection InfoSup, Dunod, 2022;
• In French, Apprentissage statistique supervisé by Fabien Moutarde in Techniques de l'Ingénieur;
• In English, Machine learning by Andrew Ng on Coursera;
• In English, The elements of statistical learning by Hastie, Tibshirani and Friedman;
• In English, Pattern recognition and machine learning by Bishop.