Control Theory and Reinforcement Learning Spring School

Programme

17 - 21 March 2025

Turing Hall, CWI, Science Park, Amsterdam

Control Theory and Reinforcement Learning: Connections and Challenges

Control theory and reinforcement learning (RL) converge on a shared objective: facilitating autonomous, real-time decision-making to optimize dynamical processes. Historically, these disciplines have diverged in assumptions regarding available prior information and in analytical techniques applied. However, recent advances bridging the two domains are fostering collaborations. The CWI research semester programme in spring 2025, themed "Control Theory and Reinforcement Learning: Connections and Challenges", comprises a spring school and workshops on various sub-topics to explore the intersections and challenges within these intertwined fields.

Spring School

This spring school emphasizes connections across control theory, reinforcement learning and stochastic approximation, enabling students to access these broader themes and start to work on cross-cutting projects. The school will be at a preparatory PhD level, suitable for advanced Master's and starting PhD students in these areas.

Programme

17 March	18 March	19 March	20 March	21 March
Registration: 08:30 - 09:00 / Welcome by Prof Ton de Kok 09:00 - 09:15	Walk-in coffee/tea 08:30 - 09:00	Walk-in coffee/tea 08:30 - 09:00	Walk-in coffee/tea 08:30 - 09:00	Walk-in coffee/tea 08:30 - 09:00
Deterministic, discrete-time & continuous-time control [lecture] Bert Kappen 09:15 - 10:45 Material video	Generalized policy iteration, planning, partial observability [lecture] Frans Oliehoek 09:00 - 10:30 Slides video	Stochastic approximation [lecture] Sean Meyn 09:00 - 10:30 Slides video	TD-learning & actor- critic methods [lecture] Sean Meyn 09:00 - 10:30 Slides video	Multi-agent games: Linear Quadratic games [lecture] Maryam Kamgarpour 09:00 - 10:30 Slides video
Break 10:45 - 11:15	Break 10:30 - 11:00	Break 10:30 - 11:00	Break 10:30 - 11:00	Break 10:30 - 11:00
Deterministic, discrete-time & continuous-time control [tutorial] Bert Kappen & TAs 11:15 - 13:00 Material	Generalized policy iteration, planning, partial observability [tutorial] Frans Oliehoek & TAs 11:00 - 12:45 colab	Stochastic approximation [tutorial] Sean Meyn & TAs 11:00 - 12:45 jupyter [Q&As]	Q&A on tutorials & lectures [tutorial] Lecturers & TAs 11:00 - 12:45	Multi-agent RL / decentralized control [lecture] Frans Oliehoek 11:00 - 12:30 Slides video
Lunch Break & Posters 13:00 - 14:00	Lunch Break & Posters 12:45 - 14:00	Lunch Break & Posters 12:45 - 14:00	Lunch Break & Posters 12:45 - 14:00	Lunch Break & Posters 12:30 - 13:30
Stochastic optimal control [lecture] Bert Kappen 14:00 - 15:30 Material video	Actors and Critics: Function Approx. & Policy Gradients in RL [lecture] Debabrota Basu 14:00 - 15:30 Slides video	Q-learning (advanced) [lecture] Sean Meyn 14:00 - 15:30 Slides video	Optimism (UCRL, PSRL, randomized VI) [lecture] Debabrota Basu 14:00 - 15:30 Slides video	Panel Discussion on Open Questions Sean Meyn, Maryam Kamgarpour, Frans Oliehoek, Debabrota Basu, Aditya Gilra & TAs 13:30 - 15:00 video
Break 15:30 - 16:00	Break 15:30 - 16:00	Break 15:30 - 16:00	Break 15:30 - 16:00
Stochastic optimal control [tutorial] Bert Kappen & TAs 16:00 - 17:45 Material	Actors and Critics: Function Approx. & Policy Gradients in RL [tutorial] Debabrota Basu & TAs 16:00 - 17:45 github [Q&As]	Q-learning (advanced) [tutorial] Sean Meyn & TAs 16:00 - 17:45 jupyter [Q&As]	Safety in RL & control - state space approach [lecture] Maryam Kamgarpour 16:00 - 17:30 Slides video
Social 17:45 - 18:30	Games evening 17:45 - 20:00	Dinner 18:30 - 20:30

Teaching Assistants (TAs): Caio Kalil Lauand, Guillaume Pourcel, Mustafa Mert Celikok, Rea Nkhumise.