Computational Intelligence in Games (Reinforcement Learning and beyond)

Description of the course

Concept of the online course: 

  • The recorded lectures are provided to you via mediasite.
  • The exercises are performed as live sessions via ZOOM.
  • There is a forum on the e-learning plattform of OVGU (moodle).
  • Registration for the exercises via LSF.
  • This webpage will be updated during the course, please regularly check for updates.

 

This course addresses the basic and advanced topics in the area of computational intelligence and games. This course has three parts:

Part one addresses the basics in Evolutionary Game Theory (EGT). In this part you will learn about simple games such as scissors/rock/paper and the main focus on the strategies for playing games.

Part two is about learning agents and we focus on reinforcement learning mechanisms. There are three questions for games:

How can we use the information from a search mechanism to learn? 
How can we use reinforcement learning to find a better strategy?
How can we use reinforcement learning as a search mechanism? 

Part three contains the advanced topics in games and artificial intelligence such as how can we program an agent who can pass a Turing test? how can we consider physical constraints of a spaceship while moving in an unknown terrain? etc. 

This course will be held in English and is for Bachelor (5CP) and Master (6CP: including extra programming assignment) students. 

 

Lectures and Tutorials

 

The lectures will start in an online format (due to the current situation). We will provide you the recorded lectures which you can access from home at anytime.

The tutorials will be done through Zoom. The Moodle page of this course gives you access to the zoom link of your tutorial. We will do regular tutorials every 2 weeks. The tutorials will start on Monday the 19th April at 11:00 for the first group.

There is an overhead group 5, which is only opened if the other groups are overly full. The date of this tutorial is subject to change depending on the needs of the students.

Please register for the tutorials using the LSF of the OvGU. The registration for the tutorials stays open till the 18th April 2021. We will give you access to Moddle based on your LSF registration. You do not need to enroll manually in Moodle!

Lecture Plan

In order to plan the lectures in a structured way, we provide you the following lecture plan. You should learn the material for the corresponding specified week by referring to the online videos and slides:

 Date  Lecture slides  Video (Password: URZ account)
 6 to 9 April  Chapter 0  no video (slides only)
 12 to 16 April  Chapter 1  Link 
 19 to 23 April   Chapter 2 , Part 1, Part 2  Part 1, Part 2
 26 to 29 April  Chapter 2, Part 3, 4 and 5  Part 3, Part 4 and Part 5
 3 to 7 May  Chapter 2 to end and Chapter 3   2 - 6,  2 - 7, 3 - 1 and 3 - 2
10 to 14 May Chapter 3 and Chapter 4 3 - 3, 3 - 4, 4 - 1 and 4 - 2
23 to 28 May Chapter 5 5-1 and 5-2
31 May to 4 June Chapter 6 6-1 and 6-2
07 to 10 June Chapter 7 7-1 and 7-2
13 to 18 June Chapter 8 8-1 and 8-2
20 to 25 June Chapter 8 and Chapter 9 8-3 and 9

 

Slides

To open the slides, you need to use a password which is specified in the recorded lecture "Organization for SS2020" (access information above).

We compiled a List of Equations and Algorithm containing important symbols, equations and algorithms. We hope this overview helps you during your study of the course topics. Please feel free to send pull requests and help us to incorporate changes and additions throughout the semester.

Password for the slides are announced in the tutorials. 

Chapter  Slide
0 Organisation
1 Introduction
2 Evolutionary Game Theory
3 Introduction to Reinforcement Learning
4 Dynamic Programming and Monte Carlo Method in RL
5 Temporal Difference Learning
6 Monte Carlo Tree Search
7 Rolling Horizon Evolutionary Algorithms
  Evolutionary Algorithms (Detailed slides from the course Evolutionary Multi-Objective Optimization(EMO)
8 Multi-Objective Decision Making and Learning in Games
9 Procedural Content Generation (PCG)

Tutorials

 Tutorials are bi-weekly done starting on the 19th April 2021. The tutorials will be done on the following dates:

  • Monday 11:00 - 13:00 even weeks Group 1 starting from 19.04.2021
  • Monday 11:00 - 13:00 odd weeks Group 2 starting from 26.04.2021
  • Friday  11:00 - 13:00 even weeks Group 3 starting from 23.04.2021
  • Friday  11:00 - 13:00 odd weeks Group 4 starting from 30.04.2021
  • Monday 11:00 - 13:00 even weeks Group 5 additionally if other groups are overly full. Date subject to change

 

 Videos and simulations related to lectures

 

 

Conditions for Certificates (Scheine) and Exams

 

Certificate (Übungsschein):There are assignment sheets published about every two weeks. You need to hand in your solutions to the assignments before the next tutorial using Moodle. If you handed in a solution you may be asked to present your solution in the tutorials (using video conferencing). The solutions need not necessarily be completely correct. However, it should become obvious that you treated the assignment thoroughly. You are granted the certificate (Schein), if (and only if) you

  • handed in at least two thirds of the assignments,
  • solve the programming assignment (this task can be solved alone or in pairs of two), and
  • pass the exam

Programming Assignment Bachelor: 

Conceptualize and implement an AI for a provided game.  The AI needs to be able to play the game and win against an easy bot provided by the lecturers.

Programming Assignment Master: 

Conceptualize and implement a n AI for a provided game. The AI needs to be able to play the game and win against an easy and a medium bot provided by the lecturers.

Both programming assignments will be handled through Moodle. You also will find additional info there.

Exam: If you intend to finish the course with an exam, your are required to meet the certificate conditions. There will be a written exam after the course.

 

Exercise Sheets Assignments:

For every tutorial a new Exercise Sheet will be uploaded, which is to be solved by the students before their next tutorial. To acquire the Exercise Sheets, upload your solutions and get access to your tutorial group visit the Moodle page of this course.

Interesting Links and further Material

 

Programming Assignment:

The framework and rules of the programming assignment is to be announced till 12th April 2021.

 Past Exam:

 Exam WS 2015/2016

 

Literature

 

  • Yannakakis, Georgios N., and Julian Togelius. Artificial Intelligence and Games. Springer, 2018. --> Link
  • Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA, 1998 --> Link
  • Nowak, Martin, Evolutionary dynamics : exploring the equations of life, Cambridge, Mass. [u.a.] : Belknap Press of Harvard Univ. Press , 2006 --> Link to OvGU Library
  • Ian Millington and John Funge, Artificial Intelligence for Games, CRC Press, 2009
  • T. L. Vincent and J. L. Brown, Evolutionary Game Theory, Natural Selection and Darwinian Dynamics, Cambridge University Press, 2012
  • Jorgen W. Weibull, Evolutionary Game Theory, MIT Press, 1997
  • Thomas Vincent, Evolutionary Game Theory, Natural Selection, and Darwinian Dynamics, Cambridge University Press, 2005
  • Josef Hofbauer, Karl Sigmund, Evolutionary Games and Population Dynamics, Cambridge University Press, 1998
  • Kalyanmoy Deb, Multi-Objective Optimization using Evolutionary Algorithms, Wiley, 2001
  • Literature about PCG: Paper1, Paper2, Paper3, Paper4
  • Kruse, Borgelt, Klawonn, Moewes, Ruß, Steinbrecher, Computational Intelligence, Vieweg+Teubner, Wiesbaden, 2011
  • Ines Gerdes, Frank Klawonn, Rudolf Kruse, Evolutionäre Algorithmen, Vieweg, Wiesbaden, 2004
  • Zbigniew Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin, 1998

Last Modification: 17.01.2024 - Contact Person: Webmaster