CS539: Probabilistic Agents

Course Description

This course will study how to construct an intelligent agent based on probabilistic reasoning. We will begin by studying reinforcement learning including both Markov decision problems (MDPs) and partially-observable Markov decision problems (POMDPs). Then we will consider several reinforcement learning algorithms including prioritized sweeping, TD(lambda), Q learning, and direct policy search. The need to understand probabilistic models of POMDPs will lead us to study belief networks (bayesian networks), algorithms for reasoning in belief networks, and special kinds of networks, particularly Hidden Markov Models (HMMs). We will study the forward-backward algorithm for reasoning in HMMs as well as Monte Carlo methods. To learn HMMs, we will first study the EM algorithm for simple, "naive bayes" networks. Then we will apply EM to learn HMMs. After taking some time to study applications of HMMs in biology, speech recognition, and robotics, we will combine HMMs with reinforcement learning to construct complete probabilistic agents. The coursework will consist primarily of reading and programming assignments along with midterm and final exams.

Prerequisites: CS530 or consent of the instructor; basic knowledge of probability

Registration Information: 4 Units. MWF 9:00-9:50 Rogers 332 CRN 25321.

Course Handouts

Viewgraphs for Lectures

Homework Assignments

Programming Assignments

Exam Solutions

Tom Dietterich, tgd@cs.orst.edu