Programming Assignment 1: Due January 19, 2000

Update: To use the priority queue code, you need the files tree.h, tree.c, queue.h and stack.h. Sorry about that!

In this assignment, you will implement and experiment with value iteration and prioritized sweeping for the "Jack's Car Rental" problem (p. 99 of Sutton and Barto). We will modify the problem statement slightly so that it is easier to complete.

This problem can be defined as follows:

I have implemented this domain in C++, and I have also implemented Value Iteration. In the assignment, your task is to implement prioritized sweeping and compare its performance with Value Iteration.

The code is organized in four files:

Your code should implement prioritized sweeping. As with my Value Iteration code, it should count the number of primitive Q backups and display this number when it terminates.

All of the code that you need for this problem is available in the following tar file. It has been tested under Solaris and on my laptop. I am using modified versions of Tim Budd's library routines for data structures, rather than STL (sorry!). If anyone wants to port this program to STL, that would be great!

Please turn in a hardcopy listing of your program and a comparison of prioritized sweeping and value iteration on this program.