Homework 1 CS539

Work the following problems in the textbook: 3.9, 3.10, 3.11, 3.12, 3.13, 4.1, 4.5. For 3.12 and 3.13, you don't need to use expected value notation unless you want to.

We will discuss the problems for Chapter 3 on Monday, January 10. We will discuss the problems from Chapter 4 on Wednesday, January 12.

Study Example 4.2 (p. 98) and write down a formal description of the states, actions, and reward functions for this problem. What are the states? How many states are there? What are the actions? How many of them are there? What are the effects of the actions? Formally describe the probability transition function. What are the rewards?