CS430/530 Midterm Study Guide

The midterm will be open books, open notes. It will cover all of the material that we have covered in class or on the homework and programming problems. We have covered most of the material in the following book sections:

1.1, 1.4, 2(all), 3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 4.1, 4.2, 4.4, 6.1, 6.2, 7.5, 7.6, 7.7, 7.8


Here is an outline of the most important points in the course so far.

1. Definitions of different kinds of agents:
  Dimensions along which to classify agents:

  * memoryless vs. using memory
    - memoryless: reflex agents
    - memory: track state, remember results of previous decisionmaking

    memoryless agents cannot be rational in many environments

  * fixed goal/tradeoff vs. accepting goal or utility function as input
    An agent with a fixed goal (or a fixed tradeoff) must be re-built
    each time the goal (or utility function) is changed.

  * implementation-methods
    - table
    - rules
    - search
    - logic
    - probability

2. Definitions of different kinds of environments
  The main two distinctions:
  * accessible vs. inaccessible
  * deterministic vs. stochastic

  Other distinctions that we haven't really worried about
  * episodic vs. non-episodic 
  * static vs. dynamic (only static so far)
  * discrete vs. continuous (mostly only discrete so far)

3. Key functions that must be implemented in a general agent:

  * mapping from percepts and current beliefs about the state of the
    environment to new beliefs about the state of the environment.
    [model of its sensors]

    percept + Bel(S) --> Bel(S')

  * mapping from state and action to beliefs about new states
    [model of actions]

    S + A --> Bel(S)

  * method for determining the quality of a state:  whether it
    is a goal state; its utility.
    [goal predicate or utility function]

    U(S)  or Goal?(S)

4. Implementing agents using Search.
  In deterministic environments, search methods can be applied to
  construct sequences of actions if the agent has a model of actions.
  In principle, multistate search problems can handle cases where the
  starting state is not known.  But in practice, these methods don't
  scale, so search-based agents work best in environments where the
  starting state is known and every action is deterministic so that a
  single-state formulation can be used. 

  * evaluating search methods
    - completeness
    - optimality
    - time complexity
    - space complexity

  * Uninformed search methods
    - breadth-first  complete, optimal (in depth), O(B^D), O(B^D)
    - uniform-cost   complete, optimal (in cost), O(B^D), O(B^D)
    - depth-first    in general spaces: incomplete, non-optimal, infinite, infinite
                     in spaces with no infinite descending paths:
                      complete, non-optimal, O(B^M), O(BM)
                     where M = deepest path in tree
    - iterative deepening
                     complete, optimal (in depth), O(B^D), O(BD)
    
    [530:  CSP problems: definition
    - depth-first search with backtracking
    - forward checking
    - arc consistency
    ]

  * Informed search methods
    - Greedy 
    - A*
    - IDA*
    An admissible heuristic must never overestimate the true cost to
    the goal.  Iterative deepening and IDA* are generally the
    preferred methods, because they make better use of available memory.

  * Iterative Improvement methods
    - Hill climbing
    - Simulated annealing

    Iterative improvement methods can be very fast if they are given a
    good starting point.  They can get stuck in local optima.

  * Drawbacks of search-based agents:  Non-determinism; inaccessible
    environments 
  
5. Implementing agents using logic
  Key ideas:
  * Use the power of disjunction and existential quantification to
    compactly represent uncertainty about the environment.
  * Use logical inference to implement the functions of the agent
  * Combine with search to create general-purpose goal-based agents.

  * Drawbacks of logic-based agents:  Non-deterministic environments,
    tradeoffs. 

Skills you should be able to demonstrate during the exam:

1. Be able to determine the best score that a pure reflex agent can
obtain on a task.
2. Be able to hand-simulate any of the search algorithms.
3. Be able to decide whether a heuristic is admissible for A*.
4. Be able to apply the rules of logical inference presented in
class.