CS 533
|
|
Date |
Topic |
Reading |
Notes |
|
Jan. 6 |
Classical STRIPS Planning, State-Space Search |
Section 11.1-11.2 |
|
|
Jan. 8 |
Partial-Order Planning (POP) Graphplan |
Section 11.3 (can skip part on unbound
variables) Section 11.4 |
|
|
Jan. 13 |
Planning as Satisfiability |
Section 11.5, 11.6, see 7 for review of propositional logic (in particular 7.6 reviews inference algorithms) |
|
|
Jan. 15 |
cont. SAT-Plan; Heuristic Search Planning |
Planning as Heuristic Search, B. Bonet and H. Geffner, Artificial Intelligence, Vol 129 (1-2), 2001 (you can skip Section 7) |
|
|
Jan. 20 |
cont. HSP; Markov Decision Processes (MDPs) | Sections 17.1-17.3 | |
|
Jan. 22 |
cont. MDPs |
||
|
Jan. 27 |
cont. MDPs; Reinforcement
Learning (RL) |
Sections 21.1-21.2 | PDF PPT |
|
Jan. 29 |
cont. RL |
|
|
|
Feb. 3 |
cont. RL: RL in Large State Spaces | Sections 21.4-21.5 | PDF PPT |
|
Feb. 5 |
cont. Large State Space: TD |
Project Ideas PDF PPT |
|
|
Feb. 10 |
cont. Large State Space: Policy
Gradient Search |
||
|
Feb. 12 |
Midterm |
Material
Covered: STRIPS Planning and MDPs (HW1-HW3) (RL not covered) |
|
|
Feb. 17 |
cont. Policy Gradient, Least Squares
Policy Iteration (LSPI) |
Optional Reading: Least-Squares Policy Iteration, Michail Lagoudakis and Ronald Parr, Accepted to the Journal of Machine Learning Research (JMLR), Vol. 4, 2003, pp. 1107-1149. |
PDF PPT |
|
Feb. 19 |
cont. LSPI |
||
|
Feb. 24 |
cont. LSPI, Simulation-Based
Planning: Evaluation |
PDF PPT | |
|
March 3 |
cont. Simulation-Based Planning:
Rollout, Approximate Policy Iteration |
Approximate Policy Iteration Note | |
|
March 5 |
cont. Simulation-Based Planning:
API, Sparse Sampling |
Optional: A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes. Michael Kearns, Y. Mansour and A. Ng. International Joint Conference on Artificial Intelligence, 1999 | PDF PPT |
| March 10 |
cont. Simulation-Based Planning:
UCT, Distribute Takehome Final |
Optional: Bandit Based
Monte-Carlo Planning. Levente Kocsis & Csaba Szepesvari.
European Conference, on Machine Learning, 2006 |
|
|
March 12 |
The stuff we didn't talk about, Q&A Session |
|
|
|
March 18 |
Final Project Presentations: 2-4 |
|
|
.