Time and Location  M 9:3011:30am, Room 4419 
Personnel 
Prof. Liang Huang (huang @ cs.qc), Instructor Kai Zhao (z.kaayy @ gmail), TA 
Office Hours (tentative) 
LH: M 11:3012pm, CS Lab KZ: M 34pm & F 45pm, CS Lab Additional office hours available before exams. 
Prerequisites 

Textbooks  Unlike other fields of CS, there isn't a standard textbook in machine learning because, (1) the field is developing so fast that classical textbooks become outdated constantly, and (2) most recent textbooks were written from a statistical or engineering perspective (rather than a CS one), which makes it unnecessarily complicated.
In light of this I recommend (for references only; this course is selfcontained):

Grading 

How can we make computers to act without being explicitly programmed and to improve with experience?
In the past decade, machine learning has given us selfdriving cars, practical speech recognition, effective web search, accurate spam filters, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it.
This course will survey the most important algorithms and techniques in the field of machine learning. The treatment of math will be rigorous, but unlike most other machine learning courses which focus on the statistical or engineering perspective with tons of equations, my course will focus on the geometric intuitions and the algorithmic perspective. I will try my best to visualize every concept.
Week  Date  Topics  Homework 
1  Jan 28  Intro to ML Intro to Python and numpy (Kai)  
2  Feb 4  slides for lectures 24: Online Learning: Perceptron and Beyond.
Linear separation; geometric computation of margin.  
3  Feb 11  Perceptron Convergence Proof; geometry vs. algebra Voted and Averaged Perceptron MIRA (geometric derivation)  
President's Day. Class moved to Wednesday.  
4  W Feb 20 
passiveaggressive; paggressive MIRA demo of perceptron vs. aggressive MIRA multiclass decision boundaries (geometry) multiclass perceptron  HW1 out: (avg) perceptron, (aggressive) MIRA, feature preprocessing.
dataset: adult50k. 
5  Feb 25  engineering tips: shuffling,
variable learning rate. feature preprocessing: rescaling/centering, binning, categorical to binary. slides for lectures 56: Kernels and Kernelized Online Learning.
Nonlinear features and feature combination; XOR  
6  Mar 4  kernelized perceptron in primal and dual (Kai) polynomial kernel on examples: XOR, circle.  
7  Mar 11  Gaussian kernel (distorts distance); geometric interpretation on the original and feature spaces; connection to instancebased learning (nearest neighbor)
Kernel properties: Mercer, gram matrix, positive semidefinite (eigenvalue); example of a nonkernel slides for lectures 78: SVM and convex optimization.
SVM optimization in primal (without bias).  HW1 due. 
8  Mar 18  SVM optimization in dual. Hildreth algorithm (Kai). softmargin. kernels; demo.  HW2 out: kernelized perceptron and SVM. dataset: spambase. 
Spring Break
HW2 due on Friday 4/5.  
11  Apr 8  slides for Lectures 910: structured prediction.
From Multiclass to Structured Classification.  
12  Apr 15  Dynamic Programming. Viterbi vs. Dijkstra. Fast Averaging Trick (Daume, 2006). Proof of Convergence. Perceptron with Inexact Search: Early Update, ViolationFixing, LaSO, etc. (Collins and Roark:2004; Daume and Marcu, 2005; Huang et al 2012)  
13  Apr 22  Unsupervised Learning kmeans  project proposal due 4/24. 
14  Apr 29  EM mixture of Gaussians  HW3 out: structured perceptron w/ exact and inexact search. data. 
15  May 6  PCA ICA  
16  May 13 last class  Final Project Midway Presentations  
17  May 20 