CS 534, Machine Learning, Fall 2017

“Equations are just the boring part of mathematics. I attempt to see things in terms of geometry.”
-- Stephen Hawking (b. 1942)

Coordinates T/Th 12-1:20pm, WNGR 149 [Registrar] [Canvas]
Instructor Liang Huang
TAs Dezhong Deng
Yilin Yang
Office Hours
(tentative)		 LH: M 9:30-10am, 5-5:30pm, KEC 2069
TAs: Dezhong T/Th 10-11am, Yilin W/F 4-5pm, both at KEC Atrium.
Additional office hours available before exams.
Recitations Tue 10/3 @ 7:40pm (linear algebra and geometry)
Thu 10/5 @ 7pm (probs/stats, python/numpy), both at KEC 1001.
Prerequisites
  • CS: algorithms and datastructures. fluent in at least one mainstream languages (Python, C/C++, Java).
    HWs will be done in Python+numpy only.
  • Math: linear algebra, calculus, and basic probability theory. good sense of geometric intuitions.

Textbooks
  • Hal Daume III. A Course in Machine Learning (CIML). default reference. easy to understand.
  • Tom Mitchell (1994). Machine Learning. a classical textbook. CS perspective. an easy read. outdated but still more helpful than most recent ones.
  • Mohri et al (2012). Foundations of Machine Learning. theory perspective. covers more recent advances such as SVMs that weren't covered in Mitchell.
  • Bishop (2007). Pattern Recognition and Machine Learning (PRML). Actually I do not recommend it, definitely not for beginners. But the figures are pretty and I use them in my slides.
Grading
  • Midterm: 25%. NO FINAL EXAM.
  • Project (groups of up to 3): 25%. No late submission is allowed. (5% proposal, 5% presentation, 15% report).
  • HWs (programming, groups of up to 3): 10% x 3 = 30%.
  • EXs (theoretical, individual): 4% x 2 = 8%.
  • Class Participation: 4%.
  • Quiz (tentatively before thanksgiving): 8%.
  • Late Penalty: Each student can be late by 24 hours only once without penalty. No more late submissions will be accepted. If a group submission is late, it is considered late for all teammates.
    E.g., if a team of A and B submits late and it's the first late submission from A and the second from B, then A will receive credit for this submission but B will not.
Machine Learning evolves around the following central question: How can we make computers to act without being explicitly programmed and to improve with experience? In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, accurate spam filters, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it.
This course will survey the most important algorithms and techniques in the field of machine learning. The treatment of Math will be rigorous, but unlike most other machine learning courses which feature tons of equations, my version will focus on the geometric intuitions and the algorithmic perspective. I will try my best to visualize every concept.
The overall structure is quite similar to previous offerings by Prof. Xiaoli Fern (Spring 2012, Spring 2013, Spring 2014, Fall 2014, Fall 2015, Fall 2016). Some new aspects in this offering include: You can study the exams from previous offerings, but do not copy HW solutions (since we have different HWs).

See also my previous offering of this course at CUNY.

Some preparatory materials:
Tentative Schedule: 
week	topic		HW/EX due
0	intro
1	perceptron	
2	perc, mira   	ex1 (perceptron theory)
3	SVM, KKT	hw1 (perceptron, logistic)
4	SVM dual        ex2 (SVM/KKT theory)
5	kernels, k-NN	hw2 (SVM, pegasos, kernel)
6	midterm		
7	struct predict	project_proposal
8	kmeans, EM	HW3 (struct perc)
9	PCA/quiz 
10	proj_present	
11	                project_report
Slides, Assignments, and Extra Readings:
Classical Papers:
Tech Giants Are Paying Huge Salaries for Scarce A.I. Talent (New York Times)
Liang Huang