Stefan Lee
News
2/2020 - Serving as area chair for ICML and NeurIPS 2020.
2/2020 - One paper accepted to CVPR-2020 on multi-task learning for vision-and-language.
12/2019 - Recordings from our Visually Grounded Interaction and Language (VIGIL) workshop @ NeurIPS 2019 are online! [1 2 3 4]
12/2019 - New arXiv paper on multi-task learning in vision-and-langage [ PDF ]. One model doing 12 tasks at state-of-the-art.
10/2019 - Recieved two LoCoBots from a Facebook AI Research grant .
About
I'm an Assistant Professor in the School of Electrical Engineering and Computer Science (EECS) at Oregon State University. My research focus is the development of agents that can perceive their environment and communicate about this understanding with humans in order to coordinate their actions to achieve mutual goals -- in short, agents that can see, talk, and act. Consequentially, I work on problems in computer vision, natural language processing, and deep learning in general.

Want to work with me? I've just started at OSU and I'm looking for enthusiastic PhD students who are interested in the intersection of vision, language, and action!

Career

2019
Assistant Professor @ Oregon State - School of Electrical Engineering and Computer Science
2017
Research Scientist II @ Georgia Tech - School of Interactive Computing
2016
Postdoctoral Associate @ Virginia Tech - Bradley Department of Electrical and Computer Engineering
2016
Earned PhD @ Indiana University - School of Informatics & Computing
Recent Press Coverage
- Facebook, Georgia Tech & OSU ‘ViLBERT’ Achieves SOTA on Vision-and-Language Tasks - SyncedReview 2019
- Meet ViLBERT, The Task-Agnostic Model Inspired From BERT for Visual Grounding - Analytics India 2019
- Facebook helped create an AI scavenger hunt that could lead to the first useful home robots - MIT Technology Review 2018
- How A Virtual Scavenger Hunt Could Train Robots To Find Things In Your Home - FastCompany 2018
- Facebook is training AI to answer questions like humans do -- Digital Journal 2018
- Research Scientist, Assistant Professor Represent IC in DARPA Risers Event - ML@GT Blog 2018
- What is Graph R-CNN? - ML@GT Blog 2018
- Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance - ML@GT Blog 2018
- Embodied Question Answering - ML@GT Blog 2018
Teaching
Oregon State
Georgia Tech
CS8903 - Special Problems (Fall 2017 - Spring 2019)
(Faculty Advisor)

Virginia Tech
EECE 5424/4425 CS 5824/4824 - Introduction to Machine Learning (Fall 2016)
(Instructor)

Indiana University
B659 - Image Processing and Recognition (Fall 2014)
(Assistant Instructor)
I399 - Research Methods for Informatics and Computing (Fall 2013)
(Graduate Mentor)
C211 - Introduction to Computer Science (Fall 2011 - Summer 2012)
(Assistant Instructor)
Publications
Recent Talks
Watch online:

Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition

Conference on Robotic Learning (CoRL) 2018

Training Embodied Agents in Semantically and Perceptually Rich Simulations

DARPA 60th Anniversary (D60) - DARPA Riser 2018


Slides:
- Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition
- Training Embodied Agents in Semantically and Perceptually Rich Simulations
- Towards Goal-Driven, Visually Grounded Dialog Agents
- Training Diverse Ensembles of Deep Networks