Stefan Lee
1/2021 - Serving as an area chair for CVPR 2021.
2/2020 - Serving as an area chair for ICML and NeurIPS 2020.
2/2020 - One paper accepted to CVPR-2020 on multi-task learning for vision-and-language.
12/2019 - Recordings from our Visually Grounded Interaction and Language (VIGIL) workshop @ NeurIPS 2019 are online! [1 2 3 4]
12/2019 - New arXiv paper on multi-task learning in vision-and-langage [ PDF ]. One model doing 12 tasks at state-of-the-art.
10/2019 - Recieved two LoCoBots from a Facebook AI Research grant .
I'm an Assistant Professor in the School of Electrical Engineering and Computer Science (EECS) at Oregon State University. My research focus is the development of agents that can perceive their environment and communicate about this understanding with humans in order to coordinate their actions to achieve mutual goals -- in short, agents that can see, talk, and act. Consequentially, I work on problems in computer vision, natural language processing, and deep learning in general.

Want to work with me? I've just started at OSU and I'm looking for enthusiastic PhD students who are interested in the intersection of vision, language, and action!


Assistant Professor @ Oregon State - School of Electrical Engineering and Computer Science
Research Scientist II @ Georgia Tech - School of Interactive Computing
Postdoctoral Associate @ Virginia Tech - Bradley Department of Electrical and Computer Engineering
Earned PhD @ Indiana University - School of Informatics & Computing
Recent Press Coverage
- Coverage of embodied AI work at Facebook including our Semantic MapNet effort - ZDNET MIT Technology Review Digital Trends VentureBeat 2020
- Coverage of our Vision-and-Language Navigation in Continuous Environments project - ZDNET 2020
- Facebook, Georgia Tech & OSU ‘ViLBERT’ Achieves SOTA on Vision-and-Language Tasks - SyncedReview 2019
- Meet ViLBERT, The Task-Agnostic Model Inspired From BERT for Visual Grounding - Analytics India 2019
- Facebook helped create an AI scavenger hunt that could lead to the first useful home robots - MIT Technology Review 2018
- How A Virtual Scavenger Hunt Could Train Robots To Find Things In Your Home - FastCompany 2018
- Facebook is training AI to answer questions like humans do -- Digital Journal 2018
- Research Scientist, Assistant Professor Represent IC in DARPA Risers Event - ML@GT Blog 2018
- What is Graph R-CNN? - ML@GT Blog 2018
- Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance - ML@GT Blog 2018
- Embodied Question Answering - ML@GT Blog 2018
Oregon State
CS539 - Natural Language Processing with Deep Learning (Winter 2021)

Georgia Tech
CS8903 - Special Problems (Fall 2017 - Spring 2019)
(Faculty Advisor)

Virginia Tech
EECE 5424/4425 CS 5824/4824 - Introduction to Machine Learning (Fall 2016)

Indiana University
B659 - Image Processing and Recognition (Fall 2014)
(Assistant Instructor)
I399 - Research Methods for Informatics and Computing (Fall 2013)
(Graduate Mentor)
C211 - Introduction to Computer Science (Fall 2011 - Summer 2012)
(Assistant Instructor)
Recent Talks
Watch online:

Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition

Conference on Robotic Learning (CoRL) 2018

Training Embodied Agents in Semantically and Perceptually Rich Simulations

DARPA 60th Anniversary (D60) - DARPA Riser 2018

- Visual Curiosity: Learning to Ask Questions to Learn Visual Recognition
- Training Embodied Agents in Semantically and Perceptually Rich Simulations
- Towards Goal-Driven, Visually Grounded Dialog Agents
- Training Diverse Ensembles of Deep Networks