Dr. Fuxin Li


2077 Kelley Engineering Center

Oregon State University

Corvallis 97331, Oregon


(+1) 541-737-5987 (office)


lif (a) eecs.oregonstate.edu

Google Scholar Profile



I am an assistant professor in the School of Electrical Engineering and Computer Science at the Oregon State University. My research direction is machine learning and computer vision, with a major interest in using and designing new machine learning algorithms to attack the structural data in images and videos, especially big data originating from videos.

Before joining OSU, I spent 4.5 years in Georgia Tech, first as a postdoc researcher supervised by Dr. Guy Lebanon, then as a research scientist working with Dr. James M. Rehg. I have worked on problems in natural language processing, recommender systems, as well as many computer vision problems such as image and video segmentation, object recognition, multi-target tracking, semantic reconstruction, etc.

Before Georgia Tech, I was working as a research scientist in the Sminchisescu group, INS, University of Bonn from 2008 to 2010. In Bonn I worked on machine learning and computer vision, looking to develop and use learning methods properly to solve conceptual and practical problems. Notably, I'm a member of the BONN-SVRSEGM team which participated in the PASCAL VOC Segmentation Challenge and (co-)won the 2009-2012 challenges.

Even before that, I got my bachelor degree on 2001 in Zhejiang University, and my Ph.D. degree on 2008 in the Institute of Automation, Chinese Academy of Sciences, with a dissertation on Euclidean Metric Learning. Besides machine learning, I have also done some proteomics algorithms and software during my Ph.D., collaborating with Professor Youhe Gao from Chinese Academy of Medical Sciences.

Research Interests (Very outdated!)

I am broadly interested in many machine learning algorithms and applications. Most recently, I have been working more on computer vision, especially segmentation-based object recognition and scene understanding. I have recently developed a significant interest in video. Video can provide spatio-temporal cues that should enable many interesting learning algorithms to work with minimal supervision. A first step is to segment the spatio-temporal objects from video. I have been working on this problem to generate efficient proposals for those objects, utilizing machine learning tools to train supervised appearance models without any human supervision. Check out my ICCV 2013 and CVPR 2015 papers for more details.

Previously, I have been working for several years on the semantic image segmentation problem. Collaborating with Joao Carreira and Cristian Sminchisescu, our object segmentation/recognition system won the PASCAL VOC 2009-2012 Segmentation Challenge (with Adrian Ion as well in 2012). I mainly work in the recognition part of the system, on how to correctly classify and generate the final segmentation from a pool of initial figure-ground segmentations. The composite statistical modeling framework I proposed with colleagues is supposed to unseat pairwise CRF approaches for higher-order inference in fully-connected graphical models.

Theoretically, I am more interested in the optimization part of machine learning, especially to unify optimization and generalization in machine learning. Some of my previous works focused on learning kernels and metrics, either from a kernel-matrix learning perspective or from the perspective of nonlinear feature selection inside a kernel. They are published in AISTATS 2009 and 2010. In particular, the trust-region inexact Newton method from our 2009 AISTATS paper was the fastest algorithm for a long while to learn a full-rank positive-definite kernel matrix.

On a larger scale, I have the drive to create the real learning machine that has more and more capabilities, that will in the end surpass human intelligence and engage in the battle with genetically heavily modified human-being in the future. But I also believe that Rome is not built in one day. Therefore, I'm also interested in alternative but practical learning paradigms. I had a big interest in semi-supervised learning, but that faded over time. I'm still quite interested in active learning, multiple-instance learning and other forms of weakly-supervised learning. In NIPS 2010 I published a paper on a convex formulation of multiple-instance learning.

Publishing under the name of Li Fuxin starting from 2019

This is a Chinese name, where Fuxin (pronounced FOO-SHEEN) is the first name and Li (pronounced LEE) is the last name.

AI/Robotics in OSU

I am a proud member of the new CoRIS institute in Oregon State University. We are a large and very strong research group in artificial intelligence, machine learning, computer vision, natural language processing and robotics. OSU is ranked 43th overall in the USA by the most up-to-date Computer Science Rankings and 22nd in AI/ML/CV/NLP. At least one ranking puts our robotics program as high as #4 in the USA. If you are using USNews rankings to guide your grad school search, take a read at this article.

Selected Recent Research Highlights:  

Apr. 2019 Code for PointConv is available on the Project Page.

Jan. 2019 We proposed HyperGAN, a generative adversarial network that can directly generate diverse, fully-trained neural networks (all the weights) from a noise vector!

Nov. 2018 We proposed PointConv, which is a complete, scalable CNN (not depthwise or other approximations) on a point cloud. We have achieved the best result on the ScanNet benchmark for a network based only on a 3D point cloud, without volumetric networks. We have also been able to achieve identical performance to CNNs on CIFAR-10 when representing the image as a point cloud.

Sep. 2018 We proposed the bilinear LSTM, which reformed the LSTM formulation according to intuitions derived from a low-rank approximation of the classical recursive least squares. This has significantly improved the capability of training a multi-target tracking network using an LSTM.

Mar. 2018 Fuxin received an NSF CAREER award.

Jul. 2017 Our GaTech-Oregon State team has obtained the 4th place in the DAVIS Video Segmentation Challenge 2017.

Jul. 2017 Our team (MHT-DAM) has obtained the best overall ranking (Avg Rank) in the Multi-Target Tracking challenge (MOT) 2017 .

Sep. 2015 Two papers accepted in ICCV 2015. We beat the state-of-the-art in both object proposals generation and multi-target tracking.

Mar. 2015 An improvement of the video segmentation framework to handle complete occlusions is accepted to CVPR 2015. Congratulations Zhengyang for the impressive achievement to publish a first-authored paper in CVPR as a junior undergraduate student! Check out the Project Website

Jun. 2014 We proposed RIGOR, an algorithm that achieves accuracy slightly better to CPMC/Object proposals, but significantly faster, generating all object proposals in 2-4 seconds on CPU-only code. Check out the Project Page for more information and downloading the code.

Oct. 2013 We exploited semantic segmentation techniques to perform unsupervised video segmentation based on tracking image segments per-frame with 1) automatic object discovery based on segment pool; 2) efficiently learnt appearance models for hundreds of objects with strong appearance features; 3) CSI refinement of objects. Check out Project Webpage, SegTrack v2 Dataset , ICCV paper.

Jun. 2013 We proposed the Composite Statistical Inference (CSI) for inference on real-valued statistics obtained on multiple high-order overlapping variable subsets, with applications in semantic segmentation. Short version, Long version .

Apr. 2013 We proposed an analytic approximation to the chi-square kernel with geometric convergence and derived from elementary methods. It's more straightforward and converges faster over previous approaches. Preprint

Group :


Zhongang Qi

Ph.D. students:

Jialin Yuan

Xingyi Li (co-advised with Xiaoli Fern)

Lawrence Neal (co-advised with Xiaoli Fern)

Robert DeBortoli (co-advised with Geoffrey Hollinger)

Neale Ratzlaff

Wenxuan Wu

Hung Nguyen

Saeed Khorram

M.S. students:

Zehuan Chen


Xinyao Wang (M.S. Graduate Dec. 2018, JD Digits)

Rahul Sawhney (Ph.D. Graduate April. 2018, Georgia Tech Postdoc)

Alrik Firl (M.S. Graduate Jul. 2018, General Electrics)

Xin Li (M.S. Graduate Sept. 2016, NVIDIA)

Zheng Zhou (M.S. Graduate Jun. 2017, Tencent AI Lab)


Winter: CS 535 Deep Learning

Winter: CS 261 Data Structures

Spring: CS 637 Computer Vision II



Neale Ratzlaff, LI Fuxin. Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks . arXiv:1804.01635.

Machine Learning and Computer Vision

Neale Ratzlaff, LI Fuxin. HyperGAN: A Generative Model for Diverse, Performant Neural Networks. ICML, 2019.

Wenxuan Wu, Zhongang Qi, LI Fuxin. PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR, 2019. Project Page

Chanho Kim, Fuxin Li, James Rehg. Multi-object Tracking with Neural Gating Using Bilinear LSTM. ECCV, 2018.

Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, Fuxin Li. Open Set Learning with Counterfactual Images. ECCV, 2018.

R. Sawhney, F. Li, H. Christensen, C. Isbell. Purely Geometric Scene Association and Retrieval - A case for macro-scale geometry. Accepted to ICRA, 2018.

R. DeBortoli, A. Nicolai, F. Li, G. Hollinger. Realtime Underwater 3D Reconstruction Using Global Context and Active Labeling. Proceedings of the International Conference on Robotics and Automation (ICRA) 2018.

Zhongang Qi, Fuxin Li. Embedding Deep Networks into Visual Explanations. arXiv:1709.05360. Short (4 pages) version published in NIPS 2017 Workshop on Interpreting, Explaining and Visualizing Deep Learning - Now what?. Code/Data

Peng Lei, Fuxin Li, Sinisa Todorovic. Boundary Flow: A Siamese Network that Predicts Boundary Motion without Training on Motion . CVPR 2018.

Xin Li, Fuxin Li. Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics . ICCV 2017.

Juan Liu, Zhengyang Wu, Fuxin Li. Ranking Video Segments with LSTM and Determinantal Point Processes. ICIP 2017.

Xingyi Li, Fuxin Li, Xiaoli Fern, Raviv Raich. Filter Shaping for Convolutional Networks. ICLR 2017.

Zhaoyang Lv, Chris Beall, Pablo F. Alcantarilla, Fuxin Li, Zsolt Kira, Frank Dellaert. A Continuous Optimization Approach for Efficient and Accurate Scene Flow. ECCV 2016.

Yu-Ying Liu, Shuang Li, Fuxin Li, Le Song, James M. Rehg. Efficient Learning of Continuous-Time Hidden Markov Models for Disease Progression. NIPS 2015.

Chanho Kim, Fuxin Li, Arridhana Ciptadi, James M. Rehg. Multiple Hypothesis Tracking Revisited. ICCV 2015 (Oral Presentation).

Ahmad Humayun, Fuxin Li, James M. Rehg. The Middle Child Problem: Revisiting Parametric Min-cut for Robust Object Proposals. ICCV 2015.

Zhengyang Wu, Fuxin Li, Rahul Sukthankar, James M. Rehg. Robust Video Segment Proposals with Painless Occlusion Handling. CVPR 2015. Project Website

Rahul Sawhney, Fuxin Li, Henrik Christensen. GASP: Geometric Association with Surface Patches. 3DV 2014.

Abhijit Kundu, Yin Li, Frank Dellaert, Fuxin Li, James M. Rehg. Joint Semantic Segmentation and 3D Reconstruction from Monocular Video . ECCV 2014.

Ahmad Humayun, Fuxin Li, James M. Rehg. RIGOR: Reusing Inference in Graph Cuts for Generating Object Regions. CVPR 2014. Project Page + Code

Fuxin Li, Taeyoung Kim, Ahmad Humayun, David Tsai, James M. Rehg. Video Segmentation by Tracking Many Figure-Ground Segments. In IEEE International Conference on Computer Vision (ICCV), 2013.
Project Webpage, SegTrack v2 Dataset
(There were a few notation typos around Eq. (7) in the official IEEE version, please use the version on this website).

Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning Contact Locations for Pushing and Orienting Unknown Objects . In IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2013.

Fuxin Li, Guy Lebanon, Christian Sminchisescu. A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG]

Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning Stable Pushing Locations. In IEEE International Conference on Development and Learning (ICDL), 2013.

Fuxin Li, Joao Carreira, Guy Lebanon, Cristian Sminchisescu. Composite Statistical Inference for Semantic Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013. Technical Report

Seungyeon Kim, Fuxin Li, Guy Lebanon, Irfan Essa. Beyond Sentiment: The Manifold of Human Emotions. In Proceedings of the 16th International Conference on Artificial Intelligence and Statistics (AISTATS), 2013.

Mingxuan Sun, Fuxin Li, Joonseok Lee, Ke Zhou, Guy Lebanon, Hongyuan Zha. Learning Multiple-Question Decision Trees for Cold-Start Recommendation. In ACM International Conference on Web Search and Data Mining (WSDM), 2013 (Spotlight presentation).

Eduards G. Bazavan, Fuxin Li, Cristian Sminchisescu. Fourier Kernel Learning. In European Conference on Computer Vision (ECCV), 2012 (Oral presentation).

Fuxin Li, Guy Lebanon, Cristian Sminchisescu. Chebyshev Approximations to the Histogram Chi-Square Kernel. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2012. Long version Code (Updated Jan. 4 2013, a few bug fixes especially for Windows)
(Note: This paper is obsolete, please check out our new arXiv (Section 4) which has real geometric convergence rate and better empirical performance: A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG])

Jaegul Choo, Fuxin Li, Keehyoung Joo, Haesun Park. A Visual Analytics Approach for Protein Disorder Prediction. Expanding the Frontiers of Visual Analytics and Visualization, Springer 2012, pp 163-174.

João Carreira, Fuxin Li, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground Hypotheses. International Journal of Computer Vision (IJCV), (First two authors contributed equally), 98(3):243-262, 2012.

Catalin Ionescu, Fuxin Li, Cristian Sminchisescu. Latent Structured Models for Human Pose Estimation. In IEEE International Conference on Computer Vision (ICCV), 2011 (Oral presentation).

Fuxin Li, Cristian Sminchisescu. Convex Multiple Instance Learning by Estimating Likelihood Ratio, Advances in Neural Processing Systems (NIPS), 2010. Supplementary Material

Fuxin Li, Catalin Ionescu, Cristian Sminchisescu. Random Fourier approximations for skewed multiplicative histogram kernels. In German Association for Pattern Recognition (Deutsche Arbeitsgemeinschaft für Mustererkennung, DAGM), 2010. DAGM prize paper. Code available

Fuxin Li, João Carreira, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground Hypotheses. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2010 (First two authors contributed equally). Per-class accuracies for our VOC 2009 final results (37.24%)

Fuxin Li, Cristian Sminchisescu. The Feature Selection Path in Kernel Methods. In Artificial Intelligence and Statistics (AISTATS), 2010.

Fen Xia, Yanwu Yang, Liang Zhou, Fuxin Li, Min Cai, Daniel D. Zeng: A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning. Pattern Recognition 42(7): 1572-1581 (2009).

Fuxin Li, Yunshan Fu, Yu-Hong Dai, Crisitian Sminchisescu, Jue Wang. Kernel Learning by Unconstrained Optimization. In Artificial Intelligence and Statistics (AISTATS), 2009.

Fen Xia, Wensheng Zhang, Fuxin Li, Yanwu Yang. Ranking with Decision Tree. Knowledge and Information Systems. 17(3):381-395 (2008)

Liang Zhou, Fuxin Li, Yanwu Yang. Path Algorithms for One-Class SVM. ISNN (1) 2008: 645-654

Fuxin Li, Jian Yang, Jue Wang. A Transductive Framework of Distance Metric Learning by Spectral Dimensionality Reduction. In Proceedings of International Conference on Machine Learning (ICML), 2007

Peng Jin, Danqing Zhu, Fuxin Li, Yunfang Wu. PKU: Combining Supervised Classifiers with Features Selection. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), 2007.

Jian Yang, Fuxin Li, Jue Wang. A Better Scaled Local Tangent Space Alignment Algorithm. Proceedings of International Joint Conference on Neural Networks (IJCNN), 2005


Chen Shao, Wei Sun, Fuxin Li, Ruifeng Yang, Ling Zhang, Youhe Gao. Oscore: a combined score to reduce false negative rates for peptide identification in tandem mass spectrometry analysis. Journal of Mass Spectrometry. 2009(14):1, 25-31.

Linjie Wang, Fuxin Li, Wei Sun, Shuzhen Wu, Xiaorong Wang, Ling Zhang, Dexian Zheng, Jue Wang, and Youhe Gao. Concanavalin A-captured Glycoproteins in Healthy Human Urine. Molecular & Cellular Proteomics. 2006(5): 560 - 562

Wei Sun, Fuxin Li, Shuzhen Wu, Xiaorong Wang, Dexian Zheng, Jue Wang, Youhe Gao. Human urine proteome analysis by three separation approaches. Proteomics. 2005(5): 4994-5001

Fuxin Li, Wei Sun, Youhe Gao, Jue Wang. RScore: A Peptide Randomicity Score For Evaluating MS/MS Spectra. Rapid Communications in Mass Spectrometry. 2004(18):14,1655-1659

Wei Sun, Fuxin Li, Jue Wang, Dexian Zheng, Youhe Gao. AMASS: Software for Automatically Validating the Quality of MS/MS Spectrum From SEQUEST Results. Molecular & Cellular Proteomics. 2004(3): 1194-1199