Dr. Fuxin Li

Address:

2077 Kelley Engineering Center

Oregon State University

Corvallis 97331, Oregon
USA

Phone:

(+1) 541-737-5987 (office)

E-Mail:

lif (a) eecs.oregonstate.edu

Google Scholar Profile

CV

Postdoc positions available

PhD position available for the 2020-2021 school year as well. For PhD, please apply to the school of EECS with my name mentioned as potential advisor. Sending emails to me directly will not have any positive effect. I will be carefully reviewing all the applications with my name mentioned.

Bio

I am an assistant professor in the School of Electrical Engineering and Computer Science at the Oregon State University. My research direction is machine learning and computer vision, with a major interest in using and designing new machine learning algorithms to attack the structural data in images and videos, especially big data originating from videos.

Before joining OSU, I spent 4.5 years in Georgia Tech, first as a postdoc researcher supervised by Dr. Guy Lebanon, then as a research scientist working with Dr. James M. Rehg. I have worked on problems in natural language processing, recommender systems, as well as many computer vision problems such as image and video segmentation, object recognition, multi-target tracking, semantic reconstruction, etc.

Before Georgia Tech, I was working as a research scientist in the Sminchisescu group, INS, University of Bonn from 2008 to 2010. In Bonn I worked on machine learning and computer vision, looking to develop and use learning methods properly to solve conceptual and practical problems. Notably, I'm a member of the BONN-SVRSEGM team which participated in the PASCAL VOC Segmentation Challenge and (co-)won the 2009-2012 challenges.

Even before that, I got my bachelor degree on 2001 in Zhejiang University, and my Ph.D. degree on 2008 in the Institute of Automation, Chinese Academy of Sciences, with a dissertation on Euclidean Metric Learning. My advisor is Jue Wang, a pioneer of artificial intelligence in China (not to be confused with Dr. Jue Wang from Megvii). Besides machine learning, I have also done some proteomics algorithms and software during my Ph.D., collaborating with Professor Youhe Gao from Chinese Academy of Medical Sciences.

Research Interests

I am broadly interested in many machine learning algorithms and applications. Most recently, I have been mainly working on 4 topics:

1) Object Segmentation and Tracking. I have been working on video object segmentation since 2013, with publications in CVPR 2015 as well as placing 4th place in the DAVIS 2017 Video Object Segmentation Challenge. On the related problem of multi-target tracking, I have been working on the MHT-DAM algorithm and most recently we have proposed MHT-bLSTM which proposes a novel formulation of LSTM for solving the multi-target tracking problem.

2) Point Clouds. We recently proposed PointConv, an approach to construct exact convolutional neural networks on point clouds, as in 2D images.

3) Understanding Deep Neural Networks. We have proposed the XNN, which explains neural network decisions with several learned concepts that are visualized to human. Most recently, we proposed I-GOS, which visualizes the important parts a deep network is looking when it makes a classification. I-GOS is "causal" in that the CNN is often capable of classifying the object from only a very small highlighted region, and no longer classifying the object if the region is removed!

4) Uncertainty and Robust Deep Neural Networks. Recently we have proposed HyperGAN, which generates complete neural networks with a GAN that does not require additional training. One can easily sample 100 or 1000 diverse, well-trained networks from a trained hyperGAN and that improves our capability of estimating uncertainty and adversarial robustness. We also have prior work on adversarial defense by convolutional filter statistics and bilateral filtering.

Previously, I have been working for several years on the semantic image segmentation problem. Collaborating with Joao Carreira and Cristian Sminchisescu, our object segmentation/recognition system won the PASCAL VOC 2009-2012 Segmentation Challenge (with Adrian Ion as well in 2012). I mainly work in the recognition part of the system, on how to correctly classify and generate the final segmentation from a pool of initial figure-ground segmentations. The composite statistical modeling framework I proposed with colleagues is supposed to unseat pairwise CRF approaches for higher-order inference in fully-connected graphical models.

Publishing under the name of Li Fuxin starting from 2019

This is a Chinese name, where Fuxin (pronounced FOO-SHEEN) is the first name and Li (pronounced LEE) is the last name. Call me Fuxin when we meet.

AI/Robotics in OSU

I am a proud member of the new CoRIS institute in Oregon State University. We are a large and very strong research group in artificial intelligence, machine learning, computer vision, natural language processing and robotics. OSU is ranked 43th overall in the USA by the most up-to-date Computer Science Rankings and 22nd in AI/ML/CV/NLP. At least one ranking puts our robotics program as high as #4 in the USA. If you are using USNews rankings to guide your grad school search, take a read at this article.

Selected Recent Research Highlights:  

Oct. 2019 Code and Demo for I-GOS available.

Jun. 2019 Code for HyperGAN is available now!

Apr. 2019 Code for PointConv is available on the Project Page.

Feb. 2019 Fuxin received an Amazon Research Award.

Jan. 2019 We proposed HyperGAN, a generative adversarial network that can directly generate diverse, fully-trained neural networks (all the weights) from a noise vector!

Nov. 2018 We proposed PointConv, which is a complete, scalable CNN (not depthwise or other approximations) on a point cloud. We have achieved the best result on the ScanNet benchmark for a network based only on a 3D point cloud, without volumetric networks. We have also been able to achieve identical performance to CNNs on CIFAR-10 when representing the image as a point cloud.

Sep. 2018 We proposed the bilinear LSTM, which reformed the LSTM formulation according to intuitions derived from a low-rank approximation of the classical recursive least squares. This has significantly improved the capability of training a multi-target tracking network using an LSTM.

Mar. 2018 Fuxin received an NSF CAREER award.

Jul. 2017 Our GaTech-Oregon State team has obtained the 4th place in the DAVIS Video Segmentation Challenge 2017.

Jul. 2017 Our team (MHT-DAM) has obtained the best overall ranking (Avg Rank) in the Multi-Target Tracking challenge (MOT) 2017 .

Sep. 2015 Two papers accepted in ICCV 2015. We beat the state-of-the-art in both object proposals generation and multi-target tracking.

Mar. 2015 An improvement of the video segmentation framework to handle complete occlusions is accepted to CVPR 2015. Congratulations Zhengyang for the impressive achievement to publish a first-authored paper in CVPR as a junior undergraduate student! Check out the Project Website

Jun. 2014 We proposed RIGOR, an algorithm that achieves accuracy slightly better to CPMC/Object proposals, but significantly faster, generating all object proposals in 2-4 seconds on CPU-only code. Check out the Project Page for more information and downloading the code.

Oct. 2013 We exploited semantic segmentation techniques to perform unsupervised video segmentation based on tracking image segments per-frame with 1) automatic object discovery based on segment pool; 2) efficiently learnt appearance models for hundreds of objects with strong appearance features; 3) CSI refinement of objects. Check out Project Webpage, SegTrack v2 Dataset , ICCV paper.

Jun. 2013 We proposed the Composite Statistical Inference (CSI) for inference on real-valued statistics obtained on multiple high-order overlapping variable subsets, with applications in semantic segmentation. Short version, Long version .

Apr. 2013 We proposed an analytic approximation to the chi-square kernel with geometric convergence and derived from elementary methods. It's more straightforward and converges faster over previous approaches. Preprint

Group :

Postdoc:

Ph.D. students:

Jialin Yuan

Xingyi Li (co-advised with Xiaoli Fern)

Lawrence Neal (co-advised with Xiaoli Fern)

Robert DeBortoli (co-advised with Geoffrey Hollinger)

Neale Ratzlaff

Wenxuan Wu

Hung Nguyen

Saeed Khorram

M.S. students:

Jay Patravali

Alumni:

Zhongang Qi (Postdoctoral Associate 2016 - 2019, Tencent Research)

Zehuan Chen (M.S. Graduate Apr. 2019, Amazon)

Xinyao Wang (M.S. Graduate Dec. 2018, JD Digits)

Rahul Sawhney (Ph.D. Graduate April. 2018, Microsoft Research Redmond)

Alrik Firl (M.S. Graduate Jul. 2018, General Electrics)

Xin Li (M.S. Graduate Sept. 2016, NVIDIA)

Zheng Zhou (M.S. Graduate Jun. 2017, Tencent AI Lab)

Teaching:  

Winter: CS 535 Deep Learning

Winter: CS 261 Data Structures

Spring: CS 637 Computer Vision II

Publications:  

Preprints

Neale Ratzlaff, LI Fuxin. Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks . arXiv:1804.01635.

Machine Learning and Computer Vision

Xingyi Li, Zhongang Qi, Xiaoli Fern, LI Fuxin. ScaleNet - Improve CNNs through Recursively Scaling Objects. AAAI , 2020.

Zhongang Qi, Saeed Khorram, LI Fuxin. Visualizing Deep Networks by Optimizing with Integrated Gradients. AAAI , 2020. Code, Demo

Xiaoling Hu, LI Fuxin, Dimitris Samaras, Chao Chen. Topology-Preserving Deep Image Segmentation. NeuRIPS, 2019.

Xinyao Wang, Liefeng Bo, LI Fuxin. Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression. ICCV, 2019.

Robert DeBortoli, LI Fuxin, Geoffrey Hollinger. Elevatenet: A Convolutional Neural Network for Estimating the Missing Dimension in 2D Underwater Sonar Images. IROS, 2019.

Neale Ratzlaff, LI Fuxin. HyperGAN: A Generative Model for Diverse, Performant Neural Networks. ICML, 2019. Code

Wenxuan Wu, Zhongang Qi, LI Fuxin. PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR, 2019. Project Page Code

Chanho Kim, Fuxin Li, James Rehg. Multi-object Tracking with Neural Gating Using Bilinear LSTM. ECCV, 2018.

Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, Fuxin Li. Open Set Learning with Counterfactual Images. ECCV, 2018.

R. Sawhney, F. Li, H. Christensen, C. Isbell. Purely Geometric Scene Association and Retrieval - A case for macro-scale geometry. Accepted to ICRA, 2018.

R. DeBortoli, A. Nicolai, F. Li, G. Hollinger. Realtime Underwater 3D Reconstruction Using Global Context and Active Labeling. Proceedings of the International Conference on Robotics and Automation (ICRA) 2018.

Zhongang Qi, Fuxin Li. Embedding Deep Networks into Visual Explanations. arXiv:1709.05360. Short (4 pages) version published in NIPS 2017 Workshop on Interpreting, Explaining and Visualizing Deep Learning - Now what?. Code/Data

Peng Lei, Fuxin Li, Sinisa Todorovic. Boundary Flow: A Siamese Network that Predicts Boundary Motion without Training on Motion . CVPR 2018.

Xin Li, Fuxin Li. Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics . ICCV 2017.

Juan Liu, Zhengyang Wu, Fuxin Li. Ranking Video Segments with LSTM and Determinantal Point Processes. ICIP 2017.

Xingyi Li, Fuxin Li, Xiaoli Fern, Raviv Raich. Filter Shaping for Convolutional Networks. ICLR 2017.

Zhaoyang Lv, Chris Beall, Pablo F. Alcantarilla, Fuxin Li, Zsolt Kira, Frank Dellaert. A Continuous Optimization Approach for Efficient and Accurate Scene Flow. ECCV 2016.

Yu-Ying Liu, Shuang Li, Fuxin Li, Le Song, James M. Rehg. Efficient Learning of Continuous-Time Hidden Markov Models for Disease Progression. NIPS 2015.

Chanho Kim, Fuxin Li, Arridhana Ciptadi, James M. Rehg. Multiple Hypothesis Tracking Revisited. ICCV 2015 (Oral Presentation).

Ahmad Humayun, Fuxin Li, James M. Rehg. The Middle Child Problem: Revisiting Parametric Min-cut for Robust Object Proposals. ICCV 2015.

Zhengyang Wu, Fuxin Li, Rahul Sukthankar, James M. Rehg. Robust Video Segment Proposals with Painless Occlusion Handling. CVPR 2015. Project Website

Rahul Sawhney, Fuxin Li, Henrik Christensen. GASP: Geometric Association with Surface Patches. 3DV 2014.

Abhijit Kundu, Yin Li, Frank Dellaert, Fuxin Li, James M. Rehg. Joint Semantic Segmentation and 3D Reconstruction from Monocular Video . ECCV 2014.

Ahmad Humayun, Fuxin Li, James M. Rehg. RIGOR: Reusing Inference in Graph Cuts for Generating Object Regions. CVPR 2014. Project Page + Code

Fuxin Li, Taeyoung Kim, Ahmad Humayun, David Tsai, James M. Rehg. Video Segmentation by Tracking Many Figure-Ground Segments. In IEEE International Conference on Computer Vision (ICCV), 2013.
Project Webpage, SegTrack v2 Dataset
(There were a few notation typos around Eq. (7) in the official IEEE version, please use the version on this website).

Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning Contact Locations for Pushing and Orienting Unknown Objects . In IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2013.

Fuxin Li, Guy Lebanon, Christian Sminchisescu. A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG]

Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning Stable Pushing Locations. In IEEE International Conference on Development and Learning (ICDL), 2013.

Fuxin Li, Joao Carreira, Guy Lebanon, Cristian Sminchisescu. Composite Statistical Inference for Semantic Segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013. Technical Report

Seungyeon Kim, Fuxin Li, Guy Lebanon, Irfan Essa. Beyond Sentiment: The Manifold of Human Emotions. In Proceedings of the 16th International Conference on Artificial Intelligence and Statistics (AISTATS), 2013.

Mingxuan Sun, Fuxin Li, Joonseok Lee, Ke Zhou, Guy Lebanon, Hongyuan Zha. Learning Multiple-Question Decision Trees for Cold-Start Recommendation. In ACM International Conference on Web Search and Data Mining (WSDM), 2013 (Spotlight presentation).

Eduards G. Bazavan, Fuxin Li, Cristian Sminchisescu. Fourier Kernel Learning. In European Conference on Computer Vision (ECCV), 2012 (Oral presentation).

Fuxin Li, Guy Lebanon, Cristian Sminchisescu. Chebyshev Approximations to the Histogram Chi-Square Kernel. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2012. Long version Code (Updated Jan. 4 2013, a few bug fixes especially for Windows)
(Note: This paper is obsolete, please check out our new arXiv (Section 4) which has real geometric convergence rate and better empirical performance: A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG])

Jaegul Choo, Fuxin Li, Keehyoung Joo, Haesun Park. A Visual Analytics Approach for Protein Disorder Prediction. Expanding the Frontiers of Visual Analytics and Visualization, Springer 2012, pp 163-174.

João Carreira, Fuxin Li, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground Hypotheses. International Journal of Computer Vision (IJCV), (First two authors contributed equally), 98(3):243-262, 2012.

Catalin Ionescu, Fuxin Li, Cristian Sminchisescu. Latent Structured Models for Human Pose Estimation. In IEEE International Conference on Computer Vision (ICCV), 2011 (Oral presentation).

Fuxin Li, Cristian Sminchisescu. Convex Multiple Instance Learning by Estimating Likelihood Ratio, Advances in Neural Processing Systems (NIPS), 2010. Supplementary Material

Fuxin Li, Catalin Ionescu, Cristian Sminchisescu. Random Fourier approximations for skewed multiplicative histogram kernels. In German Association for Pattern Recognition (Deutsche Arbeitsgemeinschaft für Mustererkennung, DAGM), 2010. DAGM prize paper. Code available

Fuxin Li, João Carreira, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground Hypotheses. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2010 (First two authors contributed equally). Per-class accuracies for our VOC 2009 final results (37.24%)

Fuxin Li, Cristian Sminchisescu. The Feature Selection Path in Kernel Methods. In Artificial Intelligence and Statistics (AISTATS), 2010.

Fen Xia, Yanwu Yang, Liang Zhou, Fuxin Li, Min Cai, Daniel D. Zeng: A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning. Pattern Recognition 42(7): 1572-1581 (2009).

Fuxin Li, Yunshan Fu, Yu-Hong Dai, Crisitian Sminchisescu, Jue Wang. Kernel Learning by Unconstrained Optimization. In Artificial Intelligence and Statistics (AISTATS), 2009.

Fen Xia, Wensheng Zhang, Fuxin Li, Yanwu Yang. Ranking with Decision Tree. Knowledge and Information Systems. 17(3):381-395 (2008)

Liang Zhou, Fuxin Li, Yanwu Yang. Path Algorithms for One-Class SVM. ISNN (1) 2008: 645-654

Fuxin Li, Jian Yang, Jue Wang. A Transductive Framework of Distance Metric Learning by Spectral Dimensionality Reduction. In Proceedings of International Conference on Machine Learning (ICML), 2007

Peng Jin, Danqing Zhu, Fuxin Li, Yunfang Wu. PKU: Combining Supervised Classifiers with Features Selection. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), 2007.

Jian Yang, Fuxin Li, Jue Wang. A Better Scaled Local Tangent Space Alignment Algorithm. Proceedings of International Joint Conference on Neural Networks (IJCNN), 2005

Proteomics

Chen Shao, Wei Sun, Fuxin Li, Ruifeng Yang, Ling Zhang, Youhe Gao. Oscore: a combined score to reduce false negative rates for peptide identification in tandem mass spectrometry analysis. Journal of Mass Spectrometry. 2009(14):1, 25-31.

Linjie Wang, Fuxin Li, Wei Sun, Shuzhen Wu, Xiaorong Wang, Ling Zhang, Dexian Zheng, Jue Wang, and Youhe Gao. Concanavalin A-captured Glycoproteins in Healthy Human Urine. Molecular & Cellular Proteomics. 2006(5): 560 - 562

Wei Sun, Fuxin Li, Shuzhen Wu, Xiaorong Wang, Dexian Zheng, Jue Wang, Youhe Gao. Human urine proteome analysis by three separation approaches. Proteomics. 2005(5): 4994-5001

Fuxin Li, Wei Sun, Youhe Gao, Jue Wang. RScore: A Peptide Randomicity Score For Evaluating MS/MS Spectra. Rapid Communications in Mass Spectrometry. 2004(18):14,1655-1659

Wei Sun, Fuxin Li, Jue Wang, Dexian Zheng, Youhe Gao. AMASS: Software for Automatically Validating the Quality of MS/MS Spectrum From SEQUEST Results. Molecular & Cellular Proteomics. 2004(3): 1194-1199