2077 Kelley Engineering Center
Oregon State University
(+1) 541-737-5987 (office)
lif (a) eecs.oregonstate.edu
I am an associate professor in the School of Electrical Engineering and Computer Science at the Oregon State University. My research direction is machine learning and computer vision, with a major interest in using and designing new machine learning algorithms to attack the structural data in images and videos, especially big data originating from videos.
Before joining OSU, I spent 4.5 years in Georgia Tech, first as a postdoc researcher supervised by Dr. Guy Lebanon, then as a research scientist working with Dr. James M. Rehg. I have worked on problems in natural language processing, recommender systems, as well as many computer vision problems such as image and video segmentation, object recognition, multi-target tracking, semantic reconstruction, etc.
Before Georgia Tech, I was working as a research scientist in the Sminchisescu group, INS, University of Bonn from 2008 to 2010. In Bonn I worked on machine learning and computer vision, looking to develop and use learning methods properly to solve conceptual and practical problems. Notably, I'm a member of the BONN-SVRSEGM team which participated in the PASCAL VOC Segmentation Challenge and (co-)won the 2009-2012 challenges.
Even before that, I got my bachelor degree on 2001 in Zhejiang University,
and my Ph.D. degree on 2008 in the Institute of Automation, Chinese Academy of
Sciences, with a dissertation on
I am broadly interested in many machine learning algorithms and applications. Most recently, I have been mainly working on 4 topics:
1) Object Segmentation and Tracking. I have been working on video object segmentation since 2013, with publications in CVPR 2015 as well as placing 4th place in the DAVIS 2017 Video Object Segmentation Challenge. On the related problem of multi-target tracking, I have been working on the MHT-DAM algorithm and most recently we have proposed MHT-bLSTM which proposes a novel formulation of LSTM for solving the multi-target tracking problem.
2) Point Clouds. We recently proposed PointConv, an approach to construct exact convolutional neural networks on point clouds, as in 2D images.
3) Understanding Deep Neural Networks. We have proposed the XNN, which explains neural network decisions with several learned concepts that are visualized to human. Most recently, we proposed I-GOS, which visualizes the important parts a deep network is looking when it makes a classification. I-GOS is "causal" in that the CNN is often capable of classifying the object from only a very small highlighted region, and no longer classifying the object if the region is removed!
4) Uncertainty and Robust Deep Neural Networks. Recently we have proposed HyperGAN, which generates complete neural networks with a GAN that does not require additional training. One can easily sample 100 or 1000 diverse, well-trained networks from a trained hyperGAN and that improves our capability of estimating uncertainty and adversarial robustness. We also have prior work on adversarial defense by convolutional filter statistics and bilateral filtering.
Previously, I have been working for several years on the semantic image segmentation problem. Collaborating with Joao Carreira and Cristian Sminchisescu, our object segmentation/recognition system won the PASCAL VOC 2009-2012 Segmentation Challenge (with Adrian Ion as well in 2012). I mainly work in the recognition part of the system, on how to correctly classify and generate the final segmentation from a pool of initial figure-ground segmentations. The composite statistical modeling framework I proposed with colleagues is supposed to unseat pairwise CRF approaches for higher-order inference in fully-connected graphical models.
May. 2021 The code for iGOS++ is available. This siginficantly improves over our previous I-GOS approach, especially under high resolutions (28x28 or 224x224).
Oct. 2020 We proposed Deep Variational Instance Segmentation, a proposal-free instance segmentation approach that directly predict instance labels with a fully convolutional network. Code can be found here.
May. 2020 Code for PointPWCNet is available!
Nov. 2019 We proposed PointPWCNet, a coarse-to-fine approach to compute scene flow from point clouds with great results. Especially, using PointConv to compute cost volume convolution saves a significant amount of computations!
Jun. 2019 Code for HyperGAN is available now!
Apr. 2019 We propose I-GOS, an approach to compute heatmaps for CNNs that generates localized heatmaps that actually correlate with the prediction at any resolution.
Feb. 2019 Fuxin received an Amazon Research Award.
Jan. 2019 We proposed HyperGAN, a generative adversarial network that can directly generate diverse, fully-trained neural networks (all the weights) from a noise vector!
Nov. 2018 We proposed PointConv, which is a complete, scalable CNN (not depthwise or other approximations) on a point cloud. We have achieved the best result on the ScanNet benchmark for a network based only on a 3D point cloud, without volumetric networks. We have also been able to achieve identical performance to CNNs on CIFAR-10 when representing the image as a point cloud.
Sep. 2018 We proposed the bilinear LSTM, which reformed the LSTM formulation according to intuitions derived from a low-rank approximation of the classical recursive least squares. This has significantly improved the capability of training a multi-target tracking network using an LSTM.
Mar. 2018 Fuxin received an NSF CAREER award.
Jul. 2017 Our GaTech-Oregon State team has obtained the 4th place in the DAVIS Video Segmentation Challenge 2017.
Jul. 2017 Our team (MHT-DAM) has obtained the best overall ranking (Avg Rank) in the Multi-Target Tracking challenge (MOT) 2017 .
Sep. 2015 Two papers accepted in ICCV 2015. We beat the state-of-the-art in both object proposals generation and multi-target tracking.
Mar. 2015 An improvement of the video segmentation framework to handle complete occlusions is accepted to CVPR 2015. Congratulations Zhengyang for the impressive achievement to publish a first-authored paper in CVPR as a junior undergraduate student! Check out the Project Website
Jun. 2014 We proposed RIGOR, an algorithm that achieves accuracy slightly better to CPMC/Object proposals, but significantly faster, generating all object proposals in 2-4 seconds on CPU-only code. Check out the Project Page for more information and downloading the code.
Oct. 2013 We exploited semantic segmentation techniques to perform unsupervised video segmentation based on tracking image segments per-frame with 1) automatic object discovery based on segment pool; 2) efficiently learnt appearance models for hundreds of objects with strong appearance features; 3) CSI refinement of objects. Check out Project Webpage, SegTrack v2 Dataset , ICCV paper.
Jun. 2013 We proposed the Composite Statistical Inference (CSI) for inference on real-valued statistics obtained on multiple high-order overlapping variable subsets, with applications in semantic segmentation. Short version, Long version .
Apr. 2013 We proposed an analytic approximation to the chi-square kernel with geometric convergence and derived from elementary methods. It's more straightforward and converges faster over previous approaches. Preprint
Xingyi Li (Ph.D. Graduate Dec. 2020, Dexterity Inc.)
Zhongang Qi (Postdoctoral Associate 2016 - 2019, Tencent Research)
Zehuan Chen (M.S. Graduate Apr. 2019, Amazon)
Xinyao Wang (M.S. Graduate Dec. 2018, JD Digits)
Rahul Sawhney (Ph.D. Graduate April. 2018, Microsoft Research Redmond)
Alrik Firl (M.S. Graduate Jul. 2018, General Electrics)
Xin Li (M.S. Graduate Sept. 2016, NVIDIA)
Zheng Zhou (M.S. Graduate Jun. 2017, Tencent AI Lab)
Winter: CS 535 Deep Learning
Winter: CS 261 Data Structures
Spring: CS 637 Computer Vision II
Erich Merill III, Stefan Lee, Li Fuxin, Thomas Dietterich, Alan Fern. Deep Convolution for Irregularly Sampled Temporal Point Clouds. arXiv:2105.00137
Xingyi Li, Wenxuan Wu, Xiaoli Z. Fern, Li Fuxin. The Devils in the Point Clouds: Studying the Robustness of Point Cloud Convolutions. arXiv:2101.07832
Neale Ratzlaff, LI Fuxin. Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks. . arXiv:1804.01635.
Saeed Khorram, Li Fuxin. Cycle-Consistent Counterfactuals by Latent Transformations. CVPR, 2022.
Vivswan Shitole, Li Fuxin, Minsuk Kahng, Prasad Tadepalli, Alan Fern. Structured Attention Graphs for Understanding Deep Image Classifications. NeuRIPS, 2021.
Bob DeBortoli, Li Fuxin, Ashish Kapoor, Geoffrey Hollinger. Adversarial Training on Point Clouds for Sim-to-Real 3D Object Detection. IEEE Robotics and Automation Letters (RA-L). 6(4), 2021, 6662-6669. (will also be presented at IROS)
Jay Patravali*, Gaurav Mittal*, Ye Yu, Li Fuxin, Mei Chen. Unsupervised Few-Shot Action Recognition via Action-Appearance Aligned Meta-Adaptation. ICCV, 2021. (Oral Presentation), (first two authors contributed equally)
Neale Ratzlaff*, Qinxun Bai*, Li Fuxin, Wei Xu. Generative Particle Variational Inference via Estimation of Functional Gradients. ICML, 2021. (first two authors contributed equally)
Xianfang Zeng, Wenxuan Wu, Guangzhong Tian, Jun Chen, Fuxin Li, Yong Liu. Deep Superpixel Convolutional Network for Image Recognition. IEEE Signal Processing Letters. Accepted.
Chanho Kim, Li Fuxin, Mazen Alotaibi, James Rehg. Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking. CVPR , 2021
Saeed Khorram*, Tyler Lawson*, Li Fuxin. iGOS++: Integrated Gradient Optimized Saliency by Bilateral Perturbations ACM Conference on Health, Inference and Learning (CHIL), 2021. (First two authors contributed equally)
Xiaoling Hu, Yusu Wang, Li Fuxin, Dimitris Samaras, Chao Chen. Topology-Aware Segmentation Using Discrete Morse Theory. ICLR, 2021 (Spotlight Presentation)
Chen Ziwen, Wenxuan Wu, Zhongang Qi, Li Fuxin. Visualizing Point Cloud Classifiers by Curvature Smoothing. BMVC, 2020 Code
Wenxuan Wu, Zhiyuan Wang, Zhuwen Li, Wei Liu, Li Fuxin. PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Clouds. ECCV, 2020 (Spotlight Presentation) Code
Neale Ratzlaff, Qinxun Bai, Li Fuxin, Wei Xu. Implicit Generative Modeling for Efficient Exploration. ICML, 2020
Jun Li, Li Fuxin, Sinisa Todorovic. Efficient Riemannian Optimization on the Stiefel Manifold via the Cayley Transform. ICLR, 2020
Xingyi Li, Zhongang Qi, Xiaoli Fern, LI Fuxin. ScaleNet - Improve CNNs through Recursively Scaling Objects. AAAI, 2020.
Zhongang Qi, Saeed Khorram, LI Fuxin. Visualizing Deep Networks by Optimizing with Integrated Gradients. AAAI , 2020. Code, Demo
Xiaoling Hu, LI Fuxin, Dimitris Samaras, Chao Chen. Topology-Preserving Deep Image Segmentation. NeuRIPS, 2019.
Xinyao Wang, Liefeng Bo, LI Fuxin. Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression. ICCV, 2019. Code
Robert DeBortoli, LI Fuxin, Geoffrey Hollinger. Elevatenet: A Convolutional Neural Network for Estimating the Missing Dimension in 2D Underwater Sonar Images. IROS, 2019.
Neale Ratzlaff, LI Fuxin. HyperGAN: A Generative Model for Diverse, Performant Neural Networks. ICML, 2019. Code
Wenxuan Wu, Zhongang Qi, LI Fuxin. PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR, 2019. Project Page Code
Chanho Kim, Fuxin Li, James Rehg. Multi-object Tracking with Neural Gating Using Bilinear LSTM. ECCV, 2018.
R. Sawhney, F. Li, H. Christensen, C. Isbell. Purely Geometric Scene Association and Retrieval - A case for macro-scale geometry. Accepted to ICRA, 2018.
R. DeBortoli, A. Nicolai, F. Li, G. Hollinger. Realtime Underwater 3D Reconstruction Using Global Context and Active Labeling. Proceedings of the International Conference on Robotics and Automation (ICRA) 2018.
Zhongang Qi, Fuxin Li. Embedding Deep Networks into Visual Explanations. arXiv:1709.05360. Short (4 pages) version published in NIPS 2017 Workshop on Interpreting, Explaining and Visualizing Deep Learning - Now what?. Code/Data
Peng Lei, Fuxin Li, Sinisa Todorovic. Boundary Flow: A Siamese Network that Predicts Boundary Motion without Training on Motion . CVPR 2018.
Xin Li, Fuxin Li. Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics . ICCV 2017.
Juan Liu, Zhengyang Wu, Fuxin Li. Ranking Video Segments with LSTM and Determinantal Point Processes. ICIP 2017.
Xingyi Li, Fuxin Li, Xiaoli Fern, Raviv Raich. Filter Shaping for Convolutional Networks. ICLR 2017.
Zhaoyang Lv, Chris Beall, Pablo F. Alcantarilla, Fuxin Li, Zsolt Kira, Frank Dellaert. A Continuous Optimization Approach for Efficient and Accurate Scene Flow. ECCV 2016.
Yu-Ying Liu, Shuang Li, Fuxin Li, Le Song, James M. Rehg. Efficient Learning of Continuous-Time Hidden Markov Models for Disease Progression. NIPS 2015.
Chanho Kim, Fuxin Li, Arridhana Ciptadi, James M. Rehg. Multiple Hypothesis Tracking Revisited. ICCV 2015 (Oral Presentation).
Ahmad Humayun, Fuxin Li, James M. Rehg. The Middle Child Problem: Revisiting Parametric Min-cut for Robust Object Proposals. ICCV 2015.
Zhengyang Wu, Fuxin Li, Rahul Sukthankar, James M. Rehg. Robust Video Segment Proposals with Painless Occlusion Handling. CVPR 2015. Project Website
Rahul Sawhney, Fuxin Li, Henrik Christensen. GASP: Geometric Association with Surface Patches. 3DV 2014.
Abhijit Kundu, Yin Li, Frank Dellaert, Fuxin Li, James M. Rehg. Joint Semantic Segmentation and 3D Reconstruction from Monocular Video . ECCV 2014.
Ahmad Humayun, Fuxin Li, James M. Rehg. RIGOR: Reusing Inference in Graph Cuts for Generating Object Regions. CVPR 2014. Project Page + Code
Fuxin Li, Taeyoung Kim, Ahmad Humayun,
David Tsai, James M. Rehg. Video Segmentation by Tracking
Many Figure-Ground Segments. In
Project Webpage, SegTrack v2 Dataset
(There were a few notation typos around Eq. (7) in the official IEEE version, please use the version on this website).
Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning
Contact Locations for Pushing and Orienting Unknown Objects .
Fuxin Li, Guy Lebanon, Christian Sminchisescu. A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG]
Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning
Stable Pushing Locations. In
Fuxin Li, Joao Carreira, Guy Lebanon, Cristian Sminchisescu. Composite
Statistical Inference for Semantic Segmentation. In
Seungyeon Kim, Fuxin Li, Guy Lebanon, Irfan Essa. Beyond
Sentiment: The Manifold of Human Emotions. In
Mingxuan Sun, Fuxin Li, Joonseok
Lee, Ke Zhou, Guy Lebanon, Hongyuan
Multiple-Question Decision Trees for Cold-Start Recommendation. In
Eduards G. Bazavan,
Fuxin Li, Cristian Sminchisescu. Fourier
Kernel Learning. In
Fuxin Li, Guy Lebanon, Cristian Sminchisescu. Chebyshev Approximations to the Histogram
Chi-Square Kernel. In
(Note: This paper is obsolete, please check out our new arXiv (Section 4) which has real geometric convergence rate and better empirical performance: A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG])
Jaegul Choo, Fuxin Li, Keehyoung Joo, Haesun Park. A Visual Analytics Approach for Protein Disorder Prediction. Expanding the Frontiers of Visual Analytics and Visualization, Springer 2012, pp 163-174.
João Carreira, Fuxin
Li, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground
Catalin Ionescu, Fuxin
Li, Cristian Sminchisescu. Latent
Structured Models for Human Pose Estimation. In
Fuxin Li, Cristian Sminchisescu. Convex Multiple Instance Learning by Estimating Likelihood Ratio, Advances in Neural Processing Systems (NIPS), 2010. Supplementary Material
Fuxin Li, Catalin Ionescu, Cristian Sminchisescu. Random Fourier approximations for skewed multiplicative histogram kernels. In German Association for Pattern Recognition (Deutsche Arbeitsgemeinschaft für Mustererkennung, DAGM), 2010. DAGM prize paper. Code available
Fuxin Li, João Carreira, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground Hypotheses. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2010 (First two authors contributed equally). Per-class accuracies for our VOC 2009 final results (37.24%)
Fuxin Li, Cristian Sminchisescu. The Feature Selection Path in Kernel Methods. In Artificial Intelligence and Statistics (AISTATS), 2010.
Fen Xia, Yanwu Yang, Liang Zhou, Fuxin Li, Min Cai, Daniel D. Zeng: A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning. Pattern Recognition 42(7): 1572-1581 (2009).
Fuxin Li, Yunshan Fu, Yu-Hong Dai, Crisitian Sminchisescu, Jue Wang. Kernel Learning by Unconstrained Optimization. In Artificial Intelligence and Statistics (AISTATS), 2009.
Fen Xia, Wensheng Zhang, Fuxin Li, Yanwu Yang. Ranking with Decision Tree. Knowledge and Information Systems. 17(3):381-395 (2008)
Liang Zhou, Fuxin Li, Yanwu Yang. Path Algorithms for One-Class SVM. ISNN (1) 2008: 645-654
Peng Jin, Danqing Zhu, Fuxin Li, Yunfang Wu. PKU: Combining Supervised Classifiers with Features Selection. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), 2007.
Jian Yang, Fuxin Li, Jue Wang. A Better Scaled Local Tangent Space Alignment Algorithm. Proceedings of International Joint Conference on Neural Networks (IJCNN), 2005
Chen Shao, Wei Sun, Fuxin Li, Ruifeng Yang, Ling Zhang, Youhe Gao. Oscore: a combined score to reduce false negative rates for peptide identification in tandem mass spectrometry analysis. Journal of Mass Spectrometry. 2009(14):1, 25-31.
Linjie Wang, Fuxin Li, Wei Sun, Shuzhen Wu, Xiaorong Wang, Ling Zhang, Dexian Zheng, Jue Wang, and Youhe Gao. Concanavalin A-captured Glycoproteins in Healthy Human Urine. Molecular & Cellular Proteomics. 2006(5): 560 - 562
Wei Sun, Fuxin Li, Shuzhen Wu, Xiaorong Wang, Dexian Zheng, Jue Wang, Youhe Gao. Human urine proteome analysis by three separation approaches. Proteomics. 2005(5): 4994-5001
Fuxin Li, Wei Sun, Youhe Gao, Jue Wang. RScore: A Peptide Randomicity Score For Evaluating MS/MS Spectra. Rapid Communications in Mass Spectrometry. 2004(18):14,1655-1659
Wei Sun, Fuxin Li, Jue Wang, Dexian Zheng, Youhe Gao. AMASS: Software for Automatically Validating the Quality of MS/MS Spectrum From SEQUEST Results. Molecular & Cellular Proteomics. 2004(3): 1194-1199