2077 Kelley Engineering Center
Oregon State University
(+1) 541-737-5987 (office)
lif (a) eecs.oregonstate.edu
I am an assistant professor in the School of Electrical Engineering and Computer Science at the Oregon State University. My research direction is machine learning and computer vision, with a major interest in using and designing new machine learning algorithms to attack the structural data in images and videos, especially big data originating from videos.
Before joining OSU, I spent 4.5 years in Georgia Tech, first as a postdoc researcher supervised by Dr. Guy Lebanon, then as a research scientist working with Dr. James M. Rehg. I have worked on problems in natural language processing, recommender systems, as well as many computer vision problems such as image and video segmentation, object recognition, multi-target tracking, semantic reconstruction, etc.
Before Georgia Tech, I was working as a research scientist in the Sminchisescu group, INS, University of Bonn from 2008 to 2010. In Bonn I worked on machine learning and computer vision, looking to develop and use learning methods properly to solve conceptual and practical problems. Notably, I'm a member of the BONN-SVRSEGM team which participated in the PASCAL VOC Segmentation Challenge and (co-)won the 2009-2012 challenges.
Even before that, I got my bachelor degree on 2001 in Zhejiang University,
and my Ph.D. degree on 2008 in the Institute of Automation, Chinese Academy of
Sciences, with a dissertation on
I am broadly interested in many machine learning algorithms and applications. Most recently, I have been working more on computer vision, especially segmentation-based object recognition and scene understanding. I have recently developed a significant interest in video. Video can provide spatio-temporal cues that should enable many interesting learning algorithms to work with minimal supervision. A first step is to segment the spatio-temporal objects from video. I have been working on this problem to generate efficient proposals for those objects, utilizing machine learning tools to train supervised appearance models without any human supervision. Check out my ICCV 2013 and CVPR 2015 papers for more details.
Previously, I have been working for several years on the semantic image segmentation problem. Collaborating with Joao Carreira and Cristian Sminchisescu, our object segmentation/recognition system won the PASCAL VOC 2009-2012 Segmentation Challenge (with Adrian Ion as well in 2012). I mainly work in the recognition part of the system, on how to correctly classify and generate the final segmentation from a pool of initial figure-ground segmentations. The composite statistical modeling framework I proposed with colleagues is supposed to unseat pairwise CRF approaches for higher-order inference in fully-connected graphical models.
Theoretically, I am more interested in the optimization part of machine learning, especially to unify optimization and generalization in machine learning. Some of my previous works focused on learning kernels and metrics, either from a kernel-matrix learning perspective or from the perspective of nonlinear feature selection inside a kernel. They are published in AISTATS 2009 and 2010. In particular, the trust-region inexact Newton method from our 2009 AISTATS paper was the fastest algorithm for a long while to learn a full-rank positive-definite kernel matrix.
On a larger scale, I have the drive to create the real learning machine that has more and more capabilities, that will in the end surpass human intelligence and engage in the battle with genetically heavily modified human-being in the future. But I also believe that Rome is not built in one day. Therefore, I'm also interested in alternative but practical learning paradigms. I had a big interest in semi-supervised learning, but that faded over time. I'm still quite interested in active learning, multiple-instance learning and other forms of weakly-supervised learning. In NIPS 2010 I published a paper on a convex formulation of multiple-instance learning.
Nov. 2018 We proposed PointConv, which is a complete, scalable CNN (not depthwise or other approximations) on a point cloud. We have achieved the best result on the ScanNet benchmark for a network based only on a 3D point cloud, without volumetric networks. We have also been able to achieve identical performance to CNNs on CIFAR-10 when representing the image as a point cloud.
Sep. 2018 We proposed the bilinear LSTM, which reformed the LSTM formulation according to intuitions derived from a low-rank approximation of the classical recursive least squares. This has significantly improved the capability of training a multi-target tracking network using an LSTM.
Mar. 2018 Fuxin got an NSF CAREER award.
Jul. 2017 Our GaTech-Oregon State team has obtained the 4th place in the DAVIS Video Segmentation Challenge 2017.
Jul. 2017 Our team (MHT-DAM) has obtained the best overall ranking (Avg Rank) in the Multi-Target Tracking challenge (MOT) 2017 .
Sep. 2015 Two papers accepted in ICCV 2015. We beat the state-of-the-art in both object proposals generation and multi-target tracking.
Mar. 2015 An improvement of the video segmentation framework to handle complete occlusions is accepted to CVPR 2015. Congratulations Zhengyang for the impressive achievement to publish a first-authored paper in CVPR as a junior undergraduate student! Check out the Project Website
Jun. 2014 We proposed RIGOR, an algorithm that achieves accuracy slightly better to CPMC/Object proposals, but significantly faster, generating all object proposals in 2-4 seconds on CPU-only code. Check out the Project Page for more information and downloading the code.
Oct. 2013 We exploited semantic segmentation techniques to perform unsupervised video segmentation based on tracking image segments per-frame with 1) automatic object discovery based on segment pool; 2) efficiently learnt appearance models for hundreds of objects with strong appearance features; 3) CSI refinement of objects. Check out Project Webpage, SegTrack v2 Dataset , ICCV paper.
Jun. 2013 We proposed the Composite Statistical Inference (CSI) for inference on real-valued statistics obtained on multiple high-order overlapping variable subsets, with applications in semantic segmentation. Short version, Long version .
Apr. 2013 We proposed an analytic approximation to the chi-square kernel with geometric convergence and derived from elementary methods. It's more straightforward and converges faster over previous approaches. Preprint
Rahul Sawhney (Ph.D. Graduate April. 2018, Georgia Tech Postdoc)
Alrik Firl (M.S. Graduate Jul. 2018, General Electrics)
Xin Li (M.S. Graduate Sept. 2016, PureStorage Inc.)
Zheng Zhou (M.S. Graduate Jun. 2017, Tencent AI Lab)
Winter: CS 535 Deep Learning
Winter: CS 261 Data Structures
Spring: CS 637 Computer Vision II
Wenxuan Wu, Zhongang Qi, LI Fuxin. PointConv: Deep Convolutional Networks on 3D Point Clouds. arXiv:1811.07246.
Neale Ratzlaff, LI Fuxin. Unifying Bilateral Filtering and Adversarial Training for Robust Neural Networks . arXiv:1804.01635.
Chanho Kim, Fuxin Li, James Rehg. Multi-object Tracking with Neural Gating Using Bilinear LSTM. ECCV, 2018.
Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, Fuxin Li. Open Set Learning with Counterfactual Images. ECCV, 2018.
R. Sawhney, F. Li, H. Christensen, C. Isbell. Purely Geometric Scene Association and Retrieval - A case for macro-scale geometry. ICRA, 2018.
R. DeBortoli, A. Nicolai, F. Li, G. Hollinger. Realtime Underwater 3D Reconstruction Using Global Context and Active Labeling. Proceedings of the International Conference on Robotics and Automation (ICRA) 2018.
Zhongang Qi, Fuxin Li. Embedding Deep Networks into Visual Explanations. arXiv:1709.05360. Short (4 pages) version published in NIPS 2017 Workshop on Interpreting, Explaining and Visualizing Deep Learning - Now what?. Code/Data
Peng Lei, Fuxin Li, Sinisa Todorovic. Boundary Flow: A Siamese Network that Predicts Boundary Motion without Training on Motion . CVPR 2018.
Xin Li, Fuxin Li. Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics . ICCV 2017.
Juan Liu, Zhengyang Wu, Fuxin Li. Ranking Video Segments with LSTM and Determinantal Point Processes. ICIP 2017.
Xingyi Li, Fuxin Li, Xiaoli Fern, Raviv Raich. Filter Shaping for Convolutional Networks. ICLR 2017.
Zhaoyang Lv, Chris Beall, Pablo F. Alcantarilla, Fuxin Li, Zsolt Kira, Frank Dellaert. A Continuous Optimization Approach for Efficient and Accurate Scene Flow. ECCV 2016.
Yu-Ying Liu, Shuang Li, Fuxin Li, Le Song, James M. Rehg. Efficient Learning of Continuous-Time Hidden Markov Models for Disease Progression. NIPS 2015.
Chanho Kim, Fuxin Li, Arridhana Ciptadi, James M. Rehg. Multiple Hypothesis Tracking Revisited. ICCV 2015 (Oral Presentation).
Ahmad Humayun, Fuxin Li, James M. Rehg. The Middle Child Problem: Revisiting Parametric Min-cut for Robust Object Proposals. ICCV 2015.
Zhengyang Wu, Fuxin Li, Rahul Sukthankar, James M. Rehg. Robust Video Segment Proposals with Painless Occlusion Handling. CVPR 2015. Project Website
Rahul Sawhney, Fuxin Li, Henrik Christensen. GASP: Geometric Association with Surface Patches. 3DV 2014.
Abhijit Kundu, Yin Li, Frank Dellaert, Fuxin Li, James M. Rehg. Joint Semantic Segmentation and 3D Reconstruction from Monocular Video . ECCV 2014.
Ahmad Humayun, Fuxin Li, James M. Rehg. RIGOR: Reusing Inference in Graph Cuts for Generating Object Regions. CVPR 2014. Project Page + Code
Fuxin Li, Taeyoung Kim, Ahmad Humayun,
David Tsai, James M. Rehg. Video Segmentation by Tracking
Many Figure-Ground Segments. In
Project Webpage, SegTrack v2 Dataset
(There were a few notation typos around Eq. (7) in the official IEEE version, please use the version on this website).
Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning
Contact Locations for Pushing and Orienting Unknown Objects .
Fuxin Li, Guy Lebanon, Christian Sminchisescu. A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG]
Tucker Hermans, Fuxin Li, James M. Rehg, Aaron F. Bobick. Learning
Stable Pushing Locations. In
Fuxin Li, Joao Carreira, Guy Lebanon, Cristian Sminchisescu. Composite
Statistical Inference for Semantic Segmentation. In
Seungyeon Kim, Fuxin Li, Guy Lebanon, Irfan Essa. Beyond
Sentiment: The Manifold of Human Emotions. In
Mingxuan Sun, Fuxin Li, Joonseok
Lee, Ke Zhou, Guy Lebanon, Hongyuan
Multiple-Question Decision Trees for Cold-Start Recommendation. In
Eduards G. Bazavan,
Fuxin Li, Cristian Sminchisescu. Fourier
Kernel Learning. In
Fuxin Li, Guy Lebanon, Cristian Sminchisescu. Chebyshev Approximations to the Histogram
Chi-Square Kernel. In
(Note: This paper is obsolete, please check out our new arXiv (Section 4) which has real geometric convergence rate and better empirical performance: A Linear Approximation to the chi^2 Kernel with Geometric Convergence. arXiv:1206.4074. [cs.LG])
Jaegul Choo, Fuxin Li, Keehyoung Joo, Haesun Park. A Visual Analytics Approach for Protein Disorder Prediction. Expanding the Frontiers of Visual Analytics and Visualization, Springer 2012, pp 163-174.
João Carreira, Fuxin
Li, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground
Catalin Ionescu, Fuxin
Li, Cristian Sminchisescu. Latent
Structured Models for Human Pose Estimation. In
Fuxin Li, Cristian Sminchisescu. Convex Multiple Instance Learning by Estimating Likelihood Ratio, Advances in Neural Processing Systems (NIPS), 2010. Supplementary Material
Fuxin Li, Catalin Ionescu, Cristian Sminchisescu. Random Fourier approximations for skewed multiplicative histogram kernels. In German Association for Pattern Recognition (Deutsche Arbeitsgemeinschaft für Mustererkennung, DAGM), 2010. DAGM prize paper. Code available
Fuxin Li, João Carreira, Cristian Sminchisescu. Object Recognition as Ranking Holistic Figure-Ground Hypotheses. In IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), 2010 (First two authors contributed equally). Per-class accuracies for our VOC 2009 final results (37.24%)
Fuxin Li, Cristian Sminchisescu. The Feature Selection Path in Kernel Methods. In Artificial Intelligence and Statistics (AISTATS), 2010.
Fen Xia, Yanwu Yang, Liang Zhou, Fuxin Li, Min Cai, Daniel D. Zeng: A closed-form reduction of multi-class cost-sensitive learning to weighted multi-class learning. Pattern Recognition 42(7): 1572-1581 (2009).
Fuxin Li, Yunshan Fu, Yu-Hong Dai, Crisitian Sminchisescu, Jue Wang. Kernel Learning by Unconstrained Optimization. In Artificial Intelligence and Statistics (AISTATS), 2009.
Fen Xia, Wensheng Zhang, Fuxin Li, Yanwu Yang. Ranking with Decision Tree. Knowledge and Information Systems. 17(3):381-395 (2008)
Liang Zhou, Fuxin Li, Yanwu Yang. Path Algorithms for One-Class SVM. ISNN (1) 2008: 645-654
Peng Jin, Danqing Zhu, Fuxin Li, Yunfang Wu. PKU: Combining Supervised Classifiers with Features Selection. In Proceedings of the 4th International Workshop on Semantic Evaluations (SemEval-2007), 2007.
Jian Yang, Fuxin Li, Jue Wang. A Better Scaled Local Tangent Space Alignment Algorithm. Proceedings of International Joint Conference on Neural Networks (IJCNN), 2005
Chen Shao, Wei Sun, Fuxin Li, Ruifeng Yang, Ling Zhang, Youhe Gao. Oscore: a combined score to reduce false negative rates for peptide identification in tandem mass spectrometry analysis. Journal of Mass Spectrometry. 2009(14):1, 25-31.
Linjie Wang, Fuxin Li, Wei Sun, Shuzhen Wu, Xiaorong Wang, Ling Zhang, Dexian Zheng, Jue Wang, and Youhe Gao. Concanavalin A-captured Glycoproteins in Healthy Human Urine. Molecular & Cellular Proteomics. 2006(5): 560 - 562
Wei Sun, Fuxin Li, Shuzhen Wu, Xiaorong Wang, Dexian Zheng, Jue Wang, Youhe Gao. Human urine proteome analysis by three separation approaches. Proteomics. 2005(5): 4994-5001
Fuxin Li, Wei Sun, Youhe Gao, Jue Wang. RScore: A Peptide Randomicity Score For Evaluating MS/MS Spectra. Rapid Communications in Mass Spectrometry. 2004(18):14,1655-1659
Wei Sun, Fuxin Li, Jue Wang, Dexian Zheng, Youhe Gao. AMASS: Software for Automatically Validating the Quality of MS/MS Spectrum From SEQUEST Results. Molecular & Cellular Proteomics. 2004(3): 1194-1199