Facial animation

This is work I did at Microsoft with Brian Guenter. It was an extension of Lance William's 2D facial animation capture system to 3D. The system used 5 calibrated cameras to track points on the face. The 3D trajectories of these points were then reconstructed by triangulation (complicated a bit by the fact that the dots tended to change color and occasionally disappear). These dot movements then drove a 3D mesh animation. Finally, the original images were back-projected onto the mesh, the dots removed (another headache) and combined to form the final texture map.

I never continued with this work because they would not release the data set to me when I left Microsoft, and I did not have the resources (or desire) to reconstruct the capture setup.

Papers

Video