Nvidia Proposes A Neural Talking-Head Video Synthesis AI Model, Making Video Conferencing 10x More Bandwidth Efficient

NVIDIA researchers introduce an AI system that generates a realistic talking-head video of a person using one source image and a driving video. The source image encodes an individual’s appearance, and the driving video directs motions in the resulting video.

The researchers have proposed a pure neural rendering approach in which a talking-head video is rendered using a deep network in a one-shot setting without using a 3D human head’s graphics model. When compared to 3D graphics-based models, 2D based methods have various advantages such as below:

1. It avoids 3D model acquisition, which is usually difficult and costly.
2. 2D-based techniques can adequately synthesize hair, beard, etc. In contrast, it is challenging to acquire accurate 3D geometries of these regions.
3. They can directly synthesize accessories in source images like eyeglasses, hats, and scarves without their 3D models.

Full Article: https://www.marktechpost.com/2020/12/05/nvidia-proposes-a-neural-talking-head-video-synthesis-ai-model-making-video-conferencing-10x-more-bandwidth-efficient/

Paper: https://arxiv.org/pdf/2011.15126.pdf

Github: https://nvlabs.github.io/face-vid2vid/

​

https://i.redd.it/xnftmsdulf361.gif