UW researchers create realistic video from audio files alone

Researchers from the University of Washington University's Graphics and Image Laboratory have developed a number of software algorithms that allow converting audio recordings into a realistic video, in which a person synchronously moves his lips in accordance with spoken words and phrases. At the heart of the system is a neural network, preliminarily trained on examples of video with human records, which speaks to any topic.

More detailed information on the audio-to-video conversion system will be presented at the conference on computer graphics 2017 SIGGRAPH. And as an example, the researchers prepared very realistic videos, on which former US President Barack Obama discusses terrorism, paternity, creating new jobs, and so on. These videos were built on the basis of existing audio recordings, and for training, the neural network, many hours of recording of public speeches by Barack Obama were used.

At present, the neural network is able to learn to simulate the speech of only one particular person. Nevertheless, the use of artificial intelligence technologies avoids the difficulties arising from the use of traditional conversion technologies, dailytechinfo.org  reports. After all, for the work of traditional technologies, studio recordings of many different people are used, repeating the same phrases, which is done to determine the correlation of individual spoken sounds with the shape, position of the lips and other parts of the person's face.

This technology of converting speech into realistic video can find application in various conferencing systems to improve their work, the researchers say. After all, to transmit only the audio signal requires a much narrower bandwidth of the communication channel, rather than for simultaneous transmission of audio and video. In addition, the new technology can provide real-time communication of people with virtual characters of computer games or historical reconstructions.

The emergence of a system that provides such a realistic result, immediately makes you think about the possibilities in the field of falsification, which it provides. However, the researchers assure, applying approximately the same approach, i.e. "Feeding" the trained neural network video for analysis, you can easily find out what it is, a real video or a movie produced by a computer.

© All rights reserved. Citing to www.ict.az is necessary upon using news