David Harwath (https://saltlab.cs.utexas.edu/)

The goal of this project is to develop machine learning models capable of automatically detecting when the audio and visual streams of a live video broadcast become temporarily out-of-sync with one another. This type of synchronization error is commonly encountered in the real world and has a negative impact on the viewer’s experience. Models that can rapidly detect when these errors arise are essential tools in helping technicians correct the problem. The new benchmark dataset for this task is comprised of real-world sports and news broadcasts, as well as the development of novel deep neural network architectures for audio-visual asynchronization detection.