Apb2Face: Audio-Guided Face Reenactment With Auxiliary Pose And Blink Signals

Audio-guided face reenactment aims at generating photorealistic faces using audio information while maintaining the same facial movement as when speaking to a real person. However, existing methods can not generate vivid face images or only reenact low-resolution faces, which limits the application value. To solve those problems, we propose a novel deep neural network named \emph{APB2Face}, which consists of \emph{GeometryPredictor} and \emph{FaceReenactor} modules. \emph{GeometryPredictor} uses extra head pose and blink state signals as well as audio to predict the latent landmark geometry information, while \emph{FaceReenactor} inputs the face landmark image to reenact the photorealistic face. A new dataset $AnnVI$ collected from YouTube is presented to support the approach, and experimental results indicate the superiority of our method than state-of-the-arts, whether in authenticity or controllability.