Pose-Guided And Style-Transferred Face Reenactment

Unlike most face reenactment that deals with minor pose variation, we propose the Pose-Guided Reenactment (PGR) framework for large-pose face reenactment. The proposed PGR is composed of a landmark detector, a landmark encoder, a face encoder, a landmark decoder, a style encoder, a face generator and two discriminators. The training of these components is divided into two phases. In Phase I, the landmark encoder is trained to encode the landmarks of the reference face to a reference landmark code, and the face encoder is trained to encode the source face to a source face code. The reference landmark code and the source face code are entered to the landmark decoder, which generates a target landmark set. In Phase II, the face generator is trained to generate the target face with the desired identity, pose and expression, given the target landmark set and the source face as input. To handle large pose, we include the large pose data in the training set and propose the pose-guided landmark switch to control the change of the facial landmarks during pose variation. Experiments on MPIE and VoxCeleb1 benchmark databases show that the proposed PGR can effectively reenact the faces with large pose, and delivers a state-of-the-art overall performance compared with other approaches.