Realistic Augmentation For Effective 2D Human Pose Estimation Under Occlusion

Occlusion is a major challenge for effective human pose estimation, occurring naturally in a high percentage of real-world images. Handling occlusion has been a difficult challenge in literature due to a lack of a proper dataset with an actual focus on occlusion, prompting researchers to create artificial datasets as a means of data augmentation. However, all of these datasets lack the features of a real-world occlusion. In this work, we introduce a new realistic data augmentation approach built on top of a base dataset (here the Human3.6m) that tackles this issue, creating realistic samples similar to those found in the wild. Arguing that CNN models pay higher attention to local as opposed to global features, we define occlusion levels, select many to-occlude objects from different categories, and blend those within the original image from the base dataset. We, then, test top-performing 2D human pose estimation models with and without this occlusion-augmented dataset (called RealOcc) to display the drop in performance under occlusion and then train them on the new dataset to show the increase in the accuracy of the model under occlusion, without any change to the models themselves.