Learning Task-aligned Mask Query for Instance Segmentation

Recently, query-based instance segmentation methods have achieved comparable performance to previous state-of-the-art methods. However, the query lacks the learning of the consistency between classification and segmentation tasks, which may lead to misalignment between classification score and mask quality (i.e., mask IoU) and can not result in a reliable ranking for predictions. In this work, we propose a novel instance segmentation method, termed AlignMask, which effectively learns task-aligned mask queries for instance end-to-end. Specifically, we propose Aligned Query Learning (AQL) to learn task-aligned features for pixel embedding and transformer decoder, which helps segmentation quality estimation of the mask query. We also use Aligned Label Assignment to explicitly align the optimization goals for classification score and mask quality of the query. Extensive experiments on MS-COCO show that our proposed AlignMask achieves competitive performance with state-of-the-art models.