A NEURAL PROSODY ENCODER FOR END-TO-END DIALOGUE ACT CLASSIFICATION

Dialogue act classification (DAC) is a critical task for spoken language understanding in dialogue systems. Prosodic features such as pitch have been shown to be useful for DAC. Despite their importance,little research has explored neural approaches to integrate prosodic features into end-to-end (E2E) DAC models which infer dialogue acts directly from audio signals. In this work, we propose an E2E neural architecture that takes into account this need of characterizing prosodic phenomena co-occurring at different levels inside an utterance. A novel part of this architecture is a learnable gating mechanism that assesses the importance of prosodic features and selectively retains core information necessary for E2E DAC. Our proposed model improves the dialogue act accuracy by 1.07% absolute across three publicly available benchmark datasets.