Learning with DNNs: Optimizer, Architecture and Formulation

Shuicheng Yan (Sea AI Lab -SAIL), Abstract: Deep Neural Networks (DNNs) have been widely explored from both theory and application perspectives. In this talk, I shall mainly introduce the latest research in the SEA AI Lab along three different lines of learning with DNNs: 1) optimizer, mainly introducing how more general and faster optimizers help improve the DNN model training speed, 2) architecture, mainly introducing how convolution and attention complement each other towards more outstanding DNN structures, and 3) formulation, mainly introducing how new problem formulations can lead to better representations and/or classifiers.