Training A Bank Of Wiener Models With A Novel Quadratic Mutual Information Cost Function

This paper presents a novel training methodology to adapt parameters of a bank of Wiener models (BWMs), i.e., a bank of linear filters followed by a static memoryless nonlinearity, using full pdf information of the projected outputs and the desired signal. BWMs also share the same architecture with the first layer of a time-delay neural networks (TDNN) with a single hidden layer, which is often trained with backpropagation. To optimize BWMs, we develop a novel cost function called the empirical embedding of quadratic mutual information (E-QMI) that is metric-driven and efficient in characterizing the statistical dependency. We demonstrate experimentally that by applying this cost function to the proposed model, our method is comparable with state-of-the-art neural network architectures for regressions tasks without using backpropagation of the error.