Tutorial - Randomization Based Deep and Shallow Learning Methods for Classification and Forecasting

Ponnuthurai Suganthan,NTU, Singapore ABSTRACT: This tutorial will first introduce the main randomization-based feedforward learning paradigms with closed-form solutions. The popular instantiation of the feedforward neural networks is called random vector functional link neural network (RVFL) originated in the early 1990s. Other feedforward methods included in the tutorials are random weight neural networks (RWNN), extreme learning machines (ELM), Stochastic Configuration Networks (SCN), Broad Learning Systems (BLS), etc. Another randomization-based paradigm is the random forest which exhibits highly competitive performances in batch mode classification. Another paradigm is based on the kernel trick such as kernel ridge regression which includes randomization for scaling to large training data. The tutorial will also consider computational complexity with the increasing scale of the classification/forecasting problems. The tutorial will also present extensive benchmarking studies using classification and forecasting datasets.