Embarrassingly parallel autoconstructive multilayer perceptron neural networks

Abstract

The present thesis proposes a method to automatically construct Multilayer Per-ceptron Artificial Neural Networks (MLP) to help non-expert users to still create robust models without the need to worry about the best combination of the number of neurons and activation functions by using specific splitting strategies, training parallelization, and multi-criteria model selection techniques. In order to do that, a data splitting algorithm (Similarity Based Stratified Splitting) was developed to produce statistically similar splits in order to better explore the feature space and consequently train better models. These splits are used to independently train several MLPs with different architectures in parallel (ParallelMLPs), using a modified matrix multiplication that takes advantage of the principle of locality to speed up the training of these networks from 1 to 4 orders of magnitude in CPUs and GPUs, when compared to the sequential training of the same models. It allowed the evaluation of several architectures for the MLPs in a very short time to produce a pool with a considerable amount of complex models. Furthermore, we were able to analyze and propose optimality conditions of theoretical optimal models and use them to automatically define MLP architectures by performing a multi-criteria model selection, since choosing a single model from an immense pool is not a trivial task. The code will be available at <https://github.com/fariasfc/parallel-mlps>.