An improved hybrid regularization approach for extreme learning machine

Abstract

Extreme learning machine (ELM) is a network model that arbitrarily initializes the first hidden layer and can be computed speedily. In order to improve the classification performance of ELM, a $\ell_2$ and $\ell_{0.5}$ regularization ELM model ($\ell_{2}$-$\ell_{0.5}$-ELM) is proposed in this paper. An iterative optimization algorithm of the fixed point contraction mapping is applied to solve the $\ell_{2}$-$\ell_{0.5}$-ELM model. The convergence and sparsity of the proposed method are discussed and analyzed under reasonable assumptions. The performance of the proposed $\ell_{2}$-$\ell_{0.5}$-ELM method is compared with BP, SVM, ELM, $\ell_{0.5}$-ELM, $\ell_{1}$-ELM, $\ell_{2}$-ELM and $\ell_{2}$-$\ell_{1}$ELM, the results show that the prediction accuracy, sparsity, and stability of the $\ell_{2}$-$\ell_{0.5}$-ELM are better than the other $7$ models.