Neural Network Pruning Using Nonlinear Oblique Subspace Projections

The increasing complexity of deep neural networks often leads to overparameterization, limiting their deployment in resource-constrained environments. This paper introduces a novel pruning methodology based on Nonlinear Oblique Subspace Projections (NObSP), an algorithm originally developed to enhance the interpretability of kernel regression models. In this work, NObSP is extended to neural networks to estimate the contribution of individual neurons and entire layers to the network's output. This is achieved by quantifying the nonlinear transformations performed by the network on its inputs and internal representations. By computing the magnitude of these variables, neurons contributing minimally to the network's output are identified. Specifically, neurons with a nonlinear effect magnitude falling below a defined threshold are pruned, effectively removing them from the network. Consequently, NObSP provides a data-driven, interpretability-based approach to neural network pruning, leading to simplified and efficient models with minor performance impact. This method offers a clear framework for understanding which components of a neural network are critical for its function, facilitating the development of more compact and interpretable deep learning models. Finally, the efficacy of NObSP for pruning is demonstrated in CIFAR-10 and MNIST classification problems, where it is compared to traditional pruning algorithms based on the L_n norm.