A Review on Pruning Techniques in Deep Neural Networks with Emphasis on Prune at Initial

With the expansion of the use of neural networks, increase deep of NN, and
increase of network parameters, as well as the limitation of computational
resources, the limitation of memory, and the incomprehensibility of these
networks, the compression of neural networks is necessary. Compression must be
intelligent, so as not to deprive us of the benefits of deep neural networks. Pruning
is one of the compression methods that eliminate unnecessary network parameters.
in recent research, Pruning has always been favored by researchers as far as a step
called pruning at the initial design that pruned the initial network to include the
benefits compression and pruning in the training and inference. this article reviews
pruning techniques in deep neural networks with emphasis on Prune at initializing.
First, the basics of pruning are discussed, then the types of pruning with the
mathematical definition of each discussed, and finally, a more detailed study of
pruning before network training has been done.