Compression of Convolutional Neural Networks Based on Kernel Redundancy

When: Friday, February 15, 2019
Where: PGH 232
Time: 11:00 AM

Speaker: Dr. Wen-Hung Liao, National Chengchi University (Taiwan)

Host: Dr. Shishir Shah

The model size and floating-point operations (FLOP) required by convolutional neural networks make it difficult to deploy these models to mobile devices or embedded systems. In this talk, we discuss a method known as the diversifying algorithm to compress CNN models. The key concept is to maintain the diversity of convolutional kernels by preserving the most representative filters in each network layer. This is achieved by expressing the network architecture as an undirected graph. The nodes in the graph denote the filters, and the weights are computed using cosine distance. Nodes are removed by considering the combined effects of several factors, including edge weights, the sum of similarity and the sum of filter weights. The compressed network is then retrained to retain accuracy with fewer model parameters and FLOPs.

We test the efficacy of the proposed diversifying algorithm on three types of CNN models, including VGG, ResNet and DenseNet using both CIFAR-10 and CIFAR-100 datasets through extensive experiments. On CIFAR-10 dataset, the proposed method is able to reduce 78.6% of total parameters and nearly 46% FLOPs in VGG16. If 1% performance loss is allowed, we can achieve 90.7% parameter and 70% FLOP reduction. On CIFAR-100 dataset, we can reduce 46% parameter and 18% FLOP. Furthermore, we can achieve 60.7% parameter and nearly 37.5% FLOP reduction if 1% accuracy loss is allowed.

Bio:

Wen-Hung Liao received his M.S. and Ph.D. in 1991 and 1996, respectively, from the Department of Electrical and Computer Engineering, at University of Texas at Austin. He joined National Chengchi University (NCCU) in Taiwan since 2000 and is currently an associate professor at the Department of Computer Science. Dr. Liao was the deputy director of the University Library at NCCU from 2012-2014. He served as the chairman of the Computer Science Department from 2014-2017. His research interests include computer vision, pattern recognition, and human-computer interaction.