A Novel Pooling Layer Method based on Gaussian Probability Distribution Function with Wavelet transform (GWT) for Convolutional Neural Network.

aqeel mohsin hamad, ammar jasim

Abstract


Convolution is represent the basic layer in the convolutional neural network, but it can results in very big size of the data at its output, so this data need to be reduced or down sampled to reduce the required operation in the network and increase its efficiency by choosing  the most important features of the input signal. Different pooling methods was used to perform down sampling the size of these features. In this paper, we have proposed a novel pooling method by using  Gaussian based  Wavelet Transform and named as (GWT). Gaussian probability distribution function is used to determine the basic and most important statistics of the signal in each pooling size, and depending on this function, the basic statistics with the most priority is determined with their weight ,which are used as coefficients for wavelet filter to extract the pooling features . According to the procedure  of extract the wavelet coefficients filter, Three method are proposed ,the first method (GWT1 ) is used the normalized values of basic statistics as weight to be multiplied by original signal, the second method (GWT2) used the determined statistics as features of the original signal and multiply it with constant weights based on half Gaussian ,while the third method  (GWT3) is work in similar way to (GWT1) except that, it depend on entire signal instead of every pool size for calculation the basic statistics. Also the proposed methods are combined with other standard methods such as max and pooling. The experiments are performed on different types of dataset , two dimension (images) and one dimension (such as ECG signal), the results are compared with other methods and show that the proposed methods perform or outperform the other methods and can increase the performance of the (CNN).


Keywords


CNN, GWT1, GWT2, GWT3 and Gaussian

References


K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR,

pages 770–778, 2016.

[A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and

H. Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications.

arXiv, 1704.04861, 2017.

M. Riesenhuber and T. Poggio. Just one view: Invariances in inferotemporal cell tuning. InNIPS, pages 215–221, 1998.

M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. NatureNeuroscience, 2(11):1019–1025, 1999.

T. Serre and T. Poggio. A neuromorphic approach to computer vision. Communications of the

ACM, 53(10):54–61, 2010.

T. Williams and R. Li. Wavelet pooling for convolutional neural networks. In ICLR, 2018.

F. Saeedan, N. Weber, M. Goesele, and S. Roth. Detail-preserving pooling in deep networks. In CVPR, pages 9108–9116, 2018.

C.-Y. Lee, P. W. Gallagher, and Z. Tu. Generalizing pooling functions in convolutional neural

networks: Mixed, gated, and tree. In AISTATS, pages 464–472, 2016.

D. Yu, H. Wang, P. Chen, and Z. Wei. Mixed pooling for convolutional neural networks. In

RSKT, pages 364–375, 2014.

M. Zeiler and R. Fergus. Stochastic pooling for regularization of deep convolutional neural

networks. In ICLR, 2013.

Takumi Kobayashi. Gaussian-Based Pooling for Convolutional Neural Networks, 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.

C.-Y. Lee, P. W. Gallagher, and Z. Tu. Generalizing pooling functions in convolutional neural

networks: Mixed, gated, and tree. In AISTATS, pages 464–472, 2016.

P. Liu, H. Zhang, W. Lian, and W. Zuo. Multi-level wavelet convolutional neural networks. CoRR, abs/1907.03128,2019.

Ajay Kumar Boyat and Brijendra Kumar Joshi., Areviw paper: Noise models in digital image processing, , Signal & Image Processing : An International Journal (SIPIJ) Vol.6, No.2, April 2015.

Yaakov Bar-Shalom, X. Rong Li, Thiagalingam Kirubarajan, ”Estimation with Applications to Tracking and Navigation,” John Wiley & Sons, 2001.




DOI: http://doi.org/10.11591/ijeecs.v20.i3.pp%25p
Total views : 16 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

shopify stats IJEECS visitor statistics