FEATURE LEVEL FUSION FRAMEWORK FOR BRAIN MR IMAGE CLASSIFICATION USING SUPERVISED DEEP LEARNING AND HAND CRAFTED FEATURES

(Received: 16-Jun.-2022, Revised: 18-Aug.-2022 , Accepted: 12-Sep.-2022)

Authors Prashantha S. J., H. N. Prakash,

Keywords #Deep-learning features #Handcrafted features #Canonical-correlation analysis #Discriminant-correlation analysis #Support vector machines

Abstract In this paper, we propose an efficient fusion framework for brain magnetic resonance (MR) image classification using deep learning and handcrafted feature extraction methods; namely, histogram of oriented gradients (HOG) and local binary patterns (LBPs). The proposed framework aims to: (1) determine the optimal handcrafted features by Genetic Algorithm (GA) (2) discover the fully connected (FC) layers’ features using fine-tuned convolutional neural network (CNN) (3) employ the canonical correlation analysis (CCA) and the discriminant correlation analysis (DCA) methods in feature-level fusion. Extensive experiments were conducted and the classification performance was demonstrated on three benchmark datasets; viz., RD-DB1, TCIA-IXI- DB2 and TWB-HM-DB3. Mean accuracy of 68.69%, 90.35% and 93.15% from CCA and 77.22%, 100.00% and 99.40% from DCA was achieved by the Support Vector Machines (SVM) sigmoid kernel classifier on RD-DB1, TCIA-IXI-DB2 and TWB-HM-DB3, respectively. The obtained results of the proposed framework outperform when compared with other state-of-the-art works.

References

[1] P. Saxena, A. Maheshwri, S. Tayal and S. Maheshwari, "Predictive Modeling of Brain Tumor : A Deep Learning Approach," Innovations in Computational Intelligence and Computer Vision, Part of the Advances in Intelligent Systems and Computing Book Series, vol. 1189, pp. 275–285, 2019.

[2] H. M. Rai and K. Chatterjee, "Detection of Brain Abnormality by a Novel Lu-Net Deep Neural CNN Model from MR Images," Machine Learning with Applications, vol. 2, p. 100004, 2020, DOI: 10.1016/j.mlwa.2020.100004, 2020.

[3] A. Kharrat, K. Gasmi and M. B. E. N. Messaoud, "A Hybrid Approach for Automatic Classification of Brain MRI Using Genetic Algorithm and Support Vector Machine," Leonardo Journal of Sciences, vol. 17, pp. 71–82, 2010.

[4] Z. Ullah, M. Umar, S. Lee and D. An, "A Hybrid Image Enhancement Based Brain MRI Images Classification Technique," Medical Hypotheses, vol. 143, no. May, p. 109922, 2020.

[5] B. Ural, "A Computer‑based Brain Tumor Detection Approach with Advanced Image Processing and Probabilistic Neural Network Methods," J. of Medical Biological Eng., vol. 38, pp. 867–879, 2018.

[6] D. J. Hemanth, J. Anitha, A. Naaji, O. Geman, D. E. Popescu and L. H. Son, "A Modified Deep Convolutional Neural Network for Abnormal Brain Image Classification," IEEE Access, vol. 7, pp. 4275-4283, DOI: 10.1109/ACCESS.2018.2885639, 2018.

[7] T. J. Alhindi, S. Kalra, K. H. Ng, A. Afrin and H. R. Tizhoosh, "Comparing LBP , HOG and Deep Features for Classification of Histopathology Images," Proc. of the 2018 IEEE Int. Joint Conf. on Neural Networks (IJCNN), DOI: 10.1109/IJCNN.2018.8489329, Rio de Janeiro, Brazil, 2018.

[8] P. K. Sethy and S. K. Behera, "A Data Constrained Approach for Brain Tumour Detection Using Fused Deep Features and SVM," Multimedia Tools and Applications, vol. 80, no. 19, pp. 28745–28760, 2021.

[9] B. Athiwaratkun and K. Kang, "Feature Representation in Convolutional Neural Networks," arXiv: 1507.02313, pp. 6–11, 2015.

[10] N. Tajbakhsh et al., "Convolutional Neural Networks for Medical Image Analysis: Fine Tuning or Full Training ?" IEEE Trans. on Medical Imaging, vol. 35, no. 5, pp. 1299–1312, 2016.

[11] Z. Zhou, J. Shin, L. Zhang, S. Gurudu, M. Gotway and J. Liang, "Fine-tuning Convolutional Neural Networks for Biomedical Image Analysis," Proc. of the 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 4761–4772, DOI: 10.1109/CVPR.2017.506, Honolulu, USA, 2017.

[12] S. Maheshwaria, V. Kanhangada, R. Pachoria, B. Sulatha V. and U. R. Acharyac, "Automated Glaucoma Diagnosis Using Bit-plane Slicing and Local Binary Pattern Techniques," Computers in Biology Medicine, vol. 105, pp. 72–80, 2019.

[13] S. Oreski and G. Oreski, "Expert Systems with Applications Genetic Algorithm-based Heuristic for Feature Selection in Credit Risk Assessment," Expert Systems with Applications, vol. 41, no. 4-2, pp. 2052-2064, 2013.

[14] S. Li, X. Kang, L. Fang, J. Hu and H. Yin, "Pixel-level Image Fusion: A Survey of the State of the art," Information Fusion, vol. 33, pp. 100–112, DOI: 10.1016/j.inffus.2016.05.004, 2016.

[15] J. Yang, J. Y. Yang, D. Zhang and J. F. Lu, "Feature Fusion: Parallel Strategy vs. Serial Strategy," Pattern Recognition, vol. 36, no. 6, pp. 1369–1381, 2003.

[16] M. Haghighat, M. Abdel-Mottaleb and W. Alhalabi, "Discriminant Correlation Analysis: Real-timeFeature Level Fusion for Multimodal Biometric," IEEE Transactions on Information Forensics and Security, vol. 11, no. 9, pp. 1984 - 1996, 2016.

[17] S. Roheda and H. Krim, "Decision Level Fusion : An Event Driven Approach," Proc. of the 26th IEEE European Signal Processing Conf. (EUSIPCO), vol. 9560, pp. 2598–2602, Rome, Italy, 2018.

[18] I. Khoulqi and N. Idrissi, "Cervical Cancer Detection and Classification Using MRIs," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 08, no. 02, pp. 141-158, DOI: 10.5455/jjcit.71-1640595124, June 2022.

[19] D. Azzouz and S. Mazouzi, "A Hyper –surface –based Modelling and Correction of Bias Field in MR Images," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 07, no. 03, pp. 223-238, DOI: 10.5455/jjcit.71-1617051919, Sep. 2021.

[20] H. Sultan, N. Salem and W. Al-Atabany, "Multi-classification of Brain Tumor Images Using Deep Neural Network," IEEE Access, vol. 7, pp. 69215–69225, DOI:10.1109/ACCESS.2019.2919122, 2019.

[21] D. Chen, K. C. Chan and X. Wu, "Gene Expression Analysis Using Genetic Algorithm Based Hybrid Approaches," Proc. of the IEEE Congress on Evolutionary Computation, pp. 963–969, Hong Kong, china, 2008.

[22] Radiopaedia, "Cases," [Online], Available: https://radiopaedia.org/cases/," Accessed on 26 June 2020.

[23] National Cancer Institute, "Cancer Imaging Archieve," [Online], Available: https://www.cancerimagingarchive.net/, Accessed on 30 June 2020.

[24] Biomedical Image Analysis Group, "IXI Dataset," [Online], Available: http://brain-development.org.ixi-dataset, Accessed on 15 June 2020.

[25] K. A. Johnson and J. Alex Becker, "The Whole Brain," Atlas, [Online], Available: http://www.med.harvard.edu/AANLIB/, Accessed on 18 June 2021.

[26] Wikipedia, "Confusion Matrix," [Online], Available: http://en.wikipedia.org/wiki/Confusion_matrix, Accessed on 20 Aug 2021.