NEWS

WEIGHTED GREY WOLF OPTIMIZER WITH IMPROVED CONVERGENCE RATE IN TRAINING MULTI-LAYER PERCEPTRON TO SOLVE CLASSIFICATION PROBLEMS


(Received: 18-May-2021, Revised: 2-Aug.-2021 , Accepted: 12-Aug.-2021)
The Grey Wolf Optimizer (GWO) is a very recently developed and emerging swarm-intelligent algorithm. The GWO algorithm was inspired by the social dominance hierarchy and hunting strategy of the grey wolves that has been successfully tailored to tackle various discrete and continuous optimization problems. During its practical implementation, however, it may be stuck in sub-optimal solutions (stagnation in local optima) due to its less exploration in the early stages that show the main drawback of this algorithm. Therefore, this research work enhances the hunting and attacking mechanism in order to modify the corresponding position updated equation and exploitation equation, respectively, to propose a novel algorithm, called Weighted Grey Wolf Optimizer with Improved Convergence Rate (WGWOIC). The effectiveness of the proposed algorithm (WGWOIC) is investigated by testing it an 33 different and fairly popular numerical benchmark functions. Although, these test functions are considered from two different benchmark datasets to assess the strength and robustness of the proposed algorithm regarding the unknown search space of the problem. In order to carry out performance analysis, moreover, the WGWOIC’s results are compared against many other state-of-the-art meta-heuristic algorithms, such as Particle Swarm Optimization (PSO), Moth-Flame Optimization (MFO), Whale Optimization Algorithm (WOA), Grey Wolf Optimizer (GWO) and very recent variants of GWO. The comparative study for WGWOIC concludes that the proposed algorithm provides very competitive results against other studied meta-heuristic algorithms. Furthermore, the hybridization of the WGWOIC meta-heuristic optimization algorithm with a Multi- Layer Perceptron (MLP) neural network is employed to improve the accuracy of the classification problem. WGWOIC trainer provides the optimal values for weight and biases to the MLP network. Further, the performance is tested in terms of classification accuracy on five popular classification datasets and assesses the efficiency of the WGWOIC trainer is assessed against many other meta-heuristics trainers. The results show that the proposed algorithm eventually provides very competitive outcomes, implying that the WGWOIC algorithm offers a better exploitation, explores the search space and effectively solves several different classification problems.

[1] D. Wolpert and W. Macready, "No Free Lunch Theorems for Optimization," IEEE Trans. on Evolutionary Computation, vol. 1, no. 1, pp. 67-82, April 1997.

[2] J. Holland, "Genetic Algorithms," Scientific American, vol. 267, no. 1, pp. 66-73, July 1992.

[3] K. Krishnakumar and D. E. Goldberg, "Control System Optimization Using Genetic Algorithms," Journal of Guidance, Control and Dynamics, vol. 15, no. 3, pp. 735-740, 1992.

[4] R. Storn and K. Price, "Differential Evolution: A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces," J. of Global Optimization, vol. 11, no. 4, pp. 341-359, 1997.

[5] D. Simon, "Biogeography-based Optimization," IEEE Trans. on Evolutionary Computation, vol. 12, no. 6, pp. 702-713, March 2008.

[6] X. Yao, Y. Liu and G. Lin, "Evolutionary Programming Made Faster," IEEE Trans. on Evolutionary Computation, Vol. 3, no. 2, pp. 82-102, July 1999.

[7] J. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1, MIT Press, 1992.

[8] N. Hansen, S. Müller and P. Koumoutsakos, "Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES)," Evolutionary Computation, Vol. 11, no. 1, pp. 1- 18, March 2003.

[9] X. Yao and Y. Liu, "Fast Evolutionary Programming," Evolutionary Programming, vol. 3, pp. 451-460, Feb. 1996.

[10] S. Hofmeyr and S. Forrest, "Architecture for an Artificial Immune System," Evolutionary Computation, vol. 8, no. 4, pp. 443-473, December 2000.

[11] K. Passino, "Bacterial Foraging Optimization," International Journal of Swarm Intelligence Research (IJSIR), vol. 1, no. 1, pp. 1-16, January 2010.

[12] E. Rashedi, H. Nezamabadi-Pour and S. Saryazdi, "GSA: A Gravitational Search Algorithm," Information Sciences, vol. 179, no. 13, pp. 2232-2248, June 2009.

[13] B. Webster and P. Bernhard, "A Local Search Optimization Algorithm Based on Natural Principles of Gravitation," Proc. of the 2003 International Conf. on Information and Knowledge Engineering (IKE’03), pp. 255–261, Las Vegas, Nevada, USA, April 2003.

[14] A. Hatamlou, "Black Hole: A New Heuristic Optimization Approach for Data Clustering," Information Sciences, vol. 222, pp. 175-184, February 2013.

[15] F. Moghaddam, R. Moghaddam and M. Cheriet, "Curved Space Optimization: A Random Search Based on General Relativity Theory," arXiv Preprint arXiv: 1208.2214, August 2012.

[16] X. Yang, "A New Metaheuristic Bat-inspired Algorithm," Nature Inspired Cooperative Strategies for Optimization (NICSO 2010), Part of Studies in Computational Intelligence Book Series, vol. 284, pp. 65-74, Springer, Berlin, Heidelberg, 2010.

[17] S. Mirjalili, "Moth-flame Optimization Algorithm: A Novel Nature-inspired Heuristic Paradigm," Knowledge- based Systems, vol. 89, pp. 228-249, November 2015.

[18] B. Mohanty, "Performance Analysis of Moth Flame Optimization Algorithm for AGC System," International Journal of Modeling and Simulation, vol. 39, no. 2, pp. 73-87, April 2019.

[19] D. Pelusi, R. Mascella, L. Tallini, J. Nayak et al., "An Improved Moth-flame Optimization Algorithm with Hybrid Search Phase," Knowledge-based Systems, vol. 191, ID: 105277, 2020.

[20] P. Singh and SK. Bishnoi, "Modified Moth-flame Optimization for Strategic Integration of Fuel Cell in Renewable Active Distribution Network," Electric Power Systems Research, vol. 197, Article ID: 107323, 2021.

[21] S. Mirjalili and A. Lewis, "The Whale Optimization Algorithm," Advances in Engineering Software, vol. 95, pp. 51-67, May 2016.

[22] B. H. Abed-alguni, "Bat Q-learning Algorithm," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 03, no. 01, pp. 52-71, DOI: 10.5455/jjcit.71-1480540385, April 2017.

[23] E. Cuevas, A. Echavarría and M. Ramírez-Ortegón, "An Optimization Algorithm Inspired by the States of Matter that Improves the Balance between Exploration and Exploitation," Applied Intelligence, vol. 40, no. 2, pp. 256-272, March 2014.

[24] X. Yang, "Flower Pollination Algorithm for Global Optimization," Proc. of International Conf. on Unconventional Computing and Natural Computation (UCNC 2012), Part of the Lecture Notes in Computer Science Book Series, vol. 7445, pp. 240-249, Springer, Berlin, Heidelberg, September 2012.

[25] A. G. Hussien, D. Oliva, E. Houssein, A. Juan and X. Yu, "Binary Whale Optimization Algorithm for Dimensionality Reduction," Mathematics, vol. 8, no. 10, 1821, October 2020.

[26] AK. Tripathi, H. Mittal, P. Saxena and S. Gupta, "A New Recommendation System Using Map-reduce-based Tournament Empowered Whale Optimization Algorithm," Complex & Intelligent Systems, vol. 7, no. 1, pp. 297-309, February 2021.

[27] J. Kennedy and R. Eberhart, "Particle Swarm Optimization," Proceedings of the IEEE International Conference on Neural Networks (ICNN'95), vol. 4, pp. 1942-1948, November 1995.

[28] K. Parsopoulos and M. Vrahatis, "UPSO: A Unified Particle Swarm Optimization Scheme," Proc. of the International Conference of Computational Methods in Sciences and Engineering (ICCMSE 2004), CRC Press, pp. 868-873, April 2019.

[29] A. Piotrowski, J. Napiorkowski and A. E. Piotrowska, "Population Size in Particle Swarm Optimization," Swarm and Evolutionary Computation, vol. 58, Article ID: 100718, November 2020.

[30] N. S. Alshdaifat and M. H. Bataineh, "Optimizing and Thinning Planar Array Using Chebyshev Distribution and Improved Particle Swarm Optimization," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 01, no. 01, pp. 31-41, December 2015.

[31] S. Parsons, "Ant Colony Optimization by Marco Dorigo and Thomas Stützle, MIT Press, ISBN 0-262-04219-3," The Knowledge Engineering Review, vol. 20, no. 1, pp. 92, 2005.

[32] XS. Yang, "Firefly Algorithm, Stochastic Test Functions and Design Optimization," International Journal of Bio-inspired Computation, vol. 2, no. 2, pp. 78-84, January 2010.

[33] X. Yang and S. Deb, "Cuckoo Search via Lévy Flights," Proc. of IEEE 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC), pp. 210-214, Coimbatore, India, December 2009.

[34] S. Mirjalili, S. Mirjalili S and A. Lewis, "Grey Wolf Optimizer," Advances in Engineering Software, vol. 69, pp. 46-61, March 2014.

[35] N. Mittal, U. Singh and B. Sohi, "Modified Grey Wolf Optimizer for Global Engineering Optimization," Applied Computational Intelligence and Soft Computing, vol. 2016, Article ID: 7950348, March 2016.

[36] N. Singh, "A Modified Variant of Grey Wolf Optimizer," International Journal of Science & Technology, Scientia Iranica, DOI: 10.24200/SCI.2018.50122.1523, 2018.

[37] N. Singh and S. Singh, "A Modified Mean Gray Wolf Optimization Approach for Benchmark and Biomedical Problems," Evolutionary Bioinformatics, vol. 13, DOI: 10.1177/1176934317729413, 2017.

[38] A. Kumar, A. Singh and A. Kumar, "Weighted Mean Variant with Exponential Decay Function of Grey Wolf Optimizer on Applications of Classification and Function Approximation Dataset," Proc. of the International Conference on Hybrid Intelligent Systems, Springer, Cham, pp. 277-290, December 2019.

[39] B. H. Abed-alguni and M. Barhoush, "Distributed Grey Wolf Optimizer for Numerical Optimization Problems," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 04, no. 03, pp. 1-20, DOI: 10.5455/jjcit.71-1532897697, December 2018.

[40] K. Price, N. Awad, M. Ali and P. Suganthan, "The 100-digit Challenge: Problem Definitions and Evaluation Criteria for the 100-digit Challenge Special Session and Competition on Single Objective Numerical Optimization," Technical Report, Nanyang Technological University, November 2018.

[41] M. Abdullah and T. Ahmed, "Fitness Dependent Optimizer Inspired by the Bee Swarming Reproductive Process," IEEE Access, vol. 7, pp. 43473-43486, March 2019.

[42] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, 2nd Edn., Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2, pp. 111–114, 2003.

[43] W. McCulloch and W. Pitts, "A Logical Calculus of the Ideas Immanent in Nervous Activity," The Bulletin of Mathematical Biophysics, vol. 5, no. 4, pp. 115-133, December 1943.

[44] G. Dorffner, "Neural Networks for Time Series Processing," Neural Network World, vol. 6, pp.447-468, 1996.

[45] G. Bebis and M. Georgiopoulos, "Feed-forward Neural Networks," IEEE Potentials, vol. 13, no. 4, pp. 27- 31, October 1994.

[46] P. Auer, H. Burgsteiner and W. Maass, "A Learning Rule for Very Simple Universal Approximators Consisting of a Single Layer of Perceptrons," Neural Networks, vol. 21, no. 5, pp. 786-795, June 2008.

[47] P. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. Dissertation, Harvard University, 1974.

[48] P. Melin, D. Sánchez and O. Castillo, "Genetic Optimization of Modular Neural Networks with Fuzzy Response Integration for Human Recognition," Information Sciences, vol. 197, pp. 1-19, August 2012.

[49] W. Gardner and S. Dorling, "Artificial Neural Networks (the Multilayer Perceptron): A Review of Applications in the Atmospheric Sciences," Atmospheric Environment, vol. 32, no. (14-15), pp. 2627-2636, August 1998.

[50] X. Yu, J. Yang and Z. Xie, "Training SVMs on a Bound Vectors Set Based on Fisher Projection," Frontiers of Computer Science, vol. 8, no. 5, pp. 793-806, October 2014.

[51] X. Yu, Y. Chu, F. Jiang, Y. Guo and D. Gong, "SVMs Classification Based Two-side Cross Domain Collaborative Filtering by Inferring Intrinsic User and Item Features," Knowledge-based Systems, vol. 141, pp. 80-91, February 2018.

[52] X. Yu, F. Jiang, J. Du and D. Gong, "A Cross-domain Collaborative Filtering Algorithm with Expanding User and Item Features via the Latent Factor Space of Auxiliary Domains," Pattern Recognition, vol. 94, pp. 96-109, October 2019.

[53] V. V. Kolisetty and D. S. Rajput, "A Review on the Significance of Machine Learning for Data Analysis in Big Data," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 06, no. 01, pp. 155- 171, DOI: 10.5455/jjcit.71-1564729835, March 2020.

[54] M. A. Ottom, "Big Data in Healthcare: Review and Open Research Issues," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 03, no. 01, pp. 38-51, DOI: 10.5455/jjcit.71-1476816159, April 2017.

[55] S.-H. Liew, Y.-H. Choo and Y. F. Low, "Fuzzy-rough Classification for Brainprint Authentication," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 05, no. 02, pp. 52-71, DOI: 10.5455/jjcit.71-1556703387, August 2019.

[56] K. M.O. Nahar, A. Jaradat, M. S. Atoum and F. Ibrahim, "Sentiment Analysis and Classification of Arab Jordanian Facebook Comments for Jordanian Telecom Companies Using Lexicon-based Approach and Machine Learning," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 06, no. 03, pp. 52-71, DOI: 10.5455/jjcit.71-1586289399, Sep. 2020.

[57] R. Z. Al-Abdallah, A. S. Jaradat, I. Abu Doush and Y. A. Jaradat, "A Binary Classifier Based on Firefly Algorithm," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 03, no. 03, pp. 32- 46, DOI: 10.5455/jjcit.71-1501152301, December 2017.

[58] M. Alweshah, L. Rababa, M. H. Ryalat, A. Al Momani and M. F. Ababneh, "African Buffalo Algorithm: Training the Probabilistic Neural Network to Solve Classification Problems," Journal of King Saud University - Computer and Information Sciences, DOI: 10.1016/j.jksuci.2020.07.004, 2020.

[59] M. Alweshah, E. Ramadan, M. H. Ryalat, M. Almi'ani and A. I. Hammouri, "Water Evaporation Algorithm with Probabilistic Neural Network for Solving Classification Problems," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 6, no. 1, pp. 1-14, March 2020.

[60] S. Tang, M. Li, F. Wang, Y. He and W. Tao, "Fouling Potential Prediction and Multi-objective Optimization of a Flue Gas Heat Exchanger Using Neural Networks and Genetic Algorithms," International Journal of Heat and Mass Transfer, vol. 152, Article ID: 119488, May 2020.

[61] M. F. Ab Aziz, S. A. Mostafa, C. F. M. Foozy, M. A. Mohammed, M. Elhoseny and A. Z. Abualkishik, "Integrating Elman Recurrent Neural Network with Particle Swarm Optimization Algorithms for an Improved Hybrid Training of Multidisciplinary Datasets," Expert Systems with Applications, vol. 183, p. 115441, June 2021.

[62] F. E. Fernandes Jr and G. G. Yen, "Pruning of Generative Adversarial Neural Networks for Medical Imaging Diagnostics with Evolution Strategy," Information Sciences, vol. 558, pp. 91-102, May 2021.

[63] A. Zannou and A. Boulaalam, "Relevant Node Discovery and Selection Approach for the Internet of Things Based on Neural Networks and Ant Colony Optimization," Pervasive and Mobile Computing, vol. 70, Article ID: 101311, January 2021.

[64] H. Zhang, H. Nguyen, X. Bui et al., "Developing a Novel Artificial Intelligence Model to Estimate the Capital Cost of Mining Projects Using Deep Neural Network-based Ant Colony Optimization Algorithm," Resources Policy, vol. 66, Article ID: 101604, June 2020.

[65] S. Mirjalili, "How Effective Is the Grey Wolf Optimizer in Training Multi-layer Perceptrons?" Applied Intelligence, vol. 43, no. 1, pp. 150-161, July 2015.

[66] H. Faris, S. Mirjalili and I. Aljarah, "Automatic Selection of Hidden Neurons and Weights in Neural Networks Using Ggrey Wolf Optimizer Based on a Hybrid Encoding Scheme," International Journal of Machine Learning and Cybernetics, vol. 10, no. 10, pp. 2901-2920, October 2019.

[67] E. Uzlu, M. Kankal, A. Akpınar and T. Dede, "Estimates of Energy Consumption in Turkey Using Neural Networks with the Teaching–learning-based Optimization Algorithm," Energy, vol. 75, pp. 295-303, October 2014.

[68] W. Yamany, M. Fawzy, A. Tharwat and A. Hassanien, "Moth-flame Optimization for Training Multi-layer Perceptrons," Proc. of the 11th IEEE International Computer Engineering Conference (ICENCO), pp. 267-272, Cairo, Egypt, December 2015.

[69] R. Singh, S. Gangwar, D. Singh and V. Pathak, "A Novel Hybridization of Artificial Neural Network and Moth-flame Optimization (ANN–MFO) for Multi-objective Optimization in Magnetic Abrasive Finishing of Aluminium 6060," Journal of the Brazilian Society of Mechanical Sciences and Engineering, vol. 41, no. 6, pp. 1-19, June 2019.

[70] R. Vasco-Carofilis, M. Gutiérrez-Naranjo and M. Cárdenas-Montes, "PBIL for Optimizing Hyperparameters of Convolutional Neural Networks and STL Decomposition," Proc. of the International Conference on Hybrid Artificial Intelligence Systems, Springer, Cham, pp. 147-159, DOI: 10.1007/978-3-030-61705-9_13, November 2020.

[71] A. Goli, H. K. Zare, R. T. Moghaddam and A. Sadeghieh, "An Improved Artificial Intelligence Based on Gray Wolf Optimization and Cultural Algorithm to Predict Demand for Dairy Products: A Case Study," International Journal of Interactive Multimedia and Artificial Intelligence, vol. 5, no. 6, pp. 15-22, March 2019.