(Received: 19-Jun.-2020, Revised: 8-Aug.-2020 , Accepted: 30-Aug.-2020)
Multi-label classification is a general type of classification that has attracted many researchers in the last two decades due to its applicability to many modern domains, such as scene classification, bioinformatics and text classification, among others. This type of classification allows instances to be associated with more than one class label at the same time. Class label ranking is a crucial problem in multi-label classification research, because it directly impacts the performance of the final classifiers, as labels with high ranks get a higher chance of being applied. This paper presents a new multi-label ranking algorithm called Multi-label Ranking based on Positive Correlations among labels (MLR-PC). MLR-PC captures positive correlations among labels to reduce the large search space and assigns the true rank per class label for multi-label classification problems. More importantly, MLR-PC utilizes novel problem transformation methods that facilitate exploiting accurate positive correlations among labels. This improves the predictive performance of the classification models derived. Empirical results using different multi-label datasets and five evaluation metrics reveal that the MLR-PC is superior to other commonly existing classification algorithms.

[1] R. Sousa and J. Gama, "Multi-label Classification from High-speed Data Streams with Adaptive Model Rules and Random Rules," Progress in Artificial Intelligence, vol. 7, pp. 1-11, 2018.

[2] C. Zeng, W. Zhou, T. Li, L. Shwartz and G. Y. Grabarnik, "Knowledge Guided Hierarchical Multi-label Classification over Ticket Data," IEEE Transactions on Network and Service Management, vol. 14, no. 2, pp. 246-260, 2017.

[3] E. Gibaja and S. Ventura, "A Tutorial on Multi-label Learning," ACM Computing Surveys, vol. 47, no. 3, pp. 1-39, 2015.

[4] J. Huang, X. Qu, G. Li, F. Qin, X. Zheng and Q. Huang, "Multi-view Multi-label Learning with View-label-specific Features," IEEE Access, vol. 7, pp. 100979-100992, 2019.

[5] S. Xu, X. Yang, H. Yu, D. J. Yu, J. Yang and E. C. Tsang, "Multi-label Learning with Label-specific Feature Reduction," Knowledge-based Systems, vol. 104, pp. 52-61, 2016.

[6] J. Huang, G. Li, S. Wang, Z. Xue and Q. Huang, "Multi-label Classification by Exploiting Local Positive and Negative Pairwise Label Correlation," Neurocomputing, vol. 257, pp. 164-174, 2017.

[7] S. J. Huang, Z. H, Zhou and Z. Zhou, "Multi-label Learning by Exploiting Label Correlations Locally," Proceedings of the 26th AAAI Conf. on Artificial Intelligence, pp. 949-955, Palo Alto, CA: AAAI, 2012.

[8] X. Kong, B. Cao and P. S. Yu, "Multi-label Classification by Mining Label and Instance Correlations from Heterogeneous Information Networks," Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 614-622, [Online], Available:, 2013.

[9] J. Lee, H. Kim, N. R. Kim and J. H. Lee, "An Approach for Multi-label Classification by Directed Acyclic Graph with Label Correlation Maximization," Information Sciences, vol. 351, pp. 101-114, 2016.

[10] M. Alluwaici, A. K. Junoh, F. K. Ahmad, M. F. M. Mohsen and R. Alazaidah, "Open Research Directions for Multi-label Learning," Proc. of IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), pp. 125-128, Penang, Malaysia, 2018.

[11] R. E. Schapire and Y. Singer, "BoosTexter: A Boosting-based System for Text Categorization," Machine Learning, vol. 39, no. 2-3, pp. 135-168, 2000.

[12] R. AlShboul, F. Thabtah, N. Abdelhamid and M. Al-Diabat, "A Visualization Cybersecurity Method Based on Features' Dissimilarity," Computers & Security, vol. 77, pp. 289-303, 2018.

[13] M. R. Boutell, J. Luo, X. Shen and C. M. Brown, "Learning Multi-label Scene Classification," Pattern Recognition, vol. 37, no. 9, pp. 1757-1771, 2004.

[14] A. Dimou, G. Tsoumakas, V. Mezaris, I. Kompatsiaris and I. Vlahavas, "An Empirical Study of Multi-label Learning Methods for Video Annotation," Proceedings of the 7th IEEE International Workshop on Content-based Multimedia Indexing, pp. 19-24, New York, NY, USA, 2009.

[15] S. Peters, L. Denoyer and P. Gallinari, "Iterative Annotation of Multi-relational Social Networks," Proc. of the IEEE International Conference on Advances in Social Networks Analysis and Mining, pp. 96-103, DOI: 10.1109/ASONAM.2010.13, Odense, Denmark, 2010.

[16] S. H. Ma, H. B. Le, B. H. Jia, Z. X. Wang, Z. W. Xiao, X. L. Cheng et al., "Peripheral Pulmonary Nodules: Relationship between Multi-slice Spiral CT Perfusion Imaging and Tumor Angiogenesis and VEGF Expression," BMC Cancer, vol. 8, no. 1, DOI: 10.1186/1471-2407-8-186, 2008.

[17] K. Trohidis, G. Tsoumakas, G. Kalliris and I. P. Vlahavas, "Multi-label Classification of Music into Emotions," In: J. P. Bellow, E. Chew and D. Turnbull (Eds.), Proceedings of the 9th International Conf. on Music Informative Retrieval, pp. 325-330, Philadelphia, PN: Drexel University, USA, 2008.

[18] Z. Barutcuoglu, R. E. Schapire and O. G. Troyanskaya, "Hierarchical Multi-label Prediction of Gene Function," Bioinformatics, vol. 22, no. 7, pp. 830-836, 2006.

[19] A. Elisseeff and J. Weston, "A Kernel Method for Multi-labelled Classification," Advances in Neural Information Processing Systems, Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic (NIPS'01), pp. 681-387, 2001.

[20] A. Skabar, D. Wollersheim and T. Whitfort, "Multi-label Classification of Gene Function using MLPs," Proceedings of the 2006 IEEE International Joint Conference on Neural Network, pp. 2234-2240, DOI: 10.1109/IJCNN.2006.247019, Vancouver, BC, Canada, 2006.

[21] A. Chan and A. A. Freitas, "A New Ant Colony Algorithm for Multi-label Classification with Applications in Bioinfomatics," Proc. of the 8th Annual Conf. on Genetic and Evolutionary Computation (GECCO'06), pp. 27-34, [Online], Available:, 2006.

[22] S. Diplaris, G. Tsoumakas, P. A. Mitkas and I. Vlahavas, "Protein Classification with Multiple Algorithms," Proc. of Panhellenic Conference on Informatics (PCI), pp. 448-456, Part of the LNCS, vol. 3746, Springer, Berlin, Heidelberg, 2005.

[23] K. Kawai and Y. Takahashi, "Identification of the Dual Action Antihypertensive Drugs Using TFS-based Support Vector Machines," Chem-Bio Informatics Journal, vol. 10, no. 9, pp. 41–51, Retrieved from:, 2010.

[24] A. Krohn-Grimberghe, L. Drumond, C. Freudenthaler and L. Schmidt-Thieme, "Multi-relational Matrix Factorization Using Bayesian Personalized Ranking for Social Network Data," Proceedings of the 5th ACM International Conference on Web Search and Data Mining, pp. 173-182, [Online], Available:, 2012.

[25] L. Tang and H. Liu, "Relational Learning via Latent Social Dimensions," Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’09), pp. 817-826, [Online], Available:, 2009.

[26] T. Li, C. Zhang and S. Zhu, "Empirical Studies on Multi-label Classification," Proc. of the 18th IEEE Int. Conf. on Tools with Artificial Intelligence (ICTAI’06), pp. 86-92, Arlington, VA, USA, 2006.

[27] O. A. Nassar and N. A. Al Saiyd, "The Integrating between Web Usage Mining and Data Mining Techniques," Proc. of the 5th IEEE International Conference on Computer Science and Information Technology, pp. 243-247, Amman, Jordan, 2013.

[28] J. Huang, Q. Feng, Z. Xiao, C. Zekai, Y. Zhixiang, Z. Weigang and H. Qingming, "Improving Multi-label Classification with Missing Labels by Learning Label-specific Features," Information Sciences, vol. 492, pp. 124-146, 2019.

[29] J. Read, B. Pfahringer, G. Holmes and E. Frank, "Classifier Chains for Multi-label Classification," Machine Learning, vol. 85, no. 3, p. 333, 2011.

[30] R. Alazaidah, F. Thabtah and Q. Al-Radaideh, "A Multi-label Classification Approach Based on Correlations among Labels," International Journal of Advanced Computer Science and Applications, vol. 6, no. 2, pp. 52-59, 2015.

[31] S. S. Ibrahiem, S. S. Ismail, K. A. Bahnasy and M. M. Aref, "Convolutional Neural Network Multi-emotion Classifiers," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 5, no. 2, pp. 97-108, Aug. 2019.

[32] R. Alazaidah, F. K. Ahmad and M. F. M. Mohsin, "A Comparative Analysis between the Three Main Approaches that are Being Used to Solve the Problem of Multi-label Classification," International Journal of Soft Computing, vol. 12, no. 4, pp. 218-223, 2017.

[33] M. L. Zhang and Z. H. Zhou, "ML-KNN: A Lazy Learning Approach to Multi-label Learning," Pattern Recognition, vol. 40, no. 7, pp. 2038-2048, 2007.

[34] J. Fürnkranz and E. Hüllermeier, "Pairwise Preference Learning and Ranking," Proc. of the European Conference on Machine Learning (ECML), pp. 145-156, LNCS, vol. 2837, Springer, Heidelberg, 2003.

[35] J. Fürnkranz, E. Hüllermeier, E. L. Mencía and K. Brinker, "Multilabel Classification via Calibrated Label Ranking," Machine Learning, vol. 73, no. 2, pp. 133-153, 2008.

[36] G. Tsoumakas, I. Katakis and I. Vlahavas, "Effective and Efficient Multi-label Classification in Domains with Large Number of Labels," Proc. of ECML/PKDD 2008 Workshop on Mining Multi-dimensional Data (MMD’08), vol. 21, pp. 53-59, 2008.

[37] J. Read, "A Pruned Problem Transformation Method for Multi-label Classification," Proc. of 2008 New Zealand Computer Science Research Student Conference (NZCSRS 2008), pp. 143-150, 2008.

[38] G. Tsoumakas and I. Vlahavas, "Random k-labelsets: An Ensemble Method for Multi-label Classification," Proc. of the European Conference on Machine Learning, pp. 406-417, LNCS, vol. 4701, Springer, Berlin, Heidelberg, 2007.

[39] M. Zhang and L. Wu, "Lift: Multi-label Learning with Label-specific Features," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 1, pp. 107-120, DOI: 10.1109/TPAMI.2014.2339815. 2015.

[40] L. Rokach, A. Schclar and E. Itach, "Ensemble Methods for Multi-label Classification," Expert Systems with Applications, vol. 41, no. 16, pp. 7507-7523, 2014.

[41] R. Alazaidah, F. K. Ahmad, M. F. M. Mohsen and A. K. Junoh, "Evaluating Conditional and Unconditional Correlations Capturing Strategies in Multi-label Classification," Journal of Telecommunication, Electronic and Computer Engineering (JTEC), vol. 10, no. 2-4, pp. 47-51, 2018.

[42] E. Frank and H. Witten, "Generating Accurate Rule Sets without Global Optimization," Proceedings of the 15th International Conference on Machine Learning (ICML '98), pp. 144–151, 1998.

[43] T. Scheffer, "Finding Association Rules that Trade Support Optimally Against Confidence," Intelligent Data Analysis, vol. 9, no. 4, pp. 381-395, 2005.

[44] R. R. Bouckaert, "Properties of Bayesian Belief Network Learning Algorithms," Uncertainty Proceedings 1994, pp. 102-109, Morgan Kaufmann, [Online], Available:, 1994.

[45] W. W. Cohen, "Fast Effective Rule Induction," Proceedings of the 12th International Conference on Machine Learning, pp. 115-123, Tahoe City, California, 1995.

[46] M. Sumner, E. Frank and M. Hall, "Speeding up Logistic Model Tree Induction," Proceedings of the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases Discovery (PKDD'05), pp. 675-683, Heidelberg: Springer-Verlag, DOI: 10.1007/11564126_72, 2005.

[47] R. Kohavi, "The Power of Decision Tables" Proceedings of the 8th European Conference on Machina Learning (ECML’95), pp. 174-189, Berlin, Heidelberg: Springer-Verlag, DOI: 10.1007/3-540-59286-5_57, 1995.

[48] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann and I. H. Witten, "The WEKA Data Mining Software: An Update, " ACM SIGKDD Explorations Newsletter, vol. 11, no.1, pp. 10-18, 2009.

[49] G. Tsoumakas, E. Spyromitros-Xioufis, J. Vilcek and I. Vlahavas, "Mulan: A Java Library for Multi-label Learning," Journal of Machine Learning Research, vol. 12, no. Jul., pp. 2411-2414, 2011.

[50] S. Zhu, X. Ji, W. Xu and Y. Gong, "Multi-labelled Classification Using Maximum Entropy Method," Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '05), pp. 274-281, [Online], available: 1076082 2005.

[51] R. Alazaidah, F. Ahmad and M. Mohsin, "Multi-label Ranking Based on Positive Pairwise Correlations among Labels," The International Arab Journal of Information Technology, vol. 17, no. 4, DOI: 10.34028/iajit/17/4/2, 2019.

[52] J. Huang, Z. Pingzhao, Z. Huiyi, L. Guorong and R. Haowei, "Multi-label Learning via Feature and Label Space Dimension Reduction," IEEE Access, vol. 8, pp. 20289-20303, 2020.

[53] F. Alshraiedeh, S. Hanna and R. Alazaidah, "An Approach to Extend WSDL-based Data Types Specification to Enhance Web Services Understandability," International Journal of Advanced Computer Science and Applications, vol. 6, no. 3, pp. 88-98, 2015.