(Received: 20-Jun.-2022, Revised: 27-Aug.-2022 , Accepted: 19-Sep.-2022)
Deep Learning is a promising domain. It has different applications in different areas of life and its application on the stock market is widely used due to its efficiency. Long Short Term Memory (LSTM) proved its efficiency in dealing with time-series data due to the unique hidden unit structure. This paper integrated LSTM with attention mechanism and sentiment analysis to forecast the closing price of two stocks; namely, APPL and TSLA, from the NASDAQ stock market. We compared our hybrid model with LSTM, LSTM with sentiment analysis and LSTM with Attention Mechanism. Three benchmarks were used to measure the performance of the models; the first one is Mean Square Error (MSE), the second one is Root Mean Square Error (RMSE) and the third one is Mean Absolute Error (MAE). The results show that the hybridization is more accurate than the LSTM model alone.

[1] M. M. Rounaghi and F. N. Zadeh, "Investigation of Market Efficiency and Financial Stability between S&P 500 and London Stock Exchange: Monthly and Yearly Forecasting of Time Series Stock Returns Using ARMA Model," Physica A: Statistical Mechanics and its Applications, vol. 456, pp. 10–21, 2016.

[2] A. A. Ariyo, A. O. Adewumi and C. K. Ayo, "Stock Price Prediction Using the ARIMA Model," Proc. of the 16th IEEE International Conference on Computer Modeling and Simulation (UKSim-AMSS), pp. 106–112, Cambridge, UK 2014.

[3] P. H. Franses and D. V. Dijk, "Forecasting Stock Market Volatility Using (Non-linear) Garch Models," Journal of Forecasting, vol. 15, no. 3, pp. 229–235, 1996.

[4] R. A. K. Cox and G.W.-Y. Wang, "Predicting the US Bank Failure: A Discriminant Analysis," Economic Analysis and Policy, vol. 44, no. 2, pp. 202–211, 2014.

[5] W. Huang, Y. Nakamori and S.-Y. Wang, "Forecasting Stock Market Movement Direction with Support Vector Machine," Computers & Operations Research, vol. 32, no. 10, pp. 2513–2522, 2005.

[6] A. V. Devadoss and T. A. A. Ligori, "Forecasting of Stock Prices Using Multi Layer Perceptron," International Journal of Computing Algorithm, vol. 2, pp. 440–449, 2013.

[7] N. Singhal, V. Ganganwar, M. Yadav, A. Chauhan, M. Jakhar and K. Sharma, "Comparative Study of Machine Learning and Deep Learning Algorithm for Face Recognition," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 7, no. 3, pp. 313–325, 2021.

[8] F. Assiri and M. Alrehaili, "Development of Ensemble Machine Learning Model to Improve COVID-19 Outbreak Forecasting," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 08, no. 2, pp. 48 – 58, DOI: 10.5455/jjcit.71-1640174252, 2022.

[9] M. Hiransha, E. Ab Gopalakrishnan, V. K. Menon and K. P. Soman, "NSE Stock Market Prediction Using Deep-learning Models," Procedia Computer Science, vol. 132, pp. 1351–1362, 2018.

[10] A. Graves, N. Jaitly and A.-R. Mohamed, "Hybrid Speech Recognition with Deep Bidirectional LSTM," Proc. of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 273–278, Olomouc, Czech Republic, 2013.

[11] C. Zhou, C. Sun, Z. Liu and F. Lau, "A C-LSTM Neural Network for Text Classification," arXiv preprint, arXiv: 1511.08630, 2015.

[12] X. Pang, Y. Zhou, P. Wang, W. Lin and V. Chang, "An Innovative Neural Network Approach for Stock Market Prediction," The Journal of Supercomputing, vol. 76, no. 3, pp. 2098–2118, 2020.

[13] J. Huang, J. Chai and S. Cho, "Deep Learning in Finance and Banking: A Literature Review and Classification," Frontiers of Business Research in China, vol. 14, pp. 1–24, 2020.

[14] D. Shah, H. Isah and F. Zulkernine, "Stock Market Analysis: A Review and Taxonomy of Prediction Techniques," International Journal of Financial Studies, vol. 7, no. 2, DOI: 10.3390/ijfs7020026, 2019.

[15] J. Chung, C. Gulcehre, K. H. Cho and Y. Bengio, "Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling," arXiv preprint, arXiv: 1412.3555, 2014.

[16] A. Vidal and W. Kristjanpoller, "Gold Volatility Prediction Using a CNN-LSTM Approach," Expert Systems with Applications, vol. 157, DOI: 10.1016/j.eswa.2020.113481, 2020.

[17] Y. Bengio, "Deep Learning of Representations for Unsupervised and Transfer Learning," Proc. of ICML Workshop on Unsupervised and Transfer Learning, vol. 27, pp. 17–36, 2012.

[18] N. Bahadur and R. Paffenroth, "Dimension Estimation Using Autoencoders," arXiv preprint, arXiv: 1909.10702, 2019.

[19] A. Essien and C. Giannetti, "A Deep Learning Framework for Univariate Time Series Prediction Using Convolutional LSTM Stacked Autoencoders," Proc. of the 2019 IEEE Int. Symposium on Innovations in Intelligent Systems and Applications (INISTA), Sofia, Bulgaria, pp. 1–6, 2019.

[20] W. Bao, J. Yue and Y. Rao, "A Deep Learning Framework for Financial Time Series Using Stacked Autoencoders and Long-short Term Memory," PLOS One, vol. 12, no. 7, p. e0180944, 2017.

[21] G. H. Merabet, M. Essaaidi, M. Ben Haddou et al., "Intelligent Building Control Systems for Thermal Comfort and Energy Efficiency: A Systematic Review of Artificial Intelligence-assisted Techniques," Renewable and Sustainable Energy Reviews, vol. 144, DOI: 10.1016/j.rser.2021.110969, 2021.

[22] V. S. Pagolu, K. N. Reddy, G. Panda and B. Majhi, "Sentiment Analysis of Twitter Data for Predicting Stock Market Movements," Proc. of the 2016 IEEE Int. Conf. on Signal Processing, Communication, Power and Embedded System (SCOPES), pp. 1345–1350, Paralakhemundi, India, 2016.

[23] J. Smailović, M. Grčar, N. Lavrač and M. Žnidaršič, "Predictive Sentiment Analysis of Tweets: A Stock Market Application," Proc. of the Int. Workshop on Human-computer Interaction and Knowledge Discovery in Complex Unstructured, Big Data (HCI-KDD 2013), vol. 7947 pp. 77–88, 2013.

[24] Z. Jin, Y. Yang and Y. Liu, "Stock Closing Price Prediction Based on Sentiment Analysis and LSTM," Neural Computing and Applications, vol. 32, pp. 9713–9729, 2020.

[25] Z. Berradi and M. Lazaar, "Integration of Principal Component Analysis and Recurrent Neural Network to Forecast the Stock Price of Casablanca Stock Exchange," Procedia Computer Science, vol. 148, pp. 55–61, 2019.

[26] M. Ismail and A. M. Awajan, "A New Hybrid Approach EMD-EXP for Short-term Forecasting of Daily Stock Market Time Series Data," Electronic Journal of Applied Statistical Analysis, vol. 10, no. 2, pp. 307–327, 2017.

[27] J. Qiu, B.Wang and C. Zhou, "Forecasting Stock Prices with Long-short Term Memory Neural Network Based on Attention Mechanism," PLOS One, vol. 15, no. 1, p. e0227222, 2020.

[28] N. Jing, Z. Wu and H. Wang, "A Hybrid Model Integrating Deep Learning with Investor Sentiment Analysis for Stock Price Prediction," Expert Systems with Applications, vol. 178, p. 115019, 2021.

[29] R. Batra and S. M. Daudpota, "Integrating Stock Twits with Sentiment Analysis for Better Prediction of Stock Price Movement," Proc. of the 2018 IEEE Int. Conf. on Computing, Mathematics and Engineering Technologies (iCoMET), pp. 1–5, Sukkur, Pakistan, 2018.

[30] S. Mohan, S. Mullapudi, S. Sammeta, P. Vijayvergia and D. C. Anastasiu, "Stock Price Prediction Using News Sentiment Analysis," Proc. of the 2019 IEEE 5th Int. Conf. on Big Data Computing Service and Applications (BigDataService), pp. 205–208, Newark, CA, USA, 2019.

[31] D. K. Kirange, R. R. Deshmukh et al., "Sentiment Analysis of News Headlines for Stock Price Prediction," Composoft: An Int. J. of Advanced Computer Technol., vol. 5, no. 3, pp. 2080–2084, 2016.

[32] J. Abraham, D. Higdon, J. Nelson and J. Ibarra, "Cryptocurrency Price Prediction Using Tweet Volumes and Sentiment Analysis," SMU Data Science Review, vol. 1, no. 3, p.1, 2018.

[33] S. Liu, C. Zhang and J. Ma, "CNN-LSTM Neural Network Model for Quantitative Strategy Analysis in Stock Markets," Proc. of the Int. Conf. on Neural Information Processing (ICONIP 2017), vol. 10635, pp. 198–206, 2017.

[34] M. R. Vargas, B. De Lima and A. G. Evsukoff, "Deep Learning for Stock Market Prediction from Financial News Articles," Proc. of the 2017 IEEE Int. Conf. on Computational Intelligence and Virtual Environments for Measurement. Sys. and Appl. (CIVEMSA), pp. 60–65, Annecy, France, 2017.

[35] C.-Y. Lee and V.-W. Soo, "Predict Stock Price with Financial News Based on Recurrent Convolutional Neural Networks," Proc. of the 2017 IEEE Conf. on Technologies and Applications of Artificial Intelligence (TAAI), pp. 160–165, Taipei, Taiwan, 2017.

[36] J. Li, H. Bu and J. Wu, "Sentiment-aware Stock Market Prediction: A Deep Learning Method," Proc. of the 2017 IEEE Int. Conf. on Service Systems and Service Management, pp. 1–6, Dalian, 2017.

[37] H. Yan and H. Ouyang, "Financial Time Series Prediction Based on Deep Learning," Wireless Personal Communications, vol. 102, no. 2, pp. 683–700, 2018.

[38] Y. Chen, J. Wu and H. Bu, "Stock Market Embedding and Prediction: A Deep Learning Method," Proc.of the IEEE 2018 15th Int. Conf. on Service Systems and Service Management (ICSSSM), pp. 1–6, Hangzhou, China, 2018.

[39] H. Y. Kim and C. H. Won, "Forecasting the Volatility of Stock Price Index: A Hybrid Model Integrating LSTM with Multiple Garch-type Models," Expert Systems with Applications, vol. 103, pp. 25–37, 2018.

[40] F. Shen, J. Chao and J. Zhao, "Forecasting Exchange Rate Using Deep Belief Networks and Conjugate Gradient Method," Neurocomputing, vol. 167, pp. 243–253, 2015.

[41] S. Hochreiter and J. Schmidhuber, "LSTM Can Solve Hard Long Time Lag Problems," Proc. of the 9th International Conference on Neural Information Processing Systems (NIPS'96), pp. 473–479, 1997.

[42] J. Pennington, R. Socher and C. D. Manning, "GloVe: Global Vectors for Word Representation," Proc. of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543, Doha, Qatar, 2014.

[43] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk and Y. Bengio, "Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation," arXiv preprint, arXiv: 1406.1078, 2014.

[44] D. Bahdanau, K. Cho and Y. Bengio, "Neural Machine Translation by Jointly Learning to Align and Translate," arXiv preprint, arXiv: 1409.0473, 2014.

[45] J. Chorowski, D. Bahdanau, D. Serdyuk, K. Cho and Y. Bengio, "Attention-based Models for Speech Recognition," arXiv preprint, arXiv: 1506.07503, 2015.

[46] Z. Berradi, M. Lazaar, O. Mahboub and H. Omara, "A Comprehensive Review of Artificial Intelligence Techniques in Financial Market," Proc. of the 2020 6th IEEE Congress on Information Science and Technology (CiSt), pp. 367–371, DOI: 10.1109/CiSt49399.2021.9357175, 2020.

[47] T. Hollis, A. Viscardi and S. Eun Yi, "A Comparison of LSTMS and Attention Mechanisms for Forecasting Financial Time Series," arXiv preprint, arXiv: 1812.07699, 2018.

[48] Z. Berradi, M. Lazaar, H. Omara and O. Mahboub. "Effect of Architecture in Recurrent Neural Network Applied on the Prediction of Stock Price," IAENG International Journal of Computer Science, vol. 47, no. 3, pp. 436–441, 2020.