An Overview of Performance Measurement for Demand Forecasting Based on Artificial Neural Networks


  • Steffen Robus Hungarian University of Agriculture and Life Sciences, Doctoral School of Management and Organizational Science, 7400 Kaposvár, Guba Sándor u. 40.,Hungary
  • Zsolt Kőmüves Hungarian University of Agriculture and Life Sciences, Institute of Agricultural and Food Economics, Department of Agricultural Management and Leadership Science
  • Virág Walter Hungarian University of Agriculture and Life Sciences, Institute of Agricultural and Food Economics, Department of Agricultural Management and Leadership Science



decision support systems, artificial neural networks, performance measurements, supply chain management


Demand forecasting is an essential task to match supply and demand. From a supplier’s view, demand forecasting is important to optimize supply chains and thus maximize profits. The ever-increasing availability of data that can be used as input factors for predictive models allows more and more sophistication for diverse forecasting tasks in the context of demand forecasting. On the one hand, increasingly complex models have been used for demand forecasting over the last years, from simple exponential smoothing methods and ARIMA models up to complex, hybrid (deep) artificial neural networks. On the other hand, little attention is paid to the methods that evaluate the forecasting performance of these models, which are essential for the selection from among potential forecasting models. In this article, we aim to answer the question of what are the most favourable measurements in recent literature on applied neural network demand forecasting for supply chain management. To this end, we analyzed 193 relevant publications in which demand forecasting was applied using artificial neural networks. We found that in artificial neural network demand forecasting used to evaluate forecasting performance, Mean Absolute Percentage Error, Root Mean Squared Error, Mean Squared Error and Mean Absolute Error are by far the most popular methods. Furthermore, we found that when forecasting performance measurements are combined, the most common combination is the combination of Mean Absolute Error, the Root Mean Squared Error and the Mean Absolute Error.

Author Biographies

  • Steffen Robus, Hungarian University of Agriculture and Life Sciences, Doctoral School of Management and Organizational Science, 7400 Kaposvár, Guba Sándor u. 40.,Hungary

    corresponding author

  • Zsolt Kőmüves, Hungarian University of Agriculture and Life Sciences, Institute of Agricultural and Food Economics, Department of Agricultural Management and Leadership Science

    associate professor

  • Virág Walter, Hungarian University of Agriculture and Life Sciences, Institute of Agricultural and Food Economics, Department of Agricultural Management and Leadership Science

    associate professor



Aggarwal, (2018). Machine Learning with Shallow Neural Networks. In C. C. Aggarwal (Ed.) Neural Networks and Deep Learning (pp. 53–104). Springer International Publishing.

Andrawis, R. R., Atiya, A. F., & El-Shishiny, H. (2011). Combination of long term and short term forecasts, with application to tourism demand forecasting. International Journal of Forecasting, 27(3), 870–886.

Anisa, M. P., Irawan, H., & Widiyanesti, S. (2021). Forecasting demand factors of tourist arrivals in Indonesia’s tourism industry using recurrent neural network. IOP Conference Series: Materials Science and Engineering, 1077(1), 012035.

Armstrong, J. S. (Ed.). (2001). Principles of forecasting: a handbook for researchers and practitioners. International Series in Operations Research & Management Science (Vol. 30). Kluwer Academic.

Armstrong, J. S., & Collopy, F. (1992). Error measures for generalizing about forecasting methods: Empirical comparisons. International journal of forecasting, 8(1), 69-80.

Babai, M. Z., Syntetos, A., & Teunter, R. (2014). Intermittent demand forecasting: An empirical study on accuracy and the risk of obsolescence. International Journal of Production Economics, 157, 212-219.

Bowerman, B. L., O'Connell, R. T., Murphree, E., Huchendorf, S. C., Porter, D. C., & Schur, P. (2003). Business statistics in practice (pp. 728-730). McGraw-Hill.

Burger, C. J. S. C., Dohnal, M., Kathrada, M., & Law, R. (2001). A practitioners guide to time- series methods for tourism demand forecasting – a case study of Durban, South Africa. Tourism management, 22(4), 403-409.

Byrne, R. F. (2012). Beyond Traditional Time-Series: Using Demand Sensing to Improve Forecasts in Volatile Times. Journal of Business Forecasting, 31(2).

Carbone, R., & Armstrong, J. S. (1982). Note. Evaluation of extrapolative forecasting methods: results of a survey of academicians and practitioners. Journal of Forecasting, 1(2), 215-217.

Carbonneau, R., Laframboise, K., & Vahidov, R. (2008). Application of machine learning techniques for supply chain demand forecasting. European journal of operational research, 184(3), 1140-1154.

Chawla, A., Singh, A., Lamba, A., Gangwani, N., & Soni, U. (2019). Demand Forecasting Using Artificial Neural Networks – A Case Study of American Retail Corporation. In Malik, H., Srivastava, S., Sood, Y., Ahmad, A. (Eds.) Applications of Artificial Intelligence Techniques in Engineering. (pp. 79-89). Springer, Singapore. 981-13-1822-1_8

Chai, T., & Draxler, R. R. (2014). Root mean square error (RMSE) or mean absolute error (MAE). Geoscientific model development discussions, 7(1), 1525-1534.

Chen, Z., & Yang, Y. (2004). Assessing forecast accuracy measures. Iowa State University.

Clements, M. P., Franses, P. H., & Swanson, N. R. (2004). Forecasting economic and financial time-series with non-linear models. International journal of forecasting, 20(2), 169-183.

Clements, M. P., & Hendry, D. F. (1993). On the limitations of comparing mean square forecast errors. Journal of Forecasting, 12(8), 617-637.

Constantino, H. A., Fernandes, P. O., & Teixeira, J. P. (2016). Tourism demand modelling and forecasting with artificial neural network models: The Mozambique case study. Tékhne, 14(2), 113-124.

Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018). Generative adversarial networks: An overview. IEEE signal processing magazine, 35(1), 53-65.

De Gooijer, J. G., & Hyndman, R. J. (2006). 25 years of time series forecasting. International journal of forecasting, 22(3), 443-473.

Diebold, F. X., & Lopez, J. A. (1996). 8 Forecast evaluation and combination. Handbook of statistics, 14, 241-268.

Diebold, F. X., & Mariano, R. S. (2002). Comparing predictive accuracy. Journal of Business & economic statistics, 20(1), 134-144.

Fildes, R., Ma, S., & Kolassa, S. (2022). Retail forecasting: Research and practice. International Journal of Forecasting, 38(4), 1283-1318.

Ghalehkhondabi, I., Ardjmand, E., Weckman, G. R., & Young, W. A. (2017). An overview of energy demand forecasting methods published in 2005–2015. Energy Systems, 8, 411-447.

Goodfellow, I. & Bengio, Y & Courville, A. (2018). Deep Learning – Das umfassende Handbuch. 1. Auflage. Verlags GmbH & Co. KG.

Granger, C. W. J., & Pesaran, M. H. (2000a). Economic and statistical measures of forecast accuracy. Journal of Forecasting, 19(7), 537-560.;2-G

Granger, C. W. J., & Pesaran, M. H. (2000b). A decision theoretic approach to forecast evaluation. Statistics and finance: An interface (pp. 261-278).

Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., & Chen, T. (2018). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354–377.

Haberleitner, H., Meyr, H., & Taudes, A. (2010). Implementation of a demand planning system using advance order information. International Journal of Production Economics, 128(2), 518-526.

Hanke,John E., Arthur G. (1995). Business forecasting. (5th ed.). Prentice Hall.

Herrera-Granda, I. D., Chicaiza-Ipiales, J. A., Herrera-Granda, E. P., Lorente-Leyva, L. L., Caraguay-Procel, J. A., García-Santillán, I. D., & Peluffo-Ordóñez, D. H. (2019). In: Rojas, I., Joya, G., Catala, A. (Eds.) Advances in Computational Intelligence. (pp. 362-373). Springer Cham.

Hodson, T. O. (2022). Root-mean-square error (RMSE) or mean absolute error (MAE): When to use them or not. Geoscientific Model Development, 15(14), 5481-5487.

Hyndman, R. J., & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International journal of forecasting, 22(4), 679-688.

Jnr, E. O. N., Ziggah, Y. Y., & Relvas, S. (2021). Hybrid ensemble intelligent model based on wavelet transform, swarm intelligence and artificial neural network for electricity demand forecasting. Sustainable Cities and Society, 66, 102679.

Ke, J., Zheng, H., Yang, H., & Chen, X. M. (2017). Short-term forecasting of passenger demand under on-demand ride services: A spatio-temporal deep learning approach. Transportation research part C: Emerging technologies, 85, 591-608.

Kilimci, Z. H., Akyuz, A. O., Uysal, M., Akyokus, S., Uysal, M. O., Atak Bulbul, B., & Ekmis, M. A. (2019). An improved demand forecasting model using deep learning approach and proposed decision integration strategy for supply chain. Complexity, 2019.

Kim, S., & Kim, H. (2016). A new metric of absolute percentage error for intermittent demand forecasts. International Journal of Forecasting, 32(3), 669-679.

Koutsandreas, D., Spiliotis, E., Petropoulos, F., & Assimakopoulos, V. (2022). On the selection of forecasting accuracy measures. Journal of the Operational Research Society, 73(5), 937-954.

Liang, Y. H. (2022). Forecasting International Tourism Demand Using the Recurrent Neural Network Model with Genetic Algorithms and ARIMAX Model in Tourism Supply Chains. International Journal of Machine Learning and Computing, 12(5).

Lewis, C. D. (1997). Demand forecasting and inventory control: A computer aided learning approach. Routledge.

Makridakis, S. (1993). Accuracy measures: theoretical and practical concerns. International Journal of Forecasting, 9(4), 527-529.

Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Parzen, E. & Winkler, R. (1982). The accuracy of extrapolation (time series) methods: Results of a forecasting competition. Journal of forecasting, 1(2), 111-153.

Maragkos, N. (2020). Retail Demand Forecasting with Artificial Neural Networks [Doctoral dissertation, Aristotle University of Thessaloniki].

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5, 115-133.

Panapakidis, I. P., & Dagoumas, A. S. (2017). Day-ahead natural gas demand forecasting based on the combination of wavelet transform and ANFIS/genetic algorithm/neural network model. Energy, 118, 231-245.

Petrucci, A., Barone, G., Buonomano, A., & Athienitis, A. (2022). Modelling of a multi-stage energy management control routine for energy demand forecasting, flexibility, and optimization of smart communities using a Recurrent Neural Network. Energy Conversion and Management, 268, 115995.

Porteiro, R., Hernández-Callejo, L., & Nesmachnow, S. (2022). Electricity demand forecasting in industrial and residential facilities using ensemble machine learning. Revista Facultad de Ingeniería Universidad de Antioquia, (102), 9-25.

Qi, J., Du, J., Siniscalchi, S. M., Ma, X., & Lee, C. H. (2020). On mean absolute error for deep neural network based vector-to-vector regression. IEEE Signal Processing Letters, 27, 1485-1489.

Rosenblatt, F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386-408.

Ryu, S., Noh, J., & Kim, H. (2016). Deep neural network based demand side short term load forecasting. Energies, 10(1), 3.

Salehinejad, H., Sankar, S., Barfett, J., Colak, E., & Valaee, S. (2017). Recent advances in recurrent neural networks. arXiv preprint arXiv:1801.01078.

Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85–117.

Schneider, P., & Xhafa, F. (2022). Anomaly Detection and Complex Event Processing Over IoT Data Streams: With Application to EHealth and Patient Data Monitoring. Academic Press.

Slimani, I., El Farissi, I., & Achchab, S. (2015). Artificial neural networks for demand forecasting: Application using Moroccan supermarket data. 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA) (pp. 266-271). IEEE.

Suganthi, L., & Samuel, A. A. (2012). Energy models for demand forecasting – A review. Renewable and sustainable energy reviews, 16(2), 1223-1240.

Swanson, N.R., Ghysels, E. & Callan, M. (1999). A Multivariate Time Series Analysisof the Data Revision Process for Industrial Production and the Composite Leading Indicator. In R.F. Engle & H. White (Eds.), Cointegration, Causality and Forecasting: A Festschrift in Honor of Clive W.J. Granger. (pp.45-75). Oxford University Press.

Thonemann, U. (2010). Operations Management, 2., aktualisierte Auflage, Pearson.

Wanchoo, K. (2019, March). Retail demand forecasting: a comparison between deep neural network and gradient boosting method for univariate time series. 2019 IEEE 5th International Conference for Convergence in Technology (I2CT) (pp. 1-5). IEEE.

Wang, B., Liu, K., & Zhao, J. (2016, August). Inner attention based recurrent neural networks for answer selection. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1288-1297). 1122

Wang, Z., & Bovik, A. C. (2009). Mean squared error: Love it or leave it? A new look at signal fidelity measures. IEEE signal processing magazine, 26(1), 98-117.

Willmott, C. J., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate research, 30, 79-82.

Willmott, C. J., Matsuura, K., & Robeson, S. M. (2009). Ambiguities inherent in sums-of- squares-based error statistics. Atmospheric Environment, 43(3), 749-752.

Yu, D., Li, Z., Zhong, Q., Ai, Y., & Chen, W. (2020). Demand management of station-based car sharing system based on deep learning forecasting. Journal of Advanced Transportation, 2020, 1-15.

Zhang, B., Li, N., Shi, F., & Law, R. (2020). A deep learning approach for daily tourist flow forecasting with consumer search data. Asia Pacific Journal of Tourism Research, 25(3), 323- 339.




How to Cite

An Overview of Performance Measurement for Demand Forecasting Based on Artificial Neural Networks. (2022). REGIONAL AND BUSINESS STUDIES, 14(2), 5-20.