A Note on the Robustness of Performance of Methods and Rankings for M4 Competition

Erol EGRIOGLU; Robert FİLDES

Article Details

A Note on the Robustness of Performance of Methods and Rankings for M4 Competition

Authors:

Erol EGRIOGLU | Robert FİLDES

Publisher:

Giresun University

Main Topics:

Mathematics

Journal:

Turkish Journal of Forecasting 2020 Vol.04 Issue 2, pp.26-32

Document Type:

Article

Publication Date:

12/2020

Document Language:

English

Document ID (Org No.):

2000637111

Frequency:

Bi-Annual

eISSN:

2618-6594

Abstract:

 

M4 forecasting competition provided some useful information to forecasting literature. The provided information is based on calculated error metrics for ranking methods in the competition. The organizers of the competition calculated arithmetic mean as a descriptive statistic for evaluating the performance of all competitors. In this paper, the effect of different descriptive statistics for ranking of methods is investigated. It is found that the distribution of error metrics for competitor forecasting methods is not symmetric. Thus, the arithmetic mean descriptive statistic is not a good metric to determine the centre of non-symmetric distributions and it will not well present centre of the distribution. In this study, it is showed that the median will well present centre of distribution for error metrics of competitor forecasting methods. When the median is used as a descriptive statistic for ranking methods, the ranks of methods is different form ranks which are calculated according to the arithmetic mean descriptive statistics. Moreover, the direction accuracy metric is calculated for the best ten methods in the competition. So, the forecasting methods are ranked according to direction accuracy and it is showed that the ranks are different from the competition results.

Article Details

A Note on the Robustness of Performance of Methods and Rankings for M4 Competition

Authors: Erol EGRIOGLU | Robert FİLDES

Publisher: Giresun University

Main Topics: Mathematics

Journal: Turkish Journal of Forecasting 2020 Vol.04 Issue 2, pp.26-32

Document Type: Article

Publication Date: 12/2020

Document Language: English

Document ID (Org No.): 2000637111

Frequency: Bi-Annual

eISSN: 2618-6594

Issues

Authors:

Erol EGRIOGLU | Robert FİLDES

Publisher:

Giresun University

Main Topics:

Mathematics

Journal:

Turkish Journal of Forecasting 2020 Vol.04 Issue 2, pp.26-32

Document Type:

Article

Publication Date:

12/2020

Document Language:

English

Document ID (Org No.):

2000637111

Frequency:

Bi-Annual

eISSN:

2618-6594