‘Meten is weten’; An evaluation of three multivariate scoring rules in the context of ARMA models.

Abstract

Over the past decades, the use of multivariate scoring rules has steadily increased. These scoring rules, which measure the quality of forecasts based on multivariate distributions, are becoming increasingly important as multivariate forecasting models are getting further developed and become more widely used. However, whilst there are several wellestablished scoring rules for univariate distributions, the multivariate scoring rules proposed until now all exhibit some known weaknesses. There still needs to be a lot of research done, so it is not always clear how well multivariate scoring rules perform in the context of certain specific forecasting models.

One of these unexplored forecasting models is the ARMA model, a widely used time series model. In this thesis, we explore the use of three multivariate scoring rules - the logarithmic score, energy score and variogram score – for multivariate Gaussian distributions arising from ARMA models. We will do this in both analytical and numerical fashion. Along the way, we derive (closed form) expressions for these scoring in the case of both general multivariate Gaussian distributions and of multivariate Gaussian distributions arising from ARMA models.