Forecasting football matches by predicting match statistics

2.1Betting odds

In this paper, betting odds are used both as potential inputs to models and as a tool with which to demonstrate profit making opportunities. Decimal, or ‘European Style’, betting odds are considered throughout. Decimal odds simply represent the number by which the gambler’s stake is multiplied in the event of success. For example, if the decimal odds are 2, a £ 10 bet on said event would result in a return of 2 × £10 = £20.

Another useful concept is that of the ‘odds implied’ probability. Let the odds for the i-th outcome of an event be O_i. The odds implied probability is simply defined as the multiplicative inverse, i.e. ri=1Oi . For example, if the odds on two possible outcomes of an event (e.g. home or away win) are O₁ = 3 and O₂ = 1.4, the odds implied probabilities are r1=13≈0.33 and R2=11.4≈0.71 . Note how, in this case, r₁ and r₂ add to more than one. This is because, whilst, conventionally, probabilities over a set of exhaustive events should add to one, this need not be the case for odds implied probabilities. In fact, usually, the sum of odds implied probabilities for an event will exceed one. The excess represents the bookmaker’s profit margin or the ‘overround’ which is formally defined as

2.2Data

This paper makes use of the large repository of data available at www.football-data.co.uk, which supplies free match-by-match data for 22 European Leagues. For each match, statistics are given including, among others, the number of shots, shots on target, corners, fouls and yellow cards. Odds data from multiple bookmakers are also given for the match outcome market, the over/under 2.5 goal market and the Asian Handicap match outcome market. For some leagues, match statistics are available from the 2000/2001 season onwards. For others, these are available for later seasons. Therefore, since the focus of this paper is forecasting using match statistics, only matches from the 2000/2001 season onwards are considered. The data used in this paper are summarised in Table 1 in which, for each league, the total number of matches since 2000/2001, the number of matches in which shots and corner data are available and the number of these excluding a ‘burn-in’ period for each season are shown. The meaning of the ‘burn-in’ period is explained in more detail in section 2 but simply omits the first six matches of the season played by the home team. All leagues include data up to and including the end of the 2018/19 season.

2.3Bivariate poisson model

Poisson models are forecasting models that use the Poisson distribution to model the number of goals scored by each team in a football match. Whilst many variants of the Poisson model have been proposed, in this paper, we consider the Bivariate Poisson model proposed by Ley et al. (2019), who compared it with nine other models and found it to achieve the most favourable forecast performance (according to the ranked probability score).

The aim of a Poisson model is to estimate the Poisson parameter for each team, which can then be used to determine a forecast probability for each outcome of a match. Whilst Poisson models typically make the assumption that the number of goals scored by each team in a match is independent, there is some evidence that this is not the case. The Bivariate Poisson includes an additional parameter that removes this assumption.

In the context of this paper, the Bivariate Poisson model has two purposes. Firstly, since it has been shown to perform favourably with respect to a number of other models, it provides a powerful benchmark for comparison in section 5.3. Secondly, it provides the basis for the Bivariate Attacking (BA) rating system described in section 3.1.2.

Let G_i,m and G_j,m be random variables for the number of goals scored in the m-th match by teams i and j, respectively, where team i is at home and team j is away. In a match between the two teams, a Poisson model can be written as

The Bivariate Poisson model is an extension of another model, also described by Ley et al. (2019), called the Independent Poisson model and it is useful to define this first. The Independent Poisson Model parametrises the Poisson parameters for a home team i against an away team j as λ_i,m = exp (c + (r_i + h) - r_j) and λ_j,m = exp (c + r_j - (r_i + h)), respectively, where c is a constant parameter, h is a home advantage parameter and r₁, . . . , r_T are strength parameters for each team.

The Bivariate Poisson model closely resembles the independent model but introduces an extra parameter to account for potential dependency between the number of goals scored by each team. Under the Bivariate Poisson model, the joint distribution for the number of goals in a match between teams i and j is given by

Both the Independent and Bivariate Poisson models are parametric models in which the parameters are estimated using maximum likelihood. However, in both cases, a slight adjustment is made to the likelihood function such that matches that happened more recently are given more weight than those that happened longer ago. To do this, the weight placed on match m is given by

Performing maximum likelihood estimation with a large number of parameters is, in general, difficult and there is a risk of falling into local optima. We follow the approach used by Ley et al. (2019) who use the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, a quasi-Newton method known for its robust properties, implemented with the ‘fmincon’ function in Matlab. Strictly positive parameters are initialised at one and each of the other parameters is initialised at zero. The sum of the team ratings r₁, . . . , r_T is constrained to zero.

A convenient property of the Poisson model is that the difference between two Poisson distributions follows a Skellam distribution and therefore match outcome probabilities can be estimated from the Poisson parameters for each team. For more details, see Karlis and Ntzoufras (2009).

3.1Ratings systems

In this paper, two different approaches are used to produce predictions for the number of goals, shots on target, shots off target and corners achieved by each team in a given football match. Each approach is described below.

3.2Constructing probabilistic forecasts

The nature of football matches is that the three possible outcomes can be considered to be ‘ordered’. Clearly, a home win is ‘closer’ to a draw than it is to an away win. As such, an appropriate model for predicting the probability of each outcome is ordinal logistic regression and this is the approach taken here.

Define an event with J ordered potential outcomes 1, . . , J. Let Y be a random variable such that p (Y = i) = p_i and ∑i=1Jpi=1 The ordinal logistic regression model is parametrised as

Combinations of the following predictor variables are used:

The home team’s odds-implied probability of winning.

Observed differences in the number of shots on target, shots off target and corners achieved by each team.

Differences in the predicted number of shots on target, shots off target, corners and goals for each team.

The home team’s odds-implied probability is included in order to assess the importance of match statistics both individually and when used alongside the other information reflected in the odds.

3.3Betting strategies

Following Wheatcroft (2020), in this paper, forecasts are constructed and used alongside two betting strategies: a simple level stakes value betting strategy and a strategy based on the Kelly Criterion. These are both described below.

Under the Level stakes betting strategy, a unit bet is placed on the i-th outcome of an event when pˆi>ri , where pˆi and r_i are the predicted probability and the odds-implied probability, respectively. The simple idea here is that, if the true probability is higher than the odds-implied probability, the bet offers ‘value’, that is the statistical expectation of the net return from the bet is positive. The idea is to use the forecast probabilities to try and find these value bets. Of course, the success of the strategy depends on the performance of the forecast probabilities in terms of uncovering such opportunities.

The Kelly strategy is based on the Kelly Criterion (Kelly Jr, 1956) and has been used in, for example, Wheatcroft (2020) and Boshnakov et al. (2017). Under this approach, the amount staked on a bet is dependent on the difference between the forecast probability and the odds implied probability. When the discrepancy between the forecast probability and the odds-implied probability is high, a greater amount of money is staked. Under the Kelly Criterion, bets are placed as a proportion of one’s wealth. For a particular outcome, the proportion of wealth staked is given by

Both the Level Stakes and Kelly betting strategies focus on the concept of ‘value’ in which bets are only taken if the forecast implies a positive expected return. It should be noted, however, that the two strategies are only guaranteed to find bets with value if the estimated probability and the true probability coincide. In practice, due to model error in the forecasts, this can never be expected to be the case and the performance of the strategies must therefore be assessed empirically.

4.1Calculation of ratings

In the following experiment, we assess the performance of differences in observed and predicted numbers of shots on target, shots off target, corners and goals as potential predictor variables for the outcomes of football matches. Different combinations of observed and predicted match statistics are then assessed both with and without the odds-implied probability of the home team (calculated using the maximum odds over all bookmakers) included as an extra predictor variable.

The experiment aims to assess the performance of observed and predicted match statistics in the forecasting of match outcomes. This is done in the context of (i) traditional variable selection (using model selection techniques), (ii) assessment of forecast performance, and (iii) betting performance. In cases (i) and (ii), observed and predicted match statistics are used as inputs to an ordinal regression model whilst, in (iii), only predicted statistics are considered. Whilst extra details of the experiment are given under the following headings, here we describe the process of producing sets of predicted match statistics using GAP and BA ratings.

We look to test forecast performance over as large a number of matches as possible. However, since we plan to use match statistics to build our forecasts and we look to assess betting performance, we are limited to those matches in which both match statistics and betting odds are available. In addition, whilst we use all matches that have this information available for the calculation of ratings, we exclude from the analysis all matches within a ‘burn-in’ period in which the home team has played six or fewer matches so far in that season to give the ratings sufficient time to ‘learn’ about the relative strengths of the teams.

For the GAP rating system, parameter estimation is performed simultaneously over all leagues and takes place between seasons such that, at the beginning of each season, optimisation is performed over all previous seasons in which the relevant statistics are available. Those parameters are then used for the entirety of the season. The first season in which match statistics are available for any of the considered leagues (2000/2001) is used only to optimise the GAP rating parameters for the following seasons, and therefore is not considered in the assessment of the performance of the forecasts or in variable selection. A team’s GAP ratings are updated each time it plays a match. However, this leaves open the question of how to initialise the ratings for each team. Whilst there are a number of approaches that could be taken, in the first season in which match statistics are available in a particular league, all GAP ratings are initialised at zero. For subsequent seasons, a team’s ratings are retained from one season to the next if they remain in the same league. Teams relegated to a league are assigned the average ratings of those teams that were promoted in the previous season and teams that are promoted are assigned the average ratings of those teams that were relegated in the previous season (note that promoted teams tend to outperform relegated teams. In the English Premier League, promoted teams have been found to achieve an average of around 8 more points than the teams they replaced (Constantinou and Fenton, 2017)). Despite this, we consider our approach to be reasonable whilst noting that more sophisticated approaches might be more effective.

For Bivariate Attacking ratings, optimisation is performed on each day in which at least one match occurs in a given league and the ratings are used for all matches on that day.

4.2Evaluating predicted match statistics

Before assessing the performance of probabilistic match forecasts, we assess the performance of the predicted match statistics in terms of how well they predict the observed statistics.

To provide a benchmark for the performance of the forecasts, a very simple alternative prediction for each match statistic is given by the sample mean of that statistic over all matches played by all teams in the data set previous to the day on which the match occurs. For the j-th match, this is given for the home and away team, respectively, by

To assess the performance of the predicted match statistics as predictors of observed statistics, we compare the mean absolute error with that achieved with the mean-benchmark model. The mean absolute error over N forecasts (predicted match statistics) and outcomes (observed match statistics) is given by

The ratio of the MAE for each approach is given by

The performance of the two approaches (GAP ratings and BA Ratings) in terms of the prediction of match statistics is assessed by comparing the value of R. The values of R for both GAP and BA ratings are shown in Fig. 1 for each of the four measures of attacking performance (goals, corners, shots on target and shots off target). For BA ratings, R is shown as a function of the chosen ‘half life’. In all cases, the GAP ratings are able to outperform the mean-benchmark model and this is generally also the case for BA ratings. Note that, due to high computational intensity, R is not shown for values of the half life longer than 135 days. However, as described in the next section, we are primarily interested in relatively short values of the half life that reflect a team’s recent performances and are able to augment the information contained in the match odds. We therefore find that the half life that maximises the performance of forecasts of the match outcome is relatively short compared with that which minimises R.

Fig. 1

Values of R for GAP ratings (straight lines) and BA ratings (curves with open circles) for each match statistic. The latter is shown as a function of half life.

There is a notably high degree of variation in the performance of the predicted statistics. Under the GAP rating system, the value of R is smallest for shots off target, whilst for goals and corners, R is not much smaller than 1. This is likely explained by the fact that there are typically a larger number of shots off target in a game than the other statistics and therefore there is more information on which to base the forecasts. BA ratings do not outperform GAP ratings for match statistics other than goals for any tested half life.

5.1Variable selection: observed match statistics

The results of variable selection when using observed match statistics are shown in Table 2. Here, the AIC for different combinations of statistics is shown both with and without the home odds-implied probability included as an additional predictor variable. Note that the AIC in each case is expressed with that of model A0 (fitted without the odds-implied probability) subtracted such that negative values imply better support for a particular combination of predictor variables than that of the model fitted without any predictor variables. The lower the AIC, the more support for that particular combination of variables.

The results yield a number of conclusions. The best AIC is achieved when the model includes all three observed match statistics both when the home odds-implied probability is included as an additional predictor variable and when it is not. That the number of shots on target should have an impact on the match result should not come as a surprise, since all goals other than own goals and highly unusual events (such as the ball deflecting off the referee or, in one case in 2009, a beachball) result from a shot on target. Interestingly, however, the inclusion of the number of corners and shots off target, which don’t usually directly result in goals, improves the model even once shots on target are considered.

It is also interesting to compare the effects of each observed match statistic as an individual predictor variable. Unsurprisingly, the number of shots on target provides the most information, followed by corners and shots off target. Interestingly, shots off target and corners do not provide much information when considered individually but add a great deal of information when combined with the number of shots on target and/or the home odds-implied probability. It is a property of generalised linear models that some predictor variables are only informative in combination with other predictor variables and this appears to be the case here.

Finally, all three match statistics add information even when the odds-implied probability is included in the model. This is perhaps not surprising since match statistics give an indication of how the match actually went.

In practice, of course, observed statistics are never available pre-match. Despite this, the results shown here have important implications. Match statistics can be predicted and, if those predictions are informative enough, it stands to reason that informative forecasts of the outcome of the match can be made.

5.2Variable selection: predicted match statistics

In section 4.2, the results of predicting match statistics using GAP and BA ratings were presented. It was shown that, in the latter case, the choice of half life has an important impact on the MAE of the predictions. Although, typically, longer half lives tend to provide better predictions for the match statistics, it may not be the case that they provide a more useful input for probabilistic forecasts of the match outcome. This is because a consistently strong team like, say, Manchester United will be expected to take a larger number of shots and corners than a weaker side over a long period of time and this will be reflected in the ratings. However, we are looking for information that is not reflected in the odds and thus to augment the information the odds provide. For example, if a team’s recent results have not reflected their performances, we look to identify that this is the case from their match statistics in recent matches. It therefore seems reasonable to expect that a shorter half life should be more useful in this case. On the other hand, looking only at more recent matches gives us a less robust reflection of a team’s strength and we therefore have a trade-off. Here, for simplicity, we choose a single half life for use in the rest of the paper based on the following fairly ad-hoc approach. Looking at the results in Fig. 2, since a half life of 45 days gives the lowest AIC for the case in which predictions of all match statistics are used in the model (bottom right panel), this value is used for all further results shown in this paper.

Fig. 2

AIC as a function of half life for forecasts produced using different combinations of (i) BA ratings (lines with points) and (ii) GAP ratings (straight horizontal lines). In both cases, the home odds-implied probability is used as an additional predictor variable.

The results of variable selection with predicted match statistics are shown in Table 3. Unsurprisingly, the AIC is generally higher than for the observed case, implying that the information content is lower. Despite this, predicted match statistics are able to provide information regarding match outcomes, even when the home odds-implied probability is included in the model. This means that, on average, both sets of predicted match statistics (from GAP and BA ratings) provide information beyond that contained in the odds-implied probabilities. However, given the universally lower AIC values, the GAP rating approach appears to be more effective.

It is of interest to note the relative importance of the different predicted match statistics. Consistent with the findings of Wheatcroft (2020), the predicted number of goals provides relatively little information when combined with the odds-implied probabilities whilst predictions of other match statistics are much more effective in improving the forecast model. It is also notable that whilst, in the observed case, the number of shots on target provides the most information about the outcome of the match, in the predicted case, shots off target is the most informative. At first, this seems counterintuitive. However, it should be noted that the information in the prediction is dependent both on the impact of the observed statistic on the match and the quality of the prediction of that statistic. Recall that Fig. 1 suggests GAP and BA rating predictions of shots off target improve more on the mean-benchmark model than those of the other match statistics and this superior prediction accuracy is the likely explanation.

Finally, it is notable that, when considered as individual predictor variables, the predicted number of shots off target and corners outperforms the equivalent observed statistics. Again, this seems counterintuitive but can probably be explained by the fact that the predicted values consider the performances of the teams over multiple past matches, gaining some information about the relative strengths of the two teams.

5.3Forecast performance

We now turn our focus onto the question of forecast performance. Though closely related to model selection, this allows us to assess the relative performance of the forecasts out-of sample and therefore as if they were produced in real time. In order to produce the forecasts, new regression parameters are selected on each day in which at least one match is played and are calculated based on all past matches which fall outside of the ‘burn-in’ period and which have shots and corner data as well as match odds available.

We compare forecast performance using two commonly used scoring rules: the Ignorance Score (Roulston and Smith, 2002; Good, 1952) and the Ranked Probability Score (Constantinou and Fenton, 2012). The ignorance score, also commonly known as the log-loss is given by

To define the Ranked Probability Score, for an event with r possible outcomes, let p_j and o_j be the forecast probability and outcome at position j where the ordering of the positions is preserved. The Ranked Probability Score (RPS) is given by

The RPS is often considered appropriate for evaluating forecasts of football matches because it takes into account the ordering of the outcomes, i.e. a home win is ‘closer’ to a draw than it is to an away win (Constantinou and Fenton, 2012). However, it has also been argued that the ordered nature of the RPS provides little practical benefit and that only the probability placed on the outcome should be taken into account, as per the ignorance score (Wheatcroft, 2019). Here, we consider it useful to evaluate the forecasts using both approaches.

To provide some context regarding the performance of the forecasts, we compare the performance with that of an alternative, strongly performing approach to forecasting football matches. The Bivariate Poisson model, described in Section 2.3, has been shown to perform favourably with respect to 9 other forecast models (Ley et al., 2019). We apply the model to our data set using the optimal half life parameter of 390 days determined by Ley et al. (2019).

Similarly to the Akaike weights confidence set used in section 5, we take a similar approach here using the Model Confidence Set (MCS) methodology proposed by Hansen et al. (2011). Here, the aim is to identify the set of models in which there is a 95 percent probability that the ‘best’ model falls, given the chosen measure of performance. We highlight the combinations of variables that fall into this set.

The mean RPS and Ignorance of each combination of variables as well as the Bivariate Poisson model are shown in Tables 4 and 5, respectively. In the latter case, the scores are given with that of model B0 subtracted such that negative scores imply better performance than the model applied with no predictor variables. The 95 percent Model Confidence Set in each case is highlighted in green. Note that, since the Bivariate Poisson model does not make use of match odds, a fair comparison is only provided by comparing these combination of variables in which the odds-implied probabilities are not included.

Similarly to the variable selection results in section 5.3, including predictions of match statistics other than goals in the model improves overall predictive performance of the match outcomes according to both scoring rules. Also consistent with the model selection results is that the model performs consistently better when match statistics are predicted using GAP ratings rather than BA ratings.

When considering the performance of the Bivariate Poisson model, it is worth noting that it only takes goals into consideration. In terms of the information used, its performance can be compared with model B8 for the case in which the odds-implied probability is not included. Here, the Bivariate Poisson model does slightly worse though the difference is small. It is when predictions of other match statistics are included that there is a large increase in performance over the Bivariate Poisson model. This suggests that much of the improvement results from the additional information in the match statistics rather than the structure of the model.

5.4Betting performance

In this section, the performance of the forecasts in section 5.3 when used alongside the Level Stakes and Kelly betting strategies described in section 3.3 is assessed. Here, it is assumed that a gambler is able to ‘shop around’ different bookmakers and take advantage of the highest odds offered on each outcome. The maximum odds over all available bookmakers are thus assumed to be obtainable (note that the actual bookmakers included in the data set vary over time). Note that bets placed on draws are not considered due to the inherent difficulty of predicting them and therefore only bets on home or away wins are allowed. The mean percentage profit obtained from the Level Stakes betting strategy when used alongside forecasts derived from each combination of predicted match statistics is shown in Table 6, along with 95 percent bootstrap resampling intervals. The resampling intervals are presented to demonstrate the robustness of the profit and, if the interval does not contain zero, the profit can be considered to be statistically significant.

It is clear from the results that including combinations of predicted match statistics as predictor variables tends to yield a profit. In addition, for all combinations, including the home odds-implied probability as an additional predictor variable yields an increase in profit. In some cases, when the home odds-implied probability is included, the profit is significant, i.e. the bootstrap resampling interval does not include zero. Whilst caution is advised in comparing the precise rankings of different combinations of variables, the best performing combinations tend to include the predicted number of shots off target. The predicted number of goals, on the other hand, tends to have limited value. When individual predicted statistics are considered, the ranking of the results is consistent with the variable selection results of Table 3 in that the best performing predicted variable is shots off target, followed by corners, shots on target and goals. It is also notable that forecasts built using BA ratings do not perform as well as those formed using GAP ratings.

The mean profit obtained from using the forecasts alongside the Kelly strategy are shown in Table 7. Here, under both the GAP and BA rating systems, notably, the mean profit is generally substantially higher than that achieved using the Level Stakes strategy. Again, including the home odds-implied probability as an additional predictor variable yields improved results for all combinations of variables. In fact, the profit is significant in all cases in which at least one predicted match statistic other than the number of goals is included alongside the home odds-implied probability. Again, the results obtained from the GAP rating approach are almost always better than under the BA rating approach.

For the remainder of this section, given the superior performance of GAP ratings relative to the BA ratings, we focus on the betting performance of forecasts formed using predicted shots on target, shots off target and corners simultaneously under this approach. We do this both with and without the home odds-implied probability as an additional predictor variable.

The cumulative profit achieved with each of the two betting strategies is shown in Fig. 3. As already shown in Tables 6 and 7, a substantial profit is made in all four cases. The figure, however, shows how each strategy performs over time and an interesting feature is that there appears to be a downturn in profit in recent seasons. Whilst this could conceivably be explained by random chance, it is perhaps more likely that something fundamental changed over that time. That predicted match statistics provide information additional to that contained in the odds suggests that, in general, the odds do not adequately account for the ability of teams to create shots and corners. However, as more data have become available and quantitative analysis has become more sophisticated, it seems a reasonable claim that such information is now more likely to be reflected in the odds on offer and it may therefore be the case that the betting opportunities available in earlier seasons simply don’t exist anymore.

Fig. 3

Cumulative profit from using the Kelly strategy (solid lines) and the level stakes strategy (dashed lines) with forecasts formed using GAP rating predictions of shots on target, shots off target and corners both when the home odd-implied probability is included as a predictor variable in the model (blue) and when it is excluded (red).

It is worth considering how the profits from each betting strategy are distributed between the different leagues and whether losses in any particular subset of leagues can explain the observed downturn. Focusing on the case in which the home odds-implied probability is included as a predictor variable, in Fig. 4 the cumulative profit made in each league is shown as a function of time. Here, the decline in profit appears to be fairly consistent over all leagues considered and therefore, if the information reflected in the odds really has increased over time, this appears to be fairly universal over the different leagues.

Fig. 4

Cumulative profit as a function of time in each league for the case in which predicted shots on target, shots off target and corners along with the home odds-implied probability are included as predictor variables.

Finally, it is important to assess the impact of the overround on the profitability of the betting strategies. In this experiment, it is assumed that the gambler is able to find the best odds on offer on each possible outcome, over a range of bookmakers. Due to increased competition, there has been a trend towards reduced profit margins in recent years. This can have a knock on effect on the overround of the best odds. A histogram of the overround of the best odds for all matches deemed eligible for betting is shown in Fig. 5. Whilst, in the majority of cases, the overround is positive, in around 18 percent of cases, it is negative. This gives rise to arbitrage opportunities, which means that a guaranteed profit can be made, without any need for a model. It is therefore important to distinguish cases in which profits are made due to the performance of the forecasts from those in which a profit could be guaranteed through arbitrage.

Fig. 5

Histogram of overrounds under the maximum odds.

To assess the importance of the overround, five different intervals are defined and the mean profit from matches whose overround falls into each one is calculated under both betting strategies. The first interval contains all matches with an overround less than zero, whilst, for matches with a positive overround, intervals with a width of 2.5 percent are defined. The interval containing matches with the largest overrounds consider those in which the overround is greater than 7.5 percent. In Fig. 6, the mean overround for matches contained in each interval is plotted against the mean profit under each of the two betting strategies. The error bars correspond to 95 percent bootstrap resampling intervals of the mean profit. In all five intervals, and under both betting strategies, the mean profit is positive. Under the Kelly strategy, three out of the five intervals yield a significant profit, whilst this is true in one interval for the Level Stakes strategy. Interestingly, the mean profit is not significantly different from zero when the overround is negative. This, however, is consistent with the decline in profit in recent seasons that has tended to coincide with lower overrounds. Overall, the fact that significant profits can be made for matches in which the overround is positive suggest that, over the course of the dataset, the forecasts in combination with the two betting strategies would have been successful in identifying profitable betting opportunities.

Fig. 6

Mean overround against mean profit under the Kelly strategy (blue) and the Level Stakes strategy (red) for each considered interval. The error bars represent 95 percent bootstrap resampling intervals of the mean.