# Round 13 Review

I had made some changes to my model on Friday and screwed things up a little bit, after finding the mistake late Friday evening (post-bounce) I noticed my simulation then predicted a Sydney victory but I was happy enough to take the gamble on the Eagles! Some tough games went my way in the tipping (thanks Saints!) but it didn’t reward me with too many bits.

I’m currently drafting some changes to my model to include a predictor variable for weather once I nail down a process for scraping and categorising weather for past matches. I’m sure this will have a positive effect on reducing my MAE; which is unacceptable.

I’m hanging on to second place in my table since I’ve gone “live”… assuming bits are more important 😛

# How do you measure a player’s form?

A key part of my model is the concept that the selected players, along with the less player-specific team performance, determine the strength of the team as a whole. Here I’m going to discuss some ways to measure a player’s form to, hopefully, accurately predict their performance in an upcoming game.

Imagine for a moment each player’s value is described by a single score; I’ll use Supercoach points for this example. How do you best predict what the player’s score will be in their next game?

Speaking of Supercoach, the site provides projected scores for upcoming matches:

“Projected scores are calculated using the Herald Sun SuperCoach AFL’s proprietary formula, which takes into account several factors including the player’s most recent performances, their direct performances against the opponent and their past performance at the venue.”

This is fairly self explanatory, it uses recent form (rolling mean from n matches), form against the opposition team and form at the game’s venue. I understand the thinking behind the latter two measures but personally I would be hesitant to use them. Most players play each team only once per year; the carryover form seems tenuous unless they happen to play on the same opponent player, against the same gameplan. Likewise, the venue form will only be useful for the home team.

The AFL Ratings system uses a long-term forecast to rate a player’s value:

“A player’s rating is determined by aggregating his points tally based on a rolling window of the previous two seasons. … only a player’s most recent 40 matches are used in the calculation of his rating. … A player’s most recent 30 matches are given greater weight in determining his rating. Matches 31 through 40 are progressively reduced in weighting”

This is a different kettle of fish. The trend over a long time is a better measure of a career value, rather than a form. As I write this, Scott Pendlebury is the 5th highest rated player in the league. He’s not in good form though (**see below), so he’s unlikely to rank that highly in his next round.

How can you handle cases like Tom McDonald and James Sicily, who have been moved to the other end of the ground and their output has changed (improved?) dramatically? I think there needs to be an element of long-term and short-term form. Let’s look at some case studies using my terrible visualisations.

Here are Scott Pendlebury’s Supercoach scores for the last few years.

The first two plots have rolling means of the last 5 and 20 games respectively (**he’s clearly out of form on this measure). The final plot I’ve separated his home and away games and plotted a rolling mean of his last 10 home/away games. Maybe his calf injury is hanging around and he’s not travelling well? Let’s check some others out.

Shannon Hurn is a nicer player to model; plenty of variation as you’d expect but a nice clean upward drift on the 20-game average; it’s probably a better predictor of future performance. Lately he’s been showing better form away than at home. Bodes well for the finals.

Finally, James Sicily has gone through that transformation from forward to back and has shown a dramatic improvement in this measure. His 20-game average is taking a while to catch up and it’s likely his 5-game average will be a better predictor.

With just a few examples it’s easy to see why there are benefits with a short-term and a long-term mean. There is possibly some benefit of considering home and away form separately too.

My model considers a weight of these three means for predicting the mean output of a player – their current form. Each of the seven (!) parameters I use to measure a player’s performance gets a mean of this form. The choice of 5- and 20-game means passes the eye test (for me, anyway) and gives good outcomes when simulating past seasons. Rookie players with n<5 games just have a mean of all of their performances, players with 5<n<20 get a 5-game mean and a n-game mean.

***

Now, what about the game-to-game variance from the mean(s)? The means will be good for giving a mean outcomes but variance will be useful in determining the probability of outcomes. In the finance industry, estimating volatility is where the money is.

I need to do more investigation on the best way of handling this, and I might write a piece when it’s a little more refined. My current method is to take a sample standard deviation of the measure over 10 games. I had been using 5 games and this was giving me some strange outcomes. Having done some more simulations, 10 seems to work pretty well as compared to larger and smaller samples.

Cheers for now!

# Round 12 Review

A bit of a tough round to tip with plenty of close games, a few upsets and a few blowouts really damaging my MAE, especially with the model’s margin underestimation. Oh well, it’s a beta!

Still measuring up well overall over the generous sample size of two live rounds. The relative indecisiveness in closer games is helping me salvage some Bits.

Onwards and upwards… after this man-flu leaves!

# An Introduction to the Model

Absolutely no-one has asked me for details on my methodology yet, and I’m happy to provide answers for these non-existent questions.

I guess the main idea behind the model is the concept of a team being more than just a sum of its players.

$\displaystyle P = P_t + \sum_i P_i$

Overall performance $P$ is a sum of team-related performance $P_t$ and the total contribution from each of the players $P_i$.

I arrived at this idea from observing the freely available statistics that are published by invaluable sites such as AFLTables and Footywire. Individual player contributions are easy to see and understand, but there are other features in the stats I was interested in; for example, does a Rebound 50 reflect on the performance of the player awarded the stat, or is it more closely related to the defensive structure of the team as a whole? Is five Rebound 50s worth as much if the opposition have had 80 inside 50s, as opposed to 40?

I divided the relevant statistics (almost all of them?) into different categories of team and player performance. I arrived at seven different categories. As is the go in footy data analysis circles I came up with a snappy acronym; SOLDIER. For each of the categories, I painstakingly weighted each relevant statistic to favour the statistics that better correlate with the outcome (winning the game). A team performance in a game is described by FOURTEEN (!) variables; seven for the sum of player performance and seven for the team performance. For each game, the difference in these fourteen variables is hypothesised to relate to the difference in the final scores, i.e. the margin.

I recognise that this model is considerably more complicated than other footy models I’ve read about online, but footy is a complicated game!

I began this project after spending some time learning about data analysis; in particular applying machine learning techniques. After doing a few beginner projects through sites like Kaggle, I figured I had enough of the basics to give this project a crack. Unlike the rest of my mathematical life where I use techniques that I have a strong base of understanding in, I have no more than a basic understanding of how machine learning actually works.

Once I have a better grasp on machine learning and refine my model, and the many parameters embedded within, I may publish more details on the categories and the statistics important to each.

I hope to be in a position to be able to predict results, rank players, make ladder predictions, but also to see if the machine learning models can give any insights into concepts such as team balance, matching up of teams with different strengths and weaknesses, etc.

This is primarily a learning exercise for me but I believe (please correct me!) that no other well-discussed footy model is using machine learning techniques, so I hope this is of interest.

# Round 11 Review

A pretty good round I think. My models seem to be under-predicting margins, something I didn’t really pick up until I started looking at individual games. Having said that, my probabilities are tending to be higher than other models around and that really helped my BITS score.

Perhaps the volatility estimation for player/team performances I’m using is not optimal and I’m getting a skinnier bell curve of simulated results than others. We shall see!

This year I’ll be focussing on tweaking my model, sussing out its strengths and weaknesses and measuring it up against others. Although I have simulated data from the first 10 rounds, that was simulated blind to the actual results, I will be measuring it against results only from this round onwards; just in case my slightly messy code managed to have prior knowledge.

# The AFL Lab

Welcome to the AFL Lab. This project is part of my ongoing education in data analysis. I love footy and numbers, so why not combine the two? I have a strong mathematical background but I’m comparatively weak on the statistics side. This is my attempt to rectify this, in a very reckless and un-rigorous way.

Normally when approaching a problem it is standard practice to start with something simple and add complexity (Occam’s Razor?), but I have gone all-in, throwing stats haphazardly at scikit-learn models. Will it work or will it explode?

My formulation is currently very unrefined, with many parameters (and probably way too many parameters) yet to be tweaked. Nevertheless, having simulated Rounds 1-10, 2018, my model has tipped 63, average margin 28.5 and a bits score of 16.61. According to the Squiggle leaderboard as of today, the leading model is on 62/28.17/14.58.

The model is not completely ready yet (it’s about 5 tips behind in a simulation of 2017), but it’s doing something right. So over the next few weeks I might write a few things about my modelling process and I’ll post round predictions/reviews and any other little fun bits I’ve found.

I’ll probably post a bit more frequently on Twitter at @AFLLab