Thursday, October 22, 2009

Methodology

My methodology for developing these rankings is based on the following three questions:

1. Given only a game's final score: if that score were the average score of an infinite number of games played between teams A and B, what are the chances team A beats B in any given game?
2. What are A's chances against B on a neutral site?
3. What are A's chances against an average team on a neutral site?

Now to delve a little further into each one.
1. Bill James came up with a fantastic formula called Pythagorean expectation.  The numbers in this formula are squared (for baseball), but only because it's convenient, and close enough; the exponent is actually a bit different, and will vary from sport to sport.  Basketball would use a variant around 15, the NFL in the mid-2s and NCAA football around 3.  (To find that number, I would simply solve for the number that gives the lowest standard deviation among the average differences of actual vs. expected win percentages.)

This comes in handy when figuring out how two teams fared against each other.  For example: if you were to take all bowl games Penn State has played in (since 1922) and only had the total number of points to go on, you'd see that they scored 855 and allowed 671.  Using the Pythagorean formula - revised for college football - you'd come out with (855 ^ 3) / ((855 ^ 3)+(671 ^ 3)) = 0.674.  You could conclude that in those 41 games, they'd win approximately 2 out of every 3.  In fact, their record is 26-13-2, or 0.658, a mere 0.016 difference.

Likewise, given any single game, you can measure their strength of victory using the same technique, to find each team's "game score."  If you were to always shutout your opponent, you'd win 100% of the time, so a shutout yields a score of 1.0.  A tie would yield a 0.5, and a 31-10 win would give you a 0.968.  In other words, if all you knew was that a game played over and over again averaged 31-10 for team A, you'd expect that team to win almost 97% of the time.

2. To make these results more relevant, we then account for home field advantage.  Since the home team, in general, wins about 60% of the time, we can build that advantage into the formula by raising the visiting team's score and lowering the home team's.  If two teams score the same number of points, but one team has home field every time, we can assume that the visiting team is actually a bit better and would win more often if playing on a neutral site or at home.  Therefore, a different formula is used for each, such that an average 20-17 win for the home team (or a game score of about 0.6) turns into a 0.5 for both teams, as they are essentially even when accounting for home field.  Here are the formulas, where GS represents the game score reached in step 1:
Home team: (GS * (1-0.6)) / ((GS * (1-0.6) + ((1-GS) * 0.6))
Away team: (GS * 0.6) / ((GS * 0.6) + ((1-GS) * (1-0.6)))

3. At this point, we can tell how each team did against each other in a given game, and average all the games together to get a pretty accurate measure of how they've done.  But we want to compare them to the league as a whole, which is where strength of schedule comes in.  By replacing 0.6 in the latter (away team) formula above with the opponent's site-adjusted game score average, we arrive at a better measure of how good a game was played by any team on any given day.  That is, if a bad team loses a close game to the best team in football, they deserve credit for not being blown out.

The problem that comes into play here is one of circular references: you can't use each team's "final" adjusted number to adjust that of another team it's played.  To avoid this, I simply run step 3 based on the numbers returned from step 2 and then use those results, and then replace them over and over again until the results get more and more narrow and any further iterations don't affect the rankings.

Keep in mind, this doesn't take wins and losses into account.  A team that has a few very close losses to good teams (like the 2009 Sooners) will still rank highly - I don't presume it will take the place of the BCS anytime soon but it is a pretty good way to rank teams' overall performance and determine how they will do in the future.

No comments:

Post a Comment