As some of you have already seen, the coach rating (CR) displayed on C division match reports looks a bit unusual. Numbers are significantly higher and the increases and decreases are quicker. This is the first step in a larger series of changes which aim to replace the core CR formula with something different (and that I believe is better).

The core formula of the old CR system was a system called Elo (named after the physics professor who came up with it, Arpad Elo), a system used widely in the world of chess. It is essentially a system designed so that winning a match against a stronger opponent gains you more rating than winning against a weaker opponent. This is done by a relatively simple formula, and functions fairly well overall (I won't go into deep detail about that here).

For FUMBBL, I adjusted the formula slightly to allow for asymmetric matches (ie, matches where the sides are not equal in strength) and it has been working well over the years. So what's the point of changing then? There are a couple of mathematical details which are less than perfect (again, glossing over this here), but the main issue is that Elo ratings don't change unless people play. This means that the top rated coaches will remain at the top indefinitely, which creates an incentive to not play if you're at the top. Clearly, this is a bit of a problem. In the past, people have suggested decaying the rating over time, which is fundamentally an interesting idea but simply does not work in an Elo based system. Removing (or adding) rating to an Elo system will inevitably cause what's called rating inflation or rating deflation, where a specific rating is worth more or less at different times (there's some of that happening in Elo regardless for subtle mathematical reasons) and is why I never added a decay into the previous rating system.

Ok, enough of the preamble. What's different? What is this new system and what does it do better?

The new formula is based on a system called Glicko 2 (named after Mark Glickman, a statistician at Harvard University), a successor to their Glicko (1) system. To simplify a complex system (I may delve into more detail in a separate blog at a later date), it can be seen as a more comprehensive variant of the Elo system (and it can, in fact, be configured to model the Elo system perfectly), where additional information is added beside the raw rating number. The most important addition is the concept of "ratings reliability" (or ratings deviation). To describe it in a more plain manner, it's essentially a measurement of uncertainty (as implied by the similarity to standard deviation), and gives an indication of how precise the number is in mathematical terms. A new coach with a rating of 1500 and a rating deviation of 150 would mean that the system thinks the person has a rating somewhere between 1350 and 1650, a fairly wide range. A more experienced coach could still be rated 1500, but have a smaller deviation (say 50), meaning the system would think the person has a rating between 1450 and 1550, a much narrower range. This deviation is taken into account, and playing against someone with a larger deviation will mean that the change in rating for a given game will be reduced given that the opponent rating is less precise.

The more a person plays, the more certain the system will become that the rating is "correct" and the deviation will be reduced. However, if someone does not play for a given period of time (called a rating period), the uncertainty will increase instead. It's worth to note that the rating itself does not change, and it's only the rating deviation that will increase.

The fact that this uncertainty moves as it does allows me to use the Glicko based system to implement "decay" without really affecting the system. Instead of displaying the raw rating, the site displays rating reduced by the deviation (in a somewhat non-linear way). This means that the apparent rating will decrease as time goes by and people who stop playing will lose some of their rating. There is an upper limit for how high that uncertainty will get, meaning there is a floor for how low your rating will drop over time.

So at its core, a high deviation means your rating will move faster and when playing enough it'll get more secure and your rating will stabilize to an extent. Taking an extended break will increase this uncertainty and you will need a few matches for the rating to settle down again. Overall, it's a very interesting system that feels "right" to me.

With all that being said, the Glicko based system is in place and gets updated as matches are played. I am mostly happy with the current configuration, but will be looking at feedback from people who are interested in CR in general. In particular, the current rating period is set to 2 days, which might be too fast but we will see how it turns out. The deviation doesn't exactly shoot into the sky just because you missed to play in a single rating period.

For now, the only place this rating is shown is in match reports for C division games, and I will be changing over various CR views to the new system over time. Clearly, the coach page needs to be updated to show it, and I need to update the rating page that shows all the different variations.

The multiple ratings we have right now will remain (and will in fact be increased with another category):

- Overall - shown on the match pages - Affected by all C games being played
- Open - affected by matches through the gamefinder
- Blackbox - affected by matches through the blackbox scheduler
- Tournament - affected by matches in tournaments (this is new)

In addition, all these exist for each roster as well (so you will have a Tournament Orc rating for example).

I may play around with the rating periods for the different categories (tournament ratings, for example, could switch to monthly or quarterly rating periods, or synchronize with majors). For now it's easier to keep it all the same, and I can absolutely go back and recalculate ratings when configuration changes are necessary.

Either way, feel free to take a look at your past C matches and let me know what you think. I know the data isn't super visible yet, but I aim to resolve that asap.

Happy cherry picking! ;)