rankOar Women's College Rowing Computer Ranking
Week 3 Computer Ranking
April 20, 2022
What is the RankOAR?
April 18th poll: We implemented a better test to make sure crews are sufficiently connected to the central mass of races, which excluded a few of teams that were previously ranked. This will improve the overall quality of the rankings by making sure that teams have comparisons to all the other teams (think: 6 Degrees to Kevin Bacon).
We saw a lot of movement in the rankings this past week as there was more racing between crews that only had distant comparisons previously. There are still a few teams with only a single weekend of racing connecting them to the rest of the teams, but the rankings as a whole have far more data than last week to make accurate comparisons.
The RankOAR is an approach to computer rankings that simulates each actual rowing race thousands of times. Each simulation is based on what are the likely outcomes if this race happened over and over, with the normal variations that happen in a race, guided by the outcome of the actual races. The mathematical basis is Luce's Choice Axiom, which is an elegant way to deal with uncertainty in comparisons between items. It is the same algorithm that makes your "what else to watch" suggestions on Netflix and that powers the Elo rankings for Chess. Like other ranking systems, these rankings are not intended to replace polls by knowledgeable experts.
Rowing, like all athletic competitions, involves a degree of uncertainty. Athletic performance in elite-level athletes generally varies as much as 1% from day to day. The problem this ranking system solves is how to properly rank broad comparisons with uncertainty about the uncertainty. This problem is simplistically described by the following results:
From a small number of races, it is hard to determine if a single event is the result of a good or a bad day by each crew. In the above example, it is not intuitive which team would likely be faster at week 4. When this problem is scaled to over a hundred teams and three months of races, it should be obvious that any rankings are not an absolute comparison.
The RankOAR makes no attempt to predict time differences in future races, but instead grades the teams with a relative number that generally falls between 0 and 10,000. Teams that are exceptionally fast will be near, or even above, 10,000. While the bottom teams will be near, or even below, 0. The numerical distance between teams signifies how close the teams are predicted to be in next week's events. Small differences generally predict more competitive races.
Methodology:
Luce's Choice Axiom is a well understood method to deal with uncertainty in comparisons. The best current generalized iteration is the Plackett-Luce Choice Axiom, which is accessible through the Choix package in Python. To better match college rowing performance, we add an age element, where older races count for less value than more recent races.
RankOAR does consider margin of victory, but only as far as it goes to determine how likely Team A is to beat Team B if that same race was repeated thousands of times. As a general guide, the simulations expect that in a given race, if Team A beats Team B by a boat length, Team A will win ~70% of the simulated races of that particular race. If the margin is three boat lengths, Team A will be expected to win >98% of the simulated races. While heats are less important than finals, the nature of the simulation generally accounts for teams 'saving themselves' for the finals if their place is secure.
Ranking results get published as soon as there is sufficient data for the model to converge on a solution. Generally, the First Varsity Eight (1V8) in Division 1 has the most early-season data and can produce a ranking before other events or divisions. All rankings points are probabilistic, meaning that there will be small variations each time the model is run, but the scale of the variations is such that only teams which are functionally tied in the rankings might be affected. NCAA Projected Finish uses the same points system that the NCAA championships use, modified for a larger pool of teams.
This is a naive model, as it does not consider underlying aspects that can affect performance, such as how far the teams must travel to the race, how long since their last race, their lane, if things went wrong on race day, what race conditions are, and who is missing from the lineup on race day.
Data:
To produce the best possible results, the model includes all available college results. The vast majority of this data is from Row2K. Standard ranked events include the 1V8, 2V8, 1V4, 2V4, and 3V8/1N8. If there is sufficient data, 3V4 and 4V8/2N8 events are also included. Boats that row in an event different from their normal event are not included, except when they faced a similar boat in that race. While lightweights are included, they are difficult to rank since they are not always entered in the same events. For example, is the lightweight boat entered in the 2V8 event a true 2V8 lightweight boat, or is it the 1V8 lightweight boat entered in the race below that school's 1V8 heavyweight boat? All divisions, including club races, are included. Due to the difficulty in cleaning the data, it is currently limited to only the women's rankings.
RankOAR history:
The rankings were developed by Nate Carmody with support from Curt Hastings. While Nate has a 20-year history in rowing, Curt has a 20-year history in Big Data and Machine Learning and once rowed 500m in 1:50 on a C2 erg. Nate coaches high school rowing out of Sandy Run Boathouse on the Occoquan and is most recognizable by his constant canine companion Teddy.