USAR Rankings
2024 Update: The USAR Team is working to transition from a mixed model towards an ELO model in order to support programming around USAR Player Status. The existing ranking currently shows results from a previous Mixed Model (more context below).
The USAR Rankings are use a results-based model based on all USAR sanctioned play - more on USAR Sanctioned events here. Quantifying sanctioned play is an important part of legitimizing our sport and helping players gauge their skill level. As more events are sanctioned, we hope this model will be an important part of the roundnet community, the way tennis’s UTR and chess’ ELO are in theirs.
Read below for info on how we developed these rankings and how we will look to improve upon them moving forward!
Check out the “Player Pages” tab to review more detailed information by player (tutorial here).
*NEW* Check out the “Player Status” tab below - learn more about how player status works here.
These rankings were built in partnership with Fwango (fwango.io)
If you notice any discrepancies in missing games, incorrect teammates, or any other areas of concern as you view these rankings, please let us know by submitting your feedback.
The Approach
When selecting the best model, we approached the task with two major considerations:
1. Objectivity
Our number one goal was to treat all players objectively. Therefore, the only information used to create USAR rankings is based only on game data and not from people's opinions.
What IS considered:
Game Win/Loss results from USAR-sanctioned events (Contender-equivalent and above divisions for Open; Top division for Women) as reported on Fwango (fwango.io)
What is NOT considered:
Results from non-USAR-sanctioned events (These are USAR only rankings)
Game score results (since the reporting of these scores has been inconsistent)
Game stats
Subjective analysis or polling
2. Accuracy
Within the above constraints, our goal was to make the most accurate model as possible. The way we assessed this was using predictive accuracy. In order to test this, we gave the model results from a portion of the 2021 season and asked it to predict the results of other 2021 tournaments. We then compared the model’s predictions to the actual game results to see its predictive accuracy.
There are a number of questions we considered along the way, such as:
How should strength of schedule be balanced with overall W/L?
How should bracket games be considered compared to pool play games?
How should recent results be considered compared to earlier results?
Should a player’s partner’s strength be considered when making ratings?
Does a player need to “prove it” in a certain number of games/tournaments before achieving a high ranking?
So far, our findings indicate that a simpler model that places players based off of their W/L record and strength of schedule has the best predictive accuracy. Thus, these are the predominant factors for the 2022 version. As we get more data, we will be testing new features and their effect on accuracy to continuously fine-tune the rankings for 2023 and beyond.
Looking Forward
Updated 2022* This model is not a perfect depiction of player skill. Notably, it is limited by the robustness of the data that drives it. Ideally one day we’ll have access to a fuller data set that includes information like each player’s break and ace rate. Until then, in many cases, we should expect subjective “eye-based” rankings to be competitive with this model. Overall, we are confident that these rankings are highly optimized given the amount and type of data we have (game-level W/L).
We are excited about the future possibilities of the model. In the short term, we plan on updating the rankings semi-weekly after USAR sanctioned tournaments. We will soon be adding an interactive web app to explore how the rankings evolve across the season. Longer term, we hope to add score data, any other match data that becomes widely available, and continuously test new features. This is just the beginning of the USAR rankings and we hope you are excited about it as we are.
If you want to learn more about the specifics of the model or how the model is selected, please check out our FAQs.
If you find any data inaccuracies or player spelling errors, please reach out to info@usaroundnet.org
*2023 documentation updates in development
FAQs
-
The model we selected is a penalized logistic regression model using the lme4 package in R. Key Terms:
• Penalized: The penalization makes it so that It takes some time to move very high (or low) on the rankings, so players won’t immediately rocket to the #1 spot after going 1-0.
• Logistic Regression: Statistical method used to predict a binary outcome (Win or Loss). Takes players ratings as inputs and then gives a probability of team 1 winning. Players are ranked based on their estimated effect on winning games. Notably, this does not include point differential, so a 21-10 win gets the same amount of credit as a 21-19 win.
What does this mean:
By default, players come in with an average (100) rating. The model will first predict the outcome of each game based on the players ratings. It then compares those predictions to actual results and calculates the loss (the difference between the prediction and the result). For example if the model predicts team 1 has a 70% chance of winning and they win, the loss is 0.3 (1 - 0.70). If the team two wins, the loss is 0.7 (0.70 - 0). The model then iteratively adjusts players ratings to minimize the differences across all games.
We chose this model over an ELO based model based on comparisons between predictions on 2021 data. Since this model looks at all the data at once, there is no clear formula for how a single game updates the rankings.
-
All players (both contender and premier) are rated on the same scale and each game is weighted equally. This is due to the long term goal of using the same rating system for players of all levels. In addition, at non-STS events such as those hosted by LROs there are Open divisions that contain a mix of contender and premier players. We see quantifying these games and the potential skill gap between divisions as an important part of our system.
This, however, does not mean a contender win counts the same as a premier win. The strength of your opponent has a big influence on the impact of your win. Beating a team rated much higher than you will most likely increase your rating while beating a team with a much lower rating won’t do much to change your ranking.
We chose a two tournament minimum to try and help the model differentiate between divisions. If a player starts in Contender and plays two tournaments they 1) Qualify at the first tournament and then play premier at the second. The games against the premier players should help place them correctly. 2) They lose games at the first tournament and don’t qualify and then play a second contender tournament. Even if they were to win all the games at the second tournament the loses to at the first should help place them lower. This is not a perfect system however as there are a number of factors that can lead to inaccuracies such as the team they lost to in the first tournament not playing more tournaments and not being bumped down, or the team losing to only highly rated premier teams. The bottom line is the more tournaments and the more games the better.
In addition, as more Open divisions at USAR Sanctioned events occur there will be crucial crossover games to help the rankings accurately rank players.
-
Player scores are scaled such that an average* player is 100, with higher scores indicating higher player skill.
As a rule of thumb, the model would expect:
• 100-rated players to win about 50% of their games if they were to play with an average partner against two average opponents
• 110-rated players to win about 75% of their games if they were to play with an average partner against two average opponents
• 120-rated players to win about 88% of their games if they were to play with an average partner against two average opponents
• 130-rated players to win about 95% of their games if they were to play with an average partner against two average opponents
The average of a team's Player Scores is a good proxy for team talent. Thus, a team comprised of two 125-rated players should be roughly equal to one comprised of one 100-rated player and one 150-rated player
*Average refers to the average player across all qualified games. For Open, this means all Contender-equivalent and above divisions at USAR Sanctioned Divisions. For Women, this means the top Women's division at USAR Sanctioned Tournaments.
-
-
Pool play, bracket play, and consolation are considered and weighted equally
-
Rankings are improved when you win games at USAR approved tournaments! (Especially against good competition.)
-
At this time we are requiring that a player play in at least two sanctioned events to be considered.
-
Since this is a USAR ranking, only US based players are considered for the rankings.
International players do still have a rating within the model and US players get credit for beating them.
-
Unfortunately, at this time we are only releasing the top player rankings. We will be including other divisions in the near future.
-
The rankings system continuously updates player ratings based on W/L record and strength of schedule. Though your W/L record doesn’t change if you don’t play, your strength of schedule still might, as the model updates its understanding of the quality of your opponents. If the team you beat last weekend then goes undefeated this weekend, your rating will likely improve, as your prior win becomes more impressive in hindsight. The reverse can also be true.
-
Most likely, your partner’s record and strength of schedule when playing without you is more impressive than your record and strength of schedule when playing without them.
-
The only tournaments considered in rankings are USAR Sanctioned events. Talk to your tournament director, and work with USAR to get your tournament sanctioned so that the results will count!
-
The score is not currently considered, due to a lack of completeness in score reporting. However, it is incredibly valuable long-term to consistently and accurately report full scores of each game, set, and match played.
-
For Players:
Play in USAR events! The more data the better.
Data hygiene on Fwango
Have only two players on your team.
Do NOT drop players from your team.
Do NOT delete your old teams.
Please record all your game scores (not just 1-0). Roundnet Germany has a great explainer about this
For Tournament Directors:
Get your tournaments sanctioned by USAR
Data hygiene on Fwango
Ensure all registered teams consist of only two players.
If a match is a three game series create space for all three scores to be entered (not just 1).
Consistently label your all non-bracket stages “Pool Play.”
Record injury forfeits as 0 to -1 and non-injury forfeits as 0 to -2.
At the end of the day: make sure that all played matches are entered into Fwango and publish final standings.