Over the past two weeks, the Pittsburgh Steelers have dropped 12 passes thrown by quarterback Ben Roethlisberger. Over the full season, they have dropped a league-high 41 passes. Unfortunately, within most analysis and metrics of quarterback accuracy such as completion percentage and completion percentage above expectation, these plays reflect poorly on Roethlisberger.
Over the past few seasons, there has been an increase in the use of more advanced measures of quarterback accuracy. These are led by the expected completion percentage models such as the NFL’s Next Gen model, SIS’s pComp model, as well as public models. However advanced these models are, they all have the common limitation of attributing the entire outcome of a pass to the quarterback. Quarterbacks are the most important position on the field, and do heavily influence the outcome of a pass, but are not the sole contributor. Receivers, defensive backs, and others all contribute to whether or not a pass is completed.
In an attempt to more accurately measure only a quarterback’s contribution, SIS has created an Expected On-Target Rate model. This model uses SIS’s extensive charting data to focus solely on the accuracy of the throw, not if the throw was actually completed or not. The separation of those two components allows for a better evaluation of quarterback play.
The Model
SIS video scouts chart three levels of quarterback accuracy on every pass attempt: Completion, Catchable, and On-Target. On-Target is the highest level of accuracy as it does not include passes in which the receiver had to significantly adjust to make a play on the ball. We chose to use On-Target as the basis for this model due to the already explained limitations of completion percentage and the fact that Catchable throws would still include acrobatic receptions where the receiver should get a higher portion of credit.
As with all expected completion models, throw depth is a main component of Expected On-Target Rate. While throw depth can explain a large part of a throw’s difficulty, it is only one of many inputs into our model. Inputs also include the throw’s horizontal location, whether the quarterback was pressured, whether the quarterback was moving, the throw’s trajectory, route type, coverage type, stadium roof type, and other factors SIS collects.
The Numbers for 2020
After creating the model, the main question is does this meaningfully change how we currently perceive quarterback accuracy using expected completion percentage? The answer to that question is yes. Using our model to create a new metric On-Target +/- and then comparing it to SIS’s pComp +/- metric, we see some significant movement in the leaderboard.
Russell Wilson takes the top spot and Carson Wentz comes in at the bottom in both metrics, but there is quite a bit of movement in between. The players most benefiting from this new measure of accuracy are Baker Mayfield who jumped from ranking 16th in pComp +/- to third in On-Target +/-, Sam Darnold who jumped from ranking 31st to 20th, and the aforementioned Ben Roethlisberger who jumped from 29th to 19th. Players who fell most in the rankings are Drew Brees, who fell from eighth to 22nd, Andy Dalton, who fell from 14th to 26th, and Jared Goff, who fell from 12th to 23rd.
The top 15 quarterbacks in On-Target +/- feature some similar names at the top as pComp +/-, but also feature some of those mentioned who make a jump in the rankings.
Top 15 QB’s in On-Tgt +/- (2020, min 200 Attempts)
| Player | SIS pComp +/- Rank | xOn-Tgt% | On-Tgt% | On-Tgt +/- |
|---|---|---|---|---|
| Russell Wilson | 1 | 73.2% | 80.6% | 7.4% |
| Teddy Bridgewater | 4 | 75.5% | 82.8% | 7.3% |
| Baker Mayfield | 16 | 70.0% | 77.1% | 7.1% |
| Josh Allen | 2 | 73.2% | 80.2% | 7.0% |
| Aaron Rodgers | 5 | 73.0% | 79.3% | 6.3% |
| Deshaun Watson | 3 | 74.8% | 80.3% | 5.6% |
| Kirk Cousins | 6 | 74.1% | 78.9% | 4.8% |
| Gardner Minshew | 11 | 71.7% | 76.3% | 4.6% |
| Ryan Tannehill | 13 | 75.5% | 79.9% | 4.4% |
| Kyler Murray | 18 | 73.9% | 78.3% | 4.4% |
| Joe Burrow | 15 | 74.0% | 78.3% | 4.4% |
| Derek Carr | 10 | 75.7% | 80.0% | 4.3% |
| Ryan Fitzpatrick | 7 | 75.0% | 78.9% | 3.8% |
| Nick Foles | 21 | 73.2% | 76.8% | 3.7% |
| Patrick Mahomes | 9 | 74.9% | 77.8% | 2.9% |
As mentioned earlier, Russell Wilson tops the ranking on On-Tgt +/-, though Teddy Bridgewater is right behind him. In the third spot is Baker Mayfield who is the largest beneficiary of the switch from expected completions to expected On-Target. Mayfield was being hurt by drops similarly to Roethlisberger, as his receivers have dropped the eighth-most passes. He is also benefiting in On-Target +/- due to being accurate on some more difficult throws outside of the pocket and having only played one game inside a dome this season.
Final Word
Expected completion percentage models have become the gold standard to measure quarterback accuracy, and for good reason. They adjust for the difficulty of the throw and are much better measures than simple completion percentage. However, they also have limitations when their results are attributed solely to the quarterback. Using charting data, we can help improve upon that limitation when evaluating quarterbacks by only looking at the accuracy of a throw and not the outcome. This is where On-Tgt +/- comes in. With this new metric, Russell Wilson still rates as the king of accuracy and Carson Wentz still rates at the bottom, but it does help improve how we perceive quarterbacks who have been hurt by the players around them.
