Total QBR?

[Fosco33]

Well - the past few years have seen an entire remake of prosport metrics - so why not alter the QB Rating with the Total QBR?

Just saw the ESPN special on it's rollout - and it makes a lot of good points. Penalizing QBs for bad decisions, sacks, fumbles, not recognizing game situations and minimizing nonimpact negative plays.

In reading about the metric, it's ridiculously complex. The KISS method has not been followed here. So we'll rely on the talking heads to tell us that Brady, Manning, Ryan, Rodgers and Vick are #1-#5 (really, after all that, really ESPN).

And to me, that's where it will both succeed and fail. It's not reproducible - and there's an element of subjectivity. So the likes of Manning - who didn't have a great year - still somehow rise above Ryan/Rodgers who had great years....:bs:

The public will surely latch onto the finite simplicity (0-100) and in time realize that a mid 60s year is pretty damn good. I dunno - not sold on it quite yet myself.

Thoughts from ratland?

[Lurker64]

It's not a good statistic. While it's generally a good idea to work things like "sacks" into the consideration of passing efficiency, the whole nebulous "clutch" aspect of the statistic is both vague and not actually a meaningful thing to measure. A player who always plays well will score less on "clutchness" than a player who usually plays poorly but plays very well in pressure situations, but which of those two players do you actually want on your team?

Ultimately the problem is that the old "Passer Rating" statistic which ESPN is trying to do away with is actually most telling as a defensive statistic, i.e. the passer rating of the other guy(s). This is a much better indicator of quality passing defense than the usual "passing yards allowed" metric. "Defensive passer rating" (and the related "Passer rating Differential", which is "Your passer rating minus defensive passer rating") has a very strong correlation with actual success, so we shouldn't be in a hurry to do away with them. Since 1940 67 of 69 NFL champions have finished in the top 10 for "defensive passer rating", whereas 68 of 69 NFL champions over the same period finished in the top 10 in "scoring differential." The Packers dynasty of the 60s lead the league in defensive passer rating in 1962, 1965, 1966 and 1967 and they lead the league in passer rating differential in 1961, 1962, 1965 and 1966; they won championships in every one of those years. The 2009 Saints who won the superbowl were 3rd in defensive passer rating and #1 in passer rating differential. The 2010 Packers who won the superbowl (I really like typing that) were #1 in defensive passer rating and #1 in passer rating differential.

So I ask, why are we in a hurry to do away with this stat?

The one thing we should probably do with the passer rating is normalize it. The arbitrary maximum of 158.33 is weird, and can be done away with by "dividing by 158.33" which will result in a 100 point scale.

[King Friday]

I'm really not for any "statistic" that you need a supercomputer to calculate. I generally like the idea of trying to incorporate everything into a rating, but I think it would be nearly impossible to come up with one that actually was a good representation at least 95%+ of the time.

To me, the ESPN rating doesn't account for the strength of a QBs supporting cast or the strength of the defense the QB is facing...which is a signifcant weakness.

[smuggler]

Not even as good as passer rating. If you want to get a look at how good a payer is, just go to football outsiders and look at their DYAR stat.

[Lurker64]

Not even as good as passer rating. If you want to get a look at how good a payer is, just go to football outsiders and look at their DYAR stat.

Yeah, TQBR basically fails in every way.

1) It's basically impenetrable unlike traditional passer rating, which anybody who can do basic math can figure on paper.

2) What it describes is not actually clear, DVOA and DYAR are basically the football versions of the classic sabermetric statistic: "how well do you do in these situations compared to a theoretical average player in your position" adjusting for the quality of defense you are facing. What does TQBR describe? Quarterback goodness? How do we calibrate "goodness of our quarterbacks" is it based on wins? If so, we should stick with a statistic that is strongly correlated to victories like Passer rating. Is it based on performance? If so, we should base it on a defense-adjusted comparison to what "average performance" looks like.

3) Not only this, but it actually gets things backwards. TQBR values "clutch performance" i.e. ability to execute in high leverage situations like the fourth quarter and overtime. But in fact this is preciselly the opposite of what you should do. Emphasizing fourth quarter stats emphasizes the performance of QBs who play for poor defenses or play poorly for the first three quarters, as these are the circumstances that get a quarterback in to situations where the offense passes exclusively (i.e. why Matt Schaub is historically a great fantasy QB: he has a good WR and he's behind all the time.) On the other hand, the quarter in which good QB performance actually correlates to victory is actually the first quarter, since a team that passes well in the first quarter and jumps out to a lead is in the best position to actually dictate the rest of the game.

You'd better believe you're going to hear about it on MNF and SportsCenter this year though.

[Fosco33]

I thought we'd find at least one person who thought this was a good idea.... guess ESPN spent all that money on creating the metric, hiring spotters for every play, analyzing every play for last 3 years, etc... But they don't seem to have asked the fans if we care.

So- what is ESPNs motivation for creating Total QBR? Are they seeing a trend in new metrics in other sports and want to own a 'metric'.

A few other points I was thinking about:
1. In blowouts, both QBs are essentially out of consideration for moving their QBR. Think about the games AR had last year and the dismantling of some teams.
2. I like traditional passer rating in limited instances - like: how well does a QB do in the 4th Q, from behind, on 3rd down, etc. The QBR can only be generated 12 hours after a game and shows the total performance...

On the flip side, ESPN showed that the winning QB had the highest QBR in 85% (ish) of cases - higher than looking at TO Margin. But when they calculate their metric based on outcome vs. expected outcome - it's like, 'No shit, sherlock.' The only cases where it's a push or QB 'gets screwed' are in Def and STs taking a lead/winning in 4th Q.

ESPN fail

[mission]

What I don't like about it is that it takes away credit to the QB on long yards-after-catch plays. Doesn't it take a perfect pass in stride to give the WR an opportunity to run with it? I think that's one of AR's strengths.

Anyway, I saw the stupid show ESPN had on the other day with Gruden and Jaws (I believe). It was almost as bad as The Decision.

[swede]

It was a transparent effort to elevate a few favored quarterbacks by working backwards statistically. A mi, no me gusta.

[pbmax]

It's not a good statistic. While it's generally a good idea to work things like "sacks" into the consideration of passing efficiency, the whole nebulous "clutch" aspect of the statistic is both vague and not actually a meaningful thing to measure. A player who always plays well will score less on "clutchness" than a player who usually plays poorly but plays very well in pressure situations, but which of those two players do you actually want on your team?

Ultimately the problem is that the old "Passer Rating" statistic which ESPN is trying to do away with is actually most telling as a defensive statistic, i.e. the passer rating of the other guy(s). This is a much better indicator of quality passing defense than the usual "passing yards allowed" metric. "Defensive passer rating" (and the related "Passer rating Differential", which is "Your passer rating minus defensive passer rating") has a very strong correlation with actual success, so we shouldn't be in a hurry to do away with them. Since 1940 67 of 69 NFL champions have finished in the top 10 for "defensive passer rating", whereas 68 of 69 NFL champions over the same period finished in the top 10 in "scoring differential." The Packers dynasty of the 60s lead the league in defensive passer rating in 1962, 1965, 1966 and 1967 and they lead the league in passer rating differential in 1961, 1962, 1965 and 1966; they won championships in every one of those years. The 2009 Saints who won the superbowl were 3rd in defensive passer rating and #1 in passer rating differential. The 2010 Packers who won the superbowl (I really like typing that) were #1 in defensive passer rating and #1 in passer rating differential.

So I ask, why are we in a hurry to do away with this stat?

The one thing we should probably do with the passer rating is normalize it. The arbitrary maximum of 158.33 is weird, and can be done away with by "dividing by 158.33" which will result in a 100 point scale.

The other problem with clutch is how they define it. A good QB on a team with a bad defense will always be chasing the last score in the 4th Quarter. Hence, it ESPNWorld, he gets clutch points.

A good QB on a good defensive team might score the most important TD in the game in the second Qt. And get no clutch points.

The other thing missing, as pointed out at Football Outsiders is adjustment for defensive opponents. Good QB in garbage division who beats up bad defenses and rarely plays a good D will have no context applied about the quality of opponent.

[King Friday]

It will be interesting to see how the other networks covering the NFL handle this. Since this rating is apparently calculated behind some curtain in Connecticut by a Wizard of Bristol, will FOX, CBS or NBC have access to calculate these ratings in game? I would assume these other networks aren't going to bother with this rating, so we will only be seeing it on MNF and ESPN programming. That is only going to confuse things for everyone...which I guess is what ESPN is hoping for since it will drive ratings to their talking head outlets on radio during the week.

[run pMc]

I think it's stupid and unnecessary. The Football Outsider stats using DYAR are better, plus there's already a QB rating that is used widespread.
Besides, I have this weird kneejerk reaction against anything ESPN promotes, and I am already bracing for the countless in-game graphics showing the TQBR.

[mraynrand]

One advantage I can see is that TQBR might give a better booth interview than Ashton Kutcher.

[pbmax]

Be prepared for ESPN to program around it like Bracketology though. Unfortunately, it elevated Matt Ryan into an elite category despite some other obvious concerns about his game. So it gives me one more reason to watch him crash and burn.

And Jaws, who in all other facets is wonderful, can get stuffed when he insists that the only thing that matters is Wins. We get it Ronnie, you are Mr. Old School. You are also better than this, by insisting that the only metric that counts for you ignores the differences among the members of the team. I think Ron, since he insists that this is the case, should only be allowed to come on camera and report the score, since that is all we need to know. Everything else is fluff Mr. Throwback.

[Smidgeon]

My problem with the TBQR is nobody else can calculate it. Whereas passer rating can be figured out anywhere by anyone.

[MadScientist]

My problem with the TBQR is nobody else can calculate it. Whereas passer rating can be figured out anywhere by anyone.

Yea, that's a problem. Even if the formula for QBR is odd, you can check quickly on a web page and do a few "what if's" based on the game. No chance with TQBR.

The other problem is that when you throw in a bunch more variables into the mix, you need a bigger sample size to make sense of it. Will a one game TQBR have any value whatsoever. At least you can get a feel for how well a QB did with the regular QBR for any given game.

[Smidgeon]

Yea, that's a problem. Even if the formula for QBR is odd, you can check quickly on a web page and do a few "what if's" based on the game. No chance with TQBR.

The other problem is that when you throw in a bunch more variables into the mix, you need a bigger sample size to make sense of it. Will a one game TQBR have any value whatsoever. At least you can get a feel for how well a QB did with the regular QBR for any given game.

And how much of the TQBR is subjective? If the answer is even just one part, then it's a poor statistic.

As for regular QBR, it originally was never intended to be a measurement of one play. It was designed so that even one pass could be measured, but the design was to measure a season's worth of play. Which is why it's interesting to me when the stat is broken down to statistically insignificant measures (i.e. on first passes of the third quarter, in 2009, the QB had a rating of 152.5).

[Upnorth]

Considering this "stat" has a element that captures a backwards cast opinion, not hard numbers, I am suprised it only has a 80% correlation. If I designed a stat that allowed for using opinion on past events I would game it to 95% correlation so I look like a genius! The people who created this "stat" failed even when cheating.