Returns to QB Experience: A graphical look

[pelagius]

Goat had a nice thread about Max Hall's expected improvement:
http://cougarguard.com/forum/showthread.php?t=17709 . I wanted to add to his analysis by taking a look at game by game passing efficiency data since 1984 (thanks for the data Indy). I am going to split passing efficiency outcomes by qp experience buckets. However, I am not going to use raw passing efficiency. I am going to modify passing efficiency by controlling for opponent quality (Massey MOV end of year computer ranking) and where the game is played. The following graph presents passing efficiency controlling opponent quality and location as a function of qb experience in terms of games played. The way to read the graph is this is the expected passing efficiency if the game was played at home and the rank of the opponent was between 60-80 (a slightly above average MWC opponent or a poor BCS opponent).




I was actually surprised a bit. I thought we would see more improvement in the first year but you can see passing efficiency is basically flat the first 12 games. This is not so true in raw passing efficiency. You do see some improvement in raw efficiency but this would be due on average to playing lower quality opponents later in the year. The big jump comes at the beginning of year two. The jump is about 20. There is some improvement after that but it is really pretty small (I wonder if the improvement is bigger when really good opponents are played).

[PaloAltoCougar]

That's a very interesting graphic. Like you, I'm surprised the improvement is fairly flat, except for the marked jump in what I assume could be described as going from season one to season two. If an uninjured Max Hall achieves a 20 +/- improvement in his PER this year, BYU will probably not be heading back to Vegas.

[Insensitive PAP]

[pelagius]

Another interesting question is how qb experience is affected by opponent quality. Do experienced qbs play a lotter better than inexperienced qbs against good opponents. This time I am going to split into opponent quality buckets (year-end massey computer rank) and look a passing efficiency (adjusted for game location) for quarterback with less than a season of experience (inxperienced QBs) and more than a season of experience (experienced QBs).



Interestingly, experienced QBs don't perform a lot better against very good competition. Experienced QBs play much better than inexperienced QBs , historically speaking, when BYU plays good and mediocre teams. Against very poor teams both types of QBs dominate. Inexperienced QB play is much less sensitive to opponent quality. You don't see big differences unless inexperienced QBs are playing against poor competition. On the other hand, experienced BYU QBs have been able to pick apart much better competition but both types play similar against really high quality competition.

This seems to support the notion the coaches probably adjust to a more conservative passing game against good competition particularly when BYU has an inexperienced QB and the adjustment starts earlier for the inexperienced QB.

[TheSizzle36]

Originally Posted by pelagius

This seems to support the notion the coaches probably adjust to a more conservative passing game against good competition particularly when BYU has an inexperienced QB and the adjustment starts earlier for the inexperienced QB.

Can you look at passer rating and correlate it to games where BYU is able to run the ball for 39 yards or less, 40-74, 75-99, 100-124, 125-149, and 150+? I would expect to see passer rating increase as BYU is able to rush for more yards, but I wonder if it would tail off some, since with a better running game, your passing yards, and most likely passing TDs wouldn't be as high, possibly skewing the numbers back down.

I'd also think that this would play into your hypothesis above as well, that when BYU plays better teams, they aren't able to run the ball as well, making it harder to pass, as well as making the passing game be more conservative.

[pelagius]

Originally Posted by TheSizzle36

Can you look at passer rating and correlate it to games where BYU is able to run the ball for 39 yards or less, 40-74, 75-99, 100-124, 125-149, and 150+? I would expect to see passer rating increase as BYU is able to rush for more yards, but I wonder if it would tail off some, since with a better running game, your passing yards, and most likely passing TDs wouldn't be as high, possibly skewing the numbers back down.

I'd also think that this would play into your hypothesis above as well, that when BYU plays better teams, they aren't able to run the ball as well, making it harder to pass, as well as making the passing game be more conservative.

I don't have rushing yards but I suspect you are right in general that efficiency numbers go down in part against good competition because the running game is often futile and defenses can just key on the pass. I also agree that the relation wouldn't be linear either. A really crappy running game probably hurts pass efficiency and so does a really great running game (touchdowns do matter a lot in the formula).

One thing I could do pretty easily is a "good" running back fixed effect. How much does having an above average running back effect passing efficiency? Its not perfect and doesn't directly answer your question but on the other hand it does avoid some of the endogeinity issues associated with rushing statistics (i.e. teams run a lot at the end of blowouts, etc) but clearly not entirely.

I think another interesting cut of the data would be to look at mechanically why the passing efficiency ratio dips. Does it dip because yards per catch go down, completion percentage drops, difficult making touchdowns, or an increase in interceptions?