Is Chip Kelly in for a Sophomore Slump?
Rejoice! The doldrums of summer, and the seemingly-endless stream of unsubstantiated speculation about the coming season is mercifully subsiding. Eagles training camp is here, and thus marks the official unofficial start of the 2014 NFL season. Huzzah! Now we have some actual news to read about!
But games that count are still quite a ways away, and something has been bugging me. While perusing the various forms of Philadelphia sports talk-radio over the last couple of weeks (Hey! Have you heard our excellent podcast yet?), I noticed that one of the more persistent topics of
unsubstantiated conjecture debate has been “Are Chip Kelly’s Eagles going to improve or decline in year two?”
The argument generally breaks down into one of two run-on sentences: 1) Chip’s offense has been torn apart in NFL film rooms all offseason, and it’s a college offense anyway, and it was sort of a flukey year, and the defense still sucks, so the Eagles will be worse in 2014, or 2) Chip is a genius, so when opposing teams make adjustments, Chip’ll adjust them right back, the team is used to Chip now, and the defense didn’t get any worse but the offense is gonna get better, so the Eagles will be better than last year.
Frankly, both sides make a somewhat cogent point. What bothers me – what always bothers me – is the lack of context…
From 2004 to 2013, there were 66 season-to-season head coach changes (not counting interim HC’s). Of those 66 lucky guys, 8 were fired before the start of their second season at the helm. Seven of the guys who kept their jobs were hired last year, which leaves us 51 head coaches over the last ten years who have had at least a two-year stint with their team.1
The circumstances under which a new coach comes to power are widely variable. There are some things that are expected: Such changes are typically accompanied by a house-cleaning, in which the present coordinators and assistant coaches are replaced by shiny new ones. Frequently, a roster purge follows. But this isn’t always the case, and these differences can make direct comparison difficult.
For example, look at Marty Schottenheimer getting the axe for Norv Turner after a 14-win season in 2006, in which the Chargers’ offense put up the most points in the league and their defense let up the 7th fewest. Then consider Norv’s subsequent squandering of that wonderful talent. You can be certain that the 11 wins Turner got from the Chargers in ’07 had a lot to do with the roster that Schottenheimer & Co. left behind.
Just to quickly confirm this common-sense judgement: among the 51 head coaches that managed to keep their jobs for two-or-more seasons, there is a significant positive correlation between the number of wins the previous coach earned in his final season, and the wins that the new coach earned in the following year (r = +.408, 51 coaches). Put plainly, the more wins the prior coach earned in his final season, the more wins the new coach was likely to earn in his first season.
Not only is this conclusion logical – it’s not often possible to turn over a full roster in one offseason – but mathematically, there is less than a 1-in-200 chance that the relationship is a fluke. By contrast, there was not a correlation between the prior coach’s final-season win total and the new coach’s second year wins (r = +.041).
The NFL can count this as a success of its overall strategy to produce competitive parity, as it seems the deeds (or misdeeds) of a prior regime are frequently nullified within two seasons. The lack of connection makes it much easier to attribute the success or failure of a sophomore head coach to that coach alone and not his predecessor.2
Since each new coach may have a very different starting point, I took a look at team rankings in four important areas: point scored, yards gained, points allowed, and yards allowed (1 league-best, 32 league-worst).3 4 As for the immediate predecessors of the 51 coaches in question, here’s what I found:
PRIOR COACH Average / Median Performance 2003-2012
Wins: 5.2 / 5
Points Scored Rank: 22.8 / 26
Yards Gained Rank: 22.3 / 25
Points Allowed Rank: 22.4 / 24
Yards Allowed Rank: 19.9 / 22
This is pretty much what you’d expect. The average team that fired its coach was in the bottom-third/bottom-quarter of the league. The median ranks for each of these categories are slightly below the mean, which indicates that most teams that fired their coaches were pretty damned bad, but there are a few cases in here where a team with good numbers fired its coach anyway and that skews the average.
Next, let’s look at the first-year outcomes of the new coaches. (Keep in mind, all of these coaches were hired from 2004 to 2012 and kept the job for at least two seasons, just like Chip Kelly.)
NEW COACH Average / Median Improvement – First Season5
Wins: +2.1 / +2
Points Scored Rank: +2.9 / +1
Yards Gained Rank: +2.0 / +2
Points Allowed Rank: +6.2 / +5
Yards Allowed Rank: +2.4 / +3
CHIP KELLY Improvement – First Season (with 2013 league rank)
Wins: +6 (10 wins)
Points Scored Rank: +25 (4th)
Yards Gained Rank: +13 (2nd)
Points Allowed Rank: +12 (17th)
Yards Allowed Rank: -14 (29th)
Quite obviously, Chip’s first season is very favorable compared to the average case and the results certainly fit the bits that he’s revealed to the public about his football philosophy. Chip doesn’t care about yards or possession outside of the tangential connection between those stats and scoring. Through Billy Davis, he installed a defense that sometimes allowed opponents to travel with impunity through the middle third of the field, but tightened up in the scoring area. All of that is reflected here.
Saying that the Eagles were an immensely improved squad last year is obvious, but in comparing Chip to the average case, the 2013 Eagles look like a truly remarkable reversal. Can we realistically expect the team to keep improving through year two?
We’ll start by looking at how the same 51 coaches above fared in their second season on average. The following is a comparison of change from year-one to year-two of the new HC.
NEW COACH Average / Median Improvement – Second Season v First Season
Wins: +0.1 / 0
Points Scored Rank: +1.9 / +2
Yards Gained Rank: +2.5 / +3
Points Allowed Rank: -1.1 / -2
Yards Allowed Rank: +0.1 / 0
Score one for the “Eagles won’t be better” side. While the average second-year coach did manage to improve again on offense, the defense stagnated or got worse. Damningly, the average coach won about the same amount of games as he did the previous season.
But Chip Kelly is no average coach. He’s an innovator, with fresh ideas and the power to do things his way. Perhaps it would be more helpful to whittle down the results to see how the second year went for coaches who had similar success in their first season. The way I see it, there are two fair comparisons for Kelly here. One would be the group of coaches that won a total of 9 games or more – potentially playoff-bound coaches. This group has a total of just 17 coaches over the last ten years (not including the 2013 class).
NEW COACH Average / Median Improvement – First Season (9 or more First Season wins only)
Wins: +4.3 / 5
Points Scored Rank: +7.2 / +6
Yards Gained Rank: +4.8 / +6
Points Allowed Rank: +10.3 / +9
Yards Allowed Rank: +3.6 / +4
Obviously all of the above coaches did pretty well for themselves. Not only did each win a minimum of 9 games, but all showed vast improvements in scoring rankings as well. Chip still outpaces this group on the offensive end, but the Eagles’ defensive performance seems to be roughly par for the course in scoring and well-below typical performance in yards. But as impressive as the first season was for this group on the whole, the second season is equally ugly.
NEW COACH Average / Median Improvement – Second Season v First Season (9 or more wins only)
Wins: -2.6 / -3
Points Scored Rank: -4.0 / -2
Yards Gained Rank: -1.2 / -3
Points Allowed Rank: -6.2 / -5
Yards Allowed Rank: -3.1 / -1
Yikes. As it turns out, more often than not the teams that performed very well in their new coach’s first season come back to earth in year two. It’s important to note that, on average, these teams were still better in year two than they were with the previous coach.
To be fair, there is also an element of sampling bias here. Since these teams were all already in the top half of the league in most of these ranking statistics, there is a lot more room to fall than improve. But for the most part, it’s starting to look like you can make a good case that it is difficult just to sustain, much less improve upon, this kind of immediate success.
However this comparison may not be the fairest to Chip. While this group contains some coaches that did make vast jumps with their team’s record, it also includes some cases where the new coach took over what was already a moderately successful team. This brings us to our second comparison group: the group of coaches whose teams improved by 5 or more wins in the first season. While this is the smallest group we’ve looked at (just 10 coaches), it’s also the one that seems most likely to accurately represent the type of coach Kelly seeks to be: innovators.
NEW COACH Average / Median Improvement – First Season (improved by 5 or more wins over prior coach)
Wins: +6.8 / 7
Points Scored Rank: +12.0 / +12
Yards Gained Rank: +11.2 / +12
Points Allowed Rank: +14.4 / +16
Yards Allowed Rank: +3.6 / +5
Finally we’ve found a comparable, albeit small, group of coaches to help put Chip’s performance in context. Overall it’s pretty solid company. In no particular order, the group includes: John and Jim Harbaugh, Sean Payton, Mike Smith, Jeff Fisher, Chuck Pagano, Tony Sparano, Nick Saban, the Mangenius, and Jim Mora. Looking at how this group fared in year two should provide us with a good overall approximation of what Chip’s Birds might do in 2014.
NEW COACH Average / Median Improvement – Second Season v First Season (improved by 5 or more wins over prior coach)
Wins: -2.5 / -3
Points Scored Rank: -1.2 / +1
Yards Gained Rank: -0.9 / -4
Points Allowed Rank: -2.5 / -2
Yards Allowed Rank: -0.6 / 0
The first thing that jumps out is not one of these coaches won more games in year two than year one. Most of them dropped at least two wins. Offensive scoring remained more-or-less the same, although defensive scoring took a bit of a hit. Then again, these are coaches that, in general, jumped over 1/3 to 1/2 of the league in rankings in their first season with the club. As such, a drop off at this level is hardly a tragic outcome. Don’t despair either; This is a group that has two Super Bowl winners (Harbaugh and Payton), one SB contender (other Harbaugh), a guy who came one Harry Douglas catch away from a SB (Smith), the guy who beat the ’07 Patriots with the Wildcat offense (Sparano), a highly esteemed NFL veteran (Fisher), and perhaps the most successful college coach in history (Saban).
(TL;DR :: No, Chip probably won’t be able to repeat or improve upon last year’s success, but that’s really not a bad thing for the Eagles’ long term outlook.)
The evidence is pretty clear. Immediate success is hard to sustain and even harder to improve upon. Historically speaking, if Chip’s Eagles were to even maintain the same performance the team earned last year, that’d be an outlier. But a marginally worse performance from the Eagles this year is no reason to despair. All told, a bit of a regression isn’t necessarily a bad thing for the long-term outlook of Chip Kelly’s coaching tenure in Philadelphia. Past coaches who came back and did a little worse overall in their second year have still gone on to accomplish the ultimate feat in this league.
1For the purposes of this exercise, coaches who were fired during their second season as HC were credited with their team’s performance for the entirety of the season.
2Yes, of course the GM is an enormous factor in coaching success/failure because his job is to fill the roster, but given that the head coach almost always has a say in what kinds of players the team gets, it’s still fair to attribute the quality of the roster – at least in part – to the HC.
3While turnover differential is a huge contributor to wins and losses, turnovers are also widely variable from year-to-year due to sheer chance. Some coaches may indeed introduce a system that consistently affects turnover differential, but that’s a bit beyond the scope of this article.
4Since the NFL game is zero-sum, a coach’s main objective (besides winning the Super Bowl) should always be to improve relative to the league in each season. Using rankings may be less precise than gross statistics in determining the exact amount of improvement or decline in a team’s performance, but it’s a much easier number to digest.
5One of the more striking results here is the improvement in points allowed. It’s fair to expect different teams to get better in different ways. Certainly when bringing in a coach like, say, Sean Payton, one would expect to see an improvement in the offensive categories. Alternatively, given a John Harbaugh, seeing an improvement in defense would seem to fit. But with a jump this big, it looks like something else has to be going on. My best guess – this is a bit outside the scope of what I’m looking at here – is that this has as much to do with bad luck on the part of the prior coach as it does skill with the new coach. This may also be partly a result of that pesky turnover differential stat, but that’s a question for another time.