What determines a play’s success?
There are a number of benchmarks that may be utilized to quantify a play’s success. It can be determined based on first down conversions or touchdowns scored. It could also be based on the pure number of yards gained. How much a particular play “helped” the team can also be used, but this method is arbitrary and subjective. Let us attempt to quantify how much a play “helps” by bringing in statistics to define the play’s success.
There are two different scopes that are important here. The top level is the drive’s success. A drive is successful based on the final result of that drive. Thus, a drive is successful if it ends in a score. Scores are accomplished by converting first downs to sustain the drive until the score. This leads us to the lower level scope of the play’s success. Individual plays build off each other to move the ball towards the chains and convert first downs. It is this scope that we will attempt to break down.
To start off, let us define a successful play as any play that eventually leads to a converted first down. Note that this can happen on the particular play or on subsequent plays. Take the following example:
1st and 10: Pass complete for 8 yards
2nd and 2: Run for 3 yards and a 1st down
In this example, the play on 1st down is successful because it leads to a first down conversion on 2nd down. The play on 2nd down is also successful because of this conversion. Sounds good so far, but this definition is flawed and incomplete. Let us look at another couple examples to demonstrate:
1st and 10: Pass incomplete for no gain
2nd and 10: Pass complete for 11 yards and a first down.
By our previous definition, the incomplete pass on 1st down is successful because it eventually leads to a first down conversion on the 2nd down play. This doesn’t seem correct as an incomplete pass shouldn’t be considered successful. Now let us look at a third example.
1st and 10: Pass complete for 9 yards
2nd and 1: Pass incomplete for no gain
3rd and 1: Run for no gain
4th and 1: Punt
In this example, a gain of nine yards is unsuccessful based on our previous definition since the offense has failed to eventually convert a first down. However, it is hard to argue that a gain of nine yards on 1st down is an unsuccessful play (ignoring the eventual outcome of the drive). Instead, the gain of nine yards on 1st down should be regarded as successful while the drive should be defined as unsuccessful.
The problem is that our definition needs to be updated to take the proper scope into account. We are looking for a measure of play success, not drive success. Therefore, we should be able to quantify a play’s success by looking at a single play without looking at the eventual result of plays that follow. Instead, let us define a successful play as one that statistically increases our probability of eventually converting a first down.
Every yard that is gained on first or second down will make it statistically more likely to convert on later downs. For example, a 2nd and 1 should have a higher statistical probability of converting than a 2nd and 5. For every play, the offense is attempting to maximize this likelihood of converting a first down by getting closer to the first down marker. We will determine how successful a play is based on the statistical increase in converting a first down.
Analyzing the Data
I looked at three seasons worth of data for USC. This included the 2008, 2009, and 2010 USC football seasons. For every 1st or 2nd down play, I noted the distance to go, the gain on the play, and whether or not a first down was eventually converted. I did not track plays that gained the first down conversion, as these plays can simply be considered as successful. Similarly, I did not track any 3rd down plays since a typical 3rd down can only be categorized as successful if it converts a first down.
Overall, there were 1,351 plays that went into the database. These plays were split between 1st down and 2nd down plays and categorized by the yardage that was gained. This yardage was categorized as a percentage of the yards-to-convert as this abstracts out a lot of the specific numbers.
After categorizing every play, the statistical likelihood that a first down would eventually be converted can be extracted for each scenario. For example, there were 50 plays that were on 1st down and gained 80-89% of the yardage required for a first down. Of these 50 plays, 41 of them (82%) eventually converted for a first down. Thus, gaining eight yards on 1st and 10 means that the offense has an 82% chance of eventually converting for a first down based on the data gathered.
So now that we’ve explained the process. What did the data actually show? Each data point was graphed to find an overall trend. The trick is to determine a cutoff point for a successful play. Let us take a look at the graph that was compiled.
As expected, the graph shows that the likelihood of eventually converting a first down increases as the team gets closer to the first down marker. Also as expected, 1st down plays inherently have a higher eventual conversion rate compared to 2nd down since there are more plays that come after 1st down.
So what does this graph tell us? It does not actually give a clear threshold in which to call a “successful” play. The graph is more or less linear in its progression. So instead, let us pick a point and call that the threshold for success. I arbitrarily picked a 75% likelihood of an eventual first down conversion. Using this conversion rate, a successful 1st down play should gain at least 50% of the yards to go and a 2nd down play should gain just over 80%. This means that a 1st and 10 should attempt to gain at least five yards while a 2nd and 10 should attempt to gain eight yards.
For comparison’s sake, Football Outsiders suggests that a college team gain at least 50% on 1st down and 70% on 2nd down (they also note 100% for 3rd and 4th down). In terms of our data, that would correspond to about a 75% chance of eventual conversion on 1st down and just under 70% for 2nd down. This seams very reasonable.
Using the data we analyzed, a baseline goal for how many yards to gain on any particular down and distance can be derived. Used in conjunction with the previous analyses (found here and here), we can paint a nice picture of what it takes to mount a scoring drive. From a play perspective, we want each play to lead towards a first down conversion to keep a drive alive. When looking at the bigger drive picture, we want to get enough first downs to score points. Hopefully these series of analyses helped give some insight into what goes into these.