The advantage of experience: Analyzing the effects of player experience on the performances of March Madness Teams

4.1Data

The majority of the data in this study were derived from www.sports-reference.com/cbb and www.teamrankings.com, two reliable online archives for historical college basketball statistics. These websites contain the postseason box scores, team roster pages, and team statistics that were used to compile a dataset consisting of all 693 March Madness games (excluding play-in games) that were played from 2007 to 2017. In addition, www.kenpom.com was used to obtain data pertaining to the effective heights of each team. All data were manually pulled from their respective sources, recorded, and then analyzed using a combination of procedures in the R 3.3.1 and IBM SPSS Statistics 20 software packages.

The timeframe used in this sample was chosen because of its recency and because all seasons were played with the “one-and-done” rule in effect (i.e., as of the 2006 NBA draft, all high school players from the United States were required to attend one year of college before entering the NBA). Each observation in the eleven-season sample corresponds to a single March Madness game. This format was chosen per the recommendations set forth by Robst, VanGilder, Berri, and Vance (2011), and Pitts (2016), which highlighted the issues of independence that can arise when the outcome of a single game is treated as two separate events.⁴ In order to avoid this dilemma, one participating team was randomly (alphabetically) assigned to be the team, i, and the other to be the opponent, j, while the explanatory statistics were recorded as the difference between the two (i – j). The explanatory variables included in this study encompassed two categories of experience and a host of variables designed to control for the basic strengths of a team. There are two outcome variables examined in this study. The first, accordingly labeled as WIN, is a binary representation of whether a team won or lost its March Madness game. This variable received a value of 1 if team i won the game and a value of 0 if team j won (i.e., team i lost). In addition, a margin of victory/defeat outcome variable (MARGIN) was included in an effort to more precisely examine the factors that are important in determining the amount by which a team wins or loses. Table 1 summarizes all of the explanatory and outcome variables that were used in this study. The following section describes the rationale for why the chosen explanatory measures were included in the empirical analyses.

4.2Explanatory variables

In an effort to more clearly depict the effects that experience may have on the outcome variables, two different measures of experience were employed in this study. The first fell within the prior March Madness experience category and was representative of the number of career minutes accumulated by a team’s players in previous March Madness games. The decision to use minutes played as the measure of postseason experience instead of the game appearances metric used in prior studies (Pitts, 2016; Tarlow, 2012) was intended to lend these variables an element of specificity that is known to yield more powerful results (Quińones et al., 1995). Rather than weighting the prior experience of a player who appeared for one minute at the end of a blowout as being equal to the prior experience of a player who played for the majority of a game, the use of minutes played allowed for a more comprehensive assessment of the impact that prior March Madness experience had on performance.

To this end, the MMEXP5 and MMEXPSUB variables were designed to capture the average amount of prior March Madness experience held by the two main tiers of players on a team. MMEXP5 took the average amount of prior March Madness minutes played by a team’s top five players in minutes played from the observed game and subtracted the opponent’s MMEXP5 value to leave the relative advantage or disadvantage that was being held by the team in terms of previous March Madness experience. The additional tier of the variable (MMEXPSUB) was included in order to account for the diminishing returns that are received from substitute players who play less minutes in a given game. This helped control for the discrepancies that may arise if the prior experience totals of fringe players are weighted as heavily as the players who appear most frequently. In order to be counted in the MMEXPSUB category, players had to have appeared for at least 4 minutes (10% of total team minutes) in the observed game. This threshold prevented players who appeared in “garbage time” at the end of a blowout win or loss from being weighted the same as the players who made more meaningful contributions. Just like the MMEXP5 variable, MMEXPSUB represented the marginal difference between a team and its opponent.

The CLASS5 and CLASSSUB variables were designed to operate in a fashion similar to the March Madness experience variables, albeit with a more general representation of experience. In many settings, experience is viewed as being synonymous with a person’s age or the number of years they have been on the job. Within the realm of college sports, experience generally coincides with the class rank of an individual student athlete. First year players are classified as freshmen, second year players as sophomores, third year players as juniors, and fourth year players as seniors. By averaging the numerical values that coincide with the respective class of each player (FR = 1, SO = 2, JR = 3, SR = 4), class rank values can be formed for a team and its opponent. As such, the CLASS5 variable represented the difference between the average class ranks of the top five players in minutes played for the two teams in the observed game. It was viewed as being important for explaining the numerous advantages that are said to arise from having key players who are more mature and more accustomed to the nuances of the college game (Lopresti, 2016; Wolstat, 2014). Subsequently, the CLASSSUB variable represented the relative difference between the average class rankings of a team’s remaining players and the coinciding CLASSSUB value of its opponent. This variable might show whether or not upperclassmen who do not play as much are still capable of contributing and inspiring as role players. Just like MMEXPSUB, the CLASSSUB variable was set to include only those players who appeared for 4 minutes or more in an observed game.

The remaining explanatory variables were primarily included to control for factors outside of experience that may be related to the relative quality of a team. The win-loss percentage (WIN%) of a team compared to its opponent was used as a broad indicator of relative performance quality. The strength of schedule (SOS) variable accounted for the relative quality of the opponents that each team faced in prior games through the use of a rating that was denominated in points above or below average, where zero was average (e.g., a team with an SOS value of 10 played against opponents who were, on average, 10 points better than the average team, controlling for home and away factors).⁵ Teams with high SOS ratings are typically those that play in “power” conferences against traditional basketball powerhouses. The offensive (ORTG) and defensive ratings (DRTG) of a team were also included in order to offer more specific measures of team quality. Offensive and defensive ratings standardize the more common points-per-game and points-allowed-per-game metrics by showing the number of points scored or allowed per 100 possessions. Because these ratings are more immune to the speed at which the game is being played, they are better measures of offensive and defensive efficiency (Kubatko, Oliver, Pelton and Rosenbaum, 2007). Lastly, variables signifying the differences between the levels of coaching experience (COACH) and the effective heights (EHT) of the teams being observed were also included. The COACH variable was included in order to control for any effects that prior March Madness coaching experience might have on a team’s chances of winning a current March Madness game. Effective height (EHT), a metric espoused by Pomeroy (2008), averages the physical heights of a team’s power forward and center positions. In variable form, it showed the relative height difference, in inches, between the two teams’ big-men and was used to control for the advantages that may arise from being a taller team.⁶

It further needs to be mentioned that each of the explanatory variables were interacted with a binary categorical variable indicating whether the observed game was taking place in the Sweet 16 or beyond (LATEROUND). Sweet 16, Elite 8, Final Four, and National Championship games received a value of 1 while Round of 64 and Round of 32 games received a value of 0. These interactions were examined because not all March Madness games are equal; that is, there will be a greater disparity in quality between teams in the early stages of the tournament than there will be later on (Bauman et al., 2010; Jacobson and King, 2009). Hypothetically, the pressure will also be building as teams get closer to the title game. Therefore, looking at these interaction terms allowed the researchers to assess whether or not the explanatory variables, and in particular the experience variables, had a statistically significant impact on game outcomes in the tournament’s later rounds.

4.3Empirical specifications

Two regression models were developed in an attempt to assess the impact that experience has on WIN and MARGIN. The first of these took the form of a probit regression model in an effort to account for the binary nature of the WIN variable and to display the marginal effect that each of the explanatory variables had on the probability of a team winning its March Madness game. As such, the following binary probit model was estimated:

where WIN_ij equals 1 if team i wins the March Madness game and 0 if it loses to team j, WIN% _ij is a variable measuring the prior win-loss record of team i relative to team j, SOS_ij is the strength of schedule rating measuring the prior strength of the opponents faced by team i relative to team j, ORTG_ij is the offensive efficiency of team i relative to team j, DRTG_ij is the defensive efficiency of team i relative to team j, MMEXP_ij is the vector of variables (MMEXP5 and MMEXPSUB) describing the prior March Madness experience of team i relative to team j, CLASS_ij is the vector of variables (CLASS5 and CLASSSUB)⁷ describing the class rank experience of team i relative to team j, COACH_ij is the number of prior March Madness games coached by the coach of team i relative to the coach of team j, EHT_ij is the average height difference, in inches, between the power forward and center positions on team i compared to team j, LATEROUND_ij is the vector of variables representing the interactions between each of the explanatory variables and a categorical variable indicating whether the game was taking place in the Sweet 16, Elite 8, Final Four, or National Championship (1), or the Round of 64 or Round of 32 (0), the β’s are parameters to be estimated, and ɛ_ij is a random error term. In total, there were 693 observations with which to estimate Equation 1.

For MARGIN, ordinary least squares (OLS) regression was identified as the appropriate method due to the relatively continuous nature of the point differentials from the observed March Madness games. The associated OLS model was estimated as:

5.1Descriptive statistics

Before examining the results of the model estimations, a look at the descriptive statistics in Table 2 helps to shed some light on the differences that may exist between winning and losing teams in the early and late stages of the tournament. Looking first at the prior experience variables (MMEXP), it is evident that winning teams tend to have more prior March Madness experience among their presumed starting five. In the Round of 64 and Round of 32, the winning teams averaged over 26 more minutes of previous March Madness playing time, per starter, than losing teams. From the Sweet 16 onward, the starters on winning teams boasted approximately 27 more minutes of prior March Madness experience than their losing opponents. The same trends held true with the substitutes, as winning teams’ non-starters averaged 9 and 14 more minutes of prior tournament experience in the opening and closing rounds, respectively.

In terms of class rank among the starting five, it appears as though the winning teams were actually marginally younger than their opposition in both samples. Winning teams in the Round of 64 and Round of 32 also had younger substitutes, while winning teams in the later stages of the tournament actually had substitutes who were slightly older. Even so, the mean difference between winning and losing teams across all players in terms of class rank was negligible. Though further analysis is needed, the descriptive statistics seem to suggest that class rank, alone, is not a variable that offers a discernable advantage at any stage of the tournament.

Looking lastly at the variables that were included to control for basic aspects of team quality, it is interesting to note how the gap closes between teams in terms of WIN%, SOS, ORTG, and DRTG as the tournament progresses. This suggests the natural movement toward parity that occurs as the higher-quality teams advance and the weaker teams are eliminated. In combination with the fact that the MMEXP variables widen from the Sweet 16 onward, the initial evidence is there to support the assumption that prior experience may come into play during the later stages of the competition. The estimated models shed more light on this phenomenon, as it pertains to both a team’s probability of winning (WIN) and the margin (MARGIN) by which it would be expected to do so.

5.2Effects on WIN

The results of the binary probit regression model⁸ estimating the probability of a team winning its March Madness game are presented in Table 3. These results are presented as marginal effects in order to show the percentage increase in a team’s likelihood of winning for every one-unit increase in the explanatory variable being examined while holding all other variables constant at their means. Marginal effects offer a more interpretable value than the standard coefficients of a probit model, which are presented as z-scores. Looking first at the marginal effects of the explanatory variables without the late round interactions, WIN%, SOS, ORTG, and DRTG, as the basic measures of team quality, appear to be the only variables that significantly affect a team’s probability of winning in the early stages. Of these, the strength of schedule measure appears to have the largest effect, signifying the advantage enjoyed by teams that have played against high-level opposition in previous games.⁹

Shifting focus to the late round interaction terms, the interaction between CLASS5 and LATEROUND was negative and marginally significant (p <  0.10), suggesting a team would suffer a 13.67% decrease in its probability of winning in the Sweet 16 or beyond for every 1-year higher its class rank was than its opponent’s. However, it is worth noting that the descriptive statistics revealed the class rank gaps between teams at these stages to be less than a full year. It would therefore be rare to encounter a team with starters who were an entire year ahead of their competitors in terms of average class rank. This makes the effect size of this variable less impactful, though the negative direction of the effect is worth highlighting. A team with starters who are of the average class rank value lower than the opposing starters will see its probability of winning increase by 1.28%. Interestingly, neither of the prior March Madness experience variables (MMEXP5 and MMEXPSUB) appeared to significantly contribute to a team’s probability of winning in the early or late stages of the tournament.

Lastly, it merits a mention that the interaction between DRTG and LATEROUND, with a marginal effect of 2.36%, appeared to show that playing stronger defense was significantly beneficial to a team’s late-round success. This stands in contrast to ORTG, which only attained marginal significance in the overall model and did not have a significant interaction with the later rounds of the tournament.

5.3Effects on MARGIN

After analyzing the effects that the explanatory variables had on a team’s probability of winning, the focus was shifted to the OLS models analyzing the effects that each variable had on a team’s margin of victory or defeat. The MARGIN outcome variable used in these models was particularly useful for interpreting the effects of the explanatory variables in terms of actual point values. The results are presented in Table 4. Much like the probit model, the base terms in the OLS model showed that basic measures of team quality (WIN%, SOS, ORTG, DRTG) were the only variables to have a significant effect on the extent to which a team won or lost its March Madness game in the early stages. The strength of schedule measure was again the most impactful, indicating that nearly a 1-point advantage existed for every 1-point harder a team’s schedule had been prior to the observed game. All four of the player experience variables were non-significant on their own.

However, when looking at the interaction terms, it is seen that prior March Madness experience among a team’s starters (MMEXP5) is one of the few significant variables from the Sweet 16 onward. The MMEXP5 effect size of 0.039 suggests that every 1-minute advantage held by a team’s presumed starters in previous tournament experience leads to a 0.039-point improvement on the scoreboard. Recalling that the average experience gap between starting-fives at this stage of the tournament was nearly 27 minutes, this improvement swells to just over a full point when analyzed at the mean. While a 1-point improvement, on average, may not seem like much, it is worth noting that the average margin of victory/defeat in the latter stages of March Madness was just 10 points. Perhaps, then, prior experience could help teams win close games that take place late in the competition.

The interaction terms also reaffirmed what was seen in the probit model, with CLASS5 attaining a marginally significant and negative effect. Indeed, the OLS model suggests that for every 1-year higher a team is in terms of class rank, its point margin would be expected to drop by just over 2.6 points. Again considering the averages, a team with a class rank of the average value lower than its opponent would see a point boost of just 0.25 points. While the opposite effects of the MMEXP5 and CLASS5 variables may seem contradictory, the discussion section that follows sheds more light on these results.

6.1Discussion of results

This study conducted its analyses in an attempt to answer some of the questions that have been left unanswered by prior studies in the field; namely, does player experience have an effect on the performances of college basketball teams whose players may be less familiar with the high stress scenarios of the postseason than professional athletes? While the regularity with which this assumption is made in television studios and press conferences around the nation may lead one to believe that this question has long been answered, research exploring this phenomenon at the professional level has argued quite the opposite. Indeed, attempts to quantify the advantages of player experience in the NBA and NFL postseasons have found few discernible links between the amount of player experience a team has and its ability to win or perform at a higher level (Pitts, 2016; Tarlow, 2012).

Aside from the inherent limitations of the previous studies, one plausible explanation for these findings is that professional athletes have already been inoculated to stressful scenarios and have developed effective coping mechanisms for dealing with the pressure. This certainly fits with the inoculation hypothesis mentioned earlier, a theory which states that prior exposure to a stressor can increase one’s ability to tolerate that stressor in the future (Eysenck, 1983). Professional athletes have been through the stress of making it to, and remaining in, the highest level of the game. They are also getting paid millions of dollars regardless of whether they win or lose. On the contrary, college athletes may be experiencing a number of stressors for the first time if they have never appeared in a major tournament. The additional media obligations and scrutiny of the national spotlight may weigh heavier on the uninitiated. As amateurs, their future income as a professional may be contingent on a successful showing in the tournament. Seeing the stark differences between collegiate and professional athletes, this study set out to examine a phenomenon that has been frequently overlooked by prior studies in the field.

While the term “experience” often carries with it a sense of ambiguity as announcers, coaches, and analysts fail to differentiate between a player’s class ranking and his previous postseason experience, the empirical analyses conducted in this study defined the parameters more clearly by analyzing two forms of experience (i.e., class rank and prior March Madness experience) and their differential effects on performance in the later rounds of the tournament. One of the key findings of this study is that experience can have an impact, but perhaps not in the ways one might expect. Indeed, class rank, a variable assumed by many to be positively predictive of performance, appears to have a negative effect on both point margins and a team’s probability of winning in the later stages of the tournament; that is, older teams actually perform significantly worse than younger teams when the pressure is highest. This stands in contrast to prior March Madness experience, which was seen in both the descriptive statistics and the OLS model to have a positive impact on team performance in the later rounds of the competition.

Clearly, though, these results cannot be considered in a vacuum, as some amount of class rank is needed in order to obtain prior experience in March Madness tournaments. A player could not simply have a greater amount of playing time in previous postseasons without having used up at least one year of eligibility. According to the descriptive statistics, the average winning team in the Sweet 16 and beyond boasted a CLASS5 value of 2.709 that would place its starters close to the junior rank. Therefore, the correct interpretation is not that class rank is unimportant, but that one of its main benefits is already being accounted for by the prior March Madness experience of its key players. Once all of the advantages of prior postseason experience are incorporated in the MMEXP measures, the class rank variables are left to essentially represent the collective “age” of a team relative to its opponent.

Taken together, these findings raise a number of interesting talking points: (1) being more experienced to the extent that a team is simply older or more mature in terms of class rank is not advantageous; rather, it is what a team has been able to do with its yearly experience (e.g., reach previous March Madness tournaments) that matters; (2) in an era full of talented, one-and-done freshmen, it might not be wholly surprising that both brands of experience have less of a positive impact than what is commonly presented; (3) keeping in line with the draft theme, one reason for why teams with a higher class rank might fare slightly worse is because their upperclassmen are not good enough to get drafted to the NBA at a younger age and may now be facing-off against teams with younger players who are more talented prospects; (4) lastly, perhaps the teams with higher class rankings fare worse because they contain a higher-than-average number of seniors who succumb to the pressures of career-ending scenarios. Moving forward, researchers could delve deeper into the impacts of one-and-done players and career-ending scenarios since these are both unique facets of the college game.

It is further worth discussing how these experience variables were not significant until the later stages of the tournament when they were interacted with the LATEROUND variable. Indeed, the early round significances of the WIN%, SOS, ORTG, and DRTG variables show that basic measures of team quality are much more vital to predicting match outcomes in the early stages of March Madness than relative measures of class rank and prior tournament experience. This makes the significance of the MMEXP5 and LATEROUND interaction in the OLS model that much more telling. After the weaker teams are eliminated in the opening two rounds and team quality becomes more even (a phenomenon that is evident in the descriptive statistics), the importance of prior experience shines through. That it is significantly and positively related to performance in what are essentially the most meaningful rounds of the tournament provides some of the clearest statistical evidence that prior experience can help college teams take command of big games where the pressure is high.

Nonetheless, while winning teams appear to have more prior March Madness experience, and advantages in the MMEXP5 category can lend additional points to a team’s effort in the Sweet 16 and beyond, having an advantage in MMEXP5 does not appear to significantly improve a team’s probability of winning, even in the later rounds. While the exact causes for the differences between the results of the probit and OLS results are unknown, it can be speculated that prior experience may be beneficial in terms of keeping games close or extending leads over an opponent, but that it may not consistently show up in a binary win-loss column. Indeed, the average tournament game is decided by about 10 points, and the average team only improves its points margin by about one point due to previous tournament experience. So, while previous tournament experience may help a team perform better, it may not always be impactful enough to change the outcome of a game.

Furthermore, even though multicollinearity issues between the explanatory variables were taken into consideration when developing the models, it could be that prior March Madness experience, like WIN%, ORTG, and DRTG, is serving as another proxy for talent. After all, one might expect a more experienced team to also be more adept at navigating the regular season, leading to a higher WIN%. As such, further analysis is warranted in order to determine if prior experience is simply representative of a more abstract, higher skill-level that is endogenous to a team that consistently appears in the tournament, or if it is actually breeding intangible qualities such as composure and focus under pressure. Regardless, the significant, late-round impact of MMEXP5 on margin of victory in the face of broad-based measures of team quality including strength of schedule (gives precedence to advancing from a power conference), win-loss percentage, offensive rating (offensive quality), and defensive rating (defensive quality), makes a case for considering its impact as the tournament progresses. While it seems as though more tangible and traditional metrics should take precedence at the beginning of any bracket, prior March Madness experience may tip an evenly-matched Sweet 16, Elite 8, Final Four, or National Championship game in favor of the team that possesses more.

Future studies in this realm can now focus on validating these findings, perhaps through the creation of more predictive models that incorporate prior experience variables in their forecasts. If this study provided any sort of insight in that area, it would be to suggest that variables denoting team quality should account for strength of schedule. Then, once teams become evenly matched in the later rounds of the competition, prior tournament experience should be introduced as a means of further separation. Subsequent examination is also needed in order to discover the more specific ways in which prior experience affects team performance. That is, are late-round tournament teams shooting poorly from the field or free-throw line because they have less experience handling the pressure? Are they turning the ball over more frequently because they are nervous? When questions like these are answered, coaches will be able to manage their teams more effectively.

Until then, those who are in charge of mentally and physically preparing the players for upcoming games will have to judiciously adjust their tactics and plans according to the degree of experience or inexperience that exists between a team and its opponents. Coaches or team personnel, for instance, could employ a variety of training methods and psychological techniques to ensure that inexperienced players are putting in the time to develop the proper mindset and appropriate coping mechanisms. Halliwell (2004), for example, highlighted the benefits of interactionist techniques that can help place athletes in the proper mood and mindset ahead of a big event. In particular, it was noted how veteran players should be instructed to “let their training and talent come out,” while the more inexperienced players should be given access to the “timeless wisdom” of the veterans through special meetings, video clips, and inspirational quotes (p. 30). Ultimately, while some amount of psychological training could prove helpful, the more data-driven, focused interventions could eventually yield more powerful results.

6.2Current limitations and recommendations for future research

One of the clear limitations of this study is that it is only focused on one specific postseason within one specific sport. While March Madness holds a prominent place in the wider realm of collegiate sport, one cannot generalize these findings to football, women’s basketball, or other team-based competitions until further analyses are conducted. Looking ahead to subsequent studies in this area, it would be interesting to see if some of the more discrete variables underlying overall performance are affected by experience. This might help quantify some of the more intangible qualities of a team (e.g., confidence, composure, and hustle). For instance, do teams with more prior experience shoot better from the foul line or make a higher percentage of their shots from the field? Do the teams with more upperclassmen turn the ball over less frequently or fight harder for rebounds? This could be viewed at the team level or by comparing relative experience across positions that match up with one another. For example, experience levels could be averaged according to each position, much like height was for centers and power forwards in Pomeroy’s (2008) effective height measure. The relative levels of experience among players at these positions could then be compared across the statistical categories that are commonly associated with them. Power forwards and centers, for example, could be compared in terms of rebounding while guards could be compared in relation to how they shoot and handle the ball.

In addition, one of the limitations of this study was that it did not incorporate any specific time or game-scenario variables. It is therefore suggested that future studies collect and incorporate some of these measures in order to provide more powerful and interpretable estimates of performance under pressure. For example, one study by Goldman and Rao (2012) on NBA players performing under pressure assigned a value of importance to each free-throw based on the time remaining in the game and the score differential between the two teams. A similar method could be adopted to include experience variables in an effort to see whether or not a player’s level of experience helped him or her perform better or worse at shooting under pressure. Another recommendation involves the use of more longitudinal, team-specific models. Instead of observing individual, head-to-head matchups, future analyses could employ methods such as multinomial logistic regression to examine the impact that the explanatory variables have on the number of March Madness games that a team can be expected to win. Methods and variables that better account for the potential latent talent issues that exist between the control variables for team quality and player experience could also be introduced in an effort to shed further light on the significant or non-significant impacts of experience on performance. Lastly, future researchers could develop variables that account for postseason experience in college basketball tournaments outside of March Madness, such as the National Invitation Tournament (NIT) or College Basketball Invitational (CBI).