What if a figure skating team event had been held at past Winter Olympic Games? An analysis of a hypothetical competition

1An Analysis of hypothetical figure skating team events based on past Winter Olympic Games

Sports analysts, mathematicians, and others commonly ask questions about how past athletes might have fared under different rules or metrics that were not in existence when these athletes competed. For instance, Davis, Perera, & Swartz (2015) created and used a novel metric for player evaluation in the sport of Twenty20 cricket. Baumer, Jensen & Matthews (2015) proposed a different way of measuring baseball players’ overall player performance than is currently used. Jensen & Turner (2014) pursued the question of what might have been the results of college football bowls if different criteria were used to assign teams to their conference affiliations. Berry, Reese, & Larkey (1999) developed a way to compare professional athletes within a sport from different eras that took into account factors such as changes in rules and quality of opponents, and applied it separately to hockey, golf, and baseball.

At the Winter Olympic Games (WOG), a figure skating team event was first separately contested in 2014. In this article, we propose a way to determine which teams might have earned medals had the team event been conducted at prior WOG. We also propose a way to evaluate the relative contributions of athletes to their countries’ hypothetical teams at past WOG, which could be useful in determining which athletes should be recognized for their contributions to the sport.

Since a team event was not separately contested at WOG held prior to 2014, we need a way to determine how points could be assigned to hypothetically formed teams. We demonstrate a method of assigning points to teams by using rankings earned in individual events when a team event was not separately skated. This method was used at the 2012 Youth Olympic Games (YOG), a competition that was sanctioned by the International Skating Union, which also sanctions the WOG figure skating events. We then use this method of assigning points to a hypothetical team event based on rankings from the 2014 WOG individual events; and we compare the results of using this method to the actual results from the 2014 WOG figure skating team event.

We then apply the 2012 YOG rules to the 2010 WOG individual results. We propose a way to answer the question, “How might countries’ teams have ranked if a figure skating team event had been conducted at the 2010 WOG?”

Next, we describe the method that we will use to determine the relative contributions of athletes to hypothetical teams that are formed. We subsequently apply this method of player evaluation to our results regarding a hypothetical team event at the 2010 WOG.

We conclude with a discussion of how these kinds of analyses could be used in selecting future team event rosters and selecting inductees into the World Figure Skating Hall of Fame and the US Figure Skating Hall of Fame.

22012 Youth Olympic Games rules applied to the 2014 Winter Olympic Games results

Fans and sports analysts might be curious about what the results would have been if, at the 2014 WOG, the 2012 YOG rules had been used (instead of the team event being separately contested). To begin answering this question, a method of assigning points to skaters representing countries must be used. At the 2014 WOG, the team figure skating event was an event held separately from the individual disciplines’ events – men’s, ladies, ice dance, and pairs. However, at WOG prior to 2014, a separate team event was not conducted. For these past WOG, we have data regarding the skaters’ rankings within each of the four disciplines. We propose to use a method of assigning points to skaters that was used at another International Skating Union (ISU) sanctioned event, the 2012 YOG. The 2012 YOG had a team event for which points were assigned for skaters on teams using their rankings in individual events. In this section, we apply the 2012 YOG rules to the 2014 WOG and determine how closely assigning points using rankings from individual events matched the results of the team event.

32012 YOG rules applied to 2010 WOG individual events

Since the actual 2010 WOG did not include a figure skating team event, there were no qualificationprocedures established to determine the countries which would have qualified to field teams in a team event. In order to produce results of a hypothetical team event, we use the 2010 WOG results (ISU, 2010) of the same ten countries that qualified to have teams compete in the 2014 WOG team event. While not all of the countries chose their top-ranking athletes within each discipline to compete in the 2014 WOG team event, for simplicity in producing 2010 WOG hypothetical team event results, we base each country’s points earned for each discipline on the rankings of the top-ranking athlete representing that country.

In 2010, five of the ten countries which competed in the 2014 WOG team event qualified to have athletes compete in all four disciplines, and the remaining five countries qualified athletes in three disciplines. China and Great Britain did not have skaters qualify to compete in the men’s event; France and Ukraine did not have skaters qualify to compete in the ladies event, and Japan did not have skaters qualify to compete in the pairs event. For the purposes of this exploration, we thus consider athletes representing these countries to have ranked last in the corresponding individual events. Since there are two such skaters in the men’s and ladies events, each country is considered to have tied for second-to-last in those disciplines and the points are accordingly assigned: the second-to-last ranked entrant receives 2 points and the last-ranking entrant receives 1 point, so in a tie situation, both entrants receive 1.5 points. In Table 3, we list the points that each of the ten countries would have earned in a hypothetical team event based on actual 2010 WOG individual event results. The underlined and italicized points listed in Table 3 indicate that the skating performance did not take place during the 2010 WOG.

There is a tie for second place since the number of points earned by the United States and Russia is the same – both countries earned 66 points in the hypothetical team event. We implement the tie-breaking procedure of using IJS score totals earned in order to break the tie. Table 4 lists the scores earned in each round in each discipline. The United States breaks the tie because the sum of its IJS score totals is higher, so the United States ranks second in the hypothetical team event.

4Method for measuring contributions in the figure skating team event

In the following section, we will analyze the contributions that entrants made toward their team achieving certain goals in the hypothetical figure skating team event that was studied in the previous section. This section describes the method we will use to measure those contributions. More specifically, it applies ideas from game theory to figure skating team events. For more details and a discussion of how this application of game theory to figure skating team events is related to analyses of voting, see Cheng & Coughlin (2017).

The entrants on a figure skating team will be modeled as players in a specific game. So, for instance, in the WOG, the set of players on a country’s team would consist of the men’s entrant (M), the ladies entrant (L), the pairs entrant (P), and the dance entrant (D) from that country. The game involves the entrants of the team earning points that count towards the team’s total number of points. We can determine the extent to which each entrant helped the team meet or exceed a meaningful objective, which will be called the threshold. This threshold is a number of points; for instance, it can be the number of points that another country’s team earned, since skating federations might want to know which entrants helped its team attain a goal of placing higher than another team. It may be important to determine the contributions that each of the four entrants earned relative to each other towards the team’s total number of points earned.

Let W_M, W_L, W_P, W_D be the number of points earned by the men’s, ladies, pairs, and dance entrants, respectively. Let W_N be the sum of the points earned by a team’s entrants. In Round 1, Great Britain’s points were W_N = 9, and the individual entrants’ points were W_M = 1, W_L = 2, W_P = 2, W_D = 4. Great Britain may be concerned about the contributions of its team’s entrants towards earning at least as many points as the last-placing team in the hypothetical team event. The threshold, q, in this situation would be 6, which is the number of points that the last-placing team of Ukraine earned.

Saari & Sieberg (2001) and Saari (2001) have suggested a method of evaluating athletes. Their method uses equations that were originally developed to describe voters’ power in determining the outcomes of voting contexts. The two best-known methods for measuring power developed in voting theory are the Shapley-Shubik Index (1954) and the Banzhaf Index (1965). Below, using an example from Great Britain’s team in the hypothetical team event described in Table 1 from individual results at the 2014 WOG, we illustrate the process by which the Shapley-Shubik and the Banzhaf Index can be computed for figure skating team event entrants.

In order to compute the Shapley-Shubik index for the four figure skating team event entrants on a team, we use the following procedure (the bulleted items illustrate how the steps are implemented with our example scenario from Great Britain at the 2014 WOG hypothetical team event from Table 1):

In order to compute the Banzhaf index for the four figure skating team event entrants on a given team, we use the following procedure (the bulleted items illustrate how the steps are implemented with our example scenario from Great Britain at the 2014 WOG hypothetical team event from Table 1):

As the example illustrates, the numbers assigned by the Banzhaf and Shapley-Shubik indices can differ. A result in Tomiyama’s (1987) article implies that the Shapley-Shubik and Banzahf indices will always have the same rankings in our model of the figure skating team.

When the numerical values assigned by the indices differ, a natural question is which of the two indices should be used. The answer depends on the purpose. For example, many countries award prize money for Olympic medalists. If a lump sum of prize money became available to the team as a reward, it would be sensible to distribute the prize money in a way such that the entries’ benefits match their contributions. By having the money divided in accordance to the Shapley-Shubik indices, that would be accomplished (Felsenthal & Machover, 1998). If the purpose is to evaluate the extent to which an entry affects whether a certain goal is achieved, the Banzhaf index is suitable (Felsenthal & Machover, 1998).

5Indices resulting from applying 2012 YOG rules to the 2010 WOG individual events

Tables 5 and 6 report a list of possible thresholds of interest for certain countries and their resulting indices for Rounds 1 and 2 respectively. In the “Index” column of these tables, “Both” indicates that the Shapley-Shubik (SSI) and Banzhaf (BI) indices yield the same numerical values.

In most of the cases considered in Tables 5 and 6, the Banzhaf and Shapley-Shubik indices yielded the same numerical values. However, in the case of Russia in Round 1, where the threshold was 24 points, the rankings derived from both indices matched but the numerical values differed.

The Banzhaf and Shapley-Shubik indices have the same numerical values for each of the thresholds considered in Table 6 for the Canadian team in Round 2. As reported in Table 3, in round 2, the point contribution of the men’s entrant is only separated by one or two points from the remaining entrants’ point contributions. However, something noteworthy occurs when one compares what happens when the threshold is 50 points with what happens when the threshold is 52 points (the calculations are detailed in Appendix A). First, observe that when the threshold is 50 points, there are big differences in the numerical values assigned to the Canadian team’s entrants: the index of the men’s entrant is only a third of the indices of the ladies or pairs entrants, and a fifth of the dance entrant’s index. Second, observe that increasing the threshold by two points, from 50 to 52 points, yields a three-fold increase in the men’s entrant’s index. Similarly, the indices for the entrants of the Russian team were considerably different for the thresholds of 50 and 52.

It is unintuitive that the same threshold of 52 points would yield different indices for the United States and Russia since they both earned the same number of total points, 66, by the conclusion of the Round 2. However, we are also taking into consideration the number of points earned in Round 1, where the United States earned two more points than Russia. Having a threshold of 52 points for the Round 2 score means that the United States must earn 19 points in Round 2 and that Russia must earn 21 points in Round 2. On the team representing the United States, the sum of the points earned by two of the entrants - men’s and dance - already meets the threshold of 19 points. As a result, the numerical values assigned by the indices for these two entrants are higher than that of the other two entrants on the team representing the United States, ladies and pairs. On the Russian team, the points earned in Round 2 by any of three of the four entrants will meet the threshold of 21 points, so the numerical values assigned by the indices to each of the four entrants are the same.

Another threshold that could be considered in Round 2 is 58 points. While in this context using 58 points as a benchmark is not meaningful, we consider this threshold to illustrate another discrepancy in the relative contributions of the athletes between the United States and Russia even though these two countries have the same point totals in Round 2. In Table 7, we report the numerical values assigned by the indices for each of the entries on these two teams resulting from the threshold of 58 points in Round 2. Having a threshold of 58 points for the Round 2 score means that the United States must earn 25 points in Round 2 and that Russia must earn 27 points in Round 2. On the team representing the United States, any three of the entrants will meet or exceed the threshold, thus, each of the four entrants contributes equally. On the Russian team, the ladies entrant makes no contribution while the remaining three entrants contribute equally towards meeting or exceeding the threshold of 58 points.

6Possible application of hypothetical team event to future team event roster selection

The exact thresholds which countries aim to meet may be unknown in advance, but they could be estimated based on results of recent prior competitions. In this section, we will explain why roster selection is significant, as well as describe a situation in which considering a hypothetical team event when determining a country’s team event roster can reap benefits for individual skaters.

There is increased possibility of injury when skaters compete in both the team event and the individual event, since these skaters would have less rest. At the 2014 WOG, the Russian skating federation selected Evgeni Plushenko to be its men’s entrant in the team event. Plushenko skated the team event, but then he was injured with the warm-up session of the 2014 WOG individual men’s event and subsequently withdrew. This example illustrates how important it is for skating federations to consider carefully whether or not to rest an athlete.

If an entrant is not needed in order for a country’s team to meet its goals, it may be sensible to rest that entrant for the sake of the entrant’s individual events. Leading up to the 2014 WOG, the German skating federation may have wondered whether they would need their highest scoring pairs team of Aliona Savchenko and Robin Szolkowy to compete in the 2014 WOG team event. Savchenko and Szolkowy won a silver medal at the 2013 World Figure Skating Championships (Worlds) in the individual pairs event, after placing 3rd in the short program and 2nd in the free skate. The 2013 Worlds competition was the penultimate large international figure skating competition organized by the International Skating Union prior to the 2014 WOG, and it would be sensible to have results from the 2013 Worlds inform roster selection for the2014 WOG.

We constructed a hypothetical team event using results from the 2013 Worlds (ISU, 2013). Table 8 displays the numbers of points that the ten teams that qualified for the 2014 WOG team event might have earned in Round 1 if the 2013 Worlds had a figure skating team event, based on the 2012 YOG team event rules.

The points assigned in Table 8 were tabulated by taking into account the number of points that the highest-placing skater from each of the countries might have earned in Round 1. The estimated points are italicized and underlined in Table 8, indicating that the skating performance did not actually take place when the country did not qualify to have a skater compete in an individual event in that discipline at the 2013 Worlds. China did not qualify to have an ice dance entrant, Japan and Ukraine did not qualify to have pairs entrants, and Great Britain did not qualify to have a men’s entrant at the 2013 Worlds. To assign points to the Japanese and Ukranian pairs teams, we averaged the number of points hypothetically earned by 9th and 10th placing teams. A traditional tie-breaking procedure cannot be used for China and Germany. The reason is that, since the team from China did not have an entrant in the ice dance event, China didn’t have any points in the International Judging System in ice dance at the 2013 Worlds.

The German team does not advance to Round 2 because only the top 5 teams advance. Furthermore, there is a five-point difference between the number of points earned by the fifth-placing team, Italy, and the sixth-placing teams China and Germany. Based on this information from the 2013 Worlds, the German team likely knew that it had a small chance of advancing to the second round of the team event, and an even smaller chance of earning a medal in the figure skating team event at the 2014 WOG. The German team also was likely aware that there was little risk of placing last in the team event.

After the 2013 Worlds, the German skating federation might have asked, do we need our highest scoring pairs team to compete at the 2014 WOG team event? The German skating federation had another pairs team, Mari Vartmann and Aaron Van Cleave, who placed last out of the remaining teams representing countries that qualified pairs entries at the 2013 Worlds. The German skating federation could send Vartmann and Van Cleave or another pairs team to compete at the 2014 WOG, allowing Savchenko and Szolkowy to rest for their individual pairs event performances. Based on this possibility, a revised hypothetical team event for Round 1 that would result is shown in Table 9.

If Savchenko and Szolkowy’s scores did not count towards the hypothetical team event, the teams placing lower than Savchenko and Szolkowy would each earn one additional point. Since the Japanese and Ukranian teams did not qualify pairs entrants to compete in that individual event while the German team qualified to have two pairs entrants at the 2013 Worlds, any German pairs team would likely earn more points than the Japanese and Ukranian pairs teams.

Would it matter which pairs team represents the German skating federation at the 2013 Worlds hypothetical team event? A threshold that the German skating federation might use in an index analysis of each entrant’s contributions to the team is the number of points required for the German team to earn more points than the next-lower placement. In the hypothetical team event summarized by Table 8, the Japanese team earned the next-lower placement with 17.5 points. In the hypothetical team event summarized by Table 9, the team from Great Britain earned the next-lower placement with 12 points. In Table 10, we report the contributions of the German entrants in both hypothetical team events.

In the hypothetical team events summarized in both Tables 8 and 9, the men’s, pairs, and dance entrants each contribute equally towards meeting or exceeding the threshold. More notably, for the threshold selected, it does not matter whether the German pairs entrant earns 8 points or 3 points in the hypothetical team event.

How did this work out for the German skaters at the 2014 WOG? The German skating federation picked the pairs team of Maylin Wende and Daniel Wende as the pairs entrants in the team event at the 2014 WOG, allowing Savchenko and Szolkowy to rest. Subsequently, German pairs team Aliona Savchenko and Robin Szolkowy earned a bronze medal in the individual pairs event at the 2014 WOG; this was the only figure skating medal earned by a German entrant at the 2014 WOG.

In Round 1 of the team event, the German team earned 17 points (5 points each from the men’s, pairs, and dance entrants; and 2 points from the ladies entrant) and did not advance to Round 2. The threshold for teams advancing from Round 1 to Round 2 was 23 points, the number of points that 5th-placed team Italy earned. The German team needed more than 6 points to advance to round 2. In hindsight, another German pairs entrant would not have tipped the balance. Any entrant can earn a maximum of 10 points, so even if the German pairs entrant had earned 10 points instead of 5 points, the German team would not have earned more than 22 points in the team event.

Teams which don’t have strong contenders in all four events are unlikely to place highly. For these teams, an important consideration might be to rest their top athletes in preparation for their individual events. The German skating federation’s team event roster selection for the 2014 Winter Olympic Games illustrates the advantages of using our approach.

7Possible application of hypothetical team event to halls of fame selection

A possible application of answering the hypothetical question of how much skaters might have contributed to their countries’ teams at past Olympic and other international competitions is towards determining who should be inducted into a hall of fame. The International Skating Union recognizes the World Figure Skating Hall of Fame, established in 1976 and managed by US Figure Skating, to be representative of the sport of figure skating as a whole (Wright, 1985). US Figure Skating also manages a second hall of fame called the US Hall of Fame (World Figure Skating Museum and Hall of Fame, 2015). Both halls of fame are located on the campus of the World Figure Skating Museum located in Colorado Springs, Colorado. The halls of fame include the names, pictures, biographies and records of all inducted members displayed prominently on a wall. When members are inducted at an annual ceremony, each member receives a plaque and a medal as tokens of membership in the Hall of Fame.

New members to be inducted into the World and US Hall of Fame are determined by an election. Anyone from the general public can submit nominees to be considered, and a group of 27 electors determined by the Hall of Fame Committee selects up to two people for each of three categories from the list of nominees each year. The first category of accomplishments which the halls of fame recognize is the following: “Skaters from throughout the world who have compiled an outstanding competitive record or who have made a noteworthy contribution in style or technique and have been retired from competitive skating for at least ten years (Wright, 1985).” Additional categories of accomplishments involve noteworthy service by skating officials and outstanding coaching by skating professionals.

In order to compare candidates nominated for the first category, electors would want to have a method for deciding what makes one competitive record more outstanding than another. In the actual hall of fame elections, explicit rubrics beyond the quoted text are not given to electors, thus each elector has the liberty to choose however he or she wishes. Each elector can base decisions on a different set of unwritten standards based on any preferred criteria. Hypothetically, if electors were required to justify their selections, they may consider developing more transparent systems for ranking skating records.

When an elector is comparing skating records, some reasonable standards could be based on medal counts, scores earned (e.g., number of perfect 6.0’s earned in the judging system used prior to the IJS, average score earned), or rankings from important competitions. However, there are some potential problems with using these standards. For example, when comparing skaters who participated in the 2014 WOG and skaters who participated in prior WOG, only examining medal counts could be misleading. There are two reasons: 1) someone who participated in 2014 had an opportunity to win a medal in an event that did not exist before, and 2) there has been no clear way to determine whether prior WOG skaters would have earned more medals if a team event had been contested. For instance, at the 2014 WOG, Jason Brown (men’s), Jeremy Abbott (men’s), Gracie Gold (ladies), Ashley Wagner (ladies), and Marissa Castelli and Simon Shnapir (pairs) all won bronze medals in the team event representing the United States, but they did not earn medals in their individual events. Some athletes who participated in the past when the team event did not exist could have been medal winners if the team event had existed. So, by using the analyses of hypothetical team events, it is possible to have a better basis for comparing past athletes with current athletes.

There are other issues that arise with only examining the potential standards mentioned above. One such issue is that skating rules are constantly evolving, and electors inevitably will be called upon to compare records of skaters from different eras. The IJS was introduced at international competitions beginning in 2003, and it was a major change from the prior judging system, the “6.0” system. Lee (2008) found that within international figure skating competition judging, judges tended to give scores which were greater standard deviations from the consensus when the IJS was used than in 6.0 judging system. Lee hypothesizes that this observed phenomenon occurs because under the IJS, the judges submit their scores anonymously (whereas in the 6.0 system, judges’ scores and their identities are made public) so there is less risk involved in producing scores that truly reflect their opinions. Even if an acceptable way of normalizing between 6.0 scores and IJS scores were developed, it could be difficult to use this normalization to decide whether a skater who competed under one judging system contributed more significantly than another skater who competed under the other judging system.

Within each discipline, comparing IJS scores between years is challenging because the Scales of Values for each discipline are revised each year. Another such issue stems from different rubrics being used for the different disciplines. Each discipline has a different set of required elements that need to be performed in a program, and these elements are scored based on discipline-specific Scales of Values in the IJS. The program lengths that are required for each discipline are also different – for instance, the maximum permitted time for the men’s free skate and pairs free skate is 4 minutes and 40 seconds, the ladies free skate and ice dance free dance is 4 minutes and 10 seconds. These facts make it difficult to use IJS scores earned as the basis of a decision of whether an entry in one discipline made more significant contributions than another entry in another discipline.

Another approach that an elector might want to use for comparing nominees in the skating accomplishment category is to quantify their contributions in a different manner. To address these concerns, we could compute an index for contributions towards the skater’s helping the country achieve a certain threshold in a hypothetical team event. Indices that were determined in the manner described earlier in this article would allow for cross-era and cross-discipline comparisons of skating records.