Queen of the skies
Which 20th century fighter was the best? To answer this, I'll turn to chess...
One of the great debates among internet nerds of the late 20th century: Where, exactly, did the different jet fighters relative to each other? To answer this question, I’ll turn to one of the great 20th century mathematical innovations in the theory of competitive ranking.
Almost 100 years ago Arpad Elo, a Hungarian-American physics professor and master chess player, was frustrated with imprecise methods of ranking chess players. He turned to statistics to invent a ratings system that took the chess world by storm. Now, Elo ratings and variations on the Elo rating system are used for measuring competitors in games ranging from tennis to League of Legends.
Today, I’m going to apply Elo ratings to the history of modern air-to-air warfare. It turns out there’s a real case for saying that the F-14 Tomcat is the most impressive fighter of the 20th century - the real “queen of the skies” in its day. In general, the record of the “teen series” fighters is very impressive, suggesting that they truly were the first fighters of a new generation.
Data quality issues
The problem with air-to-air combat is that it’s really hard to figure out what’s going on. Detection and evasion are a significant part of a duel between two fighters, and it’s common not to be able to confirm that a fighter was downed in action without comparing the accounts from both sides.
In this analysis, I have mostly relied on data from the Air Combat Information Group.1 With a few exceptions, I have restricted analysis to confirmed kills.2 I also removed helicopters and cases where fighters downed aircraft that had no confirmed air-to-air kills within the dataset, ranging from bombers to passenger jet liners.3 The two largest sources of data are the Korean War and the Vietnam War, but even in those conflicts, recorded claims related to air-to-air kills disagree significantly.
Finding the Top Gun
Elo ratings adjust match by match, updating based on statistics. If a high-rated player defeats a low-rated player, this is an expected result and neither player’s rating is adjusted much. However, if a low-rated player defeats a high-rated player, ratings are adjusted strongly. Over time, a player’s rating tends to reflect their current level of skill.4
Running the aptly-named “elo” package on our data, the top fighter at the end of the day is the F-14 Tomcat - the swing-wing fighter featured in the movie Top Gun. This is an interesting result, because the F-14 Tomcat was discontinued from service … and, unlike its “teen series” rivals, has been confirmed shot down in air-to-air duels.
While the top rated three fighters are all American, two other contenders emerge with high Elo ratings. The fourth-highest rated fighter is possibly a statistical fluke: The Gnat, which had a short but impressive record in the Indo-Pak War of 1965. The fifth-highest rated fighter is the Mirage series.5
In particular, with the parameters commonly used in chess, the F-14 Tomcat has earned a rating of 1680, while the F-15 Eagle places second at 1623 and the F-16 Falcon trails further behind at 1473.6 What’s going on?
Well, there are two things. One important factor is that in the data given, the F-15 and F-16 simply haven’t been as thoroughly tested. Within the data used in this analysis, the record of the F-15 and F-16 are 89-0 and 44-0, respectively. At 115-4, the Tomcat is more thoroughly tested. It may not have a perfect record, but it has a winning record. This is sensitive to the choice of k parameter used in the Elo rating system.
Second, the F-14 Tomcat faced higher-rated opponents than the F-15 Eagle. This becomes visible if we chart the fighters with the highest peak Elo ratings. The F-14 Tomcat’s record is mostly linked to the Iran-Iraq War.
The F-5E Tigers and F-4 Phantom IIs of the Iranian Air Force had losing records against Iraqi MiGs. As seen in the Vietnam War, the Phantom II versus MiG match-up is not an easy one, and Iranian pilots did not have the same level of training and support as American pilots. The Elo rating attached to MiGs climbed during the early part of the Iran-Iraq War. Some Iraqi pilots did quite well in “outdated” MiG-21s initially.
The Iraqis also imported the Mirage F1 from France. At this point in time, the Mirage series had the best record of any fighter in service. In particular, Israeli pilots in Mirage fighters had done extremely well against Arab pilots. Within the ACIG confirmed kill list, the F-14 racked up a 32-2 record against the Mirage F1, as opposed to a combined total of 6-0 for the F-15 and the F-16. The Tomcat had not only more kills, but more kills against higher-rated opponents.
It’s true that Iraqi pilots had issues with training, although there’s little reason to believe that Iranian pilots were exceptionally well-trained either.7 By contrast, the F-15 and F-16 rarely dueled Mirages, mostly downing poorly-rated MiGs at a time when the MiG model was increasingly outdated. Although not accounted for in the Elo ratings I’ve calculated, the fact that American and Israeli pilots flying the F-15 and F-16 were trained to higher standards than Arab pilots makes the match even less surprising.
The “MiG menace” was real
In terms of final ratings, the early MiG series fighters - the MiG-15, MiG-17, MiG-19, and MiG-21 - do the worst. This is true whether they are analyzed separately or grouped into a single series.8 Yet in the Korean War, there was a lot of talk of a “MiG menace.” Later, in the Vietnam War, the Vietnamese Air Force claimed it had nineteen ace MiG pilots (mostly MiG-21 pilots) compared to five American aces in F-4 Phantom IIs.
The “MiG Menace” was real. The early MiG series fighters had a peak rating of 1581, fourth-highest of any fighter in this analysis. Early MiGs weren’t badly designed; they just stayed in active service long after they became obsolete. The fighter that ended up countering the MiG-15 in the Korean War, the F-86 Sabre, rose to a peak Elo rating of 1872. The Sabre’s higher ending Elo rating than the early MiG series is mostly due to the fact that the Sabre was effectively retired from service earlier; the Sabre would probably have done just as badly against Israeli Mirages and Iranian Tomcats.
Generations of jet fighters
Jet fighters are usually divided into generations:
First generation: Transonic fighters with swept wings. Includes the MiG-15 and the F-86 Sabre.
Second generation: Early supersonic fighters. Includes the MiG-21, the “century series” fighters like the F-104 Starfighter, and the earliest Mirage fighters.
Third generation: Multirole supersonic fighter/bombers. Includes the F-4 Phantom II, the MiG-23, and later Mirage fighters.
Fourth generation: Advanced multirole supersonic fighter bombers. Includes the “teen series” fighters and the fighters designed to answer them.
If we look at their Elo rating records, there is a point when second-generation jeh fighters take over from first-generation jet fighters. However, the third-generation jets don’t really stand out from the second-generation jets - not in early service
(c.f. the Vietnam War) and not in later service (after the introduction of the “teen series” fighters).
In fact, looking at Elo ratings based on confirmed kill records isn’t enough information to discern significant differences in performance between second-generation jet fighters kept in service and updated with modern avionics and missiles, third-generation jet fighters still in service, and fourth-generation Soviet fighters.9
The MiG series is a particularly clear example of this. Each successive MiG had real and material upgrades over the predecessor, with incremental increases in performance - speed, agility, range, and payload. Some sources place the MiG-17 in the same generation as the MiG-21; others place the MiG-17 in the same generation as the MiG-15. The MiG-29 may have been the last and deadliest Soviet MiG, but it still lacked many of the key innovations that made the “teen series” fighters so deadly.
Variants of the F-5 are in design and manufacture in Taiwan and Iran; the Chinese version of the MiG-21 was manufactured through 2013. When the F-14 Tomcat was retired from American service in 2006, it did so as the queen of the skies - with a proven record only rivaled by its “teen series” sisters.
This is a now-defunct website that was fortunately mostly archived. Other sources include a personal website listing Israeli air-to-air-kills, as I was able to find ACIG data only for the Egyptian, Syrian, and Iraqi side of that conflict. I also looked on Wikipedia for post-Vietnam aerial kills by the United States and added a small number of known recent air-to-air engagements.
The major exceptions are the Ethiopian-Eritrean War and a few recorded but not confirmed engagements in the Russian invasion of Ukraine, which involve contests between different Soviet-designed jet fighters. This mostly helps with establishing the record of the MiG-29 and Su-27 against older Soviet fighters and each other.
Strictly speaking, this excluded some fighters and included some aircraft that aren’t fighters, such as the B-29 Superfortress.
For most competitive chess players, this means rapid improvement in early years, slower improvement during adulthood as an experienced player, and then an age-related decline that sets in some time around age 35-45.
I’ve run this analysis with the Mirage III, V, and F1 separated as well as lumped together. The main motivation for grouping together the Mirages and MiGs is that while the series does have some distinctive features, the most advanced versions unambiguously end up with the lowest ELO ratings due to the fact that they came out after the heyday of lightweight agile fighters.
Starting ELO of 1200 and k=40, which is what the FIDE uses for younger players.
Remember, the same Iranian Air Force had a losing record with its F-4 Phantom IIs and F-5 Eagles against the same opponents!
If separated, paradoxically, the MiG-21 gets the lowest rating. I’ve also grouped together the MiG-23 and MiG-27 as being fairly similar to each other. The MiG-23 was a variable-geometry design, representing a major departure from prior designs, with the variable-geometry MiG-27 closely based on the MiG-23. The MiG-29 reverted to a fixed-wing design.
The Su-27 reportedly cleaned the MiG-29’s clock in the Ethiopian-Eritrean War, and might actually have fourth-generation capabilities.