Which Frequent Flyer Program is Best? Don’t Be Fooled By US News!

US News has come out with a ranking of best frequent flyer programs. (HT: Dallas News Aviation Blog)

Here are the rankings:

1. TrueBlue, JetBlue Airways, 4.34
2. Rapid Rewards, Southwest Airlines, 3.92
3. Mileage Plan, Alaska Airlines, 3.81
4. MileagePlus, United Airlines, 3.74
5. AAdvantage, American Airlines, 3.71
6. Elevate, Virgin America, 3.57
7. SkyMiles, Delta Air Lines, 3.46
8. Hawaiian Miles, Hawaiian Airlines, 2.81
9. EarlyReturns, Frontier Airlines, 2.23
10. Free Spirit, Spirit Airlines, 1.44

But this ranking is based on a severely flawed methodology that has very little to do, even, with frequent flyer programs. And is coupled with even worse advice. Here’s their methodology.

What They Say About Their Survey is Just Wrong

Here’s how the survey is described:

To help you weigh your options, U.S. News evaluated 10 leading frequent flier programs using an unbiased methodology that takes into account each program’s average flight prices, earning ratios and daily flight volume (among other features). Our ranking and detailed program profiles are intended to help you choose the program that’s best for you.

They call it an unbiased survey, but it has little to do with the quality of frequent flyer programs as I’ll explain below.

Most importantly, though, the bias comes from the criteria itself.

It appears as though the survey designers do not understand their subject even. For instance, they offer this advice:

If you prefer to fly first class, you should consider a program that awards benefits based on dollars spent rather than miles flown.

That might be true if you prefer to buy paid first class tickets, you may earn more in a program that rewards you for spending more. That’s not always true. But it is decidedly not true that if you prefer to redeem for first class you will be better off earning points in a program that prices awards based on the cost of a paid ticket.

That’s a distinction they don’t make, so they likely contribute to confusion rather than understanding.

Sixty Percent of the US News Scoring Is About the Airline and Not the Frequent Flyer Program

10% of the score is the number of daily flights an airline has. Domestic flights only are included, so airlines that operate primarily domestically are advantaged here.

10% of the score is the geographic coverage of the airline (of course, what’s relevant for frequent flyer miles is the worldwide reach of the ability to earn and burn miles, but this ranking does not include airline partnerships or alliances in the mix at all). The number of airports served is the metric — not even metropolitan regions. Serving Washington Dulles, National, and Baltimore count as being 3 times as good as serving just one of those those. Serving little-used Essential Air Service communities is as good as serving New York, Chicago, and Los Angeles.

20% of the score is flight frequency, allowed for booking. Airlines get a score based on the number of flights for the cities they fly between the most that you can use points for. But since most airlines let you use (some number of) points for any seat on any flight, this comes down to giving points to airlines that fly a lot between the cities they serve. (Only United does not offer all customers access to the last available seat for some number of miles — a ‘privilege’ they reserve for elites and co-brand credit card holders.)

20% of the score is the quality of the airline. It doesn’t matter that I can use my Alaska Airlines miles to fly Delta or Cathay Pacific or Emirates. What matters here is how good an airline is Alaska to fly? Which of course is entirely an airline quality rating and not a frequent flyer program rating as the study purports to offer.

The 40% of the Score That’s Actually About the Frequent Flyer Program Tells Us Very Little

30% of the score is supposed to be ‘ease of earning a free roundtrip flight’. But it isn’t. They take the average price of domestic coach awards for just 5 routes for each airline. They searched the same travel dates for each. That means they took no account of seasonality, or day of travel effects.

What’s more, that tells you nothing about how easy it is to earn a free roundtrip, since it looks only at the number of points required but not the ease of earning those points. Different airlines, especially the ‘revenue-based’ ones, have an entirely different point scale.

10% of the score is ‘additional benefits’ described as follows:

The Additional Benefits score comprises the number of ways members can earn and use points (other than for booking flights), whether or not points expire and additional perks that come from memebership.

Not all airlines were given an additional benefits score.

The Survey Doesn’t Tell Us What it Tells Us it Tells Us

The important thing is to understand what a survey does and does not show. There’s no one answer to which program is best, it depends very much on the needs of the consumer, what they subjectively value, although there are broad generalizations.

While the Freddie Awards offers one window into the programs that are best, I try to be as clear as possible about what it shows and does not show — and that it doesn’t offer one final answer for all cases. It is merely an attempt to amalgamate the preferences and opinions of members writ large. It’s what program members in general express that they value most in a given year.

The US News survey tells us, based on very limited data, what its editors think is important in a US domestic airline product and they then say they’re ranking frequent flyer programs. They aren’t.

What’s more, they aren’t accounting for how easy it is to earn points, the scale of those points, airline alliances and partnerships, international premium cabin awards, and any myriad number of things that actually factor into what is important in a frequent flyer program.

Unfortunately there will be those who pick up the ranking and believe it means more than it does.


About Gary Leff

Gary Leff is one of the foremost experts in the field of miles, points, and frequent business travel - a topic he has covered since 2002. Co-founder of frequent flyer community InsideFlyer.com, emcee of the Freddie Awards, and named one of the "World's Top Travel Experts" by Conde' Nast Traveler (2010-Present) Gary has been a guest on most major news media, profiled in several top print publications, and published broadly on the topic of consumer loyalty. More About Gary »

More articles by Gary Leff »

Pingbacks

Comments

  1. Any type of survey from U.S. News has to be taken with a grain of salt. Their annual university rankings are equally as flawed as they will include irrelevant criteria like endowments, but will ignore campus safety. Don’t be fooled by any of their surveys!

  2. Considering I am a member of about 20 frequent flyer programs, live in the United States, and redeem dozens of free tickets a year, the fact that I don’t even bother to belong to TrueBlue would suggest that it is not the best frequent flyer program in the country. 🙂

  3. @ iahphx. I had to count for myself too.
    I am maintaining point balances in 18 programs and have membership in 24 (extra ones just to be able to search like Qantas)
    Yupp, don’t even have memberships in #1, #6 & #10. LOL

  4. The first two are completely wrong, but the rest of the rankings are actually how I would probably rank them.

  5. I’ve said it before. I’ll say it again. You understand the methodology. Do a Kickstarter, raise the money, hire a PhD student or something, and create the study of your dreams. Then market it as an industry metric that means something. Use your influence to do more than help families get to India over Christmas… keep the whole industry honest!!!

  6. Current PhD student in a quantitatively-oriented field at a US News Top 10 school. How much are you offering? 😉

  7. Gary,

    The fact that I didn’t read the report, nor did I even know it existed (along with 99.9%) of the frequent flier community should tell you all you need to know about the quality journalism that is US News. I’m positive that you, me and the other 6 commenters before me were indeed the largest spike in traffic for all of 2014 for that awesome “publication”

  8. It’s OK that reports like this aren’t accurate. It provides more of a level playing field and makes the really good programs continue to try harder. It’s like calling the airline on mistake fares. Don’t draw attention to yourself.

  9. Neither the airline or hotel survey gave any indication of the true worth or status of the programs. Typical of most so called news these days, neither informs nor entertains.

  10. They just screwed up the numbers – give ’em a break! Make #1 an 11 and make #2 a 12 and it all fits!

  11. Now, now….#2 actually is the best choice for domestic flyers who earn Companion Pass. And they definitely got #10 right!! Wonder if Spirit will highlight their ranking in their ads??

  12. hahaha…and there’s a reason why I don’t own the following:

    #1 – Low sign-up/hard to earn points
    #6 – Low sign-up/hard to earn points
    #7 – SkyPESOS? Really?
    #8 – Low sign-up/hard to earn points
    #9 – Need to fly out of Denver each time and I live in LA.
    #10 – Are you serious? and yes, they should come up with a catch phrase to celebrate being #10!

    What a bogus report from US News.

  13. I never understand why anyone and everyone is always trying to rank FF programs… there is simply no one-fit-all answer. the only value these reports have is entertainment.

Leave a Reply

Your email address will not be published. Required fields are marked *