JP Power is out with its 2016 ranking of hotel loyalty programs and media seems to be taking it at face value without digging into the methodology — they report the rankings without understanding that it’s garbage in, garbage out. And that’s dangerous, as illustrated by industry site Skift:
Interestingly, the Starwood Preferred Guest program, beloved among frequent travelers, ranked near the bottom, and it’s never fared well in past J.D. Powers’ reports. So, why is it so admired, and why does Marriott want it so badly?
Here are JD Power’s 2016 rankings:
This isn’t simply one of those “these things are subjective” or “it depends on whom you ask” kinds of things. The survey is just fundamentally silly. That begins with the factors that go into their ranking:
- account maintenance/management (23%)
- ease of redeeming points/miles (22%)
- ease of earning points/miles (18%)
- reward program terms (16%)
- variety of benefits (16%)
- customer service (5%)
The most heavily-weighted factor in the rankings — nearly a quarter of it — was ‘account maintenance’. That simply isn’t the most meaningful factor in the value of a hotel loyalty program. Although it’s heartening for Hyatt Gold Passport, that if they could only get their IT right they might do better in these rankings, it doesn’t speak to what turns customers into evangelists and leads them to make repeat purchases independent of the value proposition on a given stay.
Note that ‘account maintenance’ isn’t “customer service” which is of course important — but is its own category worth only 5% of the survey’s weight.
Meanwhile account maintenance and terms and conditions are drivers (.pdf) of Hilton HHonors’ success:
Hilton HHonors scores high in the account maintenance/management factor (756) and also performs particularly well in the reward program terms and variety of benefits factors.
Things that are missing from the survey’s weightings:
- Elite benefits, entirely, other than ‘variety of benefits’. JD Power doesn’t tell us what that means but it appears different from quality or richness of benefits.
- Quality and value of redemptions. They focus on how easy it is to earn and burn, not how good what you actually get is. So it’s not about the value proposition.
This survey is about earn and redeem only, not about how well a program treats you (elite benefits) and strangely for something focusing only on redemptions it cares about ease of redemption but not what you actually get.
Meanwhile they draw such strong conclusions from a small sample of 3096 consumers, which is about 20% fewer people than they were even surveying two years ago.
And that lets them conclude that La Quinta Returns is better than Hyatt Gold Passport, and Drury Gold Key Club is better than Starwood Preferred Guest. We don’t have the underlying data set, but how many of those 3096 people could have been familiar with Drury? The results of the survey are self-refuting.
**Grabs popcorn waiting for everyone’s favorite boardingarea commenter**
Th primeval scream you hear is @Gary’s decrying the results of the prestigious and academically acclaimed JD Power survey of customer satisfaction with loyalty programs, which every year completely upends his and other travel bloggers’ biased ranking of the same programs. He thus has this post ready for cutting and pasting, so I should warn you that this is a rehash of discredited claims. I have said many times that travel bloggers get things backwards and this is the annual independent proof.
As, at best, a work-in-progress, HGP raking in the bottom half is about right, and as the most expensive program in the business that also just went belly, the next to last place ranking for SPG is a surprise. I would have guessed dead last… 😉
lol. Hilton Honors is my program of choice – mainly because their hotels meet my cost and location needs most of the time, but their “account management/maintenance” is horrible if you ask me. It seems to be impossible to separate my company name from my home address… and the website will log you out of your account if you happen to navigate via a path it doesn’t like.
So not only do they over weight a rather meaningless feature – they get the rankings of the feature wrong.
@Gary — Remember that JD Power surveys represent a RANDOM sampling of loyal programs members at ALL levels and not just of top elites or other travel animals who haunt loyalty blogs and tend to be highly opinionated. I thought I would mention that simple fact about scientific surveys, which you ignore but prestigious surveys do not.
It is because of results like these from prestigious polls of ALL guests that Marriott’s top brass is not as impressed with SPG as that program’s top elite members are. As a result, the combined Marriott Rewards/SPG program is likely to be more like Marriott rewards than SPG…
I am a leisure traveler and I don’t consider JD Powers surveys “prestigious.” When I see their articles I almost never click on them. The way I use a hotel loyalty program is that I have several paid stays in to visit family and kid’s travel volleyball club. Then I use those paid stays (along with other tips Gary’s blog provides) to stay free when I go to MLE, JNB, BKK, HKG, CDG, PRG, etc. etc. I like to take 2 big trips a year and there is no way JD Powers, USA Today or NYTimes provides better information than Gary does on how to accomplish it.
@Bob sez: ” Hilton Honors is my program of choice — mainly because their hotels meet my cost and location needs most of the time…”
Well, that is what the survey found! A bunch of customers that are MOSTLY satisfied like you!
BTW, if you are running M$ Windows 8.x or higher, I would suggest getting the desktop version of the HH app for smartphone. It has the look and feel of a smartphone app, it is slick, it is fast, and it is “account management/maintenance” at its best…
G’day!
@DCS Marriott Rewards, Hilton HHonors, or anything else may be fantastic. But the methodology as I explain about does nothing whatsoever to demonstrate that. Saying you love a program, and it happens to match what JD Power found, does nothing at all to suggest anything other than a coincidence. Because, logic.
And calling JD Power ‘prestigious and academically acclaimed” is.. funny.
Having a non-transparent methodology means it’s all a big ¯\_(ツ)_/¯ when it comes to any validation of the results.
That being said, there are some measures where SPG’s program is pretty objectively awful compared to competitors, which could easily be one of the factors at work here in their low rating. At least this guy seemed to think so.
(SPG being a pretty “meh” program for rebates and me not being a road warrior capable of top-tier status is why I’m usually very disinterested in the program, unless there’s a killer promo I can use.)
I’m with JD Power on this one, they are independent and not paid to promote one or the other, Hilton is fabulous, apart from the devaluation fraud they committed against their millions of members a few years back. Oh, and who can forget them throwing Diamond status around like its going out of fashion, to anyone who gets a credit card with a Yank bank, so that those of us who actually earned it through loyalty and stays get less benefits because so many Yanks are abusing it.
@DCS:
Yeah, but as far as I can tell the survey didn’t measure satisfaction with cost/convenience. Getting it “right” by completely the wrong methods doesn’t make it good survey.
I am sure as heck not choosing my hotel program based on “account management/maintenance” which JD Powers seems to think is the single most important factor.
And, no, I won’t be downloading the desktop app. Don’t need to clutter things up when the website and phone app is adequate for making bookings.
“Gary — Remember that JD Power surveys represent a RANDOM sampling of loyal programs members at ALL levels ”
Actually, no , it doesn’t. It represents responses from people who will respond, and it’s not possible to get a random sample if the survey takes more than a few minutes–if that.
For example, I purchased a Hyundai Equus. The JD Power survey they kept sending me with a crisp $1 for the hours of time it would have taken to fill the thing out was a joke.
MAYBE they are accurate on short surveys……with people who will work for 25 cts per hour.
BTW, I would not buy a Hyundai Equus again. There’s my survey answer. I should have just written that across the top and sent it in, but it was not worth $1 of my time to mail it.
Well you know, I actually do feel SPG is overrated around travel forums and Hilton is underrated, simply by virtue of the ease of accruing points. This doesn’t matter to people like Gary, who has huge stocks in each account, but it does matter to us little people.
With that said, however, I agree completely that the quality of redemption should be taken into account, and any published work which has Best Western above Hyatt is prima facie wrong
Rewards program != Loyalty program.
HH will give anyone with $75 or whatever solid mid-tier status….thus, no loyalty is needed. It’s a rebate program.
For high-stay, higher-spend loyal customers, SPG still comes out on top. Hopefully Marriott will learn from that.
You can make all the claims you’d like. The survey is integrity and methodology are unassailable.
McGraw Hill Financial’s businesses are:
Standard & Poor’s Ratings Services
Standard & Poor’s Ratings Services helps investors and market participants manage risk through credit ratings, research and analytics.
S&P Capital IQ
Financial professionals turn to S&P Global Market Intelligence for high-value content across all asset classes.
S&P Down Jones Indices
Investors use the S&P 500, Dow Jones Industrial Average and thousands of other indices produced by S&P Dow Jones Indices across asset classes to measure and monitor global markets.
Platts
Traders, risk managers, analysts and industry leaders use Platts’ price assessments and insights to enable informed decision-making across energy, petrochemical, metals markets and agriculture markets.
J.D. Power
Companies from more than a dozen industries turn to J.D. Power’s quality and customer satisfaction research for actionable consumer insights to help produce better products and services.
@ DCS says “The survey is integrity and methodology are unassailable.”
Yeah, no. Hilton HHonors reports that it has “more than 50 million members ” And that’s just Hilton. Yes, there will be duplicates and non-active accounts in there. But there is no way JD Powers’ 3096 particpants is a representative sample.
@Bob — That is confusion. What’s the US population today. How many registered voters are there? How many of those to national surveys of, say, choice for president or presidential nominee are sampled? You need stats precisely because it is impossible and it would be prohibitively costly to poll everybody. But if the sampling is RANDOM, a small survey can accurately reflect the preference(s) of the population at large.
Stats 101.
G”day!
@DCS – the sample also has to be statistically significant in addition to random. 10 people can be random, but not representative. Stats 101.
There’s no evidence here that this survey is either truly random or statistically significant.
None of that changes the fact the weighting “Account Management” as the most important factor is patently absurd.
I’m done here.
@DCS the survey’s methodology is clearly not unassailable, because I’ve in fact criticized it. In this post. And you don’t go explaining how it makes sense.
And oh by the way JD Power licenses its logo to the winners, and that represents a substantial portion of its earnings. Surveys where the winner licenses the logo tend to be the ones that get repeated year after year it seems.
@ DCS says “The survey is integrity and methodology are unassailable.”
I’m assailing.
McGraw Hill owns S&P, as you point out.
They really gave great, unassailable opinions on all sorts of financial products that managed to go from AAA to default in no time at all. Gee, totally unassailable rating on AIG at AAA. Truth is, they had no idea how to figure out the interlocking web of companies reinsuring each other.
So, I call BS on most ratings.
JD Power is just a marketing gimmick. You know, they do get paid by someone. Consumer Reports this isn’t.
@eponymous coward this post isn’t about ‘who is best’ it’s simply that we aren’t helped in knowing by the JD power survey. I have always said that the SPG rebate for in-hotel spend is weak.
God damn, that was some good popcorn!
LOL. Shorter @Gary, the abuser of superlatives: “This survey doesn’t support my biased opinion so, therefore, it is no good.”
From the above, I think that Gary, Ep. Coward, & I all agree that the main DISadvantage of the present SPG is that hotel stays earn a measly 2 points per night! Or 3 for Elites. Which tilts the program heavily in favor of earning via credit-card spend! Given how many Starpoints it takes to get a free room vs. at Hyatt, Marriott, or IHG, they should offer 4 pp$ on stays (before credit-card,) 5 or 6 for Elites!
Starwood: 3 +2 = 5 pp$ for Amex members vs. HHonors 10 +5 +2.5 +7= 24.5 pp$. So is a 10K point room at Starwood = a 50K room at Hilton?? Maybe.
@DCS No, “the survey is no good so it neither supports nor undercuts my views or anyone else’s”
Why is it that simple claims become mangled as something completely different, that you seem to respond to straw men only?
JD Power rankings are bought and paid for. Completely meaningless
@Gary:
Which is why I pointed out that non-transparent rating system. What you pointed out in your earlier blog post COULD be part of the reason SPG is low-rated. I’ve seen similar things happen in other rating systems for travel loyalty, where suddenly Southwest or JetBlue gets great ratings because their points can quite literally be used for ANY flight.
Or it could be any other amount of “secret sauce”. But without what’s knowing in the secret sauce, the claims are fairly useless for any kind of in-depth examination because you can’t replicate their findings, you just basically have to trust them. It’s like having researchers announce findings of experiments but not give you their experimental protocols so you can reproduce them.
@Gary sez: “Why is it that simple claims become mangled as something completely different, that you seem to respond to straw men only?”
There are no straw men here at all. Anyone who has ever set their cyber-foot on this site knows the mantra: HGP and SPG have the “best” elite benefits in the business; they are small; so they have to try hard; that makes them the “best” loyalty programs out there.
HGP breakfast has been given mythical status. DSUs are “confirmed” and the “best” upgrades in the business. Starpoints are the single “best” points currency there ever was.
Then when a survey comes out that challenges those claims and superlatives, of course, you’ll think they are wrong. You simply must reject any survey that does support your biased superlatives or you come out looking silly. Therefore, you would be biased regardless and are, therefore, not credible. It is that simple. Those who live by superlatives, die by mirrored superlatives, when claims of “best” are shown to be “worst”.
In short, there are no straw men at all when I said:
LOL. Shorter @Gary, the abuser of superlatives: “This survey doesn’t support my biased opinion so, therefore, it is no good.”
It went straight to the point of the post, well as to the point of the same post last year, the one for the year before, and the one before that, for as long as JD Power did not rank HGP or SPG tops, meaning never… 😉
G’day!
“And oh by the way JD Power licenses its logo to the winners, and that represents a substantial portion of its earnings. ”
Red herring to imply quid pro quo without a shred of evidence…
Q: Do they license the logo — which AMEX, DISCOVER, and other fine winners have proudly displayed — AFTER or BEFORE declaring a winner?
The timing here is very important.
Correction: “You simply must reject any survey that does NOT support your biased superlatives or you come out looking silly.”
Gary, don’t feed the dogmatic trolls. Not worth your time.
@DCS I have made very specific arguments as to why THIS survey does not show what it claims to show. You have not even attempted to engage with those.
This survey is an affront to the practice of statistics and to the science of measurement. J.D. Power’s “six factors” and weights are completely arbitrary; the “1,000-point scale” is a veneer of precision and objectivity.
A legitimate, mathematical approach to scoring hotel loyalty programs would have involved factor analysis and latent variable (random effects) modeling. And, no scientific method is completely objective or “unassailable.”
J.D. Power does offer one clue, in fine print, toward the preposterous invalidity of its report. “Rankings are based … not necessarily on statistical significance.” Any quantitative report that lacks even internal validity, the simplest hurdle for any scientist, is a massive joke — not to mention that statistical significance, or lack thereof, is orthogonal to qualitative research methods, design, and analyses.
I concur with the general opinion of DCS that HHonors is a good loyalty program, underrated on the blogs, but I vehemently distance myself from his praise of the J.D. Power report. As a rigorous quantitative scientist by trade, I assert that the J.D. Power numbers are no more useful than the output of a random number generator.
@Gary sez: “DCS I have made very specific arguments as to why THIS survey does not show what it claims to show. You have not even attempted to engage with those.”
What do you think I have been doing for the last two years? My arguments are now well established, e.g., that starpoints are NOT the single greatest points currency in the history of the hobby, or that just 4 DSUs/yr are not the single best elite suite upgrades in the business. These subjective things that you keep trying to pass for dogma by abusing superlatives simply do not resonate with most regular people, which is really what it comes down to as far as for-profit hospitality companies are concerned. They want to fill hotels with paying customers and not with self-entitled elites! It is this type of “customer satisfaction” that JD Power assesses.
In fact, I just argued that by constantly touting the metaphysically high “value” of starpoints, travel bloggers might have contributed to the demise of Starwood and SPG. Below is the argument that occurred to me and I made over at InsideFlyer after someone suggested one scenario that might have led to Starwood/SPG being the catastrophic failure that they currently are.
So, I will ask here the question that I tried to answer in that post, which I reproduce below: “Did travel bloggers contribute to the demise of SPG by encouraging people to earn starpoints without ever setting foot in a single Starwood hotel?”
I believe they did. Here’s my argument, just slightly edited to protect anyone who might not want to be mentioned:
“Catastrophic failure, indeed. Another way in which SPG might have been wrecked — and the bloggers’ touting of the metaphysically high value of unbonused spend on the SPG AMEX and the transferability of the resulting starpoints might have contributed — is that people no longer needed to stay at Starwood properties to earn starpoints! They just needed to get the SPG AMEX and put all their general spend on it to earn starpoints without ever stepping in a Starwood hotel! Regrettably, that boneheaded model was Starwood’s own creation. How?
Is it not strange that a hotel loyalty point currency’s greatest selling point is that it can be transferred to miles for redeeming for airline award tickets? Well, it is the starpoint’s claim to fame and that is no accident. SPG made it that way on purpose…or by accident. Like all companies that run a loyalty program, Starwood always has to keep decreasing their financial “liability” due to all the starpoints that they award with every sale but cannot claim full cash credit equivalent to the monetary value of the points awarded until the points have been redeemed, forfeited or simply “broken” [never spent for whatever reason]. Although it can tie up billions of a company’s revenue, Starwood did not want to decrease this financial liability by encouraging members to redeem their points for FREE stays like all other programs do (Gosh! Giveaways to those spoiled, insufferable members? Free stays at St. Regis? No way!), so they made their hotel awards, especially “aspirational” ones, very expensive to redeem starpoints for. To make sure that starpoints were redeemed before the liability broke the company, they made it very favorable to transfer the points to miles for redeeming for airline award tickets [and bloggers went wild with the b.s. about how their transferability makes starpoints the “single most valuable points currency there is!”]. Marriott, Hilton and other programs are different: they decrease their liability by encouraging members to spend their points on free stays at their properties [or on innocuous stuff like magazine subscriptions], which is the better hotel loyalty program model, IMHO. Starwood’s model was flawed to the core. Conned by travel bloggers, the SPG model led to people preferring to earn starpoints through general spend on the SPG AMEX rather than getting them through revenue stays at the ridiculously expensive Starwood hotels!
So, the flawed model that encouraged people to earn starpoints without ever staying at Starwood properties, the company’s growth got anemic, stockholders bitched, a CEO got canned as a result, and on the auction block Starwood found itself!”
Chime in!
Last paragraph over-edited. It should be :” So, DUE TO the flawed model that…”
Oh JJ, that was priceless! Of course, as soon as I was halfway through Garys piece, I has thinking “omg, here comes Mr.Hilton, aka DCS”, but to see your comment, followed immediately by his first, super-predictable comment was laugh out loud funny.
But, now, having skimmed them all (I no longer read dcs–hell, I could WRITE his thin-skinned sanctimonious rants), I have to say: no one can eat that much popcorn!
@mbh – agreed, that’s why I call him the Marco Rubio of the blogs (3 repeated talking points, never changes) with a dash of Donald Trump mixed in (unable to rationalize, never admits his faulty points). It’s good for a laugh for the rest of us!
And, the theory that somehow SPG AmEx use contributed to the “downfall” of Starwood (the company) is just about the dumbest one I’ve ever seen. Loyalty card usage has exploded across all brands in different transportation categories.
Equally stupid is another theory that “the single best thing of SPG is airline transfers”…a myth continually repeated by a single poster. Is it a nice benefit, and a way to create a bonus on unbonused spend? Of course. I can name a half dozen other SPG benefits that I and others would say are much higher priority however.
It’ a good thing the “Carly For President” PAC was across the hall from my office and left thousands of microwave popcorn bags when they deserted . I mean, eating popcorn and reading this–hell, eating popcorn and doing ANYTHING else, has to be better than eating popcorn and watching a DVD about the history of Carly’s life. Oh, yeah, there are a thousand or so of those DVD’s if anyone wants them.
just like mbh uttered, JJ you are spot on man! Ironically I am a lifetime HHonors guy but DCS’ bs posts here just really irritate me. Keep up your good work Gary.
@Mickey in PDX et al. — Do you know what would give your bitching some credibility? If could you just once put a few coherent thoughts together and tried to rebut DCS’s so-called ‘bs posts’, instead of the constant childish temper tantrums. You could begin by rebutting the long and provocative post just up-thread…
BTW, do you know why kids of throw temper tantrums? No? Well, most of the time it’s because they can’t get someone to give in to their puerile demands. Since you’re not likely to get me to bow to your will and I will keep posting, you will just keep getting more and more frustrated. But don’t lose all hope because learning to deal with frustration is a skill that children gain over time.
G’day.
Here’s what credibility looks like – explain why HH is a good program, when most think it’s not, and even those who say it’s at least ok, can’t really say why. And do that instead of ragging on SPG for a change.
Take your meds, and G’day! LOL.
I have a prepared list of 100 internet insults that i’m selling for $9.95 good for any internet argument.
Here’s a taste:
10) I got bored after the first paragraph, largely because you aren’t as fascinating to me as you are to yourself. I bet they would adore you at reddit.