Opinion: Is JD Power's hotel loyalty ranking hopelessly flawed?

WM Circle Logo

By: RickFerguson |

Posted on April 12, 2016

The Boarding Area is hosting a lively discussion about the value of JD Power's recently-released ranking of hotel loyalty programmes. The main article, written by frequent travel guru Gary Leff, doesn't mince words- the ranking, he says, is based on flawed methodology and is largely worthless. The article's commenters, however, don't necessarily agree. Let's pop some popcorn, kick back, and watch the sparks fly.

Gary Leff is a heavy hitter in the frequent-travel space, so he knows whereof he speaks. He has harsh words for the JD Power 2016 ranking of hotel loyalty programmes, which ranks Hilton and Marriott numbers one and two, respectively, while ranking the well-regarded Starwood programme second-to-last. Given how beloved is Starwood's programme among its elite members, Gary feels compelled to ask: What gives? The problem, he concludes, is the survey's methodology. Money quote:

"This isn�t simply one of those 'these things are subjective' or 'it depends on whom you ask' kinds of things. The survey is just fundamentally silly. That begins with the factors that go into their ranking...The most heavily-weighted factor in the rankings � nearly a quarter of it � was �account maintenance�. That simply isn�t the most meaningful factor in the value of a hotel loyalty programme. Although it�s heartening for Hyatt Gold Passport, that if they could only get their IT right they might do better in these rankings, it doesn�t speak to what turns customers into evangelists and leads them to make repeat purchases independent of the value proposition on a given stay."

Leff is right on: there's simply no precedent for making "account maintenance" the most heavily weighted component of your survey. The survey also ignores the importance of soft benefits - often the most critical component of the programme value proposition for elite travelers - and gives customer service short shrift at only five percent of the weighted survey results. Structure a survey with those factors weighted appropriately, and you might come up with a dramatically different ranking.

Boarding Area commentators, however, aren't necessarily buying Leff's criticism. Money quote from a commentator:

"Remember that JD Power surveys represent a RANDOM sampling of loyalty programme members at ALL levels and not just of top elites or other travel animals who haunt loyalty blogs and tend to be highly opinionated. I thought I would mention that simple fact about scientific surveys, which you ignore but prestigious surveys do not...It is because of results like these from prestigious polls of ALL guests that Marriott's top brass is not as impressed with SPG as that programme's top elite members are. As a result, the combined Marriott Rewards/SPG programme is likely to be more like Marriott rewards than SPG�"

Another commentator points out that Leff published almost exactly the same opinion of the JD Power survey in April of 2015. Will JD Power finally heed Leff's criticism and revamp its survey for 2017? Stay tuned. In the meantime, Starwood may have some explaining to do about their account maintenance functions to their new bosses at Marriott.

- Rick Ferguson

More Info: 

http://www.jdpower.com