On Untappd and beer ratings generally

Recently I was speaking to a friend who is opening a brewery in Massachusetts. Like any new brewery in the northeast, he is going to launch with hazy IPAs, but he also loves classic British ales and German lagers. One of his concerns is how his beers will be reviewed and rated on Untappd.

Untappd is a mobile application where users can share, log, review, and rate the beer they drink. The ratings are a simple one-to-five star rating system where users can rate a beer in 0.25 star increments. There are no instructions in how to rate beers in the app. Untappd was conceived as being unstructured; an app that could be enjoyed by casual beer drinkers and hardcore craft beer nerds. The upshot is that as the app and its user base has evolved the highest rated beers are mostly hazy double IPAs, barrel-aged stouts, and the like.

My friend's concern is that even if he brews the best Ordinary Bitter in the world it still won't score as highly as a mediocre hazy IPA. That in-turn will drag down his brewery's overall rating, and deter people from buying his beer. Lots of other brewers share his concern. Some have thought of ways to try to game Untappd ratings. My friend has considered putting up signs or table tents requesting people rate his beers to style.

Initially I nodded at the idea, but the more I think about it the more I think is a little bit of an ask. The fact that a non-distributed brewery like Tree House was the most checked-in brewery in 2019 makes me think that the Untappd user base currently skews heavily toward the beer nerd crowd. While that may be true there are still casual beer drinkers on the app. I've been on Untappd since 2011, over a year before I started brewing. At that time I would have told you I was knowledgeable about beer. I didn't know what I didn't know.

The issue with beer ratings is not the ratings in and of themselves. The issue is how people use and interpret beer ratings. At their best beer ratings are a guide. Going back to the days working at my uncle's car dealership, in my experience human beings are terrible at using guides of any kind. Too many people treat guides like guides are gospel. Guides are opinion and not fact. An expert opinion in a magazine is still an opinion. Aggregated user ratings on a mobile application are nothing more than aggregated opinion.

If Beer A has a 4.07 rating on Untappd it is not definitively a better beer than Beer B which only has a 3.87 rating. Where brewery owners are rightfully concerned is when a drinker is looking at a beer list or inside a cooler and pulls out a phone before making a decision. The only solution I see is for consumers to be smarter. That means being aware of the biases in the ratings, and confident enough to form their own opinions.

As I became aware of the biases in the ratings I mostly stopped using them to inform my buying decisions. I still log most of my beers on Untappd. I use it mostly as a journal so I can look back to what beers I have enjoyed when and where. I used to really enjoy collecting badges on the app from drinking different beers, but so many badges have been added over the years the accomplishment of collecting badges felt watered down.

I stopped rating beers for the most part on the app for a few reasons. When I started working in the industry I felt weird about rating customer's or prospective customer's beers. Mostly I noticed how compressed my own ratings were. How useful is a rating if on a 1-5 scale the majority of the ratings are clustered between 3.5 and 4.0?

My ratings graphed. Not a lot of useful information there. 

If you enjoy rating beer, by all means continue rating beer. Use Untappd the way it was intended which is however you want to use it. My only suggestion is to keep an open mind when tasting a beer and don't be afraid to form your own opinion even if it differs from the crowd.

I don't envy brewers who have to sweat ratings and also comments. I can only imagine what it is like to put your heart and soul into something to have it be torn down by someone who clearly does not know what they are talking about. It must be really annoying to have a user say a beer is great, and then give it three stars. If I was a commercial brewer I'd monitor the ratings and comments to get a feel for how my beer was being received. If a beer is getting similar feedback from numerous users that's information I would want to have.

There isn't a solution for the biases toward certain beer styles. The styles that are the most highly rated are the styles that consumers like the most. Why that is the case is a post for another day.

Since I've been talking about my Untappd account, I don't see a reason to keep my account private. Feel free to follow me.

Follow me on Twitter @JChalifour
Like The Would-be Brewmaster on Facebook