Five Star Blog Post!

On sites that sell books (and other things), reviews are usually accompanied by a star rating, typically ranging from one (worst) to five (best). But if you start looking at the distribution of star ratings, it often looks odd. This, I think, is what led Netflix to announce that they are abandoning star ratings for simpler “thumbs up” or “thumbs down” feedback.

Star ratings tend to cluster at the two extremes for a variety of reasons. Some books are polarizing, resulting in readers who either love it or hate it. Reviews are more likely to be written by people with strong opinions that they want to share. The end result is an odd distribution of ratings, typically with relatively few lower than four stars. If you think about it, this makes sense. People rarely buy books blindly. They may be buying because they are already familiar with (and like) the author. Or it may have been recommended by a trusted friend. Or perhaps they read an excerpt first before committing to the purchase. In any event, most books are purchased with the expectation that the reader will enjoy them.

In an ideal world, the ratings would be objective and the reviews fair. Only great works of literature would receive five star ratings. In reality, star ratings represent people’s opinions and biases. Five stars means they really liked it, and probably not that it’s one of the best books ever written. One star might indicate a badly-written book, or it might simply indicate that the reader didn’t like the style of writing or hated the ending. Or maybe they’re just a troll. 🙂

Amazon’s Braindead Review Policy

In October of 2016, Amazon reacted to accusations of posting biased reviews by making a significant change to their review policy. The goal was to eliminate reviews that were incentivized in some manner, particularly those provided in exchange for receiving a free or discounted item. And this included book reviews. The way that Amazon implemented this change was to summarily remove any review that contained a disclaimer indicating possible bias, and then to ban the reviewer. If this approach seems reasonable to you, then think again. Allow me to illustrate that the new policy not only doesn’t work, but has actually made things worse.

Consider two buyers—Alpha and Beta. Both receive a free item (which might simply be an ebook) in exchange for agreeing to post a review. Note that they have not promised to write a favorable review. Alpha is honest, and includes a disclaimer with the posted review. Beta simply posts a review without revealing the possible bias. In the past, Amazon would have left both reviews up. Now, they key on disclaimers, so Alpha’s review would be removed, while Beta’s less honest review remains. In other words, Amazon’s reviews are now, on average, less trustworthy, because dishonesty is rewarded while honesty results in punishment. The incentivized reviews are still there, but they are now impossible to recognize.

I ended up being caught out by this. I had just self-published my first book and had asked friends and family to post reviews. Because I am an honest person, I suggested that they might include a disclaimer that they knew the author. The ones that did had their reviews summarily deleted by Amazon.

So learn from my mistake. Tell your reviewers to be dishonest, because that allows Amazon to feel better about itself. 🙂