Holding the Last Star Hostage

There’s a trope on the Android Marketplace: ★★★★☆. A reviewer haughtily provides a four out of five star review to an app, holding the last star hostage until the developer provides a final feature the user demands. The may seem petty, but I think it’s a good indication that the product review tools are insufficient.

It’s as if the reviewer believes developers are hunting methodically through reviews, looking to garner additional stars every way they can. The developer will kiss babies, hand out lollipops, and provide tablet support to wrest that last star from the most demanding users. Obviously, that’s not how developers work – but you could see how a reviewer might assume that they do.

This is also the reviewer community inventing a signal for something that can’t be communicated in the provided format. When you’re reviewing, you get a text box and a rating. That’s it. There’s little room for subtlety, and little room for concrete, structured feedback. So if I think something is going to distinguish my review from the torrent of careless one- and five-star reviews, I’ll do it. For want of a “Feature Request” flag, or some other formal signal, I’ll make it a four-star review.

This happens on free apps, and not so much on paid apps. I suppose that could be because having paid for an application, you’ve tacitly agreed that whatever that app provided was sufficient. It’s a bit silly to go back and churlishly withhold a star when you’ve already given them your money. I think it’s more likely that customers are using the developer’s support forums or ticketing system instead, where requests for features can be handled in an orderly way and the customer has some expectation of a response.

This lack of expression in the review experience hurts the other way, as well. I imagine a developer reading the reviews. It would be tedious to extract all the latent feature requests and bug reports hiding in those reviews. You’ll even see app descriptions pleading with users to not submit bug reports through the company’s own ticketing system instead of the app store review.

What’s funny is that Google’s robots understand reviews pretty well. They even honor a special format for product reviews which are displayed in the search results. You can include a rating, the number of reviews, and a host of information about where the software works, what operating systems it supports, and (most important) where you can buy it:

I think what’s missing from that schema is a way of expressing bug reports or feature requests. It wouldn’t have to be rigorous, just a couple extra text fields: what was broken, and what you wish would work. Just that extra bit of structure would allow developers to slurp up this casual feedback more easily. A reviewer would no longer (naïvely) rely on a rating for attention, freeing them to give a more honest number of stars.

I can even imagine bug tracking software that could automatically create tickets based on these reviews. Developers could respond to the reviewers, pointing back to the tickets that the user’s created, publicly demonstrating their responsiveness for new users. Going even further, the app store could display statistics on the number of bug fixes and feature requests that have been resolved from product reviews.

That’s an app store everyone could use.