Recently the Podcasts app had a sub-2 star rating on the App Store. That seemed reasonable to many (I noted just 3 weeks ago that it had a 1.8-star rating), but some suggested that this was just because angry people took to the App Store to complain and that people actually liked the app quite a bit. Now of course some other Apple apps had great scores, so I’m not sure how that logic extended to those apps buuuuuuuuut anyway…

Recently the Podcasts app started showing users the random “please rate this app” pop up while using it, and tons more reviews flowed in, and guess what, the score skyrocketed!” Maybe the Podcast app skeptics were wrong and people did actually love the app.

But the devil is in the details because when you look at the reviews, it immediately becomes clear that these reviews aren’t telling us what you would expect.

“Review Us in the Podcasts App”

If you’ve ever listened to a podcast, you’ve surely heard a host ask you to review the show on iTunes or the Apple Podcasts app because it helps people discover the show.

So when listeners saw a pop up in their podcast app asking them to leave a review, well you can probably see where this is going.

They’re Not Reviewing the App

There are too many user reviews to go through manually, but I went through the last 100 reviews as of Friday, November 19, 2021, and here’s what I found. It’s maybe not a perfect sample size, but I was hoping it would give us a fuzzy view of the data.

Of the last 100 reviews, 59 were reviewing the Podcast app, and 41 were reviewing the podcast they were presumably listening to when they saw the alert.

But when we look at the scores given for these reviews, people reviewing the app averaged a 2-star review, while every single person who was reviewing the show gave it 5-stars.

What’s Going On? Is Apple Evil?

No, come on. It’s pretty clear to me that Apple wanted more reviews so they added the pop-up, but users who had been asked hundreds of times to “give us 5-stars on the Podcasts app,” and when they saw a pop up asking them to give stars, they reviewed the show. I assume there is at least one product person on the Apple Podcasts app muttering to themselves, “of course that happened 🤦🏻‍♂️”

Do Other Podcast Apps Have This Problem?

Again, I looked at the last 100 reviews for a bunch of popular third-party podcast apps on iOS and it looks like Apple's app is far from immune from this problem.

At first, I looked at the "enthusiast" group of apps: Overcast, Pocket Casts, and Castro. In the 300 total reviews I looked at for those apps, all 300 were explicitly reviewing the app, not a show. If I stopped there, I might think this is specifically an Apple problem, but then I dug deeper…

The Podcast App seems to be huge, with 210K reviews, and 31% of the reviews I saw there were for shows, not the app itself. Spotify is the big dog in town and only 4% of its 100 most recent reviews were for content rather than the app.

Takeaway

At this point, I've looked at 600 user reviews, and I think I'm done reading poorly-worded user reviews. Based on the random selection I got from these apps, my takeaways are:

1. Apple's app is not alone in having people think they're reviewing shows, not the app. Mainstream users are confused by this review pop-up way more than I expected.
2. That said, Apple does have the biggest proportion of these errant reviews in my research samples.
3. Enthusiast apps that I use don't have this problem at all, likely because the audience using those apps is savvier to what these pop-ups are asking them.
4. Spotify doesn't seem to have this problem nearly to the extent of other mainstream apps, and I wonder what if anything they're doing differently in terms of timing their review prompts.

This is largely just how user reviews work everywhere. People leave reviews for all sorts of reasons that don't really do what user reviews are meant to do. I saw people giving 5-star reviews and ripped the app to shreds in their comments. I saw people give 1-star reviews and say it was the best app in the world. I saw people give a 1-star review because they had a bad day. I saw countless political diatribes that didn't seem to have to do with anything, but there was a text box with a publish button, so people were sure as shit going to use it to advance their opinion.

Always take user reviews with a grain of salt. Often they do add up to a total that's pretty accurate to how people feel overall, but there's always baggage with those reviews. Sometimes they're an anomaly and the aggregate still makes sense, but sometimes the entire thing can be thrown off. Sometimes it's coordinated review-bombing, and sometimes it's users unclear on what exactly they're meant to review.