Mastodon

Inadvertently Load-Testing This Site

Posted by Matt Birchler
— 2 min read

Yesterday I wrote an article that attempted to capture the feelings I've seen from a lot of people about the mixed messages related to the environment with the new iPhone 12 line and accessories. I wrote it in like 5 minutes, thought it was pretty good, and posted it without a second thought.

While there are certainly better ways to track website performance, I'm not using any of them, so what I have are CPU usage logs from the past 24 hours on this server. This site runs on Ghost and is self-hosted from a "droplet" with DigitalOcean. Generally, my CPU is pretty calm, hovering well below 10% utilization most of the time.

However, that CPU usage scales up generally linearly with usage, so the more traffic I get, I would expect to get a proportionally similar rise in CPU usage.

Yesterday, I got to find out how true that was.

You can see three separate bumps, each more substantial than the last.

A little after 10am my time, The Loop linked to the post, and I saw a modest jump in usage. A few hours later, John Gruber linked to the post as well, and there was a bigger spike. Then things got out of control when it hit the front page of Hacker News. As you can see from the chart, that's where we hit the "uh oh" point, as CPU usage hit 100% and stayed close to that for the better part of an hour. From what I could tell, the site never crashed, but some people during that period may have seen page load times noticeably more than the 1 second I would normally expect for the post with no images.

At the peak, I was seeing over 700 simultaneous users on the site, which is unusual, to say the least. Long story short, I had about the same number of people visit the site yesterday as had been here in the first 26 days of October combined.

Takeaway

I ought to figure out a better way to cache the site since I got right up to the edge of things falling apart this time. That said, this happens so rarely that unless it's simple and free, I will likely just risk it and not do anything.

That may not be the most satisfying takeaway, but frankly I think it's the most practical.