[UPDATE] Since writing this, I had another huge spike in traffic (20,000 visitors in a day), which again caused a crash. As a result, I installed WP-Super Cache, which is much more robust that WP-Cache. I also adjusted the theme to use a static stylesheet (style.css) rather than a dynamic one (style.php) -. This totally eliminates all load on the PHP and MySQL backend, vastly improving the situation. The use of style.php vs. style.css varies by theme, it’s an option in the tiga theme used here, but for many themes is unnecessary.
…..
Some time this morning my blog started acting a bit wonky, giving me errors about not being able to connect to the database, and then finally becoming inaccessible. I could not even get into my server control panel, and had to request a power cycle.
After it came back up, I found out why it was having problems. I’d been linked from reddit.com, a user-generated news site, and I was getting a new connection every few seconds, already several thousand connection had been made, and the sheer number of requests was crashing the server. All of the request were for this page:
https://cowboyprogramming.com/2007/01/04/mature-optimization-2/
But why was this making the server crash? Well, the most immediate problem was that wp-cache was not enabled. This meant that every page view had to be re-built from the MySQL database using PHP. If the requests are coming in faster than the page can be built, then they’ll get backed up. Eventually thing crunch to a stop, seeing as the basic workings of WordPress also require database connections. And since cPanel also uses a database, it too fails to work. I could probably improve this somewhat by increasing the number of connections, but really they should be cached.
So I re-enabled the cache, and things seemed to go swimmingly. I then took a look at what was being downloaded as part of the page. I was rather surprised to discover that the SyntaxHighlighter plugin was serving up a bunch of JavaScript, even though it was not used on this particular page. That seems like a bit of a bug. I was able to trim this down quite a bit by disabling every language except for C++. later I hacked it to only include the JavaScript if needed.
I then noticed that my header image and my background image were about 100K each, which is rather silly. I took them both into photoshop, and tweaked them down to about 15K. The background image was a GIF, but this was still compressable by a significant amount by just setting “lossy” to a small number like 4 or 5, and there was no perceptible degradation.
Finally, my Reddit effect was seemingly having a knock-on effect in South Korea, where I had linked to a 2.5MB pdf file that was hosted at Seoul National University. This was currently inaccessible. I’m not sure, but suspect a few people were trying to download it (I was at 5,000 visitors by then). I wanted to re-host it locally, but I could not even read it myself. So I just deleted the link, then 20 minutes later things had died down, so I was able to grab the file. I re-hosted it on my server, and put up a new link.
So my server has been up about six hours now, and since then it’s been continued to be hit fairly constantly from Reddit. In that time there has been 70,000 accesses (3.17 per second) , accounting for 724MB. It’s about 20 times the normal daily traffic for this site.
Here’s a graphical look at what happened:
As you can see, bandwidth is not really the problem. It peaks at 1Mb/sec, and my server can serve at 100Mb/sec. You could probably handle this load on a 768K DSL line.
The Server degrades severely at around 8:00AM, then around 8:20 is when I reboot it. I re-enable the cache a few minutes after this, but this does not have any effect on the bandwidth, seeing as the pages being served are identical, just less load on the PHP/MySQL back end.
Then at around 10:00AM I start optimizing, shrinking the background and header images, and trimming the syntax highlighter, which more than halves the bandwidth.
I have been a frequent visitor of this blog for some time now, so I thought it would be a good idea to leave you with my thanks.
Regards,
Jim Mirkalami
Comment by Jim Mirkalami — February 6, 2008 @ 4:02 pm
If you’re on a hosting plan with tight bandwidth restrictions, submitting content to social networks could be more harmful to you than good. This article has saved me a lot of hassle in handling with vast amounts of requests easily and a lot less painfully; thanks.
Comment by David — March 7, 2008 @ 3:50 pm
i use super cache to nice plugin
Comment by eddy — March 21, 2008 @ 3:11 pm
Very good plugin.
Comment by burun — March 21, 2008 @ 10:19 pm
I think using the stylesheet file as style.css causes a fast web page load. Great idea. Thanks a lot.
Comment by estetik — March 28, 2008 @ 12:12 am
i’ve never heard the super caching plugin for wp.. better to try :) thanks for sharing..
Comment by pdf mania — March 30, 2008 @ 1:09 am
Thanks for the great information, it was very helpful.
Comment by Steve Smith — April 1, 2008 @ 10:09 am
I recommend this plugin. Very useful for me.
Comment by kulak estetiði — April 2, 2008 @ 12:14 am
thanks for this proofing :)
Comment by rzrarti — April 2, 2008 @ 10:42 am
thanks for plugin its useful
Comment by john — April 2, 2008 @ 9:25 pm
Very useful plugin, thx.
Comment by diablo 2 cd key — April 5, 2008 @ 8:06 pm
thanks for such a useful plugin….
Comment by varun — April 6, 2008 @ 12:34 am
what i found help s reduce the strain on wordpress or any server in general is caching (as you have mentioned) and enabeling gzip.
though I know the plugin has sissues with gziped content.
What you could also try is to cache sql queries (ask your host about this).
Comment by Don — April 6, 2008 @ 5:16 am
WordPress does seem to put a massive strain on my sites when one of the articles goes viral and I get a massive surge of traffic. I think I’m going to follow some of your tips to see if they help
Comment by Gene — April 11, 2008 @ 1:33 pm
Usefull post. Thank you.
Does WP Super cache works also with new wp 2.5?
Comment by Vids — April 12, 2008 @ 3:31 pm
Yes, it works with 2.5, you just need to make sure you have the latest version (0.6.2)
Comment by Mick West — April 12, 2008 @ 4:05 pm
@Mick: Is WP-Super Cache still running well for you?
Comment by Jen — April 13, 2008 @ 9:33 am
Seems fine. I’ve not really had any huge spikes though. I’m pretty sure it could handle a slashdotting though.
Comment by Mick West — April 13, 2008 @ 10:12 am
Ok, thanks Mick. Going to install it right now.
Comment by Jen — April 13, 2008 @ 10:34 am
wow huge traffic
Comment by Wordpress Themes Park — May 4, 2008 @ 3:33 pm
what i found help s reduce the strain on wordpress or any server in general is caching (as you have mentioned) and enabeling gzip.
though I know the plugin has sissues with gziped content.
Thank you
Comment by dizi izle — May 5, 2008 @ 12:10 am
Wow…real huge traffic. The highest I ever got was around 8,000 till now in my new site: http://www.enfotainer.com/
Well, is this Super Cache recommended even if we don’t have huge traffic in our website?
Thanks in Advance!
Comment by Manish — May 30, 2008 @ 7:11 am
Generally, my site has only 200 visits per day, but if this SuperCache is going to increase their experience in surfing my page, then I’ll surely install it too…LOL
Comment by Manish — May 30, 2008 @ 7:13 am
thanks for sharing the info there..it really helps me to understand reddit traffic more
Comment by red heart yarn — June 12, 2008 @ 6:03 am
thanks ;)
Comment by abcteach.com — June 12, 2008 @ 8:03 am
Do you host this on your own servers or with a hosting company?
Comment by Denise — August 4, 2008 @ 1:25 am
It was a virtual private server at GoDaddy
Comment by Mick West — August 4, 2008 @ 6:37 am
Great post, This is why I have my site images as small as possible while not being graphic intensive. I am going to look into WP-Super Cache. Any significant problems you’ve faced since installing this plugin?
Anyone interested in video tutorials please visit: PCLicious – Video Tutorials For Personal Computing
Comment by JGG — August 12, 2008 @ 11:14 am
WP-Super Cache seems to have worked just fine since I installed it.
Comment by Mick West — August 12, 2008 @ 11:42 am
wow huge traffic , i want too :P
Comment by webdesign — February 1, 2009 @ 6:17 pm
my site images as small as possible while not being graphic intensive. I am going to look into WP-Super Cache. Any significant problems you�ve faced since installing this plugin?
Comment by perfect — June 19, 2009 @ 1:15 pm
wow huge traffic
Comment by götten sikis — September 4, 2009 @ 3:02 pm
That traffic is amazing! Untill I start getting anywhere near that will I need super cache so you think??
Comment by james @ train horns for sale — September 11, 2009 @ 10:48 pm
I use WP Cache plugin, it works great.
Comment by Adam — January 11, 2010 @ 3:07 pm
I use only google analytics with my wordpress.
Comment by SEO Content Writing Service — February 1, 2011 @ 11:39 pm
Google analytics is even better for this purpose.
Comment by lost files recovery — February 5, 2011 @ 3:01 am