15/03/2009
How much can misconfigured wordpress plugins stall your server’s performance
A couple of days ago I’ve met a guy who has a high traffic blog about tech stuff. The guy was telling me that he has hosting problems and that his blog is getting slower and slower by the day. I’ve offered to help him by providing hosting for him in one of the servers that I administer. After making the transition from his old hosting to my server, which was not an easy thing to do due to latin1 to utf8 conversions that had to be made – it deserves a post of it’s own, I started to notice increased load on my server. Sure his blog had heavy traffic…but could it be that bad ?
The first thing I noticed was that the disk usage was raising FAST.
Since there was no heavy network traffic the data must have been created internally by the server. I started checking on the /var/log/ directory.
# du -sh /var/log/
4.6G /var/log/
That needed further investigation for sure!
# du --max-depth=1 -h /var/log/
[..snip..]
8.0K /var/log/postgresql
4.6G /var/log/mysql
12K /var/log/fsck
[..snip..]
4.6G /var/log/
OK…the culprit was mysql for a reason. The directory mostly contained mysql binary logs.
# ls -Fla /var/log/mysql/ | grep mysql-bin | wc -l
58
# ls -Fla /var/log/mysql/ | grep mysql-bin | grep 03-14| wc -l
46
Out of the 58 binlogs, 46 were produced on the day of the transition. While the previous dates had binlogs of around 30-40Mb, the day of the transition had 46 binlogs of around 100Mb each! That’s 4.6Gb vs 40Mb for just one day!
I’ve opened up a few of those using mysql: # mysql -u root -p
SHOW BINLOG EVENTS IN 'mysql-bin.000337';
The binlogs were full of some wordpress queries by the newly installed blog.
I then asked for a wordpress admin account on the blog to check on the plugins. I found a lot of plugins installed. Two of those were WP-Cache and DB-Cache.
I started checking on WP-Cache. When I went to Settings->WP-Cache, it complained that it was misconfigured. It couldn’t create the cache files due to wrong permissions on some directories. So I changed the proper permissions and I could immediately see a difference on server load and on blog loading time. The blog took over 2.5 seconds to load when WP-Cache was misconfigured but it could now load in under 2 seconds. Still the mysql binlogs continued to increase in size very very fast.
DB-Cache on the other hand did not complain about anything when I went to Settings->DB-Cache. Only when I tried changing some settings it notified me that there was again a permissions’ problem. Fixing that started showing again some results. Blog’s load time fell down to 1.5-1.7 seconds. There were also less queries on the binlogs but still there were many. Many of those looked like:
UPDATE wp_options SET option_value = 'O:8:\"stdClass\":3:{s:7:\"updates\";a:1:{
i:0;O:8:\"stdClass\":5:{s:8:\"response\";s:7:\"upgrade\";s:3:\"url\";s:30:\"http://wordpress.org/download/\";s:7:\"package\";s:40:\"
http://wordpress.org/wordpress-2.7.1.zip\";s:7:\"current\";s:5:\"2.7.1\";s:6:\"locale\";s:5:\"en_US\";}}s:12:\"last_checked\";i:1237
062227;s:15:\"version_checked\";s:3:\"abc\";}' WHERE option_name = 'update_core'
I have no idea why wordpress with db-cache keeps doing those queries when DB-cache is enabled.
Anyway, I disabled DB-Cache so the blog could only rely on my own cache settings for mysqld. The results were magnificent. Blog load time reduced to 1.0-1.2 seconds!! Mysql got a boost on cached queries and server load avg is also very low again.
(ignore the spike at the end and notice the difference that happens sometime around saturday evening when the cached hits, purple, surpass the select, green)
Everybody is happy, client has a fast blog and I’ve got a very responsive and fast server 🙂
Filed by kargig at 13:49 under Linux
Tags: caching, db-cache, Linux, mysql, wordpress, wp-cache
10 Comments | 19,409 views
Really interesting post.
Do you usually keep binlogs enabled in a non cluster environment?
The mysql queries graph is done with cacti?
Do you have any link to the plugin or howto?
Well…I usually do keep binary logs in the servers that I am administering. I make them expire after 5 or 10 days, it depends on the server and the clients on them. There are ways to recover corrupted mysql databases using these binary logs, so I keep them just in case…
The mysql queries graph is done automagically by munin 🙂 Just apt-get install munin on a debian server. It’s a wonderful tool for simple graphs like this. If you need more advanced stuff…then go to a solution like cacti.
[…] my pixels to one side today, WordPress plugins to be written for a client!! # # Photoshop Ninja How much can misconfigured wordpress plugins stall your server’s performance – void.gr 03/15/2009 A couple of days ago I’ve met a guy who has a high traffic blog about tech […]
[…] See more here: How much can misconfigured wordpress plugins stall your server’s … […]
Very interesting article. Thanks for writing it.
You investigated the issue by checking /var/log/. I assume this was because of experience and logical thought. I was wondering though if you happen to know any other ways to determine which applications might have caused the excessive writing to the filesystem? For instance, an utility that would monitor the access to the filesystem for a while and then print a list of processes that had caused the most disk write operations would be useful while investigating this issue. Are there any such things out there?
[…] from: How much can misconfigured wordpress plugins stall your server’s … Posted in News | Tags: archives, caching, chat, db-cache, internet, last-fm, links, linux, […]
@George: iotop could come in handy. As a last resort I would do something like: “echo 1 > /proc/sys/vm/block_dump” and monitor /var/log/debug for the names of the process causing the write operations.
Good to see others investigating which plugins cause issues. I’ve written quite a bit on this subject but I still get problems occasionally. Personally, I’ve enabled DB cache but I haven’t seen a performance hit, possibly because my MYSQL is on a shared hosting server, separate from my file and php server. Still, perhaps it might be worth to give disabling it a try and see if it makes any difference.
Btw, WP-Cache is ancient and not supported anymore. I recommend you move either to WP-Super Cache or my personal recommendation Hyper Cache.
@Giorgos: Thanks. Getting my hands on iotop right now. Will report back soon from my blog.
@Db0: I use WP-Super-Cache without issues, but will check HyperCache.
Wow! Thank you! I always wanted to write in my blog something like that. Can I take part of your post to my site? Of course, I will add backlink?