Benchmarks, LifeType vs Wordress

[RedMagic]

I was looking for some decent benchmarks for a couple of blogging platforms online and couldn't really find any. At least any current ones. Since I'm basically considering opening a blog on LifeType vs Wordpress I wanted these two tested. Personally I'd prefer LifeType for the clean code, architecture and Smarty, but I'd go with Wordpress if it proved speedier.

Now since I didn't find benchmarks, I did my own with the little ApacheBench program. I'm putting the results here in hope I'd get some comments or possible suggestions of improving the setup.

The setup

Tested versions were LifeType 1.1.6 and Wordpress 2.1.1. The server is a FreeBSD 6.0, Pentium III 866, 128M RAM, 80GB PATA WD Caviar SE disk machine, with Apache 2.2.4, PHP 5.2.1 and MySQL 5.0.33. No other load was put on the server while testing aside from the benchmarks. The test was made from a separate laptop, Pentium M 1.6, 1.2GB RAM, connected over LAN to the server.

I did the default installations for both platforms and left them this way with a single, default post each. I'll probably redo the benchmark with some data added to each.

The test itself: ab.exe -n500 -k http://<address>/

Results

Code: Select all

                                                    [b]LifeType[/b]                [b]Wordpress[/b]
Requests per second:               3.80 [#/sec] (mean)       2.75 [#/sec] (mean)
Time per request:                    263.439 [ms] (mean)     364.023 [ms] (mean)

Percentage of the requests
served within a certain time (ms)
  50%                                                  260                     360
  66%                                                  260                     360
  75%                                                  270                     360
  80%                                                  270                     370
  90%                                                  270                     380
  95%                                                  270                     380
  98%                                                  280                     390
  99%                                                  290                     390
 100%                                                 370                     1562 (longest request)

To be honest, I am a bit surprised with the outcome. I said before, I like LifeType but Wordpress somehow always appeared to me as the lighter platform.

Comments anyone?

[jondaley]

I actually am surprised too, since we have gotten so much flak for running too slow.

phunkphorce has been making lots of improvements even in the last week on performance and memory usage, etc. so I would expect to see 1.2 even better than the benchmarking that you ran here.

LT does have a larger load than some other blogs when dealing with various plugins, and calculating the recent posts and comments on each page, like some templates do. Lots of other blog software have basically static pages once the post is created, and so it is easier to serve up the static page. If you add in more posts and comments, and install a couple plugins, you will obviously see LT's performance go down due to the extra processing. I don't know what WP's default settings are, and what happens when you install more plugins, I would expect the same sort of behavior.

Real programmers don't work from 9 to 5. If any real programmers are around at 9am it's because they were up all night. Anon.

Nice.

[phunkphorce]

The results here look very good but they're also a bit unrealistic. A client hammering a LT blog with only one post is a fantastic scenario for us: the first request will force LT to load the data and compile the template but all other requests after that will load the cached version of the template. I suppose that if you configured your WP with a plugin such as WP-Cache 2.0 (http://mnm.uib.es/gallir/wp-cache-2/), then results would be favourable again to WP.

On the other hand if you managed to recreate a scenario where cached data (both cached templates and cached objects from the database) is reset every now and then in some sort of random fashion as it would happen in a big busy blogging site (as if users kept posting articles and comments), then you'd get figures more adjusted to the real world. I don't know how well we'd fare in that kind of tests, I haven't stress tested our code for a while now...

[RedMagic]

I wanted to specifically test dynamically created blogs, hence I didn't put any interest in WP-Cache or any sort of that functionality. I did a test in the meantime with more data in LT, using one of the blogs created and filled with the generateData.php tool from the repo. Results were identical.

Does LT by default use template caching or just template compiling?

Considering a typical user probably wouldn't bother to add any special caching or speed tweaks, LT still wins by having a more efficient default install.

[phunkphorce]

We do both template caching and object caching, so that we don't have to load some data over and over from the database (unless it changes) And all this is enabled out of the box by default.

[tszming]

hello,

i think it would be nice if LifeType provide some figures abt its performance.

anyway, my suggestion to the above benchmarks:

1. try to use APC or eaccelerator, i believe LT will gains some benefits
2. maybe can also compare their memory usage per request?
3. as oscar said, use WP-Cache for a fair comparision
4. try to test without caching (i.e. spiders will not fetch the same page for 500 times)

after all, the main slowdown for LT codes are resserver & plugin as authimage, you might test them as well.

also, you might test their performance against spam, such as post to the comment form with wrong authcode, spam trackback etc.

this usually will be the most significant slowdown in real world suitations where your hosting company might complain about

[phunkphorce]

Luckily the problem with the resource server are going away in LT 1.2, now that we've got rid of it for good

[markwu]

also, you might test their performance against spam, such as post to the comment form with wrong authcode, spam trackback etc.

That's why we integrate bad-behavior into the core. ^_^

It block the spam before it hit lifetype.

Mark

[tszming]

also, you might test their performance against spam, such as post to the comment form with wrong authcode, spam trackback etc.

That's why we integrate bad-behavior into the core. ^_^

It block the spam before it hit lifetype.

Mark

1. any performance figures? such as requests per second drop from X to Y after using bad-behavior?

2. i havn't tried bad-behavior, seems it only deal with User-Agent string, so how reliable it is?

[markwu]

1. any performance figures? such as requests per second drop from X to Y after using bad-behavior?

I tested it my local machine, it will increse 20ms to each request. So, I think it acceptable.

2. i havn't tried bad-behavior, seems it only deal with User-Agent string, so how reliable it is?

Not only User-Agent string, it use some http request header analysis technique to decide it is a spam or not.

In my test, it can block almost 95% of spam .. so, it worth to use it.

Mark

[tszming]

Not only User-Agent string, it use some http request header analysis technique to decide it is a spam or not.

if spam developers look at the source of bad-behavior, is it difficult to write a spam to pass thru?

[phunkphorce]

Not only User-Agent string, it use some http request header analysis technique to decide it is a spam or not.

if spam developers look at the source of bad-behavior, is it difficult to write a spam to pass thru?

Maybe. But so far no anti-spam system has proven to be 100% reliable and everlasting so we'll just move onto the next thing...

[jondaley]

Same thing as spamassassin and every other spam catcher. The spammers work hard to figure out a way around it, and eventually they will succeed. But, some things like greylisting (for email spam) and badbehavior timeouts make spammers spend more of their computing resources to go slow enough to make it past - even if they look at the source.

[tszming]

okay, maybe i should try to find out more abt badbehavior...

btw, if badbehavior target is to prevent spam (i.e. comment, trackback), then you might consider moving comment posting function out of the index.php, maybe create a comment.php, so badbehavior can be removed from index.php, which represent most of the normal requests (read only)

[jondaley]

I originally thought that was a good idea, but badbbehavior uses index.php read-requests to determine some things about spammer behavior.