phpBB3 benchmarks : GZip; XCache vs eAccelerator vs APC, Zen

[Highway of Life]

I decided to do some basic benchmark tests on phpBB3 to see if it does better with or without GZip enabled.
I also pitted XCache, eAccelerator, APC and Zend Optimizer (and various combinations) against each other to see what required the least amount of resources to deliver the fastest pages.
In these results, lower Memory usage is always better, but sometimes it can come as a price for higher load times.
Since the test was done on a default install of phpBB3 using prosilver, the pages are light, so differences of a tenth of a second are quite dramatic.
Even a tiny change as a tenth of a second can mean a lot of load time and resource usage on a larger board.
The larger the board, the bigger impact these tiny differences will make... so look closely.

Here is the result.
System test was performed on:

Spoiler:

CPU Type: PowerPC G4 (3.3)
Number Of CPUs: 1
CPU Speed: 1 GHz
L2 Cache (per CPU): 256 KB
L3 Cache (per CPU): 1 MB
Memory: 1.5 GB
Bus Speed: 167 MHz

Load Average: 1.38, 1.68, 1.74
Apache/2.0.59 (Unix) PHP/5.2.1, MySQL(i) 5.0.37

i.e. My personal computer (localhost)

Spoiler:

All tests started with phpBB3 initially cached

Code: Select all

No Cache + GZIP: off
Time : 0.9190649986s | 9 Queries | GZIP : Off | Memory Usage: 2.51 MB | Thu May 10, 2007 8:20 pm | /startrekguide.com/area51/index.php
Time : 1.7405049801s | 10 Queries | GZIP : Off | Memory Usage: 2.53 MB | Thu May 10, 2007 8:20 pm | /startrekguide.com/area51/viewforum.php
Time : 1.1966149807s | 11 Queries | GZIP : Off | Memory Usage: 2.65 MB | Thu May 10, 2007 8:21 pm | /startrekguide.com/area51/viewtopic.php
Time : 1.8275160789s | 9 Queries | GZIP : Off | Memory Usage: 2.54 MB | Thu May 10, 2007 8:21 pm | /startrekguide.com/area51/memberlist.php
Time : 0.8444240093s | 8 Queries | GZIP : Off | Memory Usage: 2.26 MB | Thu May 10, 2007 8:21 pm | /startrekguide.com/area51/search.php
No Cache + GZIP: on
Time : 0.8469359875s | 9 Queries | GZIP : On | Memory Usage: 2.55 MB | Thu May 10, 2007 8:26 pm | /startrekguide.com/area51/index.php
Time : 1.7494890690s | 9 Queries | GZIP : On | Memory Usage: 2.57 MB | Thu May 10, 2007 8:26 pm | /startrekguide.com/area51/viewforum.php
Time : 0.9030089378s | 11 Queries | GZIP : On | Memory Usage: 2.69 MB | Thu May 10, 2007 8:26 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.8689858913s | 9 Queries | GZIP : On | Memory Usage: 2.58 MB | Thu May 10, 2007 8:26 pm | /startrekguide.com/area51/memberlist.php
Time : 1.6883308887s | 8 Queries | GZIP : On | Memory Usage: 2.3 MB | Thu May 10, 2007 8:26 pm | /startrekguide.com/area51/search.php
eAccelerator + Zend Optimizer + GZIP: On
Time : 0.6561939716s | 9 Queries | GZIP : On | Memory Usage: 553.71 KB | Thu May 10, 2007 8:02 pm | /startrekguide.com/area51/index.php
Time : 0.4231798649s | 9 Queries | GZIP : On | Memory Usage: 575.4 KB | Thu May 10, 2007 8:02 pm | /startrekguide.com/area51/viewforum.php
Time : 0.7690079212s | 11 Queries | GZIP : On | Memory Usage: 626.54 KB | Thu May 10, 2007 8:02 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.6278460026s | 9 Queries | GZIP : On | Memory Usage: 592.16 KB | Thu May 10, 2007 8:03 pm | /startrekguide.com/area51/memberlist.php
Time : 1.2196669579s | 8 Queries | GZIP : On | Memory Usage: 548.26 KB | Thu May 10, 2007 8:03 pm | /startrekguide.com/area51/search.php
APC + GZIP: On
Time : 1.5606360435s | 9 Queries | GZIP : On | Memory Usage: 2.55 MB | Thu May 10, 2007 8:03 pm | /startrekguide.com/area51/index.php
Time : 0.2111289501s | 9 Queries | GZIP : On | Memory Usage: 559.98 KB | Thu May 10, 2007 8:03 pm | /startrekguide.com/area51/viewforum.php
Time : 0.4212610722s | 11 Queries | GZIP : On | Memory Usage: 688.68 KB | Thu May 10, 2007 8:03 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.2865588665s | 9 Queries | GZIP : On | Memory Usage: 575.76 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/memberlist.php
Time : 0.7034621239s | 8 Queries | GZIP : On | Memory Usage: 534.09 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/search.php
APC + GZIP: On (again)
Time : 0.4294347763s | 9 Queries | GZIP : On | Memory Usage: 540.84 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/index.php
Time : 0.1895101070s | 9 Queries | GZIP : On | Memory Usage: 561.13 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/viewforum.php
Time : 0.6315841675s | 11 Queries | GZIP : On | Memory Usage: 608.73 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.2746441364s | 9 Queries | GZIP : On | Memory Usage: 577.87 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/memberlist.php
Time : 0.1968629360s | 8 Queries | GZIP : On | Memory Usage: 536.45 KB | Thu May 10, 2007 8:04 pm | /startrekguide.com/area51/search.php
XCache + GZIP: On
Time : 1.2385489941s | 9 Queries | GZIP : On | Memory Usage: 2.69 MB | Thu May 10, 2007 8:05 pm | /startrekguide.com/area51/index.php
Time : 0.2582309246s | 9 Queries | GZIP : On | Memory Usage: 514.92 KB | Thu May 10, 2007 8:05 pm | /startrekguide.com/area51/viewforum.php
Time : 0.2992441654s | 11 Queries | GZIP : On | Memory Usage: 652.86 KB | Thu May 10, 2007 8:05 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.2340888977s | 9 Queries | GZIP : On | Memory Usage: 532.59 KB | Thu May 10, 2007 8:05 pm | /startrekguide.com/area51/memberlist.php
Time : 0.1591839790s | 8 Queries | GZIP : On | Memory Usage: 492.2 KB | Thu May 10, 2007 8:05 pm | /startrekguide.com/area51/search.php
eAccelerator + GZIP: On
Time : 0.3264420033s | 9 Queries | GZIP : On | Memory Usage: 515.04 KB | Thu May 10, 2007 8:06 pm | /startrekguide.com/area51/index.php
Time : 1.1080911160s | 9 Queries | GZIP : On | Memory Usage: 535.77 KB | Thu May 10, 2007 8:06 pm | /startrekguide.com/area51/viewforum.php
Time : 0.3271198273s | 11 Queries | GZIP : On | Memory Usage: 582.49 KB | Thu May 10, 2007 8:06 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.5398509502s | 9 Queries | GZIP : On | Memory Usage: 552.59 KB | Thu May 10, 2007 8:06 pm | /startrekguide.com/area51/memberlist.php
Time : 0.1658101082s | 8 Queries | GZIP : On | Memory Usage: 512.71 KB | Thu May 10, 2007 8:06 pm | /startrekguide.com/area51/search.php
XCache + Zend Optimizer + GZIP: On
Time : 1.1705460548s | 9 Queries | GZIP : On | Memory Usage: 2.69 MB | Thu May 10, 2007 8:07 pm | /startrekguide.com/area51/index.php
Time : 0.2914500237s | 9 Queries | GZIP : On | Memory Usage: 514.92 KB | Thu May 10, 2007 8:07 pm | /startrekguide.com/area51/viewforum.php
Time : 0.2551391125s | 11 Queries | GZIP : On | Memory Usage: 652.86 KB | Thu May 10, 2007 8:07 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.2007980347s | 9 Queries | GZIP : On | Memory Usage: 532.59 KB | Thu May 10, 2007 8:07 pm | /startrekguide.com/area51/memberlist.php
Time : 0.1539480686s | 8 Queries | GZIP : On | Memory Usage: 492.2 KB | Thu May 10, 2007 8:07 pm | /startrekguide.com/area51/search.php
APC + GZIP: off
Time : 1.2511529922s | 9 Queries | GZIP : Off | Memory Usage: 2.51 MB | Thu May 10, 2007 8:10 pm | /startrekguide.com/area51/index.php
Time : 0.2296040058s | 9 Queries | GZIP : Off | Memory Usage: 520.02 KB | Thu May 10, 2007 8:10 pm | /startrekguide.com/area51/viewforum.php
Time : 0.3193957806s | 11 Queries | GZIP : Off | Memory Usage: 648.58 KB | Thu May 10, 2007 8:10 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.2240550518s | 10 Queries | GZIP : Off | Memory Usage: 535.84 KB | Thu May 10, 2007 8:10 pm | /startrekguide.com/area51/memberlist.php
Time : 0.6598730087s | 8 Queries | GZIP : Off | Memory Usage: 494.33 KB | Thu May 10, 2007 8:10 pm | /startrekguide.com/area51/search.php
eAccelerator + Zend Optimizer + GZIP: off
Time : 0.6129839420s | 9 Queries | GZIP : Off | Memory Usage: 512.4 KB | Thu May 10, 2007 8:11 pm | /startrekguide.com/area51/index.php
Time : 1.3299858570s | 9 Queries | GZIP : Off | Memory Usage: 535.3 KB | Thu May 10, 2007 8:11 pm | /startrekguide.com/area51/viewforum.php
Time : 0.3783009052s | 11 Queries | GZIP : Off | Memory Usage: 586.45 KB | Thu May 10, 2007 8:11 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.4857170582s | 9 Queries | GZIP : Off | Memory Usage: 552.09 KB | Thu May 10, 2007 8:11 pm | /startrekguide.com/area51/memberlist.php
Time : 1.1067769527s | 8 Queries | GZIP : Off | Memory Usage: 508.16 KB | Thu May 10, 2007 8:11 pm | /startrekguide.com/area51/search.php
XCache + Zend Optimizer + GZIP: off
Time : 1.1289560795s | 9 Queries | GZIP : Off | Memory Usage: 2.65 MB | Thu May 10, 2007 8:13 pm | /startrekguide.com/area51/index.php
Time : 0.3423521519s | 9 Queries | GZIP : Off | Memory Usage: 474.66 KB | Thu May 10, 2007 8:13 pm | /startrekguide.com/area51/viewforum.php
Time : 0.2700951099s | 11 Queries | GZIP : Off | Memory Usage: 612.77 KB | Thu May 10, 2007 8:13 pm | /startrekguide.com/area51/viewtopic.php
Time : 0.2644279003s | 9 Queries | GZIP : Off | Memory Usage: 492.51 KB | Thu May 10, 2007 8:13 pm | /startrekguide.com/area51/memberlist.php
Time : 0.3408930302s | 8 Queries | GZIP : Off | Memory Usage: 452.13 KB | Thu May 10, 2007 8:13 pm | /startrekguide.com/area51/search.php

Let me know if you would like me to perform a specific test with any of these parameters.

[mastercrap]

You don't have enough "basic" tests (XCache w/ no GZip, eA w/ no GZip, etc.)

[Highway of Life]

Actually... I do... (I think)
There are 11 tests.

Here they are in graphs, so they might be easier to read for some.
Click on the thumbnails to see the full-size graphs.

The first one is the parse times of each test...


The second is the memory consumption of each test...

[LEW21]

So:

1. everywhere we should use GZip
2. for index.php we should use eAccelerator and for other pages XCache + Zend Optimizer.

But the main problem is: I don't think that it's possible

[Highway of Life]

The reason they shoot up on the index page is because the cache was initialized first on that page... so it’s not very fair.
I really should exclude that one from the list... it’s misleading...

[Highway of Life]

index page was removed from results since it was misleading... take a look again -- it’s also easier to read now.

I’m going to post results using the various alternative phpBB3 ACM methods as well in combination with the server to see what we get.
I’ll run six tests on each method, and exclude the first since it’s always initialized on the first test.

[hurry]

Thanks HoL. That was very useful. I already have eAcc and memcache installed. Can I install Zend Optimizer also now? Will it make a difference? I have cpanel linux VPS apache server.

[CardsFanInChiTown]

Why does GZip make things faster? I thought it only saved bandwidth but used processor?

[Highway of Life]

eA works best with Zend Optimizer IMO, and I think every bit helps.
You can run the two together, so you should, if possible. But it’s not critical if you already have a cache method.
You can’t run APC with Zend, however it does not seem to really require it. -- pretty much just flip a coin.

Gzip: In some cases it can be faster if it does not take so long to deliver the content -- Which is what GZip does: It helps deliver the content faster by compression.
Of course, there are many variables in every setup, and while Gzip was a bit faster in all my benchmarks, it’s possible that it may be slower on another setup.
It does use more processor power, but as long as your server can handle it, delivery should be quicker.
BTW, STG index page uses 71,808 bytes of markup, but the compressed size is 11,013 bytes using GZip compression.

As a basis for comparison, it takes 0.084 seconds to load up the STG index page in prosilver (including all the extra stuff with the chatbox) with Gzip on.
phpBB.com forum takes 0.085 seconds to load up the index page in that version of prosilver (without any extra "stuff") with Gzip off.
and phpBB.com uses about half the size of markup: 47,836 bytes
And STG still delivers faster.

Could it be because of Gzip? perhaps.

[greggh]

eA works best with Zend Optimizer IMO, and I think every bit helps.

I don't understand how you came to that conclusion. By your own tests, eA + Zend has some pretty bad load times with viewforum and search. xCache + gzip according to your tests looks to work the best overall.