I run into a little test of Plone 2.1 today. And the whole stuff, seems to be slower than Plone 2.0.5. I decided to do a little benchmark, and yes Plone 2.1 seems to be really slower..
Plone 2.0.5 (python 2.3.4)
Total transferred: 239130 bytes HTML transferred: 234620 bytes Requests per second: 5.52 [#/sec] (mean) Time per request: 181.261 [ms] (mean) Time per request: 181.261 [ms] (mean, across all concurrent requests) Transfer rate: 128.54 [Kbytes/sec] received
Plone 2.1 (python 2.3.5)
Total transferred: 240740 bytes HTML transferred: 237440 bytes Requests per second: 3.54 [#/sec] (mean) Time per request: 282.476 [ms] (mean) Time per request: 282.476 [ms] (mean, across all concurrent requests) Transfer rate: 83.19 [Kbytes/sec] received
Grr, here is two point to read:
- The 2.1 version is really slower look at the transfer rate..
- But this is astomish slow
The bench was done on:
- pure zope httpd (no apache or squid frontend)
- without debug enable
- with default HTTPCache and RAMCache enable (default install setting)
- a dual Xeon at 2.6Ghz with 1Go of RAM ! (I know python only use 1 CPU but !!!)
I first think my setup is buggy, but no.. nobody with a good CMS in stock ?
Compare that to a purpose built Quixote app:
target: a moderately complex internationalized document object in a CMS:
Target: Root page, only a few objects resolving and minor security check
These tests run on a single CPU development workstation running X / xfce at the same time and a swack of other stuff; 1GB ram, 2.4ghz processor.
Of course, with Plone/Zope I could inherit a swack of software and perhaps write very little myself; so there’s real time and productivity saved there… at the same time, I’d have to invest a lot of energy in learning the platform before proficiency could be achieved. Plusses and minuses.
Quixote – or any of the simpler frameworks – certainly have a place.
You might be interested in having a look at funkload to do automated benchmark of your favorite CMS.
http://public.dev.nuxeo.com/~ben/funkload/
Patches, bug reports, feature requests should go to :
http://svn.nuxeo.org/trac/pub/newticket
under the "funkload" component.
One Question:
Why do you run Zope in Debug-Mode for Benchmarking? This may have a significant impact on your Performance.
And, yes with ZEO your System will run remarkably faster if you have more than 1 CPU…
Kind Regards
Maik
Sorry for the debug statement, I read "with debug enabled" :-o
I Should get another cup of coffee quick ;-)
What program did you use and what arguments did you pass to said program? Just curious…
Ab2 -n 10 (I don’t test concurrent request, it will be worst..)
In fact, Plone doesn’t seem to suffer from concurrency:
Here’s the same, Quixote, medusa. It doesn’t suffer from a big load at it either. Lightweight has its dividends ;-)
Server Software: Medusa/1.11 Document Length: 6743 bytes
And then to extremes. The system is very responsive to a real user while this runs:
And if your blog "preview" worked I might be able to format that! :D
Cheers…
The preview works .. except that firefox or moz block it ..
running plone 2.0.5, zope 2.7.5 with 512mb ram was fine.
running plone 2.1, zope 2.8.0 on the same machine gives a noticeable lag and is pretty much IMHO "unuseable".
I don’t know enough about your Quixote app to say this, but you may be comparing apples to oranges, here. For your Quixote app (and the Plone app) that you tested, was it a ‘portalish’ application, which would need to do the necessary security/permission checks, did it need to aggregate different information based upon the personalization rules of the authenticated user’s roles, etc. as would be the case with a test of an out-of-the-box Plone site?
Also, did you have debug mode enabled for the Quixote app? I don’t think that would cause the variance in requests/second that you’re reporting here, but enabling debug mode for any performance tests (of most any CMS or framework) would pretty much be a useless comparison to what you would expect to see when debug is disabled.
I think you may want to compare Quixote to other frameworks such as Django, rather than to full-fledged CMS systems, for a good comparison of like tools.
On the other hand, if you were to compare Plone’s out-of-the-box performance with that of say Documentum, Interwoven, Vignette, Joomla/Mambo, Bricolage or other CMS tools (with none in debug mode), I think that may be more meaningful as Plone is more functional than a mere framework (where you build the functionality), and with that additional functionality comes a requirement to know what one is doing when configuring for high-volume application scenarios. It can surely be done, is documented, and the community seems very helpful whenever I’ve needed to ask for help in this area.
I think as a rule of thumb, just about any framework can beat out just about any CMS in terms of performance. You hit it on the head when stating that you have to weigh the time to build features versus the time to configure the CMS for optimal performance.
Cheers!