How to install Joomla! on your Ubuntu/Linux server with basic security

The instructions may apply for other versions too.

Prerequisitements

What you need to have in order to use this how to

• Basic UNIX file permissions knowledge
• Basic UNIX shell knowledge
• You have a Linux server (Ubuntu / Debian) for which you have root user access and you plan to use this server to host one or several Joomla! sites
• Apache and MySQL instaleld on your server

User setup

Set-up an UNIX user on a dedicated server for Joomla! hosting. The user can SSH in the box and write to his home folder, /tmp and /var/www site folder.

We create a user called “user” in this instructions. Replace it with the username you desire. We also use the example site name (www).yoursite.com.

sudo adduser user # Asks for the password and created /home/user

sudo mkdir /var/www/user
sudo chmod -R user:user /var/www/user # Only user has writing access to this folder

Setup MySQL user account

Install MySQL as per Debian/Ubuntu instructions.

Login as MySQL admin user (may vary depending how your MySQL is configured). Note that first you will be asked for sudo password, then for MySQL administrative user password.

sudo mysql -u admin -p

CREATE DATABASE user DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

GRANT ALL ON user.* TO 'user'@'localhost' identified by 'zxc123zxc'; 

Extract Joomla! installation files

Enter the folder which will contain web site PHP files.

sudo -i -u user # pose yourself as UNIX user who runs the site

cd /var/www/user

wget http://joomlacode.org/gf/download/frsrelease/12350/51111/Joomla_1.5.18-Stable-Full_Package.zip

unzip Joomla_1.5.18-Stable-Full_Package.zip

exit

Set file permission

Note that during Joomla’s browser based installation Apache’s www-data must have write access to folder in order to create configuration.php file. We will later remove this access right.

We will set Joomla! files under UNIX group group www-data so that Apache can read them. Certain files are set to be writable. This must be done as root user.

sudo chown -R user:www-data /var/www/user # Make user group to www-data

sudo chmod g+wrx /var/www/user # Read only access to www-data user. Write access for installation, will be later removed.

Now ls -l command in /var/www/user should give you something like this for fil masks:

drwxr-xr-x 11 user www-data    4096 2010-05-28 10:22 plugins
-rwxr--r--  1 user www-data     304 2010-05-28 10:21 robots.txt
drwxr-xr-x  6 user www-data    4096 2010-05-28 10:22 templates

Creating Apache configuration

sudo nano /etc/apache2/sites-enabled/yoursite.conf

   ServerName yoursite.com
   ServerAlias www.yoursite.com
   ServerAdmin info@yourcompany.com

   LogFormat       combined
   TransferLog     /var/log/apache2/yoursite.log

   # Make sure this virtual host if capable of executing PHP5
   Options +ExecCGI
   AddType application/x-httpd-php .php .php5

   # Point to www folder where Joomla! is extracted
   DocumentRoot /var/www/yoursite

   # Do not give illusion of safety
   # as PHP safe_mode really is a crap
   # and only causes problems
   php_admin_flag safe_mode off

   #
   # This entry will redirect traffic www.yoursite.com -> yoursite.com
   # Assume mod_rewrite is installed and enabled on Apache
   # 301 is HTTP Permanent Redirect code
   RewriteEngine On
   RewriteCond %{HTTP_HOST} ^www\.yoursite\.com [NC]
   RewriteRule (.*) http://yoursite.com$1 [L,R=301]

</VirtualHost>

Faking the DNS entry

sudo gedit /etc/hosts

# Force this hostname to go to your server public IP address from your local computer

123.123.123 yoursite.com www.yoursite.com

Start Joomla! browser based installation

http://yoursite.com

Secure the configuration

sudo chmod -R g-w /var/www/user # Remote write permission
sudo rm -rf /var/www/user/installation # Remove installation directory
# Add write permission to folders which contain writable files
sudo chmod -R g+x /var/www/user/logs
sudo chmod -R g+x /var/www/user/images
sudo chmod -R g+x /var/www/user/tmp
sudo chmod -R g+x /var/www/user/images

Setting up htaccess files

Joomla! comes with a sample htaccess file which has some security measurements by having RewriteRules to prevent malformed URL access.

To install this file do the following

sudo -i
cd /var/www/user
cp htaccess.txt .htaccess
chmod user:www-data .htaccess # Set file permission to be readable by Apache and writable by the UNIX user

Then we create a .htaccess file which we will place in all folders with Joomla! write access to prevent execution of PHP files in these folders. First we create htaccess.limited file which we use as a template.

sudo -i
cd /var/www/user
nano htaccess.limited # Open text editor

Use the following htaccess.limited content

# secure directory by disabling script execution
AddHandler cgi-script .php .pl .py .jsp .asp .htm .shtml .sh .cgi
Options -ExecCGI -Indexes

And put the master template htaccess.limited  to proper places

cp htaccess.limited media/.htaccess
chown -R user:www-data media/.htaccess 

cp htaccess.limited tmp/.htaccess
chown -R user:www-data tmp/.htaccess 

cp htaccess.limited logs/.htaccess
chown -R user:www-data logs/.htaccess

cp htaccess.limited images/.htaccess
chown -R user:www-data images/.htaccess

Start using the site

Setting outgoing email

Test outgoing email

Maintaining file permission

sudo usermod -a -G www-data user # Add user to www-data group so that it can set group permissions

sudo -i -u user # Login as your UNIX user
chgrp -R www-data templates libraries # Fix group ownership
chmod -R g+rx libraries templates # Set read access for the group

Read our blog  Subscribe mFabrik blog in a reader Follow us on Twitter Mikko Ohtamaa on LinkedIn

Integrating and theming WordPress with your CMS site using XDV

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
      xmlns:tal="http://xml.zope.org/namespaces/tal"
      xmlns:metal="http://xml.zope.org/namespaces/metal"
      xmlns:i18n="http://xml.zope.org/namespaces/i18n"
      lang="en"
      metal:use-macro="here/main_template/macros/master"
      i18n:domain="plone">

<body>

    <div metal:fill-slot="content">

        <div id="wordpress-content">
                <!-- Your WordPress "left column" will go there -->
        </div>

    </div>

</body>
</html>

<?xml version="1.0" encoding="UTF-8"?>
<rules xmlns="http://namespaces.plone.org/xdv"
       xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
       xmlns:css="http://namespaces.plone.org/xdv+css">

    <!-- Remove WordPress CSS by filtering out <style> tags-->
    <drop css:content="style" />

    <!-- Make sure that WordPress metadata is present in <head> section -->
    <append css:content="head link" css:theme="head" />

    <!-- note: replace does not seem to handle multiple meta tags very well -->
    <drop css:theme="meta" />
    <append css:content="head meta" css:theme="head" />

    <!-- Use blog title instead of Plone page title -->
    <replace css:content="title" css:theme="title" />

    <!-- Put WordPress sidebar to Plone's portlets section -->
    <append css:content="#r_sidebar" css:theme="#portal-column-one .visualPadding" />

    <!-- Place wordpress content into our theme content area -->
    <copy css:content="#contentleft" css:theme="#wordpress-content" />

    <!-- This mixes in WordPress specific CSS sheet which is applied for pages
         served from WordPress only and does not concern Plone CMS.
         This stylesheet will theme WordPress specific tags,
         like blog posts and comment fields.
         We keep this file in Plone, but this could be served from elsewhere. -->
    <append css:theme="head">
        <style type="text/css">
           @import url(http://mfabrik.com/++resource++plonetheme.mfabrik/wordpress.css);
        </style>
    </append>

    <!-- This stylesheet is used by special spam protection plug-in NoSpamNX -->
    <append css:theme="head">
        <link rel="stylesheet" href="http://blog.mfabrik.com/wp-content/plugins/nospamnx/nospamnx.css" type="text/css" />
    </append>

    <!-- Remove Google Analytics script used for CMS site -->
    <drop css:theme="#page-bottom script" />

    <!-- Rebuild our Google Analytics code, using a different tracker id this time
         which is a specific to our blog.
      -->
    <append css:theme="#page-bottom">

        <script type="text/javascript">
                var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
                document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
        </script>

        <script type="text/javascript">
                try {
                       var pageTracker = _gat._getTracker("UA-8819100-2");
                       pageTracker._trackPageview();
                } catch(err) {
                }
        </script>
    </append>

</rules>

/* Font and block style fixes */

#wordpress-content h1 {
        border: 0;
}

#wordpress-content .post-end {
        margin-bottom: 60px;
}

#wordpress-content pre {
        width: 600px;
        overflow: auto;
        background: white;
        border: 1px solid #888;
}

#wordpress-content ul {
        margin-left: 20px;
}

#wordpress-content .post-info-date,
#wordpress-content .post-info-categories,
#wordpress-content .post-info-tags {
        font-size: 80%;
        color: #888;
}

/* Make sure that posts and comments look sane in our theme */

#wordpress-content .post {
        margin-top: 15px;
}

#wordpress-content .commentlist li {
        margin: 20px;
        background: white;
        padding: 10px;
}

#wordpress-content .commentlist li img {
        float: left;
        margin-right: 20px;
        margin-bottom: 20px;
}

#wordpress-content #commentform {
        margin: 20px;
}

#wordpress-content {
        margin-left: 20px;
        margin-right: 20px;
}

/* Make WordPress "sidebaar" look like Plone "portlets */

.template-wordpress_listing #portal-column-one ul {
        list-style: none;
        margin-bottom: 40px;
}

.template-wordpress_listing #portal-column-one ul#Recent li {
        margin-bottom: 8px;
}

.template-wordpress_listing #portal-column-one ul#Categories a {
        line-height: 120%;
}

.template-wordpress_listing #portal-column-one h2 {
        background: transparent;
        border: 0;
        font-weight:normal;
        line-height:1.6em;
        padding:0;
        text-transform:none;
        font-size: 16px;
        color: #9b9b9b;
        border-bottom:4px solid #CDCDCD;
}

"""

 This command line Python script compiles your rules.xml to XDV XSL

 Modify it for your own needs.

 It assumes your buildout.cfg has xdv section and generated XDV
 commands under bin/

 To compile, execute in the buildout folder::

     python src/plonetheme.mfabrik/xdv.py

 To build test HTML::

     python src/plonetheme.mfabrik/xdv.py --test

 To build test HTML and preview it in browser, execute in buildout folder::

     python src/plonetheme.mfabrik/xdv.py --preview

"""

import getopt, sys
import os
import webbrowser

# rules XML for theming
RULES_XML = "src/plonetheme.mfabrik/deliverance/etc/rules.xml"

# Which XSL file to generate for compiled XDV
OUTPUT_FILE = "theme.xsl"

# Which file to generate applied theme test runs
TEST_HTML_FILE = "test.html"

# Our "theme.html" is a remote template served for each request.
# Because we are doing live integrattion, this is a HTTP resource,
# not a local file.
THEME="http://mfabrik.com/news/wordpress_listing/"

#
# External site you are theming.
# Note: must have ending slash (lxm cannot handle redirects)
#
SITE="http://blog.twinapex.fi/"

try:
    opts, args = getopt.getopt(sys.argv[1:], "pt", ["preview", "test"])
except getopt.GetoptError, err:
    # print help information and exit:
    print str(err) # will print something like "option -a not recognized"

# Convert options to simple list
opts = [ opt for opt, value in opts ]

print "Compiling transformation"
value = os.system("bin/xdvcompiler -o " + OUTPUT_FILE + " " + RULES_XML +" " + THEME)
if value != 0:
    print "Compilation failed"
    sys.exit(1)

if "-p" in opts or "--preview" in opts or "-t" in opts or "--test" in opts:
      print "Generating test HTML page"
      value = os.system("bin/xdvrun -o " + TEST_HTML_FILE + " " + OUTPUT_FILE + " " + SITE)
      if value != 0:
          print "Page transformation failed"
          sys.exit(1)

if "-p" in opts or "--preview" in opts:
    # Preview the result in a browser
    # NOTE: OSX needs Python >= 2.5 to make this work

    # Make sure test run succeeded
    url = "file://" + os.path.abspath(TEST_HTML_FILE)
    print "Opening:" + url

    # We prefer Firefox for preview for its superious
    # Firebug HTML debugger and XPath rule generator
    try:
        browser = webbrowser.get("firefox")
    except webbrowser.Error:
        # No FF on the system, or OSX which can't find its browsers
        browser = webbrowser.get()

    browser.open_new_tab(url)

python src/plonetheme.mfabrik/xdv.py

bin/xdvrun -o theme.html theme.xsl http://blog.twinapex.fi
firefox theme.xhtml

python src/plonetheme.mfabrik/xdv.py --preview

sudo -i
apt-get install libxslt1-dev libapache2-mod-apreq2 libapreq2-dev apache2-threaded-dev
wget http://html-xslt.googlecode.com/files/mod-transform-html-xslt.tgz
wget http://html-xslt.googlecode.com/files/mod-depends-html-xslt.tgz
tar -xzf mod-transform-html-xslt.tgz
tar -xzf mod-depends-html-xslt.tgz
cd mod-depends-html-xslt ; ./configure ; make ; make install ; cd ..
cd mod-transform-html-xslt ; ./configure ; make ; make install ; cd ..

a2enmod filter
a2enmod ext_filter

LoadModule depends_module /usr/lib/apache2/modules/mod_depends.so
LoadModule transform_module /usr/lib/apache2/modules/mod_transform.so

/etc/init.d/apache2 force-reload

<VirtualHost *>

    ServerName blog.mfabrik.com
    ServerAdmin info@mfabrik.com

    LogFormat       combined
    TransferLog     /var/log/apache2/blog.mfabrik.com.log

    # Basic WordPress setup

    Options +Indexes FollowSymLinks +ExecCGI

    DocumentRoot /srv/www/wordpress

    <Directory /srv/www/wordpress>
        Options FollowSymlinks
        AllowOverride All
    </Directory>

    AddType application/x-httpd-php .php .php3 .php4 .php5
    AddType application/x-httpd-php-source .phps

    # Theming set-up

    # This chain is used for public web pages
    FilterDeclare THEME
    FilterProvider THEME XSLT resp=Content-Type $text/html

    TransformOptions +ApacheFS +HTML
    # This is the location of compiled XSL theme transform
    TransformSet /theme.xsl

    # This will make Apache not to reload transformation every time
    # it is performed. Instead, a compiled version is hold in the
    # virtual URL declared above.
    TransformCache /theme.xsl /srv/plone/twinapex.fi/theme.xsl

    # We want to apply theme only for
    # 1. public pages (otherwise WordPress administrative interface stops working)
    <Location "/">
        FilterChain THEME
    </Location>

    # 2. Admin interface and feeds should not receive any kind of theming
    <LocationMatch "(wp-login|wp-admin|wp-includes)">
        # The following resets the filter chain
        # http://httpd.apache.org/docs/2.2/mod/mod_filter.html#filterchain
        FilterChain !
    </LocationMatch>

</VirtualHost>

#!/bin/sh
#
# Periodically update WordPress theme to reflect changes on CMS site
#

# Recompile theme
sudo -H -u twinapex /bin/sh -c cd /srv/plone/twinapex.fi ; python src/plonetheme.mfabrik/xdv.py

# Make Apache aware of theme changes
sudo apache2ctl graceful

# Make WordPress XDV theme to reflect changes on CMS
0,15,30,45 * * * * /srv/plone/twinapex.fi/update-wordpress-theme.sh

// http://codex.wordpress.org/Editing_wp-config.php#WordPress_address_.28URL.29
define('WP_HOME','http://blog.mfabrik.com');
define('WP_SITEURL','http://blog.mfabrik.com');

Building a mobile site and applications with Django and Python

Recently we created a mobile site for an interactive bicycle tour. oulugo.mobi (you need to use mobile browser to access the site or you’ll get a redirect) is a multimedia enriched bicycle tour through the historic parts of the city of Oulu. All content is provided by OnGo.

The route, which you can bicycle through is drawn on Google Maps. There are nine  action points where the user can listen to streaming audio clips, with still images, in his/her mobile phone. This is sort of  augmented reality experience: The user sees the real world (where he/she is now bicycling) combined with the historic events (audio playback narrative). For example, at Linnansaari (a location on the route) you’ll see the actual 17th century castle ruins and the narrator tells how the castle exploded when fire, caused by a lighting, reached gunpowder warehouse… boom. The explosion caused stones fly over 400 meters.

Alternatively, the clips are available as podcasts from Oulu Tourism pages. You can download them into your iPod for offline listening and use in conjuction with a paper map. This demostrates interesting mix of multichannel publishing: paper, web, mobile and podcasts.

The tour is bilingual in Finnish and English.

There exists unreleased iPhone application, based on PhoneGap, which allows the user to track his/her location real-time on the web page. We didn’t see it worth of trouble to go through Apple iPhone application review process. When location based service support comes for the browser this feature is indended to be included as the standard HTML5 feature of the service.

There also exists Nokia Series 60 mobile application, based on PyS60 and Series 60 BrowserControl API, which allows the user to track his/her location in real-time. The application provides wrapper around Series 60 WebKit control and allows Javascript to access phone native functions (GPS) over localhost socket communication. Like with Apple, we didn’t see real-time tracking feature interesting enough to go through Symbian Signed process to get our application released. Also, BrowserControl had seriousquality problems and we didn’t consider it stable enough for the end users. Some work is available in PyS60 Community Edition repository.

The service is hosted on Python specific virtual server on Twinapex services server farm.

Features

• Premium content tailored for audio listening
• Dubbed in English and Finnish by a professional voice actor
• Bilingual: English/Finnish
• Adapts for smartphones (WebKit based browsers) and low end phones (XHTML mobile profile browsers)
• Streaming video and audio (RTSP / progressive HTTP download forv iPhone). Different audio quality is provided on depending on the handset features.
• Screen resolution detection based on user agent sniffing. Three different version of images are used.
• Custom Google Maps component for mobile is used. The component adapts for different mobile phones based on sniffing. Features include zoom, show action point, show the current location, search street address name. This component can be published on a request.
• Management interface features include video upload, video transcoding different mobile versions and editing bilingual content
• Apex Vertex handset database is used to detect the user’s mobile phone capabilities
• Apex Vertex logging and traffic analytics capabilities are used for the site statistics

Software stack

• Ubuntu 8.04 Hardy Heron virtual server
• Apache 2.2 / mod_python
• Python 2.5
• Django 1.0
• Django-CMS 1.0
• mobile.sniffer Python package to provide abstraction over different handset databases
• Apex Vertex streaming solution (RTSP based on Darwin streaming server by Apple)
• TinyMCE WYSIWYG editor
• Darwin streaming server
• ExtJS is extensively used in Apex Vertext management interface

Development effort

Development time: Around 100 hours. Three different developers where involved. Used development tools: Eclipse, PyDev, Subclipse, Subversion. There were around five meetings between the content provider and the technology provider. Few beta testing rounds using iPhone application were performed by bicycling in -10 celcius degrees weather (north and so on…). No polar bears were harmed during the creation of this mobile service.

The service is linked in from Oulu Tourism pages and thousands of paper brochures printed for Oulu summer season 2009.

About the author Mikko Ohtamaa

• LinkedIn
• Twitter

Zope Zeo vs. standalone setups

We do some Plone development here at Redi. As known, Plone is a powerful, but unfortunately quite a heavy CMS which is best suited for Intranets. Thus, we are always looking for speed increase.

Enter Zeo cluster – a feature that nowadays comes bundled with Zope and allows one database (practically Data.fs) to be used by multiple Zope instances, or more accurately Zeo clients. In standalone installation only one CPU / CPU core can be used for processing requests (as Zope / Python implementation is single-threaded AFAIK). So if there are any concurrent requests the database (ZODB, the Zope Object Database) usually has to wait for the request processing before it is asked for the data and only part of the processing power is used as requests are queued. Using Zeo server-client architecture however, each Zeo client can do the processing on their own CPU/core (thus efficiently using the whole CPU prosessing power available) and also minimize the hard disk idle time by asking for data in an ~asynchronous manner (in separate queues). Actually ZODB even serves the same object simultaneously to different client processes for performance reasons. This might raise database ConflictErrors, which are nothing to fear of, however, as noted some paragraphs below.

Similarly, you could also deploy Zeo clients on different computers in local network (or wherever you want), but that’s not the scope of this article. Having clients running on different machines is a similar case with the same performance basis, but there are connection lags, bandwith limits and such that decrease performance.

Theory vs. practice

Deploying a Zeo cluster instead of standalone Zope instance should theoretically increase the performance by factor of extra available CPUs / CPU cores. There might be some overheads from this setup though, so we tested it out using ApacheBenchmark – the benchmarking module that comes bundled with Apache nowadays. But first something about…

Setting up Zeo & converting from standalone mode

In the easiest scenario, setting Zeo up is rather easy: the unified installer supports Zeo-server setup out of the box (=there is a recipe for it). Just run the unified installer like:

$ ./install.sh zeo

Luckily, the unified installer uses buildout from Plone 3.1 onwards. Thus, converting your current buildout instances to Zeo cluster is nothing but change of buildout configuration. Where you would normally need ‘instance’ section in your buildout.cfg you will now need the following:

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = 127.0.0.1:12000
#effective-user = __EFFECTIVE_USER__

[client1]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
zeo-client = true
zeo-address = ${zeoserver:zeo-address}
# The line below sets only the initial password. It will not change an
# existing password.
user = admin:mysecretpassword
http-address = 12001
#effective-user = __EFFECTIVE_USER__
#debug-mode = on
#verbose-security = on

# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = ${buildout:eggs} ${plone:eggs} my.package
eggs =
    ${buildout:eggs}
    ${plone:eggs}

# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml =

products =
    ${buildout:directory}/products
    ${productdistros:location}
    ${plone:products}

To add more clients (which is quite the point here), append as many times the extra client sections like this:

[client2]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
zeo-client = true
zeo-address = ${zeoserver:zeo-address}
user = ${client1:user}
http-address = 12002
#effective-user = __EFFECTIVE_USER__
#debug-mode = on
#verbose-security = on
eggs = ${client1:eggs}
zcml = ${client1:zcml}
products = ${client1:products}

That minimizes the need for retyping user names, password etc. These examples were taken from Plone unified installer buildout.cfg with ports changed.

Starting, stopping & restarting

Now, to start your Zeo-powered Plon clients you could type:

bin/zeoserver start
bin/client1 start
bin/client2 start
...same for all the clients...

However, the unified installer has a recipe which automatically generates nice and simple shell scripts to control your cluster. In the end of your buildout.cfg, add:

[unifiedinstaller]
recipe = plone.recipe.unifiedinstaller
user = ${client1:user}
primary-port = ${client1:http-address}

That should generate the scripts. In fact, it propably does also something else, something which I’m not aware of. However, I didn’t bump into any problems, yet Anyway, to start the whole cluster (server & clients), type:

bin/startcluster.sh

And that does it (it start server and the clients). Shut it down via:

bin/shutdowncluster.sh

And restart:

bin/restartcluster.sh

ConflictErrors – not that errerous

As noted before, in Zeo mode the ZODB might serve the same objects to two more clients at the same time. If one client manipulates the object before others (ie. edits values and saves changes) the other requests will propably fail. This raises ConflicError which looks like this:

ConflictError: database conflict error (oid 0x0f39, class HelpSys.HelpSys.ProductHelp)

In this case ZODB tries to reprocess the failed requests. This should be common database approach and thus a feature, not a bug (although Zope might want to tell that in error message!). For more accurate explanation see Plone discussion.

Parsing it together with web server

The Zeo components (server and clients) talk to each other via standard Internet protocols (TCP or UDP, not sure). In the default setup, the Zeo server listens to port 8100 and Zeo clients to 8080, 8081, etc. Thus, to access the separate clients as ‘one site’ we need to serve the requests to multiple clients. This can be achieved with load balancers. Apache has at least one: mod_proxy_balancer which should do exactly what we need. Apache isn’t the best choice for achieving high requests per second values, but it will do for our tests (compare to more lightweight but also more limited lighttpd). Just remember that there are other alternatives/methods available, like using squid as load balancer.

Our configuration is as follows (inside VirtualHost-directive):

  <Proxy balancer://lb>
    BalancerMember http://127.0.0.1:12001/
    BalancerMember http://127.0.0.1:12002/
    BalancerMember http://127.0.0.1:12003/
    BalancerMember http://127.0.0.1:12004/
  </Proxy>

  <Location /balancer-manager>
    SetHandler balancer-manager
    Order Deny,Allow
    Allow from all
  </Location>

  ProxyPass /balancer-manager !
  ProxyPass             / balancer://lb/http://localhost/VirtualHostBase/http/www.mydomain.com:80/plonesite/VirtualHostRoot/
  ProxyPassReverse      / balancer://lb/http://localhost/VirtualHostBase/http/www.mydomain.com:80/plonesite/VirtualHostRoot/

This setup also allows us to use the balancer-manager (accessible at /balancer-manager) that comes with mod_proxy_balancer. It’s useful for checking if the configuration is working and balancer is dividing the requests equally. In my setup the balancer is using the default Request Counting -algorithm which divides the requests numerically equally between the instances, but you might want to also try Weighted Traffic Counting, which should be for actual use. In our test only the frontpage is accessed however, so each request’s data transfer is equal and the weighted traffic counting isn’t of use.

The test

The server machine

• Ubuntu 8.04 virtual server
• Intel Xeon 2.0Ghz (4 cores)
• 2 GB of RAM
• Hard disk drive (7200rpm?)

The setup

• Standalone Plone instance
• Plone via Zeo server with 4 clients (as many clients as cores in processor)
• Plone via Zeo server with 6 clients (for curiosity)

The tests where run locally in development environment to minimize the network lag (was 0-1ms).

The test commands

ApacheBenchmark commands:

$ ab -n N -c C myurl

where N was either 1000 or 9000 (requests) and C 1, 10, 100 or 1000 (concurrent requests).

The results

You can download the more in-depth test sheet Plone Standalone vs. Zeo installation (PDF).

To put it simple: theory and practise meet well – Zeo server is a lot more powerful with concurrent requests. On non-concurrent requests the results are about the same.

Having as many Zeo clients as CPUs / CPU cores can boost the performance up to number of extra CPUs/cores. For example, in our quad-core server with Zeo setup we gained nearly 4 times the requests per second of standalone installation (~370% to be accurate). Increasing Zeo clients to 6 didn’t help any as there’s no processing power left from 4 heavily stressed client processes. Also to be noted is that the waiting times for clients nearly tripled (median jumped from 126 to 305 ms) when raising concurrency from 1 to 10. This isn’t bad though – those are still low figures compared to standalone’s median of 1215 ms! Only when raising concurrency to 100 we began to see some 3,6 seconds waiting times (6 seconds for standalone). Increasing concurrency didn’t bring down the requests/second rates much (less than 5%) as expected.

Overall, the results were expected, but now we have evidence of it: under concurrent request load Zeo server is a good option to multiply the performance of your site. With very low traffic sites which rarely get more than 1 request at time this doesn’t matter.

One bad word about the resource requirements though: The used RAM increase for 6 client Zeo setup (standard Plone 3.1.2 + 12 additional Products) was whopping 621 MB (1132 MB -> 1753 MB). That means about 100 MB per Zeo client as the Zeo server memory intake was only about 12-15 MB. Thus, only use as many Zeo clients as absolutely necessary or you might find your beloved server machine under very serious Zope flu!