About

mFabrik Blog is about mobile and web software development, open source and Linux. We tell exciting tales where business, technology, web and mobile convergence.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Sauna Sprint day 1: Sunday (technical & motivational)

Now it has begun: Sauna Sprint 2011 – an open source event in Finland for Plone & Python hackers and EESTEC students.

The first day of Sauna Sprint was spent introducing new EESTEC people to Plone. Almost everyone got Ubuntu installed, there was a short crash course for IRC and then we started installing Plone.

We had a bunch of EESTEC students as a test subject and our goal was to map obstacles in the process of installing Plone for local computer for development. We got some interesting results.

The data was collected on whiteboard where everyone was free to express any problem he or she had during the installation. The advisors also monitored the user group for possible problems and made notes.

Aperture Science test subjects installing Plone in hope they’ll get a cake if they success


Good job, but the cake was a lie.

Plone is #%!!”€&! difficult, but we can fix it

The comments below are collected from the whiteboard.

Don’t make assumptions in the documentation – people are tabula rasa (i.e. how to use cd command and so on)

Documentation needed to install gcc, PIL dependencies etc. on OS level

No good instructions how to set-up Python with virtualenv (was being fixed in the sprint). Buildout actually does not work with the system Python out of the box: you need to break your OS http://stackoverflow.com/questions/5818100/buildout-tries-to-update-system-wide-distribute-installation-and-refuses-to-run

People try to run buildout by “cd bin ; ./buildout” – it does not work and the error message is confusing

Unified Installer – ZEO installation is broken (need clarification what was broken)

Detect Python version in buildout and don’t try to run on incompatible version

When creating first Plone site input field has label “Path identifier” – people try to enter Plone FS path there

README.txt in Unified Installer needs updates

ZopeSkel template plone4_buildout should autodetect the latest Plone version (now was 4.0.1)

Update the out of the box Plone installation front page to contain more developer friendly instructions too

plone.org/downloads: Show add-on version compatibility as the first item on the page – if now compatibility set warn that the compatibility is unknown (same kind of warning if no updates for one year)

Have an easy way to report broken packages

Force a process to improve add-on documentation in p.org, especially regarding installation. Only 1/5 themes was successfully installed by the test personel without guidance, mostly due to lack of instructions and version incompatibilities.

Some work in progress on PiratePad regarding the installation of Plone.

Evening program

We had Sauna time in Hervanta, student style. EESTEC lived up to their reputation as party animals.

Delicious pizza was baked, though the audience loudly disagree when the pizza was burnt enough to be eaten

Magic Toni Sause

 

 

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Plone 4 upgrade: results and steps

We upgraded one production site to Plone 4. The site is a web front for a retail chain, rich with big images. Plone 4′s support for blob storage of files has especially helpful. Below are some results.

Download speed

We see around 33% – 40% increase

Response time

Though this is little harder to read, due to the sampler having issues, the response time has dropped from 250 ms to near 100 ms.

The update process

Here is the process what we did.

Create a duplicate of buildout.cfg with Plone 4 based recipes and run bootstrap.py with Python 2.6.

Uninstall a lot of unneeded add-ons which had collected cruft over the years: plone4artists audio, video, LinguePlone.

Uninstall Products.CacheSetup. This is replaced with plone.app.caching in Plone 4.

Use wildcard.fixpersistentutilities to clean up leftovers of broken add-on uninstallers (Singing & Dancing, p4a).

Remove broken persistent tools (Notifica, Search & Replace, etc.).

Make sure that the site custom theme extends Plone 4 Classic Theme.

Replace old collective.slideshow with Products.Carousel. The migration script is available in collective.developermanual.

Replace old collective.fancyzoomview with Products.PipBox. The migration script and PipBox code are available in collective.developermanual.

Copy Data.fs from Plone 3.3 to new Plone 4 buildout.

… in this point we move from Plone 3 buildout to Plone 4 buildout …

Run portal_migration.

Converted Language portal_catalog index from LanguageIndex to FieldIndex (instructions available in c.developermanual).

Rebuild the catalog.

Reset theme on portal_skins properties tab. Shuffle portal_skins layers until they work.

Migrate custom Archetypes image scale sizes from monkey-patch to plone.app.imaging and imaging_properties.

Remove kukit-devel in portal_javascripts.

In portal_transforms -> safe_html, disable_transform=1.

Change custom folder listing from atct_album_view32 to atct_album_view.

Futher info

Some of these upgrade tips are collected to collective.developermanual.

 

 

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Session affinity (sticky) with Nginx load balancing

These are instructions for setting up session affinity with Nginx web server and Plone CMS. Nginx can be used as a load balancer at the front of several Plone front-end processes (ZEO processes). By default, Nginx does not do session affinity, a.k.a. sticky sessions.  If you have site functionality which stores user specific data on the front end session data, let’s say an eCommerce site shopping cart, you must always redirect the user to the same ZEO process or they will have have different random shopping cart for each front end client.

Instructions assume you are installing Nginx via buildout.

Nginx sticky session can be loaded from here

Manually extract nginx-sticky-module under src folder:

cd src
wget http://nginx-sticky-module.googlecode.com/files/nginx-sticky-module-1.0-rc2.tar.gz

Then add it do Nginx configure options in buildout part:

[nginx-build]
recipe = zc.recipe.cmmi
url = http://sysoev.ru/nginx/nginx-0.7.65.tar.gz
extra_options = --add-module=${buildout:directory}/src/nginx-sticky-module-1.0-rc2

Now test recompling Nginx via buildout:

mv parts/nginx-build/ parts/nginx-build-old # Make sure full rebuild is done
bin/buildout install nginx-build

See that it compiles without erros. Here is the line of compiling sticky module:

gcc -c -O -pipe  -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Wunused-function -Wunused-variable -Wunused-value -Werror -g   -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I src/mail \
        -o objs/addon/nginx-sticky-module-1.0-rc2/ngx_http_sticky_module.o \

Now add sticky to the load balancer section of nginx config:

[balancer]
recipe = gocept.nginx
nginx = nginx-build
...
http {
        client_max_body_size 64M;
        upstream zope {
                sticky;
                server ${hosts:client1}:${ports:client1} max_fails=3 fail_timeout=30s;
                server ${hosts:client2}:${ports:client2} max_fails=3 fail_timeout=30s;
                server ${hosts:client3}:${ports:client3} max_fails=3 fail_timeout=30s;
        }

Reinstall nginx balaner configs and start-up scripts:

bin/buildout install balancer

See that generated configuration is ok:

bin/nginx-balancer configtest

Restart Nginx:

bin/nginx-balancer stop ;bin/nginx-balancer start

Check that some (non-anonymous) page has route cookie set by sticky module:

Huiske-iMac:tmp moo$ wget -S http://yoursite.com/sisalto/secret-travel-destination-infoa
--2011-03-21 21:31:40--  http://yoursite.com/sisalto/secret-travel-destination-infoa
Resolving yoursite.com (yoursite.com)... 12.12.12.12
Connecting to yoursite.com (yoursite.com)|12.12.12.12|:80... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Server: nginx/0.7.65
  Content-Type: text/html;charset=utf-8
  Set-Cookie: route=7136de9c531fcda112f24c3f32c3f52f
  Content-Language: fi
  Expires: Sat, 1 Jan 2000 00:00:00 GMT
  Set-Cookie: I18N_LANGUAGE="fi"; Path=/
  Content-Length: 41471
  Date: Mon, 21 Mar 2011 19:31:40 GMT
  X-Varnish: 1979481774
  Age: 0
  Via: 1.1 varnish
  Connection: keep-alive

Now test it by doing manual session related activity and see that your shopping cart is not “lost”.

More info

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Installing MySQL-python connector on OSX

Preconditions

  • You are using MySQL installation from mysql.com (and not e.g. from Macports)
  • You want to install MySQL-python egg using easy_install or buildout

You’ll get an error

Getting distribution for 'MySQL-python'.
install_dir /Users/moo/project/eggs/tmp8-j13e
sh: mysql_config: command not found
Traceback (most recent call last):
 File "<string>", line 1, in ?
 File "/Users/moo/code/collective.buildout.python/python-2.4/lib/python2.4/site-packages/distribute-0.6.14-py2.4.egg/setuptools/command/easy_install.py", line 1855, in main
 with_ei_usage(lambda:
 File "/Users/moo/code/collective.buildout.python/python-2.4/lib/python2.4/site-packages/distribute-0.6.14-py2.4.egg/setuptools/command/easy_install.py", line 1836, in with_ei_usage
 return f()  
...

 31, in <lambda>
 {'__file__':setup_script, '__name__':'__main__'}
 File "setup.py", line 15, in ?
 File "/var/folders/O8/O8pt7q52F7Oi+P3O0pNqq++++TI/-Tmp-/easy_install-g5cvxl/MySQL-python-1.2.3/setup_posix.py", line 43, in get_config
 File "/var/folders/O8/O8pt7q52F7Oi+P3O0pNqq++++TI/-Tmp-/easy_install-g5cvxl/MySQL-python-1.2.3/setup_posix.py", line 24, in mysql_config
EnvironmentError: mysql_config not found
An error occured when trying to install MySQL-python 1.2.3.Look above this message for any errors thatwere output by easy_install.
While:
 Installing instance.
 Getting distribution for 'MySQL-python'.
Error: Couldn't install: MySQL-python 1.2.3

The reason is that MySQL installer puts MySQL under /usr/local and does not expose MySQL utilites directly to path. MySQL-python expects to find utility mysql_config to configure compile time paths when building native bindings using GCC.

Workaround:

sudo -i
cd /usr/bin
ln -s /usr/local/mysql-5.1.46-osx10.5-x86_64/bin/mysql_config .

Then rerun installer.

Though it is a bit harsh solution to modify system folder, it gets the thing done in simple manner.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter