Plone is a byproduct of community members' co-operation

These are notes from the Sauna Sprint daily debriefing #2. It was intention we’d done this already yesterday, but unfortunately we had problems with boat and swamp.

The current head count is around 27 living souls, plus or minus few depending on how you define living. We split to different teams, each working on its own topic.

Team tutorial video

This team works to create new Plone video installation and usage tutorials. It will be in the form of screencast with voice over narration.

The team has created the first voice over, screencast and now editing it together.

Team add-ons fail

Plone add-ons come with various quality of instructions. This team is working to make Plone add-on experience more user-friendly: installing, uninstalling, plone.org add-on area, individual add-on instructions.

Around 200 lines of issues collected.

Team blogging

This team collects and publishes a blog posts, participant information, etc. motivational information about the sprint and the sprinters.

Both EESTECers and Plonistans are being covered.

Currently the team has collected the bio of everyone and prepares to put the information out to a wiki page.

Team auto restart

This team is working to make Plone auto-restart perform correctly and timely manner. You just edit .py file and hit the browser refresh button and the changed code is loaded correctly. The team is utilizing the fork trick introduced in this blog post. The resulting work goes under a project name sauna.relaod.

Currently the team has managed to make the file-system monitor using Python WatchDog backend. Already 13 seconds have been shoveled off from Zope restart time of 30 seconds.

Team Logistics

This team feeds the coders with firewood, food and beer.

Yesterday the team prepared went to a shop, prepared a lunch and brought a guitar (for live music).

The day plan for Wednesday is to collect blueberries for pies and mushrooms and later cook them in the big oven of the cabin.

The new beer is on its way, though has temporary taken some steps backwards.

Team Greek

This ad-hoc team prepared the dinner.

It takes four Greeks to create one omelette.

Team collective.table

This is Google Summer of Code team is working on new collective.table add-on for Plone. Yesterday was API talk, but the progress was slow due to lack of caffeine.

Team TinyMCE

This is a Google Summer of Code team and is working on making Plone’s TinyMCE experience better.

Yesterday was spent testing on a JS/CSS compressor to make TinyMCE load faster.

Uncompressed: 19 request / 900 kb. Compressed 50% less requests, 700 kb.

Team hardcore

This team of EESTEC member was formed to introduce hardcode Plone development for newbies.

The team members actually wanted it themselves. They will regret this decision later.

Sometimes you need solo heroes

Plone butterfly preparing to hatch

A lonely coder in the forest

Fueling the hungty developers with potatoes

A statue honoring Plone developers

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Prerequisites

What you need in order to benefit from these instructions

• Basic UNIX command-line knowledge
• SSH access to the server
• MySQL access to the database
• LAMP stack ready on the new server

These instructions have been tested on Ubuntu/Debian/Linux but they should work in other environments too.

Write down database access information

Get password from config.php file on the old server:

cd /var/www/phpBB3
cat config.php

Write down database name, username and password.

Copy files

Use rsync to remotely copy forum files to a new computer. On new computer, in /var/www folder

rsync -av --compress-level=9 user@oldserver.com:/var/www/phpBB3 .

Dump and copy database

Execute the following command on the new server. It takes SSH connection to the old server and dumps phpBB3 database to the new server over the SSH connection.

ssh user@oldserver.com -C -o CompressionLevel=9 mysqldump -u databaseuser --password=databasepassword --skip-lock-tables --add-drop-table databasename > phpbb3.sql

Create a new database

Use the old access information from config.php to create a database with identical access information on the new server. You need a MySQL root access to create new databases.

mysql -uroot -p

Create database and grant access to phpBB3 user for it.

mysql> create database databasename;

mysql> GRANT ALL ON databasename.* TO 'databaseuser'@'localhost' identified by 'databasepassword';

Load the database on the new server from the dump file:

mysql> connect databasename;

mysql> source phpbb3.sql

Configure Apache virtualhost for the new server

The last step is to set-up Apache virtual host on the new server, so you can access the phpBB3 using a domain name. Note that this doesn’t need to be a real domain name, but you can spoof the domain name using /etc/hosts file on your local workstation.

Add file /etc/apache2/sites-enabled/phpbb3.conf (or pick a filename based on forum name if you host multiple forums)

<VirtualHost *>
 ServerName yourdomainname.com

 DocumentRoot /var/www/phpBB3
 <Directory />
   Options FollowSymLinks
   AllowOverride None
 </Directory>

</VirtualHost>

Note that <virtualhost *> may change depending on how Apache has been set up to listen IP addresses and ports. Also if you are using a shared hosting package or VPS you might need to use the server control panel (cPanel) to do this step.

Then check if your new config file is ok and restart Apache:

apache2ctl configtest
apache2ctl graceful

Hosts spoofing trick

If you are not having a DNS server of your own which you can use for the copy you can always use /etc/hosts file trick to spoof domain names. This way you can make Apache to serve the forum from the server even if the forum is not connected to any real domain name yet.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

To have the über-experience of customer engagement with mobile push notifications you need

• A mobile application (iOS, Android 2.2)
• RSS feed to notifications middleware server (our solution)
• RSS feeds themselves
• Windows/UNIX server running the middleware

How it is put together

Tornado web server is used to handle incoming HTTP requests in scalable manner.

feedparser library fetches RSS feeds and processes them to client notifications.

BitReader (post, source) library is used to create messages to interact with Apple push notification service (APNs). The protocol is bit based protocol running directly over TCP/IP. Apple service has been designed to handle high volumes of traffic – it does not use anything like stateless HTTP to waste bandwidth.

Django models are used to store the state of each individual subscriber. Django’s ORM abstraction allows us to use the same middleware for small distributions (< 1000 clients, SQLite database) or big ones (millions of clients, MySQL database). The stored state information includes the subscriber id and the current badge number – the red circle on the app icon showing the count unread posts. When the application is launched, it can decrease its badge number by doing a HTTP call to the server.

Django settings are used to put together required certificates and whether the application is run in sandbox mode.

Walkthrough

There is a core IO loop, running in a separate process, called stream observer. This loops updates fetches RSS feeds’ status and passes updates to Tornado server over HTTP.  With this arrangement, any HTTP capable client can send push notifications.

Tornado handles incoming updates, updates the related subscribe status – how many unread notifications, etc. through exposed Django views. The notification is formatted according to the variables available on the subscriber mobile platform. In Apple’s case, the notification message gets title, badge, sound and a launch image. Payload is checked against hard 256 byte limit.

Then the payload is pushed to Apple servers over TCP/IP protocol. SSL certificaties needed.

A subscriber is registered  when the mobile application is launched. The application asks a subscriber id from Apple servers. Then, this subscriber id is delivered to our middleware over normal HTTP call.

The middleware also handles feedback service which gives you list of devices which have unsubscribed from your service. This way you can cut off notifications from unsubscribed clients. This is also done using BitReader and TCP/IP.

Future

The architecture is built so that different push backends can be included in the service. Android support is on the roadmap and we probably will have Blackberry and Meego support (when/if Nokia announces such a service).

We have currently tested this solution with RSS streams from WordPress and Plone.

We may release source code when it’s ready.

More info

• http://blog.boxedice.com/2009/07/10/how-to-build-an-apple-push-notification-provider-server-tutorial/

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

If your Mac loses power on a wrong moment FileVault might become corrupted. This will lead to a situation that you cannot login to the system anymore. The login dialog will just tell “Volume is corrupted. Would you like to repair?” And then it says something along lines “login failed”.

Don’t lose your hope here. You can usually still fix the corrupted FileVault sparsebundle file as root user.

To create a root user

• Use OSX installation DVD to add new admin user to the system with non-protected folder
• …or in the case you have non-admin user accounts on the system already, use them to login, navigate to System preferences -> Accounts and create a new admin account.  Note that you can use your credentials from the corrupted user to make admin level system changes even if the logged in user is not admin – just click the lock icon in the System preferences.
• An admin user can enable root user. To enable root user, you need to navigate to /Macintosh HD/System/Library/CoreServices open Directory Utility and choose Enable root user from the menu (Snow Leopard)
• Logout. Then login dialog should have new option “Other” where it allows you to type in username and password. Now login as root with the password you set for it earlier.

To fix the damaged sparsebundle file

• Login as root
• Open Application -> Utilities -> Disk utility
• From menu choose Open
• Choose /Macintosh HD/Users/corruptedusername/corruptedusername.sparsebundle file
• Disk Utility will complain about something it cannot mount volume, but the volume still will appear in the right hand disk list
• In the disk list choose the corrupted volume
• Hit repair
• The repair process will take minutes, but completes succesfully

After the repair process is complete, logout as the root user and login as the user who had damaged FileVault. This time the login success.

More info

• Repairing a FileVault protected Home folder

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Copy contacts

• Register at ScheduleWorld (7 days free trial)
• Install Dropbear SSH on N900 (makes typing less painful)
• Install syncevolution (tested with command line version) on N900
• Create syncevolution templates using syncevolution -c scheduleworld (work over SSH from desktop computer using)
• Change username and password in ~/.config/syncevolution/scheduleworld/config.php (use nano shell based text editor)
• Change sync to cliet to server one way in ~/.config/syncevolution/scheduleworld/addressbook/config.php and ~/.config/syncevolution/scheduleworld/calendar/config.php
• Run syncevolution: syncevolution scheduleworld
• Check that scheduleworld.com received your data using web browser
• Install Funambol app on Android device from Android Market
• Run it. Use credentials: user = numeric scheduleworld.com username, password = given in scheduleworld.com registration email, server url = http://sync.scheduleworld.com/funambol/ds
• Alternative 2: export contacs in scheduleworld.com and import .vcf to fresh gmail account
• Alternative 3: there exist apps to read .vcf files from SD card – Scheduleworld can export these files.

Copy calendar

Note: Android 2.1 owners might need to reset their device to factory settings as 2.1 supports synchronizing calendar only from the Google account used to active the device. You need to be able to import calendar entries to Google/Gmail account you initially used to active the device / Android market. Android 2.1 devices do not support syncing calendar from multiple accounts (even if they are Google’s)

• (Register a dummy gmail.com account)
• Go to scheduleworld.com. From Tools menu choose Export -> Events and Task
• Download iCal (.ics) file locally
• Go to gmail.com account
• Open calendar
• Go to Settings -> Calendars -> Import Calendar
• Upload .ics file
• Now when device syncs with Google Calendar it will receive the calendar entries

I succesfully 600 phone numbers and 60 calendar entries.

Back up SMS

If you want to back up SMS messages from the device just copy SQLite database over SCP. You can later convert SQLite entries to any format using sqlite command line tool.

scp user@192.168.1.146:/home/user/.rtcom-eventlogger/el.db .

That’s all folks!

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

I found this nice library from Denis Laprise. However, it had a bug with fragment extractor and lacked reconstruction possibilies. So I decided to make a new version.

Download urlparse.js version 0.2. thank you

Couple of examples:

var p = new Poly9.URLParser('http://user:password@poly9.com/pathname?arguments=1#fragment');
p.getHost() == 'poly9.com';
p.getProtocol() == 'http';
p.getPathname() == '/pathname';
p.getQuerystring() == 'arguments=1';
p.getFragment() == 'fragment';
p.getUsername() == 'user';
p.getPassword() == 'password';

var p = new Poly9.URLParser("http://localhost:8080/path);

p.setQuerystring("foo=bar");
p.setFragment("anchor");
p.setPort(7070);

var url = p.getURL() // http://localhost:7070/path?foo=bar#anchor

Country and city tourist offices, as well as other organizations publishing information for tourists and travelers, traditionally have lots of information available on the internet. Background information, history, practical information like about transportation, sights, restaurants, bars and nightlife, not forgetting maps and other content types like videos and audios. Unfortunately this information is not easily usable, if at all, by mobile phones and mobile phone browsers thus making it practically unusable for tourist after s/he has left home and started her/his travel.

Its actually surprise how little there’s usable tourism and travel information which can be used with mobile phones. Here in Finland Helsinki has its own mobi site but, well, not so attractive service and could have more content. Especially if compared for the mobile service provided by eg. SAS, even their focus is a bit different.

What’s the problem? Most likely that those tourism offices and boards are just so traditional, slowly moving elephants. This have seen already when internet, as term and a “channel”, was new and how slowly tourism and travel communications moved into the net. Now the good sides, more cost effective and up to date communication, has been seen but hopefully value of mobile phones and services for them wont take so much time as it did with web services. Because the tourists and travelers are here, now, and willing to use information if that’s just available.

For your Plone launcher, set environment variables (space separated list)

PTS_LANGUAGES=en mylanguagecodehere

If your Data.fs is not fresh (i.e. you have an existing Plone instance) there is still one task to do. Go to Placess Translation Service in Zope. Delete all translation catalogs. If there exists a translation catalog entry in ZODB a reload event seem to be triggered even though PTS_LANGUAGES settings is effective. Restart Zope. Maybe this is a bug? Do this on a development box only – this code seems to be quite new.
The magic code is in PlacelessTranslationService/load.py.

I just learnt that Firefox web page screen capture tools (any of them) can’t take web page screenshots higher than short 16-bit interger (32768) pixels. Crash crash crash. But I hope I am alone with my problem.

So, thanks for being so supportive… and it’s not fully rendered (over 250 of them), since after 3200 the image was cut. Excuse me if I am not willing to support your cause.

This post is update to previous Eclipse web developer plug-in memo post. New versions are available and some plug-ins have become deprecated. This blog post reflects those changes.

These instructions are good for:

• Python developer
• PHP developer
• Java developer

Choosing Eclipse distribution

• On Window, use EasyEclipse
• On Linux, use Eclipse provided by the distribution – Eclipse links against the embedded Mozilla browser and this is distribution specific – EasyEclipse has some issues here. For Ubuntu users:

sudo apt-get install sun-java6 eclipse

• On OSX, use EasyEclipse

EasyEclipse bundles some of the stuff listed here with it – when using EasyEclipse you don’t need to have separate PyDev and Subclipse downloads.

Eclipse for 64-bit Linux has various problems. You might want to run 32-bit Eclipse (another relevant blog post). When you use Linux distribution specific Eclipse install, all your personal Eclipse files go to .eclipse folder under your home folder.

Installing plug-ins

Eclipse has internal updater/web installer. All plug-ins are downloaded as ZIP files and extracted to Eclipse folder or installed through the internal updater. Paste Eclipse update site URLs to menu Help -> Software updates -> Find and Install, New Remote Location.

Python

PyDev is a plug-in for Python and Jython development. It has enhanced commercial extensions for professional developers with more intelligent autocomplete and debugger.

Site URL: http://pydev.sourceforge.net

PyDev Eclipse update URL: http://pydev.sourceforge.net/updates/

PyDev extensions Eclipse update URL (this commercial, but worth of every penny): http://www.fabioz.com/pydev/updates

PDT

PDT download provides Eclipse, HTML editor, PHP editor and CSS editor.

Site URL: http://www.eclipse.org

Eclipse update site URL: http://download.eclipse.org/tools/pdt/updates/

Java

If you need to do J2EE development use IBM’s Web Tools Platform. If you don’t need Java capabilities don’t install these, since they just bloat Eclipse and make the start up time worse.

Subclipse

Subclipse provides Subversion version control integration to Eclipse.

Eclipse update site URL: http://subclipse.tigris.org/update_1.4.x/

In the installer, uncheck the integration modules checkbox or the installer will complain about missing modules.

Aptana Studio

Aptana Studio is state-of-the-art Web 2.0 development suite for Eclipse. It has Javascript, CSS and HTML editors. It supports various Javascript libraries out of the box and has support for Firefox and IE in-browser Javascript debugging.

Eclipse update site URL: http://update.aptana.com/update/studio/3.2/site.xml

ShellEd

Syntax coloring for Unix shell scripts

Project site: http://sourceforge.net/projects/shelled

SQL Explorer

SQL terminal and SQL editor with some GUI capabilities.

Eclipse update site URL http://eclipsesql.sourceforge.net/

SQL Explorer needs MySQL JDBC driver. Download from here. Install MySQL connector by extracting the file and adding it from SQL Explorer preferences.