About

mFabrik Blog is about mobile and web software development, open source and Linux. We tell exciting tales where business, technology, web and mobile convergence.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Python 2.3, Python for Series 60 1.4.5 still in use… and insight into Symbian deployment process and user experience

Nokia’s Python for Series 60 has a long history. It is a Python interpreter, originally escaped from Nokia prototype labs, running in your phone. It is said to been awesome to show mobile/embedded developers, who were love with their static C compilers and 4 hours built times, opening a Python prompt in your phone and typing import audio; audio.say(“your phone loves Python”) by keypad (Nokia Series 60 phones come with a speech synthetizer). Python for Series 60 is the best tool of building a simple proof of concept mobile applications. The lack of speed, lack of good UI libraries and difficult deployment problems makes it challenging to use it in production grade environments.

PyS60 has also a history of staying in archaid Python version – namely Python 2.3. It was not until this February when stable PyS60 2.0.0 with Python 2.5 was released (1.9.x was considered experimental according to the release notes). Luckily looks like new winds are blowing (Qt acquisition, Meego/Maemo) and Python is getting higher priority. For example, PySide Qt bindings is very high profile project. Based on this, we hope to expect Python to the first class citizen in the future Meego and Symbian devices.

My company had a little side venture with PyS60 Community Edition when we were still betting that Symbian and Python would rock the world – the era before iPhone changed the game.  PyS60 community edition was effectively a revamped PyS60 1.4.x with Python 2.3 toolchain which actually made PyS60 application production deployment possible. Possible…? -you ask. Madness… no. It is Symbian. It is certification and signing and obscure error messages. Basically vanilla PyS60 is being shipped as an external SIS (Symbian package format) and Symbian platform security makes it impossible to deploy two production signed applications using vanilla PyS60 on the same device. The only cure was statically building Python for both apps from the scratch, which is exactly what PyS60 Community Edition was doing.

But this all was long long time ago. Aeons in mobile time. So I was today surprised when I got email from a person (David) using PyS60 Community Edition. We never upgraded PyS60 Community Edition to Python 2.5 . In fact we haven’t touched the project about two years. David was effectively using Python 2.3 and asked questions about the tool chain internals.

My first answer was a question Why on Earth you are still using Python 2.3? I thought maybe the guy had somehow missed the last two years or was a stuck with an old phone.

However… this was not the case and the answer was very insigtful.

Yes, I’m aware of PyS60 2.0.0, but I prefer PythonCommunity, at least for the moment: no OpenC neither Platform Services dependencies; smaller .SIS size and memory footprint. I think that the final .SIS produced with PythonCommunity, with everything necessary to run contained in it and with a clean installation without multiple dependencies, is a better fit for a mass-market than the files produced by PyS60 2.0.0, above all taking into account that people don’t know what S60 or Symbian are.

Also, the runtime deployment on the new PyS60 isn’t automatic for S60_3rd and S60_FP1 devices, so in the worst case scenario, users may end having to learn to install the different files (pips.sis, ssl.sis, stdioserver.sis, Python_2.0.0.sis, PythonScriptShell_2.0.0.sis) in the correct order, which is a big no-no for a mass-market deployment.

So…. I hope someone in Nokia is reading this blog entry carefully. Do it like Apple does. Make your application deployment static. Make OpenC static. Make every freaking library which is not shipped with the device statically buildable. It should be possible now when everything is open source. It will consume precious device RAM, but at least it will make mass market application development possible. SIS hell is worse hell than deb hell, or DLL hell, as the end user cannot fix it due to device security.

In the related news SIS smart installer was announced few weeks ago. Personally I wouldn’t bet it can deal with all the problems of versioning and Symbian platform security. Forum reports aren’t promising and looks like very Symbianish user experience can be expected. In positive light, it seems that Python is being considered for this process.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Building a mobile site and applications with Django and Python

Recently we created a mobile site for an interactive bicycle tour. oulugo.mobi (you need to use mobile browser to access the site or you’ll get a redirect) is a multimedia enriched bicycle tour through the historic parts of the city of Oulu. All content is provided by OnGo.

The route, which you can bicycle through is drawn on Google Maps. There are nine  action points where the user can listen to streaming audio clips, with still images, in his/her mobile phone. This is sort of  augmented reality experience: The user sees the real world (where he/she is now bicycling) combined with the historic events (audio playback narrative). For example, at Linnansaari (a location on the route) you’ll see the actual 17th century castle ruins and the narrator tells how the castle exploded when fire, caused by a lighting, reached gunpowder warehouse… boom. The explosion caused stones fly over 400 meters.

Alternatively, the clips are available as podcasts from Oulu Tourism pages. You can download them into your iPod for offline listening and use in conjuction with a paper map. This demostrates interesting mix of multichannel publishing: paper, web, mobile and podcasts.

The tour is bilingual in Finnish and English.

There exists unreleased iPhone application, based on PhoneGap, which allows the user to track his/her location real-time on the web page. We didn’t see it worth of trouble to go through Apple iPhone application review process. When location based service support comes for the browser this feature is indended to be included as the standard HTML5 feature of the service.

There also exists Nokia Series 60 mobile application, based on PyS60 and Series 60 BrowserControl API, which allows the user to track his/her location in real-time. The application provides wrapper around Series 60 WebKit control and allows Javascript to access phone native functions (GPS) over localhost socket communication. Like with Apple, we didn’t see real-time tracking feature interesting enough to go through Symbian Signed process to get our application released. Also, BrowserControl had seriousquality problems and we didn’t consider it stable enough for the end users. Some work is available in PyS60 Community Edition repository.

The service is hosted on Python specific virtual server on Twinapex services server farm.

Features

  • Premium content tailored for audio listening
  • Dubbed in English and Finnish by a professional voice actor
  • Bilingual: English/Finnish
  • Adapts for smartphones (WebKit based browsers) and low end phones (XHTML mobile profile browsers)
  • Streaming video and audio (RTSP / progressive HTTP download forv iPhone). Different audio quality is provided on depending on the handset features.
  • Screen resolution detection based on user agent sniffing. Three different version of images are used.
  • Custom Google Maps component for mobile is used. The component adapts for different mobile phones based on sniffing. Features include zoom, show action point, show the current location, search street address name. This component can be published on a request.
  • Management interface features include video upload, video transcoding different mobile versions and editing bilingual content
  • Apex Vertex handset database is used to detect the user’s mobile phone capabilities
  • Apex Vertex logging and traffic analytics capabilities are used for the site statistics

Software stack

Development effort

Development time: Around 100 hours. Three different developers where involved. Used development tools: Eclipse, PyDev, Subclipse, Subversion. There were around five meetings between the content provider and the technology provider. Few beta testing rounds using iPhone application were performed by bicycling in -10 celcius degrees weather (north and so on…). No polar bears were harmed during the creation of this mobile service.

The service is linked in from Oulu Tourism pages and thousands of paper brochures printed for Oulu summer season 2009.

About the author Mikko Ohtamaa

Logging with LogMan

This is my first post on our company blog and I thought I’d tell you something about LogMan, which is developer’s utility for getting logging messages from Symbian device over a USB cable. It is written by me and mostly on my own time. I started the project because I had to do a Symbian excercise for university course and I thought I’d do something useful instead of quickly tinkering something small and easy.

LogMan supports both C++ and Python. With LogMan, you can send data to the same location from Python and C++ in real-time instead of using log file(s). Browsing through multiple log files can be tedious and you can’t see the debug output while using your application, because on Symbian you must read the file after the debugging session – Symbian cannot share opened files between applications. It is also possible that you create too much log and you run out of Phone internal memory. LogMan helps by removing the use of log files and you never run out of disk space because messages can be stored directly to PC. On simulator, the messages are also sent to RDebug (%TEMP%\epocwind.out). Surprisingly I have not seen a Python module, which would enable use of RDebug. Even though it is very easy to implement. With RDebug, there is no need to open a serial port on simulator for reading the logging messages.

Of course, I tried to use RDebug on device first, but I never got it working. I also tried REcmt, which is supported on S60 only and the service just kept on crashing on my phone. This is why I decided to write LogMan. Plus both are closed software, which effectively prevented me from fixing the problems.

Just wondering what kind of benefit Symbian or Nokia gets from keeping development tools such as these closed? What is there so secret about them? It didn’t take me very long to write the first working version. *sigh*

The use of LogMan is similar to RDebug. There are static class methods, which are a bit slower but easier, and instance methods. Check the project’s homepage for more examples.


#include "LogMan.h" //RLogMan
RLogMan::Log( _L("Hello world ") );

I recently added a new feature for LogMan, which allows you to log stack and heap usage of the current thread with one function. When you are unsure about your heap or stack usage, these might come handy. Of course there is some memory used when calling these so take that into consideration. Python can access MemoryInfo only, which logs both stack and heap (Well, I got a bit lazy at that point). There are equivalent macros for these, so check them out from “logmanutils.h”.


// Store this as a member of your class, for example
RLogMan logman;

logman.HeapInfo();
logman.StackInfo();
logman.MemoryInfo(); // Both stack and heap
logman.Close();

The output from MemoryInfo is something like this:

StackInfo
Free:1039000, Used:9576, Size:1048576
HeapInfo
Free:25856, Used:101004, Size:126860

Browser for PyS60

I have been trying, unsuccessfully, to get Browser Control working on PyS60. In a nutshell, CBrCtlInterface wrapper for Python. I have developed it against PyS60Community version in Launchpad. See /src/appui/appuifw/. I have used LogMan extensively to debug the extension so if you want a real example, check out “browsercontrol.cpp”.

Browser Control would allow one to embed a browser into his PyS60 application, which would be quite cool. No need to do user interfaces with “appuifw”, which is not very portable. With Browser Control, one could create his user interface with html and javascript, which are a lot more portable indeed. Less work leads to more time. And what is time?… it’s money. Or so I have heard. And being able to handle events with Python instead of C++ is another bonus.

Unfortunately, the API is not very stable as you can see by searching for “CBrCtlInterface” at Forum Nokia. The browser worked fine on the simulator with small pages such as “www.google.com”, but it crashed miserably with larger pages. The crash happens in browsercontrol.dll when calling e32.Ao_sleep() in Python. On device it was unable to open any page and crashed instantly when trying to load one. With LogMan, I was able to verify that the crash happened in e32.Ao_sleep() on device also. This reminds me to test on device all the time, which I didn’t do for the first versions. But this is why I added the memory logging feature to LogMan, but it only revealed that I was not out of stack or heap. At least not before the browser started to mess around.

I tried to compile the WebKit myself to see what is going on, but the build instructions didn’t work and the build scripts are written in Perl(my eyes started to hurt). What a mess. I don’t wonder anymore if there are bugs in browsercontrol.dll. I finally gave up because my idea pool dried up. Any help getting the wrapper working would be very much appreciated.

Plans for LogMan

I’m planning to add remote shell interface so that you could control your phone from PC. I want access to the file system first. Transfer files, list folders and such. If you have TCB rights ( or hacked phone ) you can speed up development remarkably by simply replacing your binaries in \sys\bin or your Python files with new versions. No need to install sis files and fiddle with certificates and play with memory cards. It would be so nice… I can use 1 day in a week for a personal project so this may happen in near future ;)

PyS60 application release build toolchain

A common question for Python for Series 60 newcomers is how to build standalone Symbian applications from Python source code. We have been using Makefile based toolchain internally. I describe it in this picture, I didn’t bother to add thumbnail for the image, since it’s a 3400 pixels wide diagram.

The diagram describes building a PyS60 application with some Python extensions (Symbian native C++) mixed in and bundling it all to one downloadable SIS file. The application will appear as any first class S60 application in the menu and the user does not know it’s running Python internally, besides bad installation experience (it challenges Microsoft installers with all those unnecessary yes/no questions), extra uninstaller entries and slow start-up time.

The biggest problems are caused by embedded SISs (SIS inside other SIS files) which are not treaded very wel by several Symbian parties.  In theory, it could be build one monolithic SIS, but you’d need to recompile PyS60 from scratch and patch UIDs inside it for your own UIDs received from symbiansigned.com. We are planning to explore SCons based build solution to address this problem, since Makefiles are a bit unflexible with tasks like PKG file and UID range generation.

Here is a PKG file example for final user distributable SIS file.

Also, see UIKludges project for additional details for PKG files of Python extensions.

You need to have

  • Ensymble tool
  • Series 60 SDK (contains some old GNU make)

You need to master

  • A build tool (make)
  • Symbian PKG file structure
  • Lots of different command line tools

Pros

  • It’s the best one we have for now

Cons

  • Symbian signing and certification companies don’t understand embedded SIS files (all SIS files must be signed prior embedding) and may have hard time signing SIS files containing only an extension DLL for Pyton. Symbian Signed test criteria has been built only UI application based SIS files in the mind.
  • You cannot cook your own patched PyS60 distribution without revamping some hardcoded UIDs and paths, since otherwise there are UID conflicts (EXE and DLL file UIDs are in Nokia’s protected range)
  • S60 installers askes extra confirmation for every embedded SIS file, even in the middle of the progress bar, so the user experience of installation is screwed up
  • There will extra uninstallation entry for every embedded SIS file in S60 application manager confusing the user
  • As you can see, most cons come from Symbian and Symbian signing limitations and have nothing to do with Python

Ps. I would have put this thing to wiki.opensource.nokia.com, but their webmaster email address is non-functional and one cannot upload images to their Wiki.