mFabrik News – download now for iPhone and Android

Why create a mobile application?

The first question is why one rather create a mobile application when the same task can be accomplished with a mobile site? Most people even prefer mobile sites over applications. From a pure engineering viewpoint, mobile applications are usually just glorified RSS readers that embed Webkit and add some native user interface bling bling over it. With an app, you are limiting your target audience, because an application is limited to one platform. Maintaining application(s) and application developers is more expensive compared to a mobile site which few (cheap) PHP junkies can throw together.

But is not always technology or price which matters. Mobile applications have prestige value – having or showing success, rank, wealth, etc. If you have a high quality brand, you probably want to have a mobile application too. When you see the brand logo swinging forth and back in an iPhone application with smooth animation running 60 frames per second, you see that it is a proper placement for the brand logo. The output is more luxury, more carefully planned, and does not look like it was thrown together by few cheap web developers.

There are even rationale reason for going after applications. First, you are in a business of making money. It is a lot of easier when the platform itself is offering you a payment solution without a monthly fees (iTunes payment). Other good reason is that there exists interaction between the application and your content beyond the browser window.  You can push messages or do things even if the user is not on your site (see more information about the push solution we implemented below).

Mobile application development and Python

As most of this post readers are probably fellow Python developers, here are some thoughts specifically aimed for them. Python itself is not a very good alternative what comes to mobile application development. Though, the application itself may not contain Python code, Python still shines on the backend side of the things. For example, we’ll hope to publish an example application using Google App Engine in the near future.

The only future proof platform where Python is 1st class citizen for building applications, is Nokia’s Meego with its Pyside and Qt bindings. Unfortunately Meego doesn’t have any shipped handsets and looks like it never will.

Android has script bindings, but they are not good enough for real application development, as interaction with the native platform happens over TCP/IP sockets. However, Android has seen some recent exciting development from PyPy project, possibly enabling native Android development for Python in the future.

iOS with Python could be a go, now when Apple has lift ban on interpreted languages. I haven’t heard anybody doing it yet, though. CTypes had some problems long time ago regarding run-time generated code for Python bindings.

Python has also a port for Series 60 (Symbian) – don’t go there if you are not prototyping. It is good platform for students for  playing around, but unfortunately it has never been considered as serious development environment by the handset manufacturer. You have tons of headaches if you actually want to release a product version of your application.  Nokia N900, soon supported. is better prototyping platform for Python than Series 60 as you get full Debian userland.

Mobile application development and wrappers

There exist various wrapper technologies which help you to wrap your HTML5 application to a native application shell. With simplistic APIs provided through Javascript bindings, you can access a limited subset of native platform APIs. Wrapper technologies are mostly aimed for web developers, who do not have any experience on application development and they might want to skip the learn experience of native development.

Wrapper technologies do their job and produce decent apps. But if you are a Python developer I recommend you skip the wrapper step and build your own native user interface and embed Webkit yourself. Designing an user interface is much is easier with Apple’s Interface Builder or Google’s  Android tools than with half-baked Javascript bindings. The fact that you are actually able to insert a real breakpoint into your code is itself worth of skipping wrappers. If you already are a Python developer you already know at least one real programming language and mastering Objective-C or Java should be an easy task for you.

Webkit itself has bugs. You will regularly hit obscrure bugs when the amount of  Javascript and CSS code grows. In the worst cases Webkit just dies under your application without a way to debug the problem – sometimes without a workaround available for the problem. This means dead end for your lovely application. You don’t want to end up to this situation. So, just to have more low level control, using native tools is good.

mFabrik News application

mFabrik News mobile application allows you to follow the latest news of mobile and web development, produced by our hacking team. The applications source the news from our Plone based web site and WordPress blog (which you are currently reading). It uses special RSS streams prepared with our Web and Mobile multichannel publishing solution: news images are optimized for mobile device screens using a handset database (Wurfl) and some other HTML preproessing is done to make the posts look better in embedded WebKit. Processing is done using mobile.sniffer and mobile.htmlprocessing Python packages which are generic Python packages and should be usable in various environments, including App Engine.

iOS mFabrik News application has push notification support. Android doesn’t yet implement push solution, but it is coming for Android 2.2 handsets.  Please see the earlier blog post how we use Apple Push Notifications with Python.

Download, give the apps a spin and report any feedback! (direct links at the beginning of the post)

We may or may not release the source code of the applications, depending if anybody thinks they actually would find it useful.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

PyS60 has also a history of staying in archaid Python version – namely Python 2.3. It was not until this February when stable PyS60 2.0.0 with Python 2.5 was released (1.9.x was considered experimental according to the release notes). Luckily looks like new winds are blowing (Qt acquisition, Meego/Maemo) and Python is getting higher priority. For example, PySide Qt bindings is very high profile project. Based on this, we hope to expect Python to the first class citizen in the future Meego and Symbian devices.

My company had a little side venture with PyS60 Community Edition when we were still betting that Symbian and Python would rock the world – the era before iPhone changed the game.  PyS60 community edition was effectively a revamped PyS60 1.4.x with Python 2.3 toolchain which actually made PyS60 application production deployment possible. Possible…? -you ask. Madness… no. It is Symbian. It is certification and signing and obscure error messages. Basically vanilla PyS60 is being shipped as an external SIS (Symbian package format) and Symbian platform security makes it impossible to deploy two production signed applications using vanilla PyS60 on the same device. The only cure was statically building Python for both apps from the scratch, which is exactly what PyS60 Community Edition was doing.

But this all was long long time ago. Aeons in mobile time. So I was today surprised when I got email from a person (David) using PyS60 Community Edition. We never upgraded PyS60 Community Edition to Python 2.5 . In fact we haven’t touched the project about two years. David was effectively using Python 2.3 and asked questions about the tool chain internals.

My first answer was a question Why on Earth you are still using Python 2.3? I thought maybe the guy had somehow missed the last two years or was a stuck with an old phone.

However… this was not the case and the answer was very insigtful.

Yes, I’m aware of PyS60 2.0.0, but I prefer PythonCommunity, at least for the moment: no OpenC neither Platform Services dependencies; smaller .SIS size and memory footprint. I think that the final .SIS produced with PythonCommunity, with everything necessary to run contained in it and with a clean installation without multiple dependencies, is a better fit for a mass-market than the files produced by PyS60 2.0.0, above all taking into account that people don’t know what S60 or Symbian are.

Also, the runtime deployment on the new PyS60 isn’t automatic for S60_3rd and S60_FP1 devices, so in the worst case scenario, users may end having to learn to install the different files (pips.sis, ssl.sis, stdioserver.sis, Python_2.0.0.sis, PythonScriptShell_2.0.0.sis) in the correct order, which is a big no-no for a mass-market deployment.

So…. I hope someone in Nokia is reading this blog entry carefully. Do it like Apple does. Make your application deployment static. Make OpenC static. Make every freaking library which is not shipped with the device statically buildable. It should be possible now when everything is open source. It will consume precious device RAM, but at least it will make mass market application development possible. SIS hell is worse hell than deb hell, or DLL hell, as the end user cannot fix it due to device security.

In the related news SIS smart installer was announced few weeks ago. Personally I wouldn’t bet it can deal with all the problems of versioning and Symbian platform security. Forum reports aren’t promising and looks like very Symbianish user experience can be expected. In positive light, it seems that Python is being considered for this process.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

• Image uploads
• Payments for subscribed content

Sounds easy, right? Well it isn’t… Below are some notes for our due diligence work which you fellow developers might find interesting.

SDKs

Mobile phone vendors are jealously and don’t want to co-operate with each other. Building application which works in all handsets is major headache.

We found some reasonable candidates for cross-platform mobile development doing HTML and Javascript. HTML and Javascript pages are converted to native application using a wrapper technology (a.k.a. appaccelerator). Doing Flash Lite or Java ME can be pretty much forgotten nowadays as they won’t run on the most hyped platform, iPhone. Flash Lite has poor support for anything except content authoring due to primitive and limited APIs. Java ME provides horrible user experience.

(X)HTML is the only common language spoken by mobile phones. Thus, there has been a rise of “appaccelerators”, technologies which allow to create mobile applications with HTML(5) and Javascript.

• Phonegap: iPhone, Android, Blackberry and possibly S60 in the future. Pluses: BSD license, very active community. Minuses: bad documentation, difficult deployment process.
• Titanium: iPhone, Android. Pluses: Professional, Apache license. Minuses: Too tightly coupled with Appacclerator Inc. company.
• Rhomobile: iPhone, Android, Blackberry, S60, Windows Mobile. Pluses: Professional, tries to build open source community, the widest platform support. Minuses: Dual licensing and tightly coupled with Rhomobile Inc.
• Nokia Web-runtime: Nokia S60 and some other Symbian based phones. Pluses: Professional, good documentation. Minuses: Not open source, impossible to extend, Nokia has little interest to make this cross-platform, Nokia doesn’t like updating old models and web-runtime is useable only in the latest S60 5th edition models.
• Palm Pre supports web applications natively. However Palm Pre application business is still taking a shape.

All these wrap the browser component (WebKit) and provide some extra Javascript APIs when your web pages as executed under the application mode.

• Locationing
• Contacts
• SMS
• Client-side database
• and so on…

Rhomobile has little different use cases  from the rest of the bunch as it provides client-side programming using Ruby and less focuses on Javascript/web applications.

Payments and in-application purchases

There are four major way to do mobile payments “inside” the application for bought content and subscriptions. The price tag on the application itself is left out on this discussion as the application stores themselves take care of it.

• Credit card
• SMS
• App Store payment (thus far Apple only)
• Direct operator payments – you have a service provider (Bango) which can directly charge items to the operator phone bill based on handset identification.

App Store payment is the most attractive as it provides the best end user experience.  It allows you to use App Store payment mechanism inside the application. It is safe and no need to hassle with external payment providers. However, App Store payment can be used only for content consumed directly inside the application. You cannot use it e.g. for ordering a pizza. I think this might be related to recent EU legislation forbidding SMS payments for services not consumed in the phone itself.

SMS payment is ok for little payments. Operators take big cut of the revenue, generally 30% – 70% depending on the country. Short code fees usually start from 500€ set-up fee + 500€ / month. SMS cannot be often send as a background, but the user is presented the normal SMS editor which reduces the user experience somehow.

For credit card payments there exists several providers. Credit card has the cheapest entry fees, but the downside is that the user needs to have the credit card. This excludes teenager audience.

Direct operator payments are not very well supported yet globally. Most western operators support them. The operator also takes a big share and the fixed fee is pretty high.

My favorite payment provider thus far is Bango which provides credit card payment starting 9€ / mo. and scales up to worldwide SMS payments which cost few grannies per month.

In most cases, the payment experience will not be smooth. You need to open the phone main browser on the payment provider page to do the payment. This usually will close your own application. Rarely you can do the payment inside the application and support multiple platforms. After doing the payment most platforms allow you to close the browse and reopen your application using a special URL handler.

Wikipedia mobile payments page is also useful.

Image upload

<input type=”file”> won’t work on iPhone and some other platforms as those don’t have user browsable file system. Also the file dialog usually doens’t have image preview making it useless.

Phonegap has a branch which supports images picking using iPhone’s own gallery browser.

In any case, there is not yet cross-platform solution for this.

Future prospects

In some time-frame we will get rid of the need to wrap HTML applications natively as the web browser applications will support all HTML5 features without extensions and probably have some proprietary extensions for mobile specific features like SMS. We already have had some taste for this:

• The first taste of this is Mozilla’s Fennec mobile browser which has locationing support.
• iPhone’s Safari already supports client-side storage and CSS3.
• All phones have support for dial-in links. The format of the link may vary. DeviceAtlas is good place to hunt for this information.
• Nokia browser supports send SMS links
• Nokia browser supports downloadable map markers (to the map application of the phone itself)

The route, which you can bicycle through is drawn on Google Maps. There are nine  action points where the user can listen to streaming audio clips, with still images, in his/her mobile phone. This is sort of  augmented reality experience: The user sees the real world (where he/she is now bicycling) combined with the historic events (audio playback narrative). For example, at Linnansaari (a location on the route) you’ll see the actual 17th century castle ruins and the narrator tells how the castle exploded when fire, caused by a lighting, reached gunpowder warehouse… boom. The explosion caused stones fly over 400 meters.

Alternatively, the clips are available as podcasts from Oulu Tourism pages. You can download them into your iPod for offline listening and use in conjuction with a paper map. This demostrates interesting mix of multichannel publishing: paper, web, mobile and podcasts.

The tour is bilingual in Finnish and English.

There exists unreleased iPhone application, based on PhoneGap, which allows the user to track his/her location real-time on the web page. We didn’t see it worth of trouble to go through Apple iPhone application review process. When location based service support comes for the browser this feature is indended to be included as the standard HTML5 feature of the service.

There also exists Nokia Series 60 mobile application, based on PyS60 and Series 60 BrowserControl API, which allows the user to track his/her location in real-time. The application provides wrapper around Series 60 WebKit control and allows Javascript to access phone native functions (GPS) over localhost socket communication. Like with Apple, we didn’t see real-time tracking feature interesting enough to go through Symbian Signed process to get our application released. Also, BrowserControl had seriousquality problems and we didn’t consider it stable enough for the end users. Some work is available in PyS60 Community Edition repository.

The service is hosted on Python specific virtual server on Twinapex services server farm.

Features

• Premium content tailored for audio listening
• Dubbed in English and Finnish by a professional voice actor
• Bilingual: English/Finnish
• Adapts for smartphones (WebKit based browsers) and low end phones (XHTML mobile profile browsers)
• Streaming video and audio (RTSP / progressive HTTP download forv iPhone). Different audio quality is provided on depending on the handset features.
• Screen resolution detection based on user agent sniffing. Three different version of images are used.
• Custom Google Maps component for mobile is used. The component adapts for different mobile phones based on sniffing. Features include zoom, show action point, show the current location, search street address name. This component can be published on a request.
• Management interface features include video upload, video transcoding different mobile versions and editing bilingual content
• Apex Vertex handset database is used to detect the user’s mobile phone capabilities
• Apex Vertex logging and traffic analytics capabilities are used for the site statistics

Software stack

• Ubuntu 8.04 Hardy Heron virtual server
• Apache 2.2 / mod_python
• Python 2.5
• Django 1.0
• Django-CMS 1.0
• mobile.sniffer Python package to provide abstraction over different handset databases
• Apex Vertex streaming solution (RTSP based on Darwin streaming server by Apple)
• TinyMCE WYSIWYG editor
• Darwin streaming server
• ExtJS is extensively used in Apex Vertext management interface

Development effort

Development time: Around 100 hours. Three different developers where involved. Used development tools: Eclipse, PyDev, Subclipse, Subversion. There were around five meetings between the content provider and the technology provider. Few beta testing rounds using iPhone application were performed by bicycling in -10 celcius degrees weather (north and so on…). No polar bears were harmed during the creation of this mobile service.

The service is linked in from Oulu Tourism pages and thousands of paper brochures printed for Oulu summer season 2009.

About the author Mikko Ohtamaa

• LinkedIn
• Twitter

Check our announcement at

http://discussion.forum.nokia.com/forum/showthread.php?t=152969

planetmobile.us was managed by Christopher Schmidt until I resurrected it few weeks ago.

Please feel free to subscribe your feed to the planet!

Ubuntu does not distribute proprietary codes. First thing you need to do is to rebuild ffmpeg.  Since Ubuntu 8.04 Hardy Heron ships with ffmpeg from 2007, which is aeons old in video codec years, you need to build libx264 and ffmpeg from SVN sources. Here are detailed, valid, instructions. Note that FFMPEG trunk is not currently stable (September 2008), so you need to use revision 15261 which needs this little patch. Indeed, this is a very difficult month to start your career in the dark world of video encoders.

To make it legal and support open source codec development,  please pay for your codecs.

Then we use this guide by Robert Swain. We have a tiny sub 2,4″ screen, we do not care about the quality and do one pass encoding. By empirical research, I have found that the following MPEG-4 profile parameters are compatible with N95 8 GB and provide the optimal result. You can vary video and audio bitrate depending on your taste.

Here is a script which recursivelu encodes all detected video files suitable for mobile format:

#!/bin/sh
#
# Optimal movie encoding for Nokia N-series mobile phones
#
# Copyright 2008 Red Innovation Ltd.
#
# Say hi if you find this useful.
# We do some professional mobile video publishing, so if you
# need a helping hand please call us.
#
# Usage: Run encode.sh in any folder and all video files are recursively converted to mobile phone suitable format
#
# Note: We expect all the source material be in 16:9 aspect ration
#
# Also see http://www.nseries.com/index.html#l=support,search,faq,general,video%20encoding,53848
#

VIDEO_BITRATE=300k

AUDIO_BITRATE=72k

# Assume locally build ffmpeg + x264 in /usr/local/bin
# http://ubuntuforums.org/showthread.php?t=786095
export LD_LIBRARY_PATH=/usr/local/lib

# Search all source AVI, MPG and WMV video files
# Place all encoded files to the same folder with the source, with added .mp4 extension
find . -iname "*.avi" -or -iname "*.wmv" -or -iname "*.mpg" | while read src ; do
        srcfile=`basename "$src"`
	srcfolder=`dirname "$src"`
	dstfile="$srcfolder"/"$srcfile".mp4

	# The magical string!
	# Size and cropping is for 16:9 source material, so that 320:240 display will have black bars.
	# Fex pixels off... note that h264 sizes must be multiplies of 16, use 256x144 for streaming
	# N95 RealMedia player does not seem to respect MPEG-4 embedded aspect ration info.
	/usr/local/bin/ffmpeg -y -i "$srcfile" -acodec libfaac -ab $AUDIO_BITRATE -s 320x176 -aspect 16:9 -vcodec libx264 -b $VIDEO_BITRATE -qcomp 0.6 -qmin 16 -qmax 51 -qdiff 4 -flags +loop -cmp +chroma -subq 7 -refs 6 -g 250 -keyint_min 25 -rc_eq 'blurCplx^(1-qComp)' -sc_threshold 40 -me_range 12 -i_qfactor 0.71 -directpred 3 "$dstfile"

done

Pythor for Series 60 Community Edition is a new open source effort to push Python for high quality mobile phone development. It aims to provide a maintained software stack for creating real mobile applications. The codebase is derived from the original Nokia’s Python for Series 60 codebase, but has been refactored for better integration with third party extensions and patches and commercial grade application deployment.

Motivation

Building and distribution

It is difficult to distribute Python for Series 60 applications to the end user with the current Nokia’s PyS60 distribution. You probably want to modify or extend PyS60 in some way. Since the build chain and deployment model is not designed for changes this would collide with the other PyS60 installations. Symbian Platform security prevents installing conflicting binaries. Thus, one can effectively have one Nokia PyS60 application in the phone once.

There are other problems: Nokia PyS60 distribution has UIDs in Nokia protected range. Embedded SIS file cause extra installation dialog and an application manager uninstall entry. Trimming down Nokia PyS60 distribution is a little bit difficult.

To overcome all these issues we created a build chain which spits out monolithic PyS60 distributions. We build only one DLL whose name and UID can be decided. Also the build chain is Scons for Symbian, scrapping the obscure, inflexible and difficult to understand Symbian ABLD once for all.

Evolution towards higher quality

Currently there is no centralized authority to co-ordinate PyS60 open source developers and maintain the repository of all the extension and patches. This effectively prevents the biggest benefit of open source: open innovation and gradual evolution of the product. It would be very nice having all those third party extensions, now scattered around the internet, under one maintained source – making PyS60 more functional out of the box.

The community maintained repositories do not have the same restrictions as ones managed by a big public corporations. It is not a probable target of a trigger happy lawyer action and ungentlemanly competition: the discussion and plans can be public and due dilugence check of the code more relax.

We started the project in the Launchpad. Launchpad provide a distributed version control system (Bazaar) which streamlines the process of integrating third party commits and patches. This should encourage contribution. The standard build system makes it easy to roll out applications and extensions from bare  C++ source up to the end user distributable SIS files.

It is yet to see what kind of co-operation possibilities between the community and Nokia exists. In the future, it should be possible to cherry pick patches from PyS60 community edition to Nokia’s own version.

Showing the commercial potential of PyS60 in the mobile application development

On Python you can write native Series 60 applications with very little effort compared to hardcore C++ banging, lowering the barries to enter the mobile application development.

We do not deny that we have an extrinct motivation called money. Of course we have also instrict motivations like thinking Python is the best programming language in the world and we all want to be most respectable gurus in it. Gurus need to eat still, though. We hope that our effort does not go unnoticed in the mobile application development world and good subcontract offers fill our inboxes.

Also, there is the John McClane effect. Unless we had done it, no one had. Somebody has to save the world, despite the hangover.

It runs on Linux

Since we are no longer dependend on .BAT/Perl/Windows hindered ABLD buildchain, we can (almost) crosscompile and build native Symbian binaries in Linux and  Python applications. All good hackers use Linux – but currently there are still kinks and you need to use WINE for some parts – all sane Symbian developers are tied to Windows based tools for now and so are these instructions.

Prerequisitements

You need all this stuff to get things running.

Install Bazaar

You need Bazaar distributed version control client. We are not planning to have fixed releases for Python for Series 60 community edition any time soon. This is because 1) the most magic happens at a compiler level and we are providing a buildchain 2) we hope this fosters incoming patches.

• Go to http://bazaar-vcs.org/Download

Why Bazaar?

• Distributed (patching made easy)
• Works in Windows
• User friendly
• Integrates with Launcpad.

Install ActiveState Perl

Series 60 SDK has ActiveState as a prerequisitement for running its installer.

http://www.activestate.com/Products/activeperl/index.mhtml

Install Series 60 SDK

Use only Series 60 3.0 maintenance release. Other releases have SDK bugs preventing correct Python compilation.

Get the Windows installer from http://forum.nokia.com. Forum Nokia Registration is required.

Please use the default installation location C:\Symbian\9.1\S603rd_MR.

Install Carbide.c++ express

Carbide.c++ comes with a Windows compiler to compile the emulator binaries. You need this only if you indend to develop and test your applications on Series 60 emulator.

http://www.forum.nokia.com/info/sw.nokia.com/id/dbb8841d-832c-43a6-be13-f78119a2b4cb.html

Forum Nokia Registration is required.

Use Software updater in Carbide.c++ to install PyDev, Python developer extensions for Eclipse.

http://pydev.sourceforge.net/

Install Python 2.5

Scons build chain and our utility scripts use Python.

http://www.python.org/download/releases/2.5.2/

Use the installer EXE and the default installation location C:\Python25. If you want to use advanced Bluetooth shell (PUTools) you also need wxPython and pyserial packages.

Install SCons

Python for Series 60 build script are based on SCons. It is a build system using Python as a recipe langauge.

http://sourceforge.net/project/showfiles.php?group_id=30337

Install Subversion

Install Subversion client for Windows. This is needed for checking out Scons for Series 60.

http://www.collab.net/downloads/subversion/

Registration to CollabNet is needed to download Windows binaries.

Scons for Series 60

SCons for Series 60 is available as a separate project. SCons for Symbian is a build toolchain intended as a replacement for Perl and MMP files used on regular Symbian projects. SCons for Symbian is not limited to build Python – You can use it to build any Series 60 C++ application.

http://code.google.com/p/scons-for-symbian/

This is later checked out during to the environment construction, so you do not need to install it now.

We have included a workaround for a problem with limited command line length on Windows.

Included tools

The following tools are included in the trunk tools folder:

• OpenSSL
• Ensymble
• SISInfo
• COG

These tools are not licensed under Apache license. Some of them are under GPL license. However, we believe that distribution them is ok, since this falls under GPL’s mere aggregation clause. However if you indent to distribute commercial applications built from PyS60 Community codebase, make sure that you understand the set of different licenses involved.

Set up build environment

Ensure that Bazaar is properly in your Windows path.

Create a workspace folder

First you need to subst (make a folder appear as a driver letter) in Windows. Open command line. Go to SDK folder.

C:\Symbian\9.1>subst t: S60_3rd_MR

T:

mkdir workspace

Now choose this folder as a workspace folder in Carbide C++ and create an Empty Symbian C++ project called ”pys60” there.

Checkout PyS60 community edition

The go to this folder

T:
cd workspace\pys60
bzr branch lp:pys60community
cd pys60community\src

Preparing the build

This needs to be done only once.

We need to pacth the existing Series 60 SDK headers which have some bugs.

T:
cd \epoc32\include
\workspace\pys60\pys60community\src\tools\patch.exe -p1 < \workspace\pys60\pys60community\src\pys60-fix-3rded-sdk.diff

EPOCROOT must be set for some Series 60 SDK tools to work. We point to T: drive root.

T:
cd workspace\pys60\src
set EPOCROOT=\

As we still have some dependencies to the legacy system, one needs to configure the build system using PyS60 setup. This will generate some files and defines for Series 60 versio 3.0.

c:\Python25\python.exe setup.py configure 30

Do not run bldmake bldfiles.

You need to convert legacy MMP build files to SCons based. First we need to possibly fix up PATH, since Carbide C++ might break it.

set PATH=c:\program files\bazaar;c:\program files\CSL Arm Toolchain\arm-none-symbian elf\bin;c:\program files\CSL Arm Toolchain\libexec\gcc\arm-none-symbianelf\3.4.3;C:\program files\CSL Arm Toolchain\bin;t:\epoc32\gcc\bin;t:\epoc32\tools;t:\epo c32\tools;C:\program files\CSL Arm Toolchain\bin;C:\Program Files\Common Files\Symbian\Tools;C:\Perl\site\bin;C:\Perl\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem

c:\Python25\python tools\mmp2scons.py
ext\\calendar\\calendar.mmp:60:7: warning: no newline at end of file
Creating recipe ext\miso\build.py
Creating recipe ext\socket\build.py
...
Done!

Checkout SCons for Symbian. We assume it lives in src tree.

"c:\Program Files\CollabNet Subversion"\svn checkout http://scons-for-symbian.googlecode.com/svn/trunk/ scons_symbian

Running the build

Now we can execute the actual Python build script. This will create one monolithic emulator DLL which has almost all the PyS60 extensions built in – some extensions need manual building, since they rely on headers not found from standard Series 60 SDK.  You might need to reset PATH to default to Carbide C++ after the previous mmp2cons step by reopening the console window.

c:\Python25\Scripts\scons

You should see the following output. As you can see, UIDs are being allocated dynamically as instructed in Scontruct UID_BASE argument.

scons: Reading SConscript files ...
EPOCROOT=\
Info: SIS creation disabled
Building winscw udeb
Defines []
Getting dependencies for e32socket.pyd
Getting dependencies for _topwindow.pyd
Getting dependencies for zlib.pyd
Getting dependencies for _locationacq.pyd
Getting dependencies for _location.pyd
Getting dependencies for _graphics.pyd
Getting dependencies for _sysinfo.pyd
Getting dependencies for Python222Config.lib
Getting dependencies for Python222.dll
Getting dependencies for Python_appui.dll
Allocated UID:0xE1000000
Getting dependencies for Python.exe
Allocated UID:0xE1000001
Getting dependencies for Python_launcher.exe
Allocated UID:0xE1000002
scons: done reading SConscript files.
scons: Building targets ...

...

scons: done building targets.

Now you should be able to start a custom built Python shell in the emulator. You should see the following applications in Installation menu: helloworld, btconsole and filebrowser. Try launch helloworld and if it opens a pop up dialog the build has been succesful.

Building a SIS file for mobile phones

To build a target distribution type:

scons release=urel compiler=gcce dosis=true

This should yield to the result:

scons: Building targets ...
ensymble(["MyPythonForSymbian.sis"], [])
scons: warning: no package version given, using 1.0.0
scons: warning: no certificate given, using insecure built-in one
scons: done building targets.

If you want to build a SIS file signed with your developer certificate:

c:\python25\scripts\scons compiler=gcce release=urel dosis=true simplesis={'--privkey':'C:\\Certificates\\PrivateKeyNoPassphrase.pem','--cert':'C:\\Certificates\\MyApp.cer','--passphrase':''}

For now, installing the SIS file works only for C drive  – we’ll fix this little issue soon.

If you do not have a symbiansigned.com developer certificate you can sign the SIS file online for one phone (one IMEI code).

1. Go to symbiansigned.com
2. Register
3. Enter your IMEI and upload the SIS file to OpenSigned Beta

Building your own application

The purpose of this project is to make rolling out your PyS60 applications possible – so here we go. Currently we make a quite bad assumption that all the application live in the same source tree with PyS60 community edition due to problems with absolute file paths with Symbian build tools – we will figure out a long term solution for this later.

PyS60 applications are stub Symbian executables which boostrap Python virtual machine and start the Python code execution. Executables are linked against a custom PyS60 DLL and they are restricted by capabilities given to the EXE file.

PyS60 applications live in applications source tree. The source tree comes with Bluetooth console, Hello world and File browser sample applications.

Scons build scripts takes as applications parameter a comma separated list which applications are included in the build.

scons builtin=all applications=helloworld,filebrowser

Applications consist of

• default.py boostrapper module
• icon.svg SVGT icon
• Any number of Python modules (.py) files

See applications\helloworld folder to examine what files are needed to build an application. All application Python modules go to the private application folder (/private/myapplicationuid). Default.py must boostrap PYTHONPATH (sys.path) for this folder – PYTHONPATH defines where Python interpreter looks for the code. Application UIDs can be chosen manually or they are picked automatically by Scons for the unprotected test range.

Note that Python Script Shell application is handled out of this flow due to its legacy heritage.

Adding in your own extension

If you have development an PyS60 extension you can drop in into the buildchain easily. Each extension is defined in ext subfolder. It consists of necessary CPP, H and Python files. The building structure is defined in build.py using SConstruct command PyS60Extension().

Build.py files can be automatically generated from legacy code using mmp2scons.py converter.

• Drop files to ext/yourmodule folder.

• Run MMP -> build.py. This needs GCC or valid C compiler for preprocssing MMP files.

c:\Python25\python.exe tools\mmp2scons.py

ext\calendar\calendar.mmp:60:7: warning: no newline at end of file
ext\progressnotes\progressnotes.mmp:38:7: warning: no newline at end of file
ext\uikludges\uikludges.mmp:37:7: warning: no newline at end of file
Creating recipe ext\\socket\build.py
Creating recipe ext\\glcanvas\build.py
Creating recipe ext\\graphics\build.py
...
...Done!

• Recompile. New ext modules are automatically picked as built in modules.

c:\Python25\scripts\scons

• Now test importing your module in the script shell in emulator

>> import applicationmanager

If your extension is using thread local storage (Dll::Tls()) you might need to figure out how to workaround with it. See socket  module for example. You may also need to play around with the init function of the Python extension – it must be init + module name.

Developing on target

If you want to develop your application on a mobile phone, you do not need to go through the full development cycle for every little change. It is possible to update Python files on a phone without SIS installation. You can either automatically synchronize changed files from your PC to Phone (the example below) or you can edit files in-place on the Phone either using PCSuite or Series 60 SMB server.

Here are short instructions how to update files using PUTools console (btconsole). PUTools is wxPython based remote Python shell which allows you to run Python console commands over a Bluetooth connection from your PC. PUTools also has a file syncrhonization feature – after editing source code on the PC changes are reflected automatically to the phone.

1. Put application Python files to a shared folder on the phone e.g. the memory card root (E:\). Add startup.py to E:\ which will modify sys.path to include your files.
2. Add incoming Bluetooth serial port on windows (Control panel -> Bluetooth -> COM ports)
3. Start Bluetooth shell on the computer (tools\putools\pcfiles\console.bat)

T:\workspace\pys60\pys60community\src\tools\putools\pcfiles>c:\python25\python.exe putools.py com5

4. Start Bluetooth shell on the phone (btconsole icon)
   
5. Edit PUTools sync.config file on PC and run sync command on Bluetooth terminal to update changes made on PC to phone
   
6. Run application launcher in the console (depends on the application structure how it is best to bootstrap in the shell):

>> import filebrowser
>> filebrowser.FileBrowser.run()

Release notes

Here is the short summary of differences with the current PyS60 community edition and one available from Nokia. This information is also available in divergence.txt file in the source folder.

2008-08-29 Mikko Ohtamaa <mikko@redinnovation.com>

    * PyS60 general

        New build chain and static config generation

        Migration tool for MMP -> Scons based extensions

        Added several tools included in the core distribution: sisinfo, ensymble, cog, patch

        Patched py2sis tool

        Contains extension: applicationmanager

        Contains extension: uikludges

        Contains extension: progressnotes

        Contains extension: miso

        Contains application: Bluetooth shell

        Contains example applications: filebrowser, helloworld

        Changed Bluetooth console bootstrap to e:\startup.py         

2008-08-15 Antti Haapala <antti@redinnovation.com>

    * e32socketmodule.cpp:

        socket.access_points has more information, two new
        fields is given per access point: isptype and bearertype,
        whose values are integers corresponding to values returned
        by CApSelect::Type and CApSelect::BearerType respectively.
        No symbolic constants are yet exported.

    * appuifwmodule.cpp:

        multi_select_list has a new argument, selected, which defaults
        to None. Given a list of integers, the items with the given indices
        are initially selected.

Conclusion

We hope this helps you to get started with PyS60 community edition. It’s still a bit complicated, since setting up the build environment on Windows is a such a pain. In the future, when the Linux based build system is reading settings up the development environment should be easier – all those boring steps happen automatically.

This might be still too difficult for some of the readers, since a lot of prerequirement work must be done before anything useful can be done. Feel free to comment the article in this blog, but we hope that you use Answers section in Launchpad to ask help and technical questions related to PyS60 community edition.

LogMan supports both C++ and Python. With LogMan, you can send data to the same location from Python and C++ in real-time instead of using log file(s). Browsing through multiple log files can be tedious and you can’t see the debug output while using your application, because on Symbian you must read the file after the debugging session – Symbian cannot share opened files between applications. It is also possible that you create too much log and you run out of Phone internal memory. LogMan helps by removing the use of log files and you never run out of disk space because messages can be stored directly to PC. On simulator, the messages are also sent to RDebug (%TEMP%\epocwind.out). Surprisingly I have not seen a Python module, which would enable use of RDebug. Even though it is very easy to implement. With RDebug, there is no need to open a serial port on simulator for reading the logging messages.

Of course, I tried to use RDebug on device first, but I never got it working. I also tried REcmt, which is supported on S60 only and the service just kept on crashing on my phone. This is why I decided to write LogMan. Plus both are closed software, which effectively prevented me from fixing the problems.

Just wondering what kind of benefit Symbian or Nokia gets from keeping development tools such as these closed? What is there so secret about them? It didn’t take me very long to write the first working version. *sigh*

The use of LogMan is similar to RDebug. There are static class methods, which are a bit slower but easier, and instance methods. Check the project’s homepage for more examples.


#include "LogMan.h" //RLogMan
RLogMan::Log( _L("Hello world ") );


I recently added a new feature for LogMan, which allows you to log stack and heap usage of the current thread with one function. When you are unsure about your heap or stack usage, these might come handy. Of course there is some memory used when calling these so take that into consideration. Python can access MemoryInfo only, which logs both stack and heap (Well, I got a bit lazy at that point). There are equivalent macros for these, so check them out from “logmanutils.h”.


// Store this as a member of your class, for example
RLogMan logman;

logman.HeapInfo();
logman.StackInfo();
logman.MemoryInfo(); // Both stack and heap
logman.Close();

The output from MemoryInfo is something like this:

StackInfo
Free:1039000, Used:9576, Size:1048576
HeapInfo
Free:25856, Used:101004, Size:126860


Browser for PyS60

I have been trying, unsuccessfully, to get Browser Control working on PyS60. In a nutshell, CBrCtlInterface wrapper for Python. I have developed it against PyS60Community version in Launchpad. See /src/appui/appuifw/. I have used LogMan extensively to debug the extension so if you want a real example, check out “browsercontrol.cpp”.

Browser Control would allow one to embed a browser into his PyS60 application, which would be quite cool. No need to do user interfaces with “appuifw”, which is not very portable. With Browser Control, one could create his user interface with html and javascript, which are a lot more portable indeed. Less work leads to more time. And what is time?… it’s money. Or so I have heard. And being able to handle events with Python instead of C++ is another bonus.

Unfortunately, the API is not very stable as you can see by searching for “CBrCtlInterface” at Forum Nokia. The browser worked fine on the simulator with small pages such as “www.google.com”, but it crashed miserably with larger pages. The crash happens in browsercontrol.dll when calling e32.Ao_sleep() in Python. On device it was unable to open any page and crashed instantly when trying to load one. With LogMan, I was able to verify that the crash happened in e32.Ao_sleep() on device also. This reminds me to test on device all the time, which I didn’t do for the first versions. But this is why I added the memory logging feature to LogMan, but it only revealed that I was not out of stack or heap. At least not before the browser started to mess around.

I tried to compile the WebKit myself to see what is going on, but the build instructions didn’t work and the build scripts are written in Perl(my eyes started to hurt). What a mess. I don’t wonder anymore if there are bugs in browsercontrol.dll. I finally gave up because my idea pool dried up. Any help getting the wrapper working would be very much appreciated.

Plans for LogMan

I’m planning to add remote shell interface so that you could control your phone from PC. I want access to the file system first. Transfer files, list folders and such. If you have TCB rights ( or hacked phone ) you can speed up development remarkably by simply replacing your binaries in \sys\bin or your Python files with new versions. No need to install sis files and fiddle with certificates and play with memory cards. It would be so nice… I can use 1 day in a week for a personal project so this may happen in near future

The diagram describes building a PyS60 application with some Python extensions (Symbian native C++) mixed in and bundling it all to one downloadable SIS file. The application will appear as any first class S60 application in the menu and the user does not know it’s running Python internally, besides bad installation experience (it challenges Microsoft installers with all those unnecessary yes/no questions), extra uninstaller entries and slow start-up time.

The biggest problems are caused by embedded SISs (SIS inside other SIS files) which are not treaded very wel by several Symbian parties.  In theory, it could be build one monolithic SIS, but you’d need to recompile PyS60 from scratch and patch UIDs inside it for your own UIDs received from symbiansigned.com. We are planning to explore SCons based build solution to address this problem, since Makefiles are a bit unflexible with tasks like PKG file and UID range generation.

Here is a PKG file example for final user distributable SIS file.

Also, see UIKludges project for additional details for PKG files of Python extensions.

You need to have

• Ensymble tool
• Series 60 SDK (contains some old GNU make)

You need to master

• A build tool (make)
• Symbian PKG file structure
• Lots of different command line tools

Pros

• It’s the best one we have for now

Cons

• Symbian signing and certification companies don’t understand embedded SIS files (all SIS files must be signed prior embedding) and may have hard time signing SIS files containing only an extension DLL for Pyton. Symbian Signed test criteria has been built only UI application based SIS files in the mind.
• You cannot cook your own patched PyS60 distribution without revamping some hardcoded UIDs and paths, since otherwise there are UID conflicts (EXE and DLL file UIDs are in Nokia’s protected range)
• S60 installers askes extra confirmation for every embedded SIS file, even in the middle of the progress bar, so the user experience of installation is screwed up
• There will extra uninstallation entry for every embedded SIS file in S60 application manager confusing the user
• As you can see, most cons come from Symbian and Symbian signing limitations and have nothing to do with Python

Ps. I would have put this thing to wiki.opensource.nokia.com, but their webmaster email address is non-functional and one cannot upload images to their Wiki.