| Testing if hostname is numeric IPv4Posted on August 31, 2010 by Mikko OhtamaaFiled Under iphone, mobile, python, technology I had to resort this hack when testing a hybrid web/mobile site which uses site hostname based device discrimination. In production mode we can have m.yoursite.com and www.yoursite.com hostnames. However, when running the site locally, on your development computer and in LAN this does not work very well: one cannot spoof hostnames for web browsers in devices like iPhone/iPod/other mobile phone unless you install a DNS server. And installing a DNS server for LAN is something you don’t want to do… So, I figured out that I can use hostname spoofing on desktop computers (/etc/hosts file) and I always access the site via numeric IP (IPv4 over ethernet) when testing over WLAN on mobile devices.
And,… dadaa,… here is my magical code to test whether hostname is numeric IPv4. I couldn’t find a ready function from Python standard library import re
ipv4_regex_source = "^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$"
ipv4_regex = re.compile(ipv4_regex_source)
def is_numeric_ipv4(str):
"""
http://answers.oreilly.com/topic/318-how-to-match-ipv4-addresses-with-regular-expressions/
@param str: Hostname as a string.
@return: True if the given string is numeric IPv4 address
"""
# ^(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$
return ipv4_regex.match(str)
Open source contribution agreement templatePosted on August 24, 2010 by Mikko OhtamaaFiled Under open source, phonegap, python, technology We are looking for creating contribution agreements for few new open source projects. IANAL, but hiring a real lawyer is freaking expensive. The thing is that we, us a company, want to guarantee that all code coming into the project is “clean”. We also want to guarantee our right to change the license in the future (GPL -> BSD, GPL -> Apache, etc.) Thus far, the best free, as in freedom and in beer, contribution agreement template we have found is Sun Contribution Agreement 1.5 which is available under Creative Commons Attribution-Share Alike 3.0 license. It is at least used by high profile Phonegap project (Nitobi as the company) if you don’t count OpenSolaris anymore as open source project. IANAL, but if I understood correctly, the agreement basically says
Since I couldn’t find the orignal document in editable form (PDF was the best I could get) I made OpenOffice.org ODS document out of it with easily replaceable identification information. Comments welcome. The agreement text pasted below. YOURPROJECT Contributor Agreement
These terms apply to your contribution of materials to the YOURCOMPANY ("us"/"our"), and set out the intellectual property rights you grant to us in the contributed materials. If this contribution is on behalf of a company, the term "you" will also mean the company you identify below. If you agree to be bound by these terms, fill in the information requested below and provide your signature.
Read this agreement carefully before signing.
1. The term "contribution" means any source code, object code, patch, tool, sample, graphic, specification, manual, documentation, or any other material posted or submitted by you to the project.
2. With respect to any worldwide copyrights, or copyright applications and registrations, in your contribution:
you assign to us joint ownership through this document, and to the extent that such assignment is or becomes invalid, ineffective or unenforceable, through this document you grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge, royalty-free, unrestricted license to exercise all rights under those copyrights. This includes, at our option, the right to sublicense these same rights to third parties through multiple levels of sublicensees or other licensing arrangements;
you agree that each of us can do all things in relation to your contribution as if each of us were the sole owners, and if one of us makes a derivative work of your contribution, the one who makes the derivative work (or has it made) will be the sole owner of that derivative work;
you agree that you will not assert any moral rights in your contribution against us, our licensees or transferees;
you agree that we may register a copyright in your contribution and exercise all ownership rights associated with it; and
you agree that neither of us has any duty to consult with, obtain the consent of, pay, or give an accounting to the other for any use or distribution of your contribution.
3. With respect to any patents you own, or that you can license without payment to any third party, through this document you grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge, royalty-free license to:
make, have made, use, sell, offer to sell, import, and otherwise transfer your contribution in whole or in part, alone or in combination with or included in any product, work or materials arising out of the project to which your contribution was submitted, and
at our option, to sublicense these same rights to third parties through multiple levels of sublicensees or other licensing arrangements.
4. Except as set out above, you keep all right, title, and interest in your contribution. The rights that you grant to us under these terms are effective on the date you first submitted a contribution to us, even if your submission took place before the date you sign these terms. Any contribution we make available under any license will also be made available under a Free Culture (as defined by http://freedomdefined.org) or Free Software/Open Source licence (as defined and approved by the Free Software Foundation or the Open Source Initiative).
5. With respect to your contribution, you represent that it is an original work and that you can legally grant the rights set out in these terms;
it does not to the best of your knowledge violate any third party's copyrights, trademarks, patents, or other intellectual property rights; and
you are authorized to sign this contract on behalf of your company (if identified below).
6. The place of performance is the registered seat of
YOURCOMPANYNAME
YOURCOMPANYADDRESS1
YOURCOMPANYADDRESS2
YOURCOUNTRY
YOURCOMPANYBUSINESSID
Any disputes concerning this agreement including the issue of its valid conclusion and its pre and past contractual effects are exclusively decided by the competent court in YOURHOMECITY, YOURCOUNTRY or, at our discretion, also by the competent court is whose district you may have your residence, your registered seat, an establishment or assets.
If available, please list your YOURPROJECT username(s) for the YOURPROJECT systems.
Username(s): __________________________________________________________________
_______________________________________________________________________________
Your contact information (Please print clearly)
Your name: ____________________________________________________________________
Your company's name (if applicable): __________________________________________
Mailing address: ______________________________________________________________
Telephone, Fax and Email: _____________________________________________________
Your signature: _______________________________________________________________
Date: _________________________________________________________________________
To complete this agreement:
email a scanned copy of a signed agreement to
fax a signed copy to + .....; or
post a signed copy to:
YOURCOPMANYNAME
YOURCOMPANYADDRESS1
YOURCOMPANYADDRESS2
YOURCOUNTRY
This agreement is based on version 1.5 of the Sun Contributor Agreement, which
can be found at:
http://www.sun.com/software/opensource/contributor_agreement.jsp
This document is licensed under a Creative Commons Attribution-Share Alike 3.0
Unported License http://creativecommons.org/licenses/by-sa/3.0
Python 2.3, Python for Series 60 1.4.5 still in use… and insight into Symbian deployment process and user experiencePosted on August 18, 2010 by Mikko OhtamaaFiled Under nokia, pys60, python, series 60, technology Nokia’s Python for Series 60 has a long history. It is a Python interpreter, originally escaped from Nokia prototype labs, running in your phone. It is said to been awesome to show mobile/embedded developers, who were love with their static C compilers and 4 hours built times, opening a Python prompt in your phone and typing import audio; audio.say(“your phone loves Python”) by keypad (Nokia Series 60 phones come with a speech synthetizer). Python for Series 60 is the best tool of building a simple proof of concept mobile applications. The lack of speed, lack of good UI libraries and difficult deployment problems makes it challenging to use it in production grade environments. PyS60 has also a history of staying in archaid Python version – namely Python 2.3. It was not until this February when stable PyS60 2.0.0 with Python 2.5 was released (1.9.x was considered experimental according to the release notes). Luckily looks like new winds are blowing (Qt acquisition, Meego/Maemo) and Python is getting higher priority. For example, PySide Qt bindings is very high profile project. Based on this, we hope to expect Python to the first class citizen in the future Meego and Symbian devices. My company had a little side venture with PyS60 Community Edition when we were still betting that Symbian and Python would rock the world – the era before iPhone changed the game. PyS60 community edition was effectively a revamped PyS60 1.4.x with Python 2.3 toolchain which actually made PyS60 application production deployment possible. Possible…? -you ask. Madness… no. It is Symbian. It is certification and signing and obscure error messages. Basically vanilla PyS60 is being shipped as an external SIS (Symbian package format) and Symbian platform security makes it impossible to deploy two production signed applications using vanilla PyS60 on the same device. The only cure was statically building Python for both apps from the scratch, which is exactly what PyS60 Community Edition was doing. But this all was long long time ago. Aeons in mobile time. So I was today surprised when I got email from a person (David) using PyS60 Community Edition. We never upgraded PyS60 Community Edition to Python 2.5 . In fact we haven’t touched the project about two years. David was effectively using Python 2.3 and asked questions about the tool chain internals. My first answer was a question Why on Earth you are still using Python 2.3? I thought maybe the guy had somehow missed the last two years or was a stuck with an old phone. However… this was not the case and the answer was very insigtful. Yes, I’m aware of PyS60 2.0.0, but I prefer PythonCommunity, at least for the moment: no OpenC neither Platform Services dependencies; smaller .SIS size and memory footprint. I think that the final .SIS produced with PythonCommunity, with everything necessary to run contained in it and with a clean installation without multiple dependencies, is a better fit for a mass-market than the files produced by PyS60 2.0.0, above all taking into account that people don’t know what S60 or Symbian are. Also, the runtime deployment on the new PyS60 isn’t automatic for S60_3rd and S60_FP1 devices, so in the worst case scenario, users may end having to learn to install the different files (pips.sis, ssl.sis, stdioserver.sis, Python_2.0.0.sis, PythonScriptShell_2.0.0.sis) in the correct order, which is a big no-no for a mass-market deployment. So…. I hope someone in Nokia is reading this blog entry carefully. Do it like Apple does. Make your application deployment static. Make OpenC static. Make every freaking library which is not shipped with the device statically buildable. It should be possible now when everything is open source. It will consume precious device RAM, but at least it will make mass market application development possible. SIS hell is worse hell than deb hell, or DLL hell, as the end user cannot fix it due to device security. In the related news SIS smart installer was announced few weeks ago. Personally I wouldn’t bet it can deal with all the problems of versioning and Symbian platform security. Forum reports aren’t promising and looks like very Symbianish user experience can be expected. In positive light, it seems that Python is being considered for this process.
Peek-a-boo – Python logo spotted in outdoor advertisement!Posted on August 17, 2010 by Mikko OhtamaaFiled Under plone, python, series 60, symbian, technology This caught my eye when coming home from the work.
The finger points straight to the snake…
Kudos to Nokia for this. Nokia’s phones are the best platform if you wish to have pleasant rapid mobile application development in Python – just stay away from Symbian and Series 40 models.
Python Interest Group Finland (PIG FI) – join nowPosted on August 13, 2010 by Mikko OhtamaaFiled Under django, plone, python, technology I have started a mailing list / Google group for Python Finnish community. http://groups.google.com/group/pigfi As far as I know such thing didn’t exist before. We discussed about the establishment of Python Finnish user group with several Python activists in Sauna Sprint few days ago (the event arranged by EESTEC and Plonistans, mid term report, final report). There used to exist Plone-related mailing list, but that’s pretty much dead now. If there are other parallel efforts I am not aware of them, but I really like to merge with the existing work whether it exists. I know at least Nokia, Finnish universities and several small enterprises are very active Python users in Finland. Our goal is to establish a vibrant, self-sustaining, Python community of Finland. We want to promote the sharing of Python knowledge across organizational borders, the adoption of Python in corporate and hobbyist cultures and have jolly good time with other Python enthusiasts. This probably means discussion (in Finnish), events and lots of, lots of, sauna. After all, the community will take the shape what suits it for the best, it is not forced externally. The short term goal is to gather enough participants to have a meaningful community. So, Join now!. You can use web interface, email or RSS to follow the discussion. Spread the word to make sure that the message reaches everyone. Tweet it. IRC it. Email it to your friends. SMS it to your parents. Go to streets and cry aloud “Rakastan Pythonia.” For most of us being Python programmer is a choice you have made or would like to take. Let’s together create an environment where this choice is fun and will generate great personal wealth for all of us.
Applying monkey patches in Django middleware layerPosted on August 12, 2010 by Mikko OhtamaaFiled Under django, plone, python, technology Monkey patching is a technique to modify module functions or class methods in Python and other dynamic languages run-time. It differs from the traditional source code patching that it does not need separate utility or compilation process to become effective. This means that you can deploy patches to codebase not under your control with your application without extra effort. Monkey patching has been made famous by Plone/Zope community where there is even collective.monkeypatcher add-on for managed monkey patching. Because monkey patches do not need a compilation stage, the patch will work with the future versions of the application, assuming the patched function or method is not changed. So you can “safely” update the patched software and the patch will apply to the new version without need to go to command-line to perform some cumbersome commands. However, it is the best practice of open source community to report the bugs and submit the fixing patches, as source code patch, in the corresponding issue trackers. In Django context, you can use monkey patching to
Patches are usually applied when Python does module imports. You have a special module called “monkeypatches.py” and when that is imported, it applies the patches when the module body level code runs. However, it is difficult to find stable import point in Django to run monkey patching. Django does some really evil magic to initialize INSTALLED_APPS, database models and stuff and doing any kind of work during import causes headache. So I figured out that you can apply monkey patches using middleware. Middleware applies the monkey patch when the first HTTP request hits the process (note that if you run preforked web server like FCGI every process has its own run-time code in memory). This technique, of course, cannot be used to monkey-patch things that happen before middleware processing, but it is not often needed. Below is an example how to monkey-patch Django CMS to normalize its unicode output. There was an issue with unicode characters and this is a stop-gap measure to fix it. (I think the proper fix would be fix related Cufon font renderin Javascript library). We add our monkey patcher to loaded middleware in settings.py. MIDDLEWARE_CLASSES = (
...
'foobar.middleware.FixCufonUnicodeNormalization',
)
The actual monkey patching happens by fiddling with the class code in process_request(). Note that in this particular case we only need to transform the output of the original function, we can simply hold a reference into it, call it and perform our transformation on the result. This way our monkey patch do not hinder the orignal function and is update safe (it does not matter if the code of PlaceholderNode.render method changes). import unicodedata
# Orignal function we monkey-patched away
_orignal_render = None
def _patched_render(self, context):
""" Normalize all unicode output of Placeholder nodes in Django templates """
content = _orignal_render(self, context)
# http://docs.python.org/library/unicodedata.html
return unicodedata.normalize("NFC", content)
class FixCufonUnicodeNormalization(object):
""" Fix issue with Cufon, user-generated HTML and unicode decomposed characters.
http://github.com/sorccu/cufon/issues/issue/133
"""
def process_request(self, request):
""" Install monkey-patch on demand.
If monkey-patch has not been run in for this process (assuming multiple preforked processes),
then do it now.
"""
from cms.templatetags.cms_tags import PlaceholderNode
global _orignal_render, _patched_render
if not _orignal_render:
# replace one of the class's method with own fixed version
_orignal_render = PlaceholderNode.render
PlaceholderNode.render = _patched_render
As far as I know, monkey patching is something PHP cannot do
Running 32-bit chroot on 64-bit Ubuntu server to reduce Python memory usagePosted on August 3, 2010 by Mikko OhtamaaFiled Under linux, plone, python, technology, ubuntu, zope Here are documented brief instructions how to run 32-bit chroot’ed environment on 64-bit Ubuntu server. chroot means that you run re-rooted and jailed system inside another system. What we do here is enabling 32-bit chroot’ed userland on 64-bit server. 32-bit userland and 32-bit Python environment reduces the memory usage of heavy website applications we are running (read: Plone/Zope), since 32-bit has only half of the pointer size and object-oriented programming is all about pointers. Zope is especially memory hungry, because it uses ZODB object database. The developer does not need to worry about when doing queries, updates or caches that much as the persistent site state is transparent to Python (objects are automatically loaded from database or cache when they are referred). Easy persistency means that almost everything is in the database and you need to have big object cache. Plone has huge client-side, in-process, cache for persistent objects. The default setting is for the cache size 5000 objects. (sidenote: since ZODB cache is in-process and Python does not do threading too well, running big Plone sites means that you need run several processes to handle parallel requests – having multiple processes with big in-process caches means loads of memory consumption) 32-bit userland is especially useful if you need to run Plone on 64-bit VPS (virtual private server) with low amount of available memory (512 MB or 1 GB). There are some brief measurements about 32-bit vs. 6-bit Python memory usage at the end of this article. Unless otherwise specified, all command here should be run as the root user of the host system. Commands here are for example only and you need to know what you are doing. If you lack advanced UNIX administration skills we gladly arrange you some commercial training or hosting support. InstallingBasic schroot installation instructions for Ubuntu can be found here https://help.ubuntu.com/community/DebootstrapChroot . We also install ZopeSkel in the chroo’ed environment for starting creating Plone sites. Note that we are using Ubuntu 8.04 which still ships with Python 2.4 – for later Ubuntus you need to compile Python 2.4 from the scratch. apt-get install debootstrap apt-get install schroot # Old schroot uses global schroot.conf, new versions have # chroot.d directory # This is a heredoc, but use what ever editor you like # to create the configuration cat <<EOF > /etc/schroot/chroot.d/hardy_i386.conf [hardy_i386] description=Ubuntu 8.04 Hardy for i386 location=/srv/chroot/hardy_i386 personality=linux32 root-users=bob run-setup-scripts=true run-exec-scripts=true type=directory users=bob,john,alice,ploneuser EOF mkdir -p /srv/chroot/hardy_i386 debootstrap --variant=buildd --arch i386 hardy /srv/chroot/hardy_i386 \ http://archive.ubuntu.com/ubuntu/ # Check that the chroot is created and working schroot -l # Enter the chroot (logged in as bob) schroot -c hardy_i386 -u root # Once inside, install python2.4-dev and other needed tools # Installing PIL with easy_install didn't work for some # reason, so we use python-imaging package. apt-get install python2.4-dev python-setuptools python-imaging easy_install-2.4 ZopeSkel And that’s about it. Chroot’ed environment will have
Chroot’ed environment will share with the host system
Chroot’ed users
…giving additional safety for shared hosting in the case of chroot’ed environment is compromised. Entering chroot’ed environment as a specific userTry schroot -c hardy_i386 -u root …or… schroot -c hardy_i386 -u plone_user # After you have set-uped normal user for chroot'ed environment All background processes you leave running in chroot’ed environment are terminated when you exit this environment, unless you create sessions as described below. Creating chroot sessionsSessions enable running commands in chroot without the need to have it constantly open. # Create a new schroot session schroot --chroot=hardy_i386 --user=ploneuser \ --session-name=plonesession \ --begin-session # Run commands in the created session schroot --chroot=plonesession --user=ploneuser --run-session \ /srv/plone/yourplonesite/bin/instance start # Ending session schroot --chroot=plonesession --user=ploneuser --end-session Note that --chroot parameter takes in both actual chroot installations and session ids. Doing resets for chrooted environmentThe session processes exist as long as the session exist. Unless you explicitly start a new session with –begin-session the processes are terminated as soon as you log out from the chroot’ed environent. # Terminate previous session if any schroot --chroot=hardy_i386 --user=plone_user --session-name=plone_session --end-session #Start the session (again) schroot --chroot=hardy_i386 --user=plone_user --session-name=plone_session --begin-session # Run a start script inside the chroot'ed environment which will start Plone # NGINX and other necessary 32-bit services schroot --chroot=plone_session --user=plone_user --run-session /srv/plone/myplonesite/restart-all.sh Running sessions at startupYou can add schroot bootstrap in real /etc/rc.local: schroot --chroot=hardy_i386 --user=ploneuser \ --session-name=plonesession --begin-session \ && schroot --chroot=plonesession --run-session \ /srv/plone/inst/bin/instance start Remember that the users have to be created outside the chrooted environment. If you set the home directory to something that exists only in the chrooted environment, use something like this adduser --no-create-home --home HOMEDIR_IN_CHROOT ploneuser Then to create the directory inside the chroot and set its ownership to the newly created user and group. 32-bit vs 64-bit memory consumptionReason why we even tried this is that some python applications, like Zope, use references heavily and moving from 32bit to 64bit references increases memory usage. (J Stahl 2010) Memory figures from a development Zope/Plone 3.3.5 server
This is far from a complete study, but it would seem that the chroot does pay off even though it has to load 32bit versions of basic libraries along. If running more than one instance on same server memory savings should increase. More information
.gitignore for Python developersPosted on July 23, 2010 by Mikko OhtamaaFiled Under git, plone, python, technology, web development, zope If you are using Git for version control for your Python egg and buildout development below are is a sample which you might want to put into your .gitignore file. *.mo *.egg-info *.egg *.EGG *.EGG-INFO bin build develop-eggs downloads eggs fake-eggs parts dist .installed.cfg .mr.developer.cfg .hg .bzr .svn *.pyc *.pyo *.tmp* Suggestions for new ignores are welcome.
Zoho integration for Python and Plone CMSPosted on July 18, 2010 by Mikko OhtamaaFiled Under plone, python, technology, zoho Zoho is a web application provider competing with Google Docs, Microsoft Office and Live. Zoho provides a very wide set of browser based applications from text editing and spreadsheets to project management and customer relationship management (highlighted items should ring a bell for small software development companies). Especially the last one, CRM, is a very attractive deal as you get a hosted complex CRM application with API services for very affordable or free price. Small organizations are not necessarily rich enough to go for Salesforce API supported edition which would be 135 € / month / user. mFabrik has been working on Zoho Python bindings as we use Zoho internally. Zoho API is HTTP GET/POST based.
mfabrik.zoho is a GPL’ed Python library which provides basic facilities for Zoho API calls. Currently the feature set is very CRM weighted, though it can be easily expanded for other Zoho applications. mfabrik.plonezohointegration is a Plone CMS add-on product which marry Plone and Zoho together. The add-on provides a control panel where you can enter Zoho API key details for Plone. Forms for CRM lead generation are provided as standalone and as a portlet (you can see them in action on our web and blog site). The source is hosted on Github, so you can easily start tailoring it for your own organization needs. I happily accept all merge requests, providing that unit tests for new features are included. If you do not feel comfortable with Python programming, but still want to integrate Zoho to your systems, please contact us for further help.
Easily install all Python versions under Linux and OSX using collective.buildout.pythonPosted on July 16, 2010 by Mikko OhtamaaFiled Under plone, python, technology, ubuntu Here are short instructions how to install all versions (2.4, 2.5, 2.6, 2.7 and 3.1) of Python interpreters on UNIX system. The instructions were tested on Ubuntu 10.04 Lucid Lynx Linux but should work on other systems as is. The installation is based of downloading, compiling and installing different Pythons and their libraries using buildout tool. A buildout configuration for doing this is maintained by a Plone community. This buildout is especially useful to get Python 2.4 properly running under the latest Ubuntu 10.04 Lucid Lynx. This is because Ubuntu repositories won’t ship with Python 2.4 packages anymore. The installation will also include static compilation of some very popular libraries. These are dependencies for other Python packages including, but not limited, to
Prerequisites
Running itsvn co http://svn.plone.org/svn/collective/buildout/python/ cd python python bootstrap.py bin/buildout Using itAll Pythons are under virtualenv installations. This means that you can activate one Python configuration for your shell once easily (python command will run under different Python versions). Activating Python 2.4 source python/python-2.4/bin/activate (python-2.4)moo@murskaamo:~/code$ python -V Python 2.4.6 Check that Python Imaging Library works python Python 2.4.6 (#1, Jul 16 2010, 10:31:46) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import PIL (No exceptions raised, Python Imaging Library works well).
|
Latest Top Shouts | Want to get mobile? |
