Old web browsers do not support Allow-acces-origin HTTP header needed to do cross-domain AJAX requests (IE6, IE7).

Below is an example how to work around this for jQuery getJSON() calls by

• Detecting browsers which do not support this using jQuery.support API
• Doing an alternative code path through a local website proxy view which uses Python urllib to make server-to-server call and return it as it would be a local call, thus working around cross-domain restriction

This example is for Plone/Grok, but the code is easily port to other Python web frameworks.

Javascript example

/**
 * Call a RESTful service vie AJAX
 *
 * The final URL is constructed by REST function name, based
 * on a base URL from the global settings.
 *
 * If the browser does not support cross domain AJAX calls
 * we'll use a proxy function on the local server. For
 * performance reasons we do this only when absolutely needed.
 *
 * @param {String} functionName REST function name to a call
 *
 * @param {Object} Arguments as a dictionary like object, passed to remote call
 */
function callRESTful(functionName, args, callback) {

    var src = myoptions.restService + "/" +functionName;

    // set to true to do proxied request on every browser
    // useful if you want to use Firebug to debug your server-side proxy view
    var debug = false;

        console.log("Doing remote call to:" + src)

        // We use jQuery API to detect whether a browser supports cross domain AJAX calls
        // http://api.jquery.com/jQuery.support/
        if(!jQuery.support.cors || debug) {
                // http://alexn.org/blog/2011/03/24/cross-domain-requests.html
                // Opera 10 doesn't have this feature, neither do IExplorer < 8, Firefox < 3.5

                console.log("Mangling getJSON to go through a local proxy")

                // Change getJSON to go to our proxy view on a local server
                // and pass the orignal URL as a parameter
                // The proxy view location is given as a global JS variable
                args.url = src;
                src = myoptions.portalUrl + "/@@proxy";
        }

        // Load data from the server
        $.getJSON(src, args, function(data) {
                // Parse incoming data and construct Table rows according to it
                console.log("Data succesfully loaded");
                callback(data, args);

     });

}

The server-side view:

import socket
import urllib
import urllib2
from urllib2 import HTTPError

from five import grok
from Products.CMFCore.interfaces import ISiteRoot
from mysite.app import options

class Proxy(grok.CodeView):
    """
    Pass a AJAX call to a remote server. This view is mainly indended to be used
    with jQuery.getJSON() requests.

    This will work around problems when a browser does not support Allow-Access-Origin HTTP header (IE).

    Asssuming only HTTP GET requests are made.s
    """

    # This view is available only at the root of Plone site
    grok.context(ISiteRoot)

    def isAllowed(self, url):
        """
        Check whether we are allowed to call the target URL.

        This prevents using your service as an malicious proxy
        (to call any internet service).
        """

        allowed_prefix = options.REST_SERVICE_URL

        if url.startswith(allowed_prefix):
            return True

        return False

    def render(self):
        """
        Use HTTP GET ``url`` query parameter for the target of the real request.
        """

        # Make sure any theming layer won't think this is HTML
        # http://stackoverflow.com/questions/477816/the-right-json-content-type
        self.request.response.setHeader("Content-type", "application/json")

        url = self.request.get("url", None)
        if not url:
            self.request.response.setStatus(500, "url parameter missing")

        if not self.isAllowed(url):
            # The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeate
            self.request.response.setStatus(403, "proxying to the target URL not allowed")
            return

        # Pass other HTTP GET query parameters direclty to the target server
        params = {}
        for key, value in self.request.form.items():
            if key != "url":
                params[key] = value

        # http://www.voidspace.org.uk/python/articles/urllib2.shtml
        data = urllib.urlencode(params)

        full_url = url + "?" + data
        req = urllib2.Request(full_url)

        try:

            # Important or if the remote server is slow
            # all our web server threads get stuck here
            # But this is UGLY as Python does not provide per-thread
            # or per-socket timeouts thru urllib
            orignal_timeout = socket.getdefaulttimeout()
            try:
                socket.setdefaulttimeout(10)

                response = urllib2.urlopen(req)
            finally:
                # restore orignal timeoout
                socket.setdefaulttimeout(orignal_timeout)

            # XXX: How to stream respone through Zope
            # AFAIK - we cannot do it currently

            return response.read()

        except HTTPError, e:
            # Have something more useful to log output as plain urllib exception
            # using Python logging interface
            # http://docs.python.org/library/logging.html
            logger.error("Server did not return HTTP 200 when calling remote proxy URL:" + url)
            for key, value in params.items():
                logger.error(key + ": "  + value)

            # Print the server-side stack trace / error page
            logger.error(e.read())

            raise e

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

To tackle the lack of console.log() problem there are several approaches.

Use dummy placeholder if console is missing

This snippet wraps console.log (need to repeat for console.error etc.):

// Ignore console on platforms where it is not available
if (typeof(window["console"]) == "undefined") { console = {}; console.log = function(a) {}; }

Pros

• Easy

Cons

• Need to add to every Javascript file
• Messes with global namespace

Use module specific log function

This makes your code little bit ugly, more Java like. Each Javascript module declares their own log() function which checks the existence of console.log() and outputs there if it’s present.

mfabrik.log =function(x) {
 if(console.log) {
 console.log(x);
 }
}

mfabrik.log("My log messages")

Pros

• Easy to hook other logg
• You can disable all logging output with one if

Cons

• Not as natural to write as console.log()
• Need to add to every Javascript module

Preprocess Javascript files

Plone (Kukit / KSS) uses this approach. All debug Javascript is hidden behind conditional comments and it is filtered out when JS files are bundled for the production deployment. (The preprocessing code is here in Python for those who are interested in it).

if (_USE_BASE2) {
 // Base2 legacy version: matchAll has to be used
 // Base2 recent version: querySelectorAll has to be used
 var _USE_BASE2_LEGACY = (typeof(base2.DOM.Document.querySelectorAll) == 'undefined');
 if (! _USE_BASE2_LEGACY) {
 ;;;     kukit.log('Using cssQuery from base2.');

Pros

• Makes production Javascript files lighter
• Make production Javascript files more professional – you do not deliver logging statements indented for internal purposes for your site visitors

Cons

• Complex – preprocessing is required

Commit hooks

You can use Subversion and Git commit hooks to check that committed JS files do not contain console.log. For example, Plone repositories do this for the Python statement  import pdb ; pdb.set_trace() (enforce pdb breakpoint).

Pros

• Very robust approach – you cannot create code with console.log()

Cons

• Prevents also legitimate use of console.log()
• Github, for example, lacks possibility to push client-side commit hooks to the repository cloners. This means that every developer must manually install commit hooks themselves. Everything manual you need to do makes the process error prone.

Other approaches?

Please tell us!

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

domgen uses a simple specification to generate the HTML. In some cases it’s more reliable to use DOM API to generate HTML instead of using innerHTML. We have had problems especially with iPhone when using innerHTML.
See for example http://pastebin.com/LLk3J0iH or google “iPhone innerHTML” for more info. In fact, innerHTML is not even part of the HTML standard even though it exists on every browser.

So if you want to add dynamic HTML properly, you should use the Javascript DOM API. Consider the following short HTML where the contents inside body had to be generated:

<body>
    <div id="mydiv">my div</div>
</body>

Using Javascript DOM API:

// Get the body tag
var body = document.getElementsByTagName("body")[0];
// Create 'div' element
var mydiv = document.createElement("div");
mydiv.setAttribute("id", "mydiv");
mydiv.innerText = "my div";
// Attach the div to body
body.appendChild(mydiv);

And using domgen:

var spec = [
    'div',
    {
        id : 'mydiv',
        _innerText : 'my div'
    }
];
domgen.generate( domgen.get("body")[0], spec);
// or with jQuery
domgen.generate( $("body")[0], spec);

To me, domgen seems a lot easier to read and work with. Get code and more examples here: https://bitbucket.org/mfabrik/domgen

Is it wise to gzip static resources (javascript and CSS) before sending them to client?

The opinion of Internet seems to be divided

• GZip decompression takes too much time at the client end and thus it is not wise (latency at the clients end)
• Bandwidth save is enough to counter the decompression latency

So if we put this to context of

• Compressed and merged CSS and Javascript files of Plone
• Assuming the users are using the state of the art browsers: Safari 4, Chrome, Firefox 3.6 and IE8
• Connections are faster than 384 Kbit/s

…and forget…

• Recompressing images (GIF, PNG and JPEG) using GZip as there is no notable save

should I enable GZip compression on the front-end server (Apache) with disk cache enabled?

Of course, I could do testing and timing myself, but I’ll simply ask for your experiences first before investing few hours for this. Also, hints how to measure how fast GZip decompression is, are welcome.

Thank you.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

http://mfabrik.com/about-us/jobs/developer-helsinki

• Image uploads
• Payments for subscribed content

Sounds easy, right? Well it isn’t… Below are some notes for our due diligence work which you fellow developers might find interesting.

SDKs

Mobile phone vendors are jealously and don’t want to co-operate with each other. Building application which works in all handsets is major headache.

We found some reasonable candidates for cross-platform mobile development doing HTML and Javascript. HTML and Javascript pages are converted to native application using a wrapper technology (a.k.a. appaccelerator). Doing Flash Lite or Java ME can be pretty much forgotten nowadays as they won’t run on the most hyped platform, iPhone. Flash Lite has poor support for anything except content authoring due to primitive and limited APIs. Java ME provides horrible user experience.

(X)HTML is the only common language spoken by mobile phones. Thus, there has been a rise of “appaccelerators”, technologies which allow to create mobile applications with HTML(5) and Javascript.

• Phonegap: iPhone, Android, Blackberry and possibly S60 in the future. Pluses: BSD license, very active community. Minuses: bad documentation, difficult deployment process.
• Titanium: iPhone, Android. Pluses: Professional, Apache license. Minuses: Too tightly coupled with Appacclerator Inc. company.
• Rhomobile: iPhone, Android, Blackberry, S60, Windows Mobile. Pluses: Professional, tries to build open source community, the widest platform support. Minuses: Dual licensing and tightly coupled with Rhomobile Inc.
• Nokia Web-runtime: Nokia S60 and some other Symbian based phones. Pluses: Professional, good documentation. Minuses: Not open source, impossible to extend, Nokia has little interest to make this cross-platform, Nokia doesn’t like updating old models and web-runtime is useable only in the latest S60 5th edition models.
• Palm Pre supports web applications natively. However Palm Pre application business is still taking a shape.

All these wrap the browser component (WebKit) and provide some extra Javascript APIs when your web pages as executed under the application mode.

• Locationing
• Contacts
• SMS
• Client-side database
• and so on…

Rhomobile has little different use cases  from the rest of the bunch as it provides client-side programming using Ruby and less focuses on Javascript/web applications.

Payments and in-application purchases

There are four major way to do mobile payments “inside” the application for bought content and subscriptions. The price tag on the application itself is left out on this discussion as the application stores themselves take care of it.

• Credit card
• SMS
• App Store payment (thus far Apple only)
• Direct operator payments – you have a service provider (Bango) which can directly charge items to the operator phone bill based on handset identification.

App Store payment is the most attractive as it provides the best end user experience.  It allows you to use App Store payment mechanism inside the application. It is safe and no need to hassle with external payment providers. However, App Store payment can be used only for content consumed directly inside the application. You cannot use it e.g. for ordering a pizza. I think this might be related to recent EU legislation forbidding SMS payments for services not consumed in the phone itself.

SMS payment is ok for little payments. Operators take big cut of the revenue, generally 30% – 70% depending on the country. Short code fees usually start from 500€ set-up fee + 500€ / month. SMS cannot be often send as a background, but the user is presented the normal SMS editor which reduces the user experience somehow.

For credit card payments there exists several providers. Credit card has the cheapest entry fees, but the downside is that the user needs to have the credit card. This excludes teenager audience.

Direct operator payments are not very well supported yet globally. Most western operators support them. The operator also takes a big share and the fixed fee is pretty high.

My favorite payment provider thus far is Bango which provides credit card payment starting 9€ / mo. and scales up to worldwide SMS payments which cost few grannies per month.

In most cases, the payment experience will not be smooth. You need to open the phone main browser on the payment provider page to do the payment. This usually will close your own application. Rarely you can do the payment inside the application and support multiple platforms. After doing the payment most platforms allow you to close the browse and reopen your application using a special URL handler.

Wikipedia mobile payments page is also useful.

Image upload

<input type=”file”> won’t work on iPhone and some other platforms as those don’t have user browsable file system. Also the file dialog usually doens’t have image preview making it useless.

Phonegap has a branch which supports images picking using iPhone’s own gallery browser.

In any case, there is not yet cross-platform solution for this.

Future prospects

In some time-frame we will get rid of the need to wrap HTML applications natively as the web browser applications will support all HTML5 features without extensions and probably have some proprietary extensions for mobile specific features like SMS. We already have had some taste for this:

• The first taste of this is Mozilla’s Fennec mobile browser which has locationing support.
• iPhone’s Safari already supports client-side storage and CSS3.
• All phones have support for dial-in links. The format of the link may vary. DeviceAtlas is good place to hunt for this information.
• Nokia browser supports send SMS links
• Nokia browser supports downloadable map markers (to the map application of the phone itself)

Say, you have variable ‘a’ and you want to pass it to a MooTools class B instance during creation. In the easiest case you’d use new B({ myReference: a}) and trust on MooTools’ Class.setOptions() to minify the need of code lines. This is what you should do… well at least that’s what I did and in this case it was a mistake.

It turns out that Class.setOptions() merges it’s arguments to this.options and then takes copy of them via $merge(). That means that any variable references you pass to setOptions() will get copied to this.options and.. well, that’s it. See lines 1170-1173 in uncompressed version of MooTools 1.2:

var Options = new Class({
    setOptions: function(){
        this.options = $merge.run([this.options].extend(arguments));

That effectively nullifies the benefits of Class.setOptions() if you want to pass in variable references..

Here’s a longer example to clarify (use Firebug):

  // The most basic MooTools class that implements options
  // ref is a variable meant for pointing at given object
  // (won't do that, however)
  var B = new Class({
    Implements: Options,
    options: {
      ref: null
    },
    initialize: function(options) {
      this.setOptions(options);
    }
  });

  // Ok let's create an instance that we can pass to B
  // It's similar with all sorts of variables
  var A = new Class({
    initialize: function() {
      this.somevar = 'untouched';
    }
  });
  var a = new A();

  // Create an instance of B and give it somevar as reference
  var b = new B({ ref: a });

  // prints out "untouched" as should
  console.log(b.options.ref.somevar);

  // Let's change the variable (direct access, bad)
  a.somevar = "changed";

  // b's reference should still point to a, right?
  // In that case the following should print "changed",
  // but because our reference object was copied instead
  // of retaining reference to it, we just get "untouched"
  console.log(b.options.ref.somevar);

I don’t know why MooTools wants to make a copy of arguments in setOptions() – propably for performance reasons.

Since this problem pops up constantly, I decided to create an easy way to deal with the situation. I decorate all functions which made have silent exceptions (e.g. one called from document load event) with a custom logger function which will first log the exception and then rethrow it.

Thus instead of writing (JQuery example)

myfunction() {
    // crash here
    var i = foobar; // missing variable foobar
}

$(document).ready(myfunction);

write

myfunction() {
    // crash here
    var i = foobar; // missing variable foobar
}

$document.ready(logExceptions(myFunction));

or

// myFunction can be bind to many events and exceptions are logged always
myfunction = logExceptions(function() {
  // crash here
    var i = foobar; // missing variable foobar
});

$document.ready(logExceptions(myFunction));

The Javascript code for the decorator:

/**
 * Enhancd Javascript logging and exception handler.
 *
 * Copyright 2008 Red Innovation Ltd.
 *
 * @author Mikko Ohtamaa
 * @license 3-clause BSD
 *
 */

// Browser specific logging output initialization
// Supports Firefox/Firebug. Other (Opera) can be hooked in here.
if(!console) {
	// Install dummy functions, so that logging does not break the code if Firebug is not present
    var console = {};
    console.log = function(msg) {};
    console.info = function(msg) {};
    console.warn = function(msg) {};
} else {
    // console.log provided by Firefox + Firebug
}

/**
 * Try print human digestable exception stack trace to Firebug console.
 *
 * http://developer.mozilla.org/en/docs/Core_JavaScript_1.5_Reference:Global_Objects:Error
 *
 * @param e: Error
 */
function printStackTrace(e) {

	var msg = e.name + ":" + e.message;

	if (e.fileName) {
		msg += " at " + e.fileName + ":" + e.lineNumber;
	}
	console.log(msg);

	if (e.stack) {
		// Extract Firefox stack information. This tells how you ended up
		// to the exception in the first place. I didn't find
		// instructions how to parse this stuff.
		console.log(e.stack);
	}
}

/**
 * Decorate function so that exceptions falling through are printed always.
 *
 * Returns a decorated function which will be used instead of the normal function.
 * The decorated function has preplaced try ... catch block which will not let
 * through any exceptions silently or without logging. Even though there is an
 * exception it is normally throw upwards in the stack after logging.
 *
 * @param func: Javascript function reference
 */
function logExceptions(func) {

	var orignal = func;

	decorated = function() {
		try {
			orignal.apply(this, arguments);
		} catch(exception) {
			printStackTrace(exception);
			throw exception;
		}
	}

	return decorated;
}

Javascript is executed run-time. Thus, any errors cannot be cached until the code is run. JSLint is a verifier tool which checks Javascript syntax offline. Of course, it cannot detect application logic errors, but it makes sure that you don’t have cross-browser compability problems in your Javascript syntax. Maybe the most famous of these problems is the extra comma before the list termination which is ok for Firefox, but kills the page on Internet Explorer.

Some argue that this does not really catch bugs and instead a comprehensive unit-test suite should be used. This might be a requirement for a platform level Javascript library code, but it is often an overkill for your little site with some flashy dialog windows.

Installing JSLint

JSLint itself is written in Javascript. So you need to offline Javascript interpreter to run it. On Linux there exists Rhino. On Ubuntu, Rhino package is broken. and you need to download the orignal Rhino JAR from here and copy it to /usr/share/java.

sudo apt-get install rhino # Note https://bugs.launchpad.net/ubuntu/+source/rhino/+bug/93885

Then get JSLint offline version from here.

Running JSLint straight from the Eclipse

We use JSEclipse plug-in in Eclipse for Javascript development. Thus, it is convenient to to execute JSLint directly from the Eclipse IDE.

Drag and drop jslint.js to your project/scripts folder in Eclipse.

Create a new External tool like in the example below.

Then you can run it by just clicking “run external tool” icon. Output comes to the console window.

Checking all Javascript files during unit test

verify_javascript.bash can be executed from shell and it checks all JS files in your project. This is a nice feature to integrate to your unit test cycle.

Example:

$ bash scripts/verify_javascript.bash source/server/media/jssource/server/media/js/jquery.js

Lint at line 11 character 30: Expected '{' and instead saw 'var'.

...