About

mFabrik Blog is about mobile and web software development, open source and Linux. We tell exciting tales where business, technology, web and mobile convergence.

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.

AJAX proxy view with Python, urllib and Plone

Old web browsers do not support Allow-acces-origin HTTP header needed to do cross-domain AJAX requests (IE6, IE7).

Below is an example how to work around this for jQuery getJSON() calls by

  • Detecting browsers which do not support this using jQuery.support API
  • Doing an alternative code path through a local website proxy view which uses Python urllib to make server-to-server call and return it as it would be a local call, thus working around cross-domain restriction

This example is for Plone/Grok, but the code is easily port to other Python web frameworks.

Note: This is not a full example code. Basic Python and Javascript skills are needed to interpret and adapt the code for your use case.

Javascript example

/**
 * Call a RESTful service vie AJAX
 *
 * The final URL is constructed by REST function name, based
 * on a base URL from the global settings.
 *
 * If the browser does not support cross domain AJAX calls
 * we'll use a proxy function on the local server. For
 * performance reasons we do this only when absolutely needed.
 *
 * @param {String} functionName REST function name to a call
 *
 * @param {Object} Arguments as a dictionary like object, passed to remote call
 */
function callRESTful(functionName, args, callback) {

    var src = myoptions.restService + "/" +functionName;

    // set to true to do proxied request on every browser
    // useful if you want to use Firebug to debug your server-side proxy view
    var debug = false;

        console.log("Doing remote call to:" + src)

        // We use jQuery API to detect whether a browser supports cross domain AJAX calls
        // http://api.jquery.com/jQuery.support/
        if(!jQuery.support.cors || debug) {
                // http://alexn.org/blog/2011/03/24/cross-domain-requests.html
                // Opera 10 doesn't have this feature, neither do IExplorer < 8, Firefox < 3.5

                console.log("Mangling getJSON to go through a local proxy")

                // Change getJSON to go to our proxy view on a local server
                // and pass the orignal URL as a parameter
                // The proxy view location is given as a global JS variable
                args.url = src;
                src = myoptions.portalUrl + "/@@proxy";
        }

        // Load data from the server
        $.getJSON(src, args, function(data) {
                // Parse incoming data and construct Table rows according to it
                console.log("Data succesfully loaded");
                callback(data, args);

     });

}

The server-side view:

import socket
import urllib
import urllib2
from urllib2 import HTTPError

from five import grok
from Products.CMFCore.interfaces import ISiteRoot
from mysite.app import options

class Proxy(grok.CodeView):
    """
    Pass a AJAX call to a remote server. This view is mainly indended to be used
    with jQuery.getJSON() requests.

    This will work around problems when a browser does not support Allow-Access-Origin HTTP header (IE).

    Asssuming only HTTP GET requests are made.s
    """

    # This view is available only at the root of Plone site
    grok.context(ISiteRoot)

    def isAllowed(self, url):
        """
        Check whether we are allowed to call the target URL.

        This prevents using your service as an malicious proxy
        (to call any internet service).
        """

        allowed_prefix = options.REST_SERVICE_URL

        if url.startswith(allowed_prefix):
            return True

        return False

    def render(self):
        """
        Use HTTP GET ``url`` query parameter for the target of the real request.
        """

        # Make sure any theming layer won't think this is HTML
        # http://stackoverflow.com/questions/477816/the-right-json-content-type
        self.request.response.setHeader("Content-type", "application/json")

        url = self.request.get("url", None)
        if not url:
            self.request.response.setStatus(500, "url parameter missing")

        if not self.isAllowed(url):
            # The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeate
            self.request.response.setStatus(403, "proxying to the target URL not allowed")
            return

        # Pass other HTTP GET query parameters direclty to the target server
        params = {}
        for key, value in self.request.form.items():
            if key != "url":
                params[key] = value

        # http://www.voidspace.org.uk/python/articles/urllib2.shtml
        data = urllib.urlencode(params)

        full_url = url + "?" + data
        req = urllib2.Request(full_url)

        try:

            # Important or if the remote server is slow
            # all our web server threads get stuck here
            # But this is UGLY as Python does not provide per-thread
            # or per-socket timeouts thru urllib
            orignal_timeout = socket.getdefaulttimeout()
            try:
                socket.setdefaulttimeout(10)

                response = urllib2.urlopen(req)
            finally:
                # restore orignal timeoout
                socket.setdefaulttimeout(orignal_timeout)

            # XXX: How to stream respone through Zope
            # AFAIK - we cannot do it currently

            return response.read()

        except HTTPError, e:
            # Have something more useful to log output as plain urllib exception
            # using Python logging interface
            # http://docs.python.org/library/logging.html
            logger.error("Server did not return HTTP 200 when calling remote proxy URL:" + url)
            for key, value in params.items():
                logger.error(key + ": "  + value)

            # Print the server-side stack trace / error page
            logger.error(e.read())

            raise e

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

Everyone loves and hates console.log()

console.log()  is the best friend of every Javascript junkie. However, the lack of it isn’t. console.log() function is only available in Webkit based browsers and with Firebug in Firefox. It’s the infamous situation that someone leaves console.log() to Javascript code, doesn’t notice its presence, commits the file and suddenly all Javascript on the production server stops working for Internet Explorer users….

To tackle the lack of console.log() problem there are several approaches.

Use dummy placeholder if console is missing

This snippet wraps console.log (need to repeat for console.error etc.):

// Ignore console on platforms where it is not available
if (typeof(window["console"]) == "undefined") { console = {}; console.log = function(a) {}; }

Pros

  • Easy

Cons

  • Need to add to every Javascript file
  • Messes with global namespace

Use module specific log function

This makes your code little bit ugly, more Java like. Each Javascript module declares their own log() function which checks the existence of console.log() and outputs there if it’s present.

mfabrik.log =function(x) {
 if(console.log) {
 console.log(x);
 }
}

mfabrik.log("My log messages")

Pros

  • Easy to hook other logg
  • You can disable all logging output with one if

Cons

  • Not as natural to write as console.log()
  • Need to add to every Javascript module

Preprocess Javascript files

Plone (Kukit / KSS) uses this approach. All debug Javascript is hidden behind conditional comments and it is filtered out when JS files are bundled for the production deployment. (The preprocessing code is here in Python for those who are interested in it).

if (_USE_BASE2) {
 // Base2 legacy version: matchAll has to be used
 // Base2 recent version: querySelectorAll has to be used
 var _USE_BASE2_LEGACY = (typeof(base2.DOM.Document.querySelectorAll) == 'undefined');
 if (! _USE_BASE2_LEGACY) {
 ;;;     kukit.log('Using cssQuery from base2.');

Pros

  • Makes production Javascript files lighter
  • Make production Javascript files more professional – you do not deliver logging statements indented for internal purposes for your site visitors

Cons

  • Complex – preprocessing is required

Commit hooks

You can use Subversion and Git commit hooks to check that committed JS files do not contain console.log. For example, Plone repositories do this for the Python statement  import pdb ; pdb.set_trace() (enforce pdb breakpoint).

Pros

  • Very robust approach – you cannot create code with console.log()

Cons

  • Prevents also legitimate use of console.log()
  • Github, for example, lacks possibility to push client-side commit hooks to the repository cloners. This means that every developer must manually install commit hooks themselves. Everything manual you need to do makes the process error prone.

Other approaches?

Please tell us!

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter

domgen – Creating HTML with Javascript without DOM API

Tired of using Javascript DOM API for generating HTML dynamically? Meet domgen. domgen is a tool for easy dynamic content generation via Javascript. It generates DOM elements from dictionary specification similar to HTML and eliminates the need for cumbersome DOM Javascript API.

domgen uses a simple specification to generate the HTML. In some cases it’s more reliable to use DOM API to generate HTML instead of using innerHTML. We have had problems especially with iPhone when using innerHTML.
See for example http://pastebin.com/LLk3J0iH or google “iPhone innerHTML” for more info. In fact, innerHTML is not even part of the HTML standard even though it exists on every browser.

So if you want to add dynamic HTML properly, you should use the Javascript DOM API. Consider the following short HTML where the contents inside body had to be generated:

<body>
    <div id="mydiv">my div</div>
</body>

Using Javascript DOM API:

// Get the body tag
var body = document.getElementsByTagName("body")[0];
// Create 'div' element
var mydiv = document.createElement("div");
mydiv.setAttribute("id", "mydiv");
mydiv.innerText = "my div";
// Attach the div to body
body.appendChild(mydiv);

And using domgen:

var spec = [
    'div',
    {
        id : 'mydiv',
        _innerText : 'my div'
    }
];
domgen.generate( domgen.get("body")[0], spec);
// or with jQuery
domgen.generate( $("body")[0], spec);

To me, domgen seems a lot easier to read and work with. Get code and more examples here: https://bitbucket.org/mfabrik/domgen

To gzip or not to gzip CSS and Javascript?

Dear Lazyweb,

Is it wise to gzip static resources (javascript and CSS) before sending them to client?

The opinion of Internet seems to be divided

  • GZip decompression takes too much time at the client end and thus it is not wise (latency at the clients end)
  • Bandwidth save is enough to counter the decompression latency

So if we put this to context of

  • Compressed and merged CSS and Javascript files of Plone
  • Assuming the users are using the state of the art browsers: Safari 4, Chrome, Firefox 3.6 and IE8
  • Connections are faster than 384 Kbit/s

…and forget…

  • Recompressing images (GIF, PNG and JPEG) using GZip as there is no notable save

should I enable GZip compression on the front-end server (Apache) with disk cache enabled?

Of course, I could do testing and timing myself, but I’ll simply ask for your experiences first before investing few hours for this. Also, hints how to measure how fast GZip decompression is, are welcome.

Thank you.

Get developers  Subscribe mFabrik blog in a reader Follow me on Twitter