mFabrik - mobile sites, apps, HTML5 and CMS software development » logging

Ghetto logging for PHP

Mikko Ohtamaa — Thu, 03 Mar 2011 12:49:55 +0000

In the case you need to quickly see if certain code path is executed on a production server for which config cannot be changed.

file_put_contents("/tmp/log.txt", "this is a test entry\n");

The proper method would be

Configure error log in php.ini
Use error_log()
…. or use logging facility of your PHP framework

Get developers Subscribe mFabrik blog in a reader Follow me on Twitter

Temporarily capturing Python logging output to a string buffer

Mikko Ohtamaa — Wed, 23 Feb 2011 17:16:56 +0000

You can capture Python logging output temporarily to a string buffer. This is useful if you want to use logging module to record
the status of long running operations and later show the resulting log to the end user, who does not have access to file system logs.

Below is an Grok framework view code example for Plone CMS.

View code:

import logging
from StringIO import StringIO

from five import grok

from xxx.objects.interfaces import IXXXResearcher
from Products.CMFCore.interfaces import ISiteRoot
from Products.statusmessages.interfaces import IStatusMessage

from xxx.objects.sync import sync_with_xxx

grok.templatedir("templates")

logger = logging.getLogger("XXX sync")

class SyncAll(grok.View):
    """
    Update all researcher data on the site from XXX (admin action)
    """

    grok.context(ISiteRoot)

    def sync(self):
        """
        Search all objects of certain type on the site and
        sync them with a remote site.
        """

        brains =  self.context.portal_catalog(object_provides=IXXXResearcher.__identifier__)
        for brain in brains:
            object = brain.getObject()
            sync_with_xxx(object, force=True)

    def startCapture(self, newLogLevel = None):
        """ Start capturing log output to a string buffer.

http://docs.python.org/release/2.6/library/logging.html

        @param newLogLevel: Optionally change the global logging level, e.g. logging.DEBUG
        """
        self.buffer = StringIO()

        print >> self.buffer, "Log output"

        rootLogger = logging.getLogger()

        if newLogLevel:
            self.oldLogLevel = rootLogger.getEffectiveLevel()
            rootLogger.setLevel(newLogLevel)
        else:
            self.oldLogLevel = None

        self.logHandler = logging.StreamHandler(self.buffer)
        formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
        self.logHandler.setFormatter(formatter)
        rootLogger.addHandler(self.logHandler)    

    def stopCapture(self):
        """ Stop capturing log output.

        @return: Collected log output as string
        """

        # Remove our handler
        rootLogger = logging.getLogger()        

        # Restore logging level (if any)
        if self.oldLogLevel:
            rootLogger.setLevel(self.oldLogLevel)

        rootLogger.removeHandler(self.logHandler)

        self.logHandler.flush()
        self.buffer.flush()

        return self.buffer.getvalue()

    def update(self):
        """ Process the form.

        Process the form, log the output and show the output to the user.
        """

        self.logs = None

        if "sync-now" in self.request.form:
            # Form button was pressed

            # Open Plone status messages interface for this request
            messages = IStatusMessage(self.request)

            try:
                self.startCapture(logging.DEBUG)

                logger.info("Starting full site synchronization")

                # Do the long running,
                # lots of logging stuff
                self.sync()    

                logger.info("Succesfully done")

                # It worked! Trolololo.
                messages.addStatusMessage("Sync done")

            except Exception, e:
                # Show friendly error message
                logger.exception(e)
                messages.addStatusMessage(u"It did not work out:" + unicode(e)) 

            finally:
                # Put log output for the page template access
                self.logs = self.stopCapture()

Related Zope page template:



 
 
 

 
 XXX site update
     

 
 Update all researches from XXX
 

 
   Sync results:

Get developers Subscribe mFabrik blog in a reader Follow me on Twitter

Enhanced Apache logging: request duration, balancer member, transferred bytes

Mikko Ohtamaa — Tue, 21 Dec 2010 18:24:02 +0000

This helps you to diagnose problems on the production environment if you are running Apache or Apache mod_balancer. Put to your

# timestamp referer useragent, server time seconds/microseconds, transfered bytes input/output, which balancer was used
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" duration:%T/%D io:%I/%O balancer:%{BALANCER_WORKER_NAME}e"
TransferLog     /var/log/apache2/yoursite.log

More info

http://www.ducea.com/2008/02/06/apache-logs-how-long-does-it-take-to-serve-a-request/

Get developers Subscribe mFabrik blog in a reader Follow me on Twitter

Logging with LogMan

Jussi Toivola — Mon, 23 Jun 2008 14:55:58 +0000

This is my first post on our company blog and I thought I’d tell you something about LogMan, which is developer’s utility for getting logging messages from Symbian device over a USB cable. It is written by me and mostly on my own time. I started the project because I had to do a Symbian excercise for university course and I thought I’d do something useful instead of quickly tinkering something small and easy.

LogMan supports both C++ and Python. With LogMan, you can send data to the same location from Python and C++ in real-time instead of using log file(s). Browsing through multiple log files can be tedious and you can’t see the debug output while using your application, because on Symbian you must read the file after the debugging session – Symbian cannot share opened files between applications. It is also possible that you create too much log and you run out of Phone internal memory. LogMan helps by removing the use of log files and you never run out of disk space because messages can be stored directly to PC. On simulator, the messages are also sent to RDebug (%TEMP%\epocwind.out). Surprisingly I have not seen a Python module, which would enable use of RDebug. Even though it is very easy to implement. With RDebug, there is no need to open a serial port on simulator for reading the logging messages.

Of course, I tried to use RDebug on device first, but I never got it working. I also tried REcmt, which is supported on S60 only and the service just kept on crashing on my phone. This is why I decided to write LogMan. Plus both are closed software, which effectively prevented me from fixing the problems.

Just wondering what kind of benefit Symbian or Nokia gets from keeping development tools such as these closed? What is there so secret about them? It didn’t take me very long to write the first working version. *sigh*

The use of LogMan is similar to RDebug. There are static class methods, which are a bit slower but easier, and instance methods. Check the project’s homepage for more examples.

#include "LogMan.h" //RLogMan RLogMan::Log( _L("Hello world ") );

I recently added a new feature for LogMan, which allows you to log stack and heap usage of the current thread with one function. When you are unsure about your heap or stack usage, these might come handy. Of course there is some memory used when calling these so take that into consideration. Python can access MemoryInfo only, which logs both stack and heap (Well, I got a bit lazy at that point). There are equivalent macros for these, so check them out from “logmanutils.h”.

// Store this as a member of your class, for example RLogMan logman;

logman.HeapInfo();
logman.StackInfo();
logman.MemoryInfo(); // Both stack and heap
logman.Close();

The output from MemoryInfo is something like this:
StackInfo Free:1039000, Used:9576, Size:1048576 HeapInfo Free:25856, Used:101004, Size:126860

Browser for PyS60

I have been trying, unsuccessfully, to get Browser Control working on PyS60. In a nutshell, CBrCtlInterface wrapper for Python. I have developed it against PyS60Community version in Launchpad. See /src/appui/appuifw/. I have used LogMan extensively to debug the extension so if you want a real example, check out “browsercontrol.cpp”.

Browser Control would allow one to embed a browser into his PyS60 application, which would be quite cool. No need to do user interfaces with “appuifw”, which is not very portable. With Browser Control, one could create his user interface with html and javascript, which are a lot more portable indeed. Less work leads to more time. And what is time?… it’s money. Or so I have heard. And being able to handle events with Python instead of C++ is another bonus.

Unfortunately, the API is not very stable as you can see by searching for “CBrCtlInterface” at Forum Nokia. The browser worked fine on the simulator with small pages such as “www.google.com”, but it crashed miserably with larger pages. The crash happens in browsercontrol.dll when calling e32.Ao_sleep() in Python. On device it was unable to open any page and crashed instantly when trying to load one. With LogMan, I was able to verify that the crash happened in e32.Ao_sleep() on device also. This reminds me to test on device all the time, which I didn’t do for the first versions. But this is why I added the memory logging feature to LogMan, but it only revealed that I was not out of stack or heap. At least not before the browser started to mess around.

I tried to compile the WebKit myself to see what is going on, but the build instructions didn’t work and the build scripts are written in Perl(my eyes started to hurt). What a mess. I don’t wonder anymore if there are bugs in browsercontrol.dll. I finally gave up because my idea pool dried up. Any help getting the wrapper working would be very much appreciated.

Plans for LogMan

I’m planning to add remote shell interface so that you could control your phone from PC. I want access to the file system first. Transfer files, list folders and such. If you have TCB rights ( or hacked phone ) you can speed up development remarkably by simply replacing your binaries in \sys\bin or your Python files with new versions. No need to install sis files and fiddle with certificates and play with memory cards. It would be so nice… I can use 1 day in a week for a personal project so this may happen in near future