Welcome to RocketMonkeys.com!

This is my personal site, where I store my rants, pictures, and movie reviews. Have a look around, register and leave comments.
-James

Paranoid Programming: no such thing as optional braces

Posted by james on Nov 22, 2009 12:00 AM

The more I program, the more paranoid I become. It's started to become more of a game of statistics and chances than cleverness or brilliance. No matter how smart or experienced I am, there's an X% chance that I'll introduce a bug. If I change the way I code, that becomes a slightly smaller percentage. It's all about methodology. When you create programs for an assignment in school, it only matters if it works once, for the specific dataset from the assignment, and then it's thrown out.

In the "real world", you have to deal with maintenance. That changes everything.

So, one programming method I've taken up most recently: always use braces. This doesn't apply to python, but it does for Javascript, C++, and other C-like languages. Clever programming gives you this:

if (some_condition)
function1();

x=3;
function2();


However, it's just too easy to screw this up later when adding lines:

if (some_condition)
function1a();
function1b();

x=3;
function2();


Now you've got a bug. Worse yet, it's a silent fail (the bane of programming), which allows the program to run but give completely wrong results.

So, always use braces:

if (some_condition) {
function1();
}

x=3;
function2();


It's harder to mess up that code. The 12% chance of introducing a bug into those 3 lines of code just became 8%. It's all about method.

Paranoid Programming: beware the "else" statement

Posted by james on Nov 18, 2009 12:00 AM

It's very typical to have something like this:

if (num_items == 0) {
// handle no items here
} else if (num_items == 1) {
// handle one item here
} else {
// handle more than one item here
}

The comments make this huge flaw very obvious. Without the comments, it'd be harder to spot: what if num_items is -1? This falls through the cracks, since we use the else as a "catch all" bucket, them assume that it's valid data. Unless we've validated the data before, this is a bug. But even if you remember most of the time to validate data, one time you forget and you've introduced a bug. So you're simply shifting the bug from one place to another.

Another variation of this:

if (num_items == 0) {
// handle no items here
} else if (num_items == 1) {
// handle one item here
} else if (num_items > 1) {
// handle more than one item here
}

This code is "more correct", but still has the same bug; if num_items is -1, then your code wont execute in any of the if() blocks. This may be intentional, but most often it's not.

So to prevent these common and subtle bugs, always use the else as a catch-all error.

if (num_items == 0) {
// handle no items here
} else if (num_items == 1) {
// handle one item here
} else if (num_items > 1) {
// handle more than one item here
} else {
// Shouldn't get here, bad input
assert(false);
}


It's so easy to add the "else { assert(false); }", and it makes your code so much safer. Validating user input doesn't replace this. If you refactor an internal function and it starts calling this function with bad data, or if you make a mistake further up in the same function, you're still sunk unless you're being paranoid.

Moral: always be explicit with if/else conditions, don't rely on else to catch the "probably is good data" unless you're sure, and use else/asserts to let you know when data isn't what you expected.

Auto reload with Apache, mod_wsgi, and Django on Windows

Posted by james on Nov 11, 2009 12:00 AM

Django has the built-in 'manage.py runserver', which is very convenient since it auto-reloads when code is modified. On unix, you can set WSGI to auto reload when code is modified, but on windows it's trickier. I've figured out how to do this by modifying the code at http://blog.dscpl.com.au/2008/12/using-modwsgi-when-developing-django.html.

Here's the line in my apache httpd.conf file:

WSGIScriptAlias /tracker c:/webpage/tracker/tracker/apache/tracker.wsgi

And in my tracker.wsgi file:

# This causes apache/WSGI to auto-reload django whenever a python file changes.
import monitor
monitor.start(interval=1.0)

This assumes that your app's main folder is in the python path, so that 'import monitor' can find monitor.py. This is my modified monitor.py. Really I just had to add the kill() function (from the python FAQs) and modify the _restart() function to use it. And yay! Now I can run django apps in apache via mod_wsgi and enjoy auto-reload as I develop. The reloading seems faster than the django runserver, which is a nice bonus.

The only downside is that django output is now in the main apache log. I'll have to figure that out later.

# This file monitors a path for changes

import os
import sys
import time
import signal
import threading
import atexit
import Queue

_interval = 1.0
_times = {}
_files = []

_running = False
_queue = Queue.Queue()
_lock = threading.Lock()

def kill(pid):
    """kill function for Win32"""
    import win32api
    handle = win32api.OpenProcess(1, 0, pid)
    return (0 != win32api.TerminateProcess(handle, 0))

def _restart(path):
    _queue.put(True)
    prefix = 'monitor (pid=%d):' % os.getpid()
    print >> sys.stderr, '%s Change detected to \'%s\'.' % (prefix, path)
    print >> sys.stderr, '%s Triggering process restart.' % prefix
    if hasattr(os, 'kill'):
        os.kill(os.getpid(), signal.SIGINT)
    else:
        # Windows
        kill(os.getpid())

def _modified(path):
    try:
        # If path doesn't denote a file and were previously
        # tracking it, then it has been removed or the file type
        # has changed so force a restart. If not previously
        # tracking the file then we can ignore it as probably
        # pseudo reference such as when file extracted from a
        # collection of modules contained in a zip file.

        if not os.path.isfile(path):
            return path in _times

        # Check for when file last modified.

        mtime = os.stat(path).st_mtime
        if path not in _times:
            _times[path] = mtime

        # Force restart when modification time has changed, even
        # if time now older, as that could indicate older file
        # has been restored.

        if mtime != _times[path]:
            return True
    except:
        # If any exception occured, likely that file has been
        # been removed just before stat(), so force a restart.

        return True

    return False

def _monitor():
    while 1:
        # Check modification times on all files in sys.modules.

        for module in sys.modules.values():
            if not hasattr(module, '__file__'):
                continue
            path = getattr(module, '__file__')
            if not path:
                continue
            if os.path.splitext(path)[1] in ['.pyc', '.pyo', '.pyd']:
                path = path[:-1]
            if _modified(path):
                return _restart(path)

        # Check modification times on files which have
        # specifically been registered for monitoring.

        for path in _files:
            if _modified(path):
                return _restart(path)

        # Go to sleep for specified interval.

        try:
            return _queue.get(timeout=_interval)
        except:
            pass

_thread = threading.Thread(target=_monitor)
_thread.setDaemon(True)

def _exiting():
    try:
        _queue.put(True)
    except:
        pass
    _thread.join()

atexit.register(_exiting)

def track(path):
    if not path in _files:
        _files.append(path)

def start(interval=1.0):
    global _interval
    if interval < _interval:
        _interval = interval

    global _running
    _lock.acquire()
    if not _running:
        prefix = 'monitor (pid=%d):' % os.getpid()
        print >> sys.stderr, '%s Starting change monitor.' % prefix
        _running = True
        _thread.start()
    _lock.release()

Trying to Do Less

Posted by james on Nov 8, 2009 12:00 AM

I've been reading Jason Fried's blog at 37Signals lately, and one of his guidelines is to do less. I've been trying to follow it lately.

Programmers like me are cursed with too much imagination. When we see a problem, we don't think of a good solution - we think of the "best" solution. But I've always thought that an engineer's task is to create the most appropriate solution for the given situation, not some overall "best" solution.

When I work on a new feature, I'm always tempted to create a robust, scalable, fancy solution. Instead I try to do less. Steve Jobs used to write "Real artists deliver". Getting caught up creating a very fancy solution that does more than what you need just ties you up and burns hours (and money).

Why spend all that time making a really good but really complicated feature? Just do what's needed and move on. You can always go back and add the parts you really need. But most likely you wont end up adding those extra features, at least not for a while. The simpler solution will work just fine, and you can move on to what's really needed.

So instead of burning hours on an overly complicated feature and waiting forever in the process, in the same time you can just do what's need and deliver the product. Most of the time it's not a matter of not having enough time for your requirements... it's a matter of giving up on creating the "best", and going with "just enough". This becomes a lot more important when I'm working on my own projects. With no guidelines, I can spend tons of time on a very interesting problem and finally come up with a clever solution... only to realize after that I could have done something much simpler and just gotten it done.