I wrote FastComplete as a solution to this problem. The tool creates a local cache of links on your hard drive to everything in your $PATH. It uses a couple of tricks to make sure all the stats remain locally, while still allowing the remote file to change without needing to update the cache. Linux should also keep this information in memory disk cache making tab completion almost instant again. Yay!

You can find FastComplete at https://github.com/mithro/rcfiles/blob/master/bin/fastcomplete It is a stand alone python program which shouldn’t have any non-core dependencies. The usage documentation is as follows;

Fast complete creates a local disk cache of your path.
It’s specifically designed to make bash tab complete run much faster. The correct fix would be to add caching to bash, but it was to hard to do so.

To find out what path fastcomplete is currently using:

> ~tansell/bin/fastcomplete # Found 3977 commands export PATH=/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db

To get fastcomplete to rebuild it’s cache:

> ~tansell/bin/fastcomplete --rebuild # Using path of '/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db' # Cache /usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a did not exist! Rebuilding.... # Looking in: /home/tansell/bin (execv) ... # Looking in: /usr/bin (symlink) ... # Looking in: /home/build/google3/ads/db (execv) # Found 3977 commands export PATH=/usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a

To use fastcomplete all the time add the following as the *LAST* line in your ~/.bashrc file. Fastcomplete will echo some output to stderr so you can see what is happening.

# Create a cache of the command eval `~tansell/bin/fastcomplete $PATH`

One of the best talks I went to Linux.conf.au last year was the Ksplice talk. This is a wonderful tool which allows people to develop “hot patches” so that you never need to reboot again. The developers have done some very cool work which means for 88% of patches a hot fix can be generated automatically. The presenter was a really great speaker too, during the talk he explains some advanced concepts (like hot to fix-up memory structures) I was able to easily understand it all. I can’t watch until I never have to reboot my Linux machines again!

I also attended Rusty‘s tutorial on hacking lguest. Two years ago I submitted a very important patch to lguest which is now included in the kernel. This year I didn’t get as far as I had wanted mostly because I had not gotten enough sleep the night before. I did however submit one patch which was accepted. I guess that makes me a kernel hacker .

Sadly, Rusty won’t be running another lguest tutorial this year, so it’s unlikely I’ll submit a third patch.

I have recently been watching and really enjoying a little known series called Starhunter. At first I thought it was an enjoyable rip off of Firefly, so i was surprised to find out that it was made first.

It has so many of the same elements, the quirky ship mechanic, the captain who doesn’t play by the rules with a darker background, raiders, etc. I have no doubt that Joss Whedon stole many the ideas for Firefly from this series and in the process improving on them in many ways. Sadly, like Firefly this series was canceled way before it reached a wide audience.

You can watch the “second season” (Starhunter 2300), which is closer to a spin off  rather then a second season (most of the characters have changed), on Youtube. I would highly recommend people who enjoyed Firefly check it out. Since its free, what have you got to loose?

While the first season is available on Joost, it is apparently horrible mangled. Sadly, the DVD’s are not available here in Australia so I’m going I’m going to have to buy them from Amazon.

The game comes out of the Open Velociraptor Per Child project’s ideas. The OVPC project aims to give every child in the world their very own Velociraptor. The game simulates what would happen if this project was ever to achieve it’s wonderful but lofty goal.

My main contribution was adding support for (very bad) collision detection, scoring and difficult levels. However, what I spent most of my time was spent making bad programmer art, that of the particularly gruesome child death animation. I like to think it was this artwork that meant we won top spot at the Gaming Miniconf competition, beating the “Shave Bdale” game.

You can download the game from the git repository. Have fun!

Ubuntu provides the mingw32 compilers in the repository so I didn’t think it would be all that hard to get working. The problem is that tpserver-cpp does not have a “native” Windows support but cygwin comes to the rescue and provides a compatibly layer. Using cygwin turned out to not be as simple as using mingw32 compiler with the cygwin headers.

I ended up using crosstool to build my own cygwin compiler. I battled for a long while with the fact that Ubuntu now enables “fortify source” by default. This breaks many versions of things like binutils and gcc (which often do naughty things which fortify source does not like). After I figured out how to disable it, I was still was only able to get an ancient version of gcc to compile (3.3.6) which meant I had to fix a lot of problems in the tpserver-cpp code. I guess someone had to do it eventually, but it was annoying that I was forced too.

I then manually downloaded a bunch of cygwin packages to build a tree for the dependencies (such as boost and guile). This was much faster then trying to compile them on my own. Finally, I was able to build tperver-cpp and create a Windows binary! I can confirm it runs fine under Wine and am now getting friends who are still shacked to Windows to test it there.

It sounds much simpler now, but it took me over a week of work to boil it down to these steps. It was like a constant game of wack-a-mole, once I had solved one problem another popped up.

So what now in this area? I want to get a recent version of the compiler working and preferably build all the dependencies ourselves (rather then rely on the cygwin compiled versions). I would ultimately like to see the cygwin compilers being packaged with Ubuntu/Debian in the same way that the mingw32 compilers are. I don’t know if any of that is likely to happen however as I never seem to have enought time. For now I have uploaded a copy of my cross compiler (It needs to be extracted so it is found in /opt/crosstool).

I hope this helps someone!

As part of speaking at the conference I had to produce a paper. This paper puts into writing a lot of what I have been talking about. I wasn’t going to post it, but after getting a email out of the blue about the topic, I’ve decided to put a copy here on my blog. I might as well also upload the presentation I gave, but it won’t be very interesting by itself. Both are released under a CC-BY-SA.

If there is one thing that any budding game developer (open source or otherwise) should take away from this talk, it is the following:

When a person is looking for normal software, they have an issue to solve.
When looking for computer games, they are looking to be entertained.

This fundamental difference in mindset should drive every aspect of your game, website and release process.

Edit: How could I forget? A big thanks goes out to Leslie Hawthorn who helped proof read the paper and making it much easier to read!

#! /usr/bin/env python # Reading the cookie's from Firefox/Mozilla. Supports Firefox 3.0 and Firefox 2.x # # Author: Noah Fontes <nfontes AT cynigram DOT com>, # Tim Ansell <mithro AT mithis DOT com> # License: MIT   def sqlite2cookie(filename): from cStringIO import StringIO from pysqlite2 import dbapi2 as sqlite   con = sqlite.connect(filename)   cur = con.cursor() cur.execute("select host, path, isSecure, expiry, name, value from moz_cookies")   ftstr = ["FALSE","TRUE"]   s = StringIO() s.write("""\ # Netscape HTTP Cookie File # http://www.netscape.com/newsref/std/cookie_spec.html # This is a generated file! Do not edit. """) for item in cur.fetchall(): s.write("%s\t%s\t%s\t%s\t%s\t%s\t%s\n" % ( item[0], ftstr[item[0].startswith('.')], item[1], ftstr[item[2]], item[3], item[4], item[5]))   s.seek(0)   cookie_jar = cookielib.MozillaCookieJar() cookie_jar._really_load(s, '', True, True) return cookie_jar   import cookielib import os import sys import logging import ConfigParser   # Set up cookie jar paths def _get_firefox_cookie_jar (path): profiles_ini = os.path.join(path, 'profiles.ini') if not os.path.exists(path) or not os.path.exists(profiles_ini): return None   # Open profiles.ini and read the path for the first profile profiles_ini_reader = ConfigParser.ConfigParser(); profiles_ini_reader.readfp(open(profiles_ini)) profile_name = profiles_ini_reader.get('Profile0', 'Path', True)   profile_path = os.path.join(path, profile_name) if not os.path.exists(profile_path): return None else: if os.path.join(profile_path, 'cookies.sqlite'): return os.path.join(profile_path, 'cookies.sqlite') elif os.path.join(profile_path, 'cookies.txt'): return os.path.join(profile_path, 'cookies.txt')   def _get_firefox_nt_cookie_jar (): # See http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/473846 try: import _winreg import win32api except ImportError: logging.error('Cannot load winreg -- running windows and win32api loaded?') key = _winreg.OpenKey(_winreg.HKEY_CURRENT_USER, r'Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders') try: result = _winreg.QueryValueEx(key, 'AppData') except WindowsError: return None else: key.Close() if ret[1] == _winreg.REG_EXPAND_SZ: result = win32api.ExpandEnvironmentStrings(ret[0]) else: result = ret[0]   return _get_firefox_cookie_jar(os.path.join(result, r'Mozilla\Firefox\Profiles'))   def _get_firefox_posix_cookie_jar (): return _get_firefox_cookie_jar(os.path.expanduser(r'~/.mozilla/firefox'))   def _get_firefox_mac_cookie_jar (): # First of all... result = _get_firefox_cookie_jar(os.path.expanduser(r'~/Library/Mozilla/Firefox/Profiles')) if result == None: result = _get_firefox_cookie_jar(os.path.expanduser(r'~/Library/Application Support/Firefox/Profiles')) return result   FIREFOX_COOKIE_JARS = { 'nt': _get_firefox_nt_cookie_jar, 'posix': _get_firefox_posix_cookie_jar, 'mac': _get_firefox_mac_cookie_jar }   cookie_jar = None try: cookie_jar = FIREFOX_COOKIE_JARS[os.name]() except KeyError: cookie_jar = None   path = raw_input('Path to cookie jar file [%s]: ' % cookie_jar) if path.strip(): # Some input specified, set it cookie_jar = os.path.realpath(os.path.expanduser(path.strip()))   if cookie_jar.endswith('.sqlite'): cookie_jar = sqlite2cookie(cookie_jar) else: cookie_jar = cookielib.MozillaCookieJar(cookie_jar)

Edit: The latest version of this code can be found at http://blog.mithis.com/cgi-bin/gitweb.cgi and includes numerous fixes and updates.

This stopped working when I upgraded to Ubuntu Intrepid. Even more strangely while the scripts are still installed, even they are never called.

It appears that thanks to moving towards HAL (which is probably a “Good Thing”) these scripts are no longer used. The scripts which are used can be found in /etc/pm/. Not only has the location changed, but the script format has too.

Previously, my script was found in /etc/acpi/resume.d/99-custom.sh looked like the following,

#! /bin/sh # Turn off the CD drive and the bluetooth device echo 1 > /sys/devices/platform/sony-laptop/cdpower echo 0 > /sys/devices/platform/sony-laptop/cdpower   echo 1 > /sys/devices/platform/sony-laptop/bluetoothpower echo 0 > /sys/devices/platform/sony-laptop/bluetoothpower

Now my script script must be found in /etc/pm/sleep.d/10-custom and looks like the following,

#!/bin/sh -e case "$1" in resume) # Turn off the CD drive and the bluetooth device echo 1 > /sys/devices/platform/sony-laptop/cdpower echo 0 > /sys/devices/platform/sony-laptop/cdpower   echo 1 > /sys/devices/platform/sony-laptop/bluetoothpower echo 0 > /sys/devices/platform/sony-laptop/bluetoothpower ;; esac

The main reason I’m posting this on my blog is that this change does not seem to be documented anywhere. Searching on Google for things like “resume script intrepid” or “/etc/acpi/resume.d intrepid” does not come up with anything useful. Hopefully some people will find this helpful.

Python dutifully respects the $LANG environment variable on the terminal. It turns out that a lot of the time this variable is totally wrong, it’s set to something like C even though the terminal is UTF-8 encoding.

The problem is that there is no easy way to change a file’s encoding after it’s open, well until this horrible hack! The following code will force the output encoding of stdout to UTF-8 even if started with LANG=C.

# License: MIT try: print u"\u263A" except Exception, e: print e   import sys print sys.stdout.encoding   from ctypes import pythonapi, py_object, c_char_p PyFile_SetEncoding = pythonapi.PyFile_SetEncoding PyFile_SetEncoding.argtypes = (py_object, c_char_p) if not PyFile_SetEncoding(sys.stdout, "UTF-8"): raise ValueError   try: print u"\u263A" except Exception, e: print e

This code is useful if you want to access something behind an authentication gateway and you also access the page through your web browser. You can also use this code to convert a sqlite database into a cookie file CURL can read.

I didn’t write this code, it was written by Noah Fontes when we where doing some scraping of the Google Summer of Code website (before I joined Google).

#! /usr/bin/env python # Protocol implementation for handling gsocmentors.com transactions # Author: Noah Fontes nfontes AT cynigram DOT com # License: MIT   def sqlite2cookie(filename): from cStringIO import StringIO from pysqlite2 import dbapi2 as sqlite   con = sqlite.connect(filename)   cur = con.cursor() cur.execute("select host, path, isSecure, expiry, name, value from moz_cookies")   ftstr = ["FALSE","TRUE"]   s = StringIO() s.write("""\ # Netscape HTTP Cookie File # http://www.netscape.com/newsref/std/cookie_spec.html # This is a generated file! Do not edit. """) for item in cur.fetchall(): s.write("%s\t%s\t%s\t%s\t%s\t%s\t%s\n" % ( item[0], ftstr[item[0].startswith('.')], item[1], ftstr[item[2]], item[3], item[4], item[5]))   s.seek(0)   cookie_jar = cookielib.MozillaCookieJar() cookie_jar._really_load(s, '', True, True) return cookie_jar