Mithro rants about stuff

For something a bit different, I decided to work on making embedding Scheme in Python easier. I’ve previously been using the cool PyScheme, however it hasn’t been updated in quite a long time (since 2004) and is quite slow.

The reason I would want to do something crazy like this is that Thousand Parsec use a subset of Scheme called TPCL. The is used to transmit information from the server to clients about rules for creating designs. Servers also need to be able to parse TPCL for “dumb clients” which can’t parse TPCL for themselves.

Recent developments by DystopicFro on his Summer of Code project, a Ruleset Development Environment have meant that he also needs the ability to parser TPCL (and specifically the ability to detect errors). This got us chatting about PyScheme and it’s inadequacies.

What I have decided to do is create a module called SchemePy (pronounced Skimpy). On platforms where speed is of no concern, we will fall back to using a modified version of PyScheme. However, we can also use C scheme systems such as Guile (or other libraries) to improve speed.

Why have multiple implementations? It stops us from using custom things in one scheme implementation which are not compatible with other implementations. It also makes installation easier for the user, as they are much more likely to already have a compatible scheme library installed. Different scheme’s also have different speed advantages.

So far I have got the Guile wrapper 95% working. It’s written mainly in Python using ctypes. I needed a small C helper module as well because of the extensive Macro’s used by Guile. So far, you can convert between Guile and Python types easily, you can register Python functions into the Guile context and exceptions are caught. There is also the ability to pass python objects thru the Scheme environment to Python functions. I would like to thank the guys who hang out on #guile for all their help, it has made doing this wrapper much easier.

I’m happy enough with the outcome. My guess it will be between 10 and 20 times faster then PyScheme, but I’ve yet to do any benchmarking. I’m going to move to wrapping mzscheme too soon enough. It should be much easier to do now that I have gotten most of the hard stuff sorted out. I think a lot of it will be common between the implementations.

What I really need to do is get a test-suit working. Once I have more then one implementation working it will be very important to make sure that they all work the same way, the only way I can see to do that properly is to have a test suite which I can run every implementation against.

One thing which might be really cool to investigate is using a similar system to lython which compiles Lisp s-expressions directly to python byte code. If this was done well it should be the fastest method as it would mean no type conversion needs to be done.

Overall this has been quite a good learning experience. I have improve my ctype skills quite a lot (this wasn’t my first ctypes wrapper, that being libmng-py, a Python wrapper around libmng). I also understand how Scheme works quite a lot better now.