Skip to content

{ Monthly Archives } February 2007

darcs almost perfect.

Darcs is a really cool SCM system. We use it for Thousand Parsec and I really like all the way it works. However there are some very annoying bugs which keep pissing me off. So now I’m going to rant about what these problems are,

Using Massive amounts of memory with Binary files
Darcs is an absolute memory hog when you try and use it to manage even small binary files. I have seen it balloon to over 700mb on a little 7mb binary file. This keeps biting me in the arse because we are using darcs to manage our web repository. It’s starting to get so bad I’m thinking of switching that repository to subversion. We have already had to convert our media repository to subversion because of this problem :-/.
Not using good terminal interaction.
Darcs is design to be used interactively. However it crashes if you send it weird control characters or other strange stuff. It also doesn’t let you use the cursor keys to change long message title and such. Currently I have to use the “add long comment” if I want to fix anything in a patch title (which is very annoying for small patches).
Darcs being written in Haskel
By writing Darcs in Haskel I am unable to try and fix the above bugs. It is also the main reason using Darcs under “alternative” operating systems (such as Windows or Mac OS X) sucks so much (getting a working Haskell compiler is a real chore). Even when you do get it working it doesn’t quite fit properly (and the console IO is even more fragile). Mathematical correctness doesn’t mean you code doesn’t have any bugs. Being unusable is very much a huge bug.

It’s starting to get to the Stage that I’m considering other SCM tools such as Mercurial or maybe even monotone.

2007 02 26 mithro darcs Comments (0)

Using Tailor – Creating Subversion Repository for Thousand Parsec

To try and get our activity rating up on SourceForge I’m in the progress of trying to mirror our darcs repositories in SourceForge’s SVN repositories.

This would also be useful for things like oholh as JLP pointed out. It might make it easier for people to track the development version under Windows/Mac as they wouldn’t need to get a working darcs version.

To do this I need to use a program called Tailor which lets you convert between a wide range of different SCM systems. It took me a while to get a combination of tailor, darcs and svn which seems to work okay.

What I ended up with is the following Darcs to Subversion Conversion script.

Each time you run it, it rebuilds the svn repository from scratch. This was useful during testing so that I could get my comment formatting correct (and fiddle with the other settings).

A ran into another problem however, to use Tailor and get nice author and actual commit date you need the special hook installed. This hook is just an empty pre-revprop-change. However, SourceForge in all their wisdom don’t have support for this yet, there is a pending change request but it hasn’t been touched in about 3 months. Here is hoping they figure out something in the near future. There have been about 20 requests if you do a search for pre-revprop-change.

2007 02 25 mithro darcs
Thousand Parsec Comments (0)

Thousand Parsec Protocol Overview

I’ve been planning to try and get tp04 into draft stage for a while now. However the AI competition and RL has kept me really busy so I haven’t got time to do the draft yet. I am going to however go over some highlights of the new tp04 protocol.

Firstly, what does tp03 currently have?

Dynamic Orders
Thousand Parsec servers can define new orders and clients can automatically discover these orders and show the user quite a bit of detail about what arguments should be given to the order.
Comprehensive Design Support with tpcl
Designing things is a very important part of most empire building games. tp03 includes support for building designs out of “components”, these components dynamical describe their properties and requirements. They can be either calculated on the server or calculated on the client. It is quite possible to have components which require other components, forbid other components via either specific exclusion or by using property values.
Dynamic Resources
Resources which are used for doing things are dynamical described by the server.
Message and Board system
Support for a wide variety of messages including referencing objects related to a message (IE This message came from this object). Servers can also have both private and public boards.
Partial Design Discover
Although no server currently implements a system where as you can only discover partial information about enemy designs, this option is total supported.

The next version of the tp protocol tp04 will have a bunch of new features. It is going to be built in an incremental way based on tp03. All the highlights of tp03 will still exist in tp04.

Some of the new highlights include.

Full XML protocol specification
The protocol will be completely specified in an XML document. This will allow more dynamic languages (such as Python, Ruby and PHP) to read in the protocol document and dynamical create the correct data. This doesn’t mean our good documentation is going away (for those people who want to implement it the “hard way”), instead it will be more accurate and contain better linking, lot more useful tables and even an index. The documentation will all be generated using XSLT from the protocol XML document so it will also always be current.
Meta Protocol definition
A definition on how to talk the “meta protocol”, IE talking to the metaserver and find local games will be specified. It will be almost identical to the current protocol specified separately.
Filter Support
The protocol will support filters such as encryption and compression (or even a 32bit aligned strings filter), there will be a way to negotiate which filters to use.
Difference Support
The protocol will include (and servers will be required to support) a proper method for downloading “what has changed” lists. This will be extended from the current “get id sequence” stuff but made so it doesn’t require downloading every single ID in the universe to find out the differences.
Dynamic Objects
Like how servers can define new a interesting order types, with tp04 servers will also be able to do the same for objects. A wide variety of object properties types will exist, from Graph like properties to just plain strings. This will rapidly allow many more advanced rulesets to exist.
Old Data support
As a side effect of Dynamic Objects, object properties will be able to be “aged”. This means that if you could detect/determine the value in the past, but can’t determine the value now, the client will be able to understand this.
Multiple Instruction queue support
As another side effect of Dynamic Objects, objects will be able to have multiple instruction queues. These will allow for things like “standing battle orders” and “research queues” (and probably plenty of other things I can’t think of at this very moment).
Media support
The current “media support” is just a hack in tpclient-pywx. The dynamic objects will allow proper specification of what media should be used for objects and such.
Research support
The protocol will include support for figuring out which “Research options” are available. It will support a wide range of research methods too (from researching for a specific object, to researching in a general area).
EOT Support
There will be support for things like saying “I’m Done” and “Please end the turn now.”. This will mean we are no longer just stuck with the EOT at a certain time problem like in tpserver-cpp (or when admin runs a special program like in tpserver-py).
Frame Type Versions
Support for changing frames (in a backward compatible way) separately. This will allow better updates of the protocol without having to do a complete new version.

tp05 will very much be another incremental version on top of tp04, the following is a list of what is planned for tp05 (and will not be included in tp04).

History Support
Servers should start worrying about storing history as it will definitely be added in tp05.
Trading or Diplomacy
Diplomacy will still be able to be done via sending messages to each other, however the server will not support guaranteed diplomacy and trading. (IE If a person says they won’t attack you, the server will have no knowledge of this and they could still attack you.)
Player / Race Separation
There will be no specific support for a player controlling multiple races or a race having multiple players until tp05.
Other stuff?
Probably plenty of other stuff which I have forgotten.
2007 02 23 mithro Highlights
Thousand Parsec Comments (5)

Compiling tpserver-cpp under windows – Part 3

Well here is a binary, I have tested under Windows 2000 and Windows XP. It should also run under Windows Vista too, but I don’t have a copy of it to test with.

It currently only supports the modules it has been linked against. This means that it will only support MiniSec and no persistence modules.

You can download it using the following link Windows Binary of tpserver-cpp.

2007 02 21 mithro Thousand Parsec Comments (0)

Compiling tpserver-cpp under windows – Part 2

Well it’s finally done. I managed to get tpserver-cpp to compile under Windows. I just need to fix a few things and then I’ll upload “unsupported” binaries.

Here is a picture of it all working!

2007 02 20 mithro Thousand Parsec Comments (0)

Compiling tpserver-cpp under windows.

I had hoped that compiling tpserver-cpp under windows would be easy. Turns out I was very, very wrong. I am even going the easy route of using Cygwin to handle all the select/signal/pthread stuff which wouldn’t work easily under Microsoft products.

Compiling it was relatively easy once I had all the dependencies installed. Cygwin comes out of the box with guile (1.6 and 1.8) which was the only dependency I was concerned about. (Everything else is pretty standard.)

However, this is where I ran into a problem. The server uses shared libraries to contain important modules like game rule data and persistence. This doesn’t work to well under Windows. At first I should it should just be an easy step of convincing the autotools to build .dll versions of the modules. Turns out dll’s aren’t at all like .so files. They have to have “no unresolved symbols” which makes it very hard to do what tpserver-cpp does (IE modules use the core functions in the main server like the logger).

Next step was to find out what other applications did, I found this library made by libtool guys called ‘libltdl’ which lets you “fake” dlopen stuff. So I “ported” tpserver-cpp to use this instead of just a raw dlopen. (This should also make tpserver-cpp more portable to such weird operating systems as BeOS and HPUX.) Dunno if Lee will like it or not 🙂

Still not done yet but it’s looking much more hopeful.

2007 02 20 mithro Thousand Parsec Comments (0)

Swap Nice?

It would be nice if you could say “swap out these applications at a last choice”.

The main reason for this is that I use the deskbar application quite a lot. I would really like the system not to swap it out so that it is always really responsive.

Searching google doesn’t turn up anything all that useful. I did recently find memlockd which would be quite useful for Servers.

2007 02 18 mithro Ideas Comments (0)

Resume

2007 02 18 mithro Uncategorized Comments (0)