Mithro rants about stuff

nMigen – a new improved Migen developed by whitequark!

mithro — Sat, 02 May 2020 10:12:03 +1000

As people may know, I’m a big supporter of the Migen and associated LiteX ecosystem. As of today if you are starting a new project today, I would instead recommend that you chose to use nMigen instead.

nMigen, A refreshed Python toolbox for building complex digital hardware is a project that whitequark has been working on for the last couple of years and I’m really liking the new improve syntax and the fact that it interfaces directly to Yosys.

There seems a lot of interesting things happening in the nMigen ecosystem;

Lambda Concept has built an 32-bit RISC-V soft processor called Minerva. The CPU core currently implements the RISC-V RV32IM instruction set, is pipelined on 6 stages and largely inspired by the LatticeMico32 processor.
Robert Baruch has a multipart series where he uses nMigen — a Python toolbox — to recreate a 6800 CPU like the one used in many vintage video games and pinball machines.
Kate Temkin is also working on creating a new USB 2.0 protocol stack in nMigen for the LUNA: a USB multitool (& nMigen library). I believe that she is also hoping to work on a USB 3.0 stack using the ECP5 high speed transceivers by rewriting the Daisho core and the work that Enjoy Digital did to make a transceivers adapter.

Using “IdentitiesOnly” without key files

mithro — Tue, 03 May 2016 12:26:52 +1000

If you want to restrict the keys that ssh tries when connecting to a server, you need to use the IdentityFile and IdentitiesOnly configuration options in your ssh_config.

However, a couple of the keys I have are auto generated, with the key being loaded directly into ssh-agent and never written to a file on disk. For hopefully obvious reasons, you can’t dump a private key back out of the agent, but it turns out that IdentityFile only needs the public key which you can get.

I ended up using the following script to dump the public keys to files;

# Dump the public keys
for KEY in $(ssh-add -l | sed -e's/[^ ]\+ [^ ]\+ \([^ ]\+\) .*/\1/'); do
  if echo $KEY | grep -q '^/'; then
    continue
  fi
  export KEY_FILE="$HOME/.ssh/agent.$(echo $KEY | sed -e's/[^A-Za-z0-9]/_/g').pub"
  echo "Saving $KEY into $KEY_FILE"
  ssh-add -L | grep $KEY > $KEY_FILE
done

Then I added the following to the ssh_config

IdentitiesOnly true IdentityFile ~/.ssh/agent.keyname.pub

When running with “ssh -vv” I see the following in the output;

debug2: set_newkeys: mode 1 debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug2: set_newkeys: mode 0 debug1: SSH2_MSG_NEWKEYS received debug1: SSH2_MSG_SERVICE_REQUEST sent debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug2: key: /home/tansell/.ssh/agent.keyb.pub (0x2257da0), explicit debug1: Authentications that can continue: publickey,password debug1: Next authentication method: publickey debug1: Offering ECDSA public key: //home/tansell/.ssh/agent.keyb.pub debug2: we sent a publickey packet, wait for reply

TimVideos.us and Google Summer of Code 2016!

mithro — Tue, 15 Mar 2016 17:32:13 +1000

This is a cross post from the HDMI2USB website.

TimVideos.us and Google Summer of Code 2016!

The TimVideos.us project is happy to announce that it has been selected to participate in the Google Summer of Code for 2016 (GSoC). GSoC is a program where students are paid to contribute to selected open source projects over the northern hemisphere summer, flip bits not burgers!

The application period is now open and students have until 25 March 19:00 UTC to apply to work with the TimVideos.us project. The list of proposal ideas includes contributing to the HDMI2USB and things related to the Numato Opsis.

Due to the focus on hardware, we are very interested in students who are interested in things like VHDL/Verilog and other HDLs, embedded C programming and operating systems and electronic circuit/PCB design. Some possible ideas include;

Working on USB 3.0 support for ultra high speed video capture.

Porting a real operating system such as Linux or RTEMS to our SoC to allow easier development of new features.

Making use of the Ethernet interface for connecting multiple devices together and alternative capture interfaces.

Creating new expansion board and supporting the professional SDI standard.

and many, many more.

If you are interested in applying, your first point of call should be our Google Summer of Code page. Even if you can’t apply to GSoC, you can also help us by forwarding this message to anyone you might think who is interested!

TimVideos.us 2016 New Year’s Resolutions

mithro — Fri, 15 Jan 2016 01:00:58 +1000

This is a cross post from the HDMI2USB website about my plans for 2016 and the TimVideos project. Last year was an exciting time for my personal projects!

TimVideos.us 2016 New Year’s Resolutions

Hello everyone,

Hope everyone has had an awesome start to 2016 so far. As is tradition in many western countries, I thought I would put together some New Year’s Resolutions and reflect on our progress in 2015. I guess more business minded people might call it a “project roadmap” 🙂

TimVideos Project

In 2015, I decided to focus the TimVideos project on the HDMI2USB project. The three key results of this focus were;

Starting and completing a rewrite of the HDMI2USB firmware based on the Migen and MiSoC system developed by M-Labs.

Launching a successful crowdfunding campaign for Numato Opsis, our first open hardware for the HDMI2USB firmware.

Having the HDMI2USB firmware on Atlys boards used in production by multiple people!

Carl Karsten from NextDayVideo in the US for both PyCon ZA and Nodevember.

The DebConf Video team for their MiniDebConf in November.

Myself for recording my own talks on the HDMI2USB project at user groups here in Sydney!

With the success of this focus in 2015, the TimVideos project is going to continue to focus on theHDMI2USB project for 2016 (and I’ll go into more detailed goals shortly).

The TimVideos project has also been mildly successful in collaborating with other open source groups doing things related to video recording and production. In 2016, I hope we can strengthen these bonds and forge new ones. Some specific goals around this include;

Getting the TimVideos project to join Software Freedom Conservancy (or similar organisation).

Figure out the right way to collaborate with the C3VOC team on voctomix and start adding missing features from gst-switch allowing that project to be retired.

Continue to work with supporting groups like NextDayVideo, the DebConf Video team and Linux Australia.

Support and help Mike “Hamster” Field continue to develop a fully open source DisplayPort core.

Collaborate with the apertus° project on high end (4k and greater!) video capture and processing.

HDMI2USB Project

As we are concentrating on the HDMI2USB project, we have some specific goals around that.

HDMI2USB firmware goals;

Refactor the HDMI core to allow support a wider range of interfaces, better debugging and addition of more features. A document about the refactor has been started here.

Add support for the high-speed GTP transceivers and Mike “Hamster” Field open source DisplayPort core.

Get Ethernet support working (on both the Atlys and Opsis boards). The two major Ethernet features are;

Ethernet supports identical capture and control feature set to the USB port.

Allowing HDMI2USB boards to act has “HDMI over Ethernet extenders”.

Support for more hardware;

miniSpartan6+

Digilent Nexys Video

New HDMI2USB designed hardware!

Stretch Goals (1)

Add support for hardware based mixing.

Have either a RTOS or Linux running on the FPGA softcore.

HDMI2USB hardware goals;

(By end of year) Development of a low cost PCI-Express capture card.

(By middle 2017) Development of an Opsis V2 based around either an high end Artix-7 or a low end Kintex-7 FPGA.

(1): Stretch goals are things we plan to try and achieve if things go well.

Hope this update give you an idea of what we have planned for 2016! We would love your help making it all possible.

Tim ‘mithro’ Ansell

First V2 “HDMI2USB Production Board” constructed!

mithro — Sun, 05 Jul 2015 14:47:57 +1000

For the last year and a half, I have been working with Numato Labs to create a “HDMI2USB Production Board” for our HDMI2USB firmware that was originally developed on the Digilent Atlys board. On Friday, they sent me a picture of the first constructed board!

HDMI2USB “Production Board” Version 2

At the end of last year, we decided to abandon our first attempt and start again from scratch, this picture is the result of that work. Some of the reasons we decided to start from scratch was;

A “de facto standard” for locking HDMI ports was established and low cost connectors became available. This meant we no longer needed to support both DVI and HDMI connectors, reducing the complexity significantly and solving some persistent issues.
Not only did the cost of Spartan 6 parts with high speed “GTP” transceivers drop but our understanding of how to use them increased. This would allow us to create a board which natively supports DisplayPort.
The idea streaming via not only USB, but also Ethernet became a stronger possibility, meaning the extra cost of adding ethernet was now worth it.

Overall, our board has the following differences with the Atlys board;

Has DDR3 memory instead of DDR2, increasing the memory bandwidth.
Has a Spartan S6LX45T with the GTP broken out to DisplayPort headers.
Has a PCI-Express style expansion connector instead of the expensive VHDCI connector, allow much cheaper expansion boards.
Has all the extra pins on the FX2, increasing the potential USB interface options.
Has control over all the HDMI functionality, including hot plug and CEC functionality.
Removes parts we don’t need such as the audio, buttons, switches and LEDs.
Mounts in any ITX style computer case.
Adds UTMI USB (as well as the Cypress FX2)
Adds MicroSD connector.

HDMI2USB – Production Board Bring Up – Day 8 (28th July 2014)

mithro — Tue, 29 Jul 2014 00:13:02 +1000

Updated the HDMI2USB variant spreadsheet with new Numato board pin out.
Successfully generated a HDMI2USB firmware for Numato board with new pin information. That fixes the constraint issue!
Successfully generated a HDMI2USB firmware for Numato board with second receive port (RX2) disabled!
- Was able to view the HDMI2USB test image output on TX2!
- Was able to capture the HDMI2USB test image via mplayer!
- Was unable to detect a screen on RX1.
Created a “flash.sh” script to use libFPGALink to flash the FPGA and then load the Cypress firmware.
Wrote a Python script to interrogate the CDC serial port for status of the firmware.

HDMI2USB – Production Board Bring Up – Day 5, 6 and 7 (25th, 26th and 27th July 2014)

mithro — Mon, 28 Jul 2014 23:39:31 +1000

25th July 2014

Setup xob‘s hacked up colormake in our build system. Nice colorized output now!

Fought more with

ERROR:Place - Constraint Resolved NO placeable site for hdmiMatri_Comp/dvi_rx1/ioclk_buf issue

Watched Paul Fenwick’s OSCon Keynote.

Streaming System Hacking

Made the configuration system better and adding a lot of comments to the json file (which isn’t really valid json any more).
Made pycon2internal.py slightly better and able to parse PyOhio format (based on the PyCon US format).
Added hack to make event2internal.py to use pycon2internal.py at some periods.
Set up a bunch of encoders in the Oregon EC2 region.
Get access to www.timvideos.us again and deployed the updated website.

26th July 2014

Went into the Adelaide Hackerspace and created a cable to interface the USB3300 ULPI chip to 2xPMOD headers.
Got streaming working for day 1 of PyOhio, it was late as we were disorganised.
aps-sids foolishly pointed out a bug in the title of the streaming system, so I taught him about how we actually flumotion for an event.

27th July 2014

Did a lot of spreadsheet hacking on the production board pin planning spreadsheet;
- Added half bank / BUFIO2 regions.
- Added dedicated clock pin information.
Started work on reshuffling the pins to fix the timing issue.
Helped aps-sids get Flumotion working on the latest Twisted release. Turned out that someone else had already done most of the work and only a couple very small fixes where needed. aps-sids reported that he was now able to run flumotion on Ubuntu Trusty!

HDMI2USB – Production Board Bring Up – Day 4 (24th July 2014)

mithro — Fri, 25 Jul 2014 12:18:12 +1000

Generally slow progress today, but some success.

Able to successfully load libFPGALink firmware onto Cypress chip and programmed the FPGA using it. Yay!
Wrote a test firmware based on http://hamsterworks.co.nz/mediawiki/index.php/DVI-D_Serdes to transmit 720p output from both the TX ports.
- Images are successfully received on both devices but there is a weird artifact in the image.

Rohit’s VGA expansion board was received, but yet to turn it on. Looking forward to seeing that working live!

Set up the streaming system for Carl and PyOhio
- Tried to figure out where www.timvideos.us was hosted.

HDMI2USB – Production Board Bring Up – Day 3 (23rd July 2014)

mithro — Thu, 24 Jul 2014 01:54:52 +1000

Looked into if Travis CI has IPv6 connectivity. It appears so!
Investigated git server side hooks as a method to run CI on build.timvideos.tv
- Travis-CI does a git push to build.timvideos.tv which does a make in the post-recieve.
- Issues;
  - For non-pull requests we could encode the private key using Travis’s encrypted environment variables.
  - What about pull requests? Mean anyone can send arbitrary code to build.timvideos.tv – Maybe use QEmu’s COW support and reboot after each build?
- Looked a jig for a nice way to write the git hooks.
  - Sent a pull request to fix some documentation.
  - Really designed to run as a “pre-commit” hook locally on a person’s machine rather than a server side hook. Logged an issue to discuss that.
  - Has a lot of nice plugins for Python development already.
  - Would allow us to add hdl-pretty as a commit hook.
- Alternatives to jig appear to be;
  - https://github.com/icefox/git-hooks – Written in bash
  - https://pypi.python.org/pypi/git-pre-commit-hook – Written in python
Got my “fake” Xilinx Platform Cable USB (Model DLC9G) working under Linux – Full instructions can be found at https://github.com/timvideos/HDMI2USB/wiki/Xilinx-Platform-Cable-USB-under-Linux
- The device was purchased from eBay on the 11th July, costed $37 USD with shipping and arrived at Joel’s house on Monday.
- First issue was /opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/setup_pcusb didn’t understand I had udev and was trying to install for the ancient hotplug.
  - Fixed by changing line 26 from TP_USE_UDEV=”0″ to TP_USE_UDEV=”1″
- Next setup_pcusb didn’t tell udev to reload the udev rules. Had to do that with udevadm control –reload-rules
- Next setup_pcusb didn’t ask me to install fxload, so I needed to install it with apt-get install fxload
- Next the rules that were installed to /etc/udev/rules.d/xusbdfwu.rules were invalid; they caused the following errors in /var/log/daemon.log

Jul 23 16:40:29 laptop udevd[841]: unknown key 'SYSFS{idVendor}' in /etc/udev/rules.d/xusbdfwu.rules:2
Jul 23 16:40:29 laptop udevd[841]: invalid rule '/etc/udev/rules.d/xusbdfwu.rules:2'
Jul 23 16:46:11 laptop udevd[841]: unknown key 'SYSFS{idVendor}' in /etc/udev/rules.d/xusbdfwu.rules:3
Jul 23 16:46:11 laptop udevd[841]: invalid rule '/etc/udev/rules.d/xusbdfwu.rules:3'

This was fixed with;
- Changing SYSFS to ATTRS
- Changing BUS to SUBSYSTEM
- Changing $TEMPNODE to $tempnode
The little status light then turned on red! Yay!
Was able to do a boundary scan in iMPACT on a Zybo development board after soldering a header onto it.

Received the HDMI2USB production boards created by Numato!
- Started with bunch of stuff to do with LEDs;
  - Logged issues about LEDs not been label intelligently. Such as the “power” LED being labeled “D33”.
  - Researched the DONE net again so I could understand the D2 LED (which should be named D_FPGA_NOT_CONFIGURED or DNCFG for short). Started adding the information to the JTAG/Reset documentation.
  - Figured out why D3 (connected to the Cypress INT1 pin) was only faintly lit.
- Logged a bunch of issue regarding small silk screen fixes to make future boards easier to understand.
- Logged an issue about adding some standoffs in the center of the board for mechanical stability.
- Logged an issue about having a good GND point to connect your probe too.
  - This page describes the two good ways to add a GND test point – http://www.robotroom.com/PCB-Layout-Tips.html
- Starting researching the 5V rail and if we could remove it totally (thus saving a bunch of stuff). Looks like we can, but needs more investigation.
- After replacing the JTAG cable was able to use iMPACT to boundary scan and it found Spartan 6 chip!
Discovered CEC is 3V3 signal.

Non HDMI2USB stuff

Tried to figure out why my home router has decided that it wants to hand out address in the 2001:44b8:31dc:8d01::/64 rather than the 2001:44b8:31dc:8d00::/64 range it use too.
Found a bunch of issues with domains served of ns1.mithis.com as the secondary servers where disabled. Root cause was an old version of PowerDNS failing on TCP zone transfers causing domains to become stale and get dropped from the secondaries. Enabled email notification when secondary disables the zones.

HDMI2USB – Production Board Bring Up – Day 2 (22nd July 2014)

mithro — Wed, 23 Jul 2014 01:32:55 +1000

Answered a bunch of work emails.
Read and answered bunch of PyCon AU related emails.
Fix DLV records for a bunch of domains.
Helped aps-sids with flumotion porting work.

Worked from Joel’s house today to receive the production HDMI2USB boards, they didn’t turn up 🙁
Create a trusty VM on server for Joel to setup HDMI2USB CI on.
Investigated origin of files in HDMI2USB code. Current status seems to be;
- hdl/edid – Written by us (Jahanzeb).
- hdl/hdmi – Mostly written by Xilinx and released in application notes. Need to figure out the exact licensing of the files (are they MIT? Only for use on Xilinx parts?) and move them to third party.
- hdl/jpeg_encoder – OpenCores mkjpeg core. Need to move to third party.
- hdl/usb – Written by us (Jahanzeb).
- ipcore_dir – Cores generated by Xilinx. Need to figure out the exact licensing of the files (are they MIT? Only for use on Xilinx parts?) and move them to third party.
- cypress – Confusing mess. We know this and plan to rewrite the firmware using only FOSS tools.
  - Bunch of proprietary binary blobs.
  - Bunch of headers of unknown copyright.
  - Bunch of code written by us.
Logged a lot of bugs about warnings Xilinx tools are generating when compiling the firmware.
Made generate_ucf.py generate a file for Atlys functionally identical to existing HDMI2USB.ucf file. Was able to successfully build the firmware with generate HDMI2USB.ucf file for the Atlys board.
Started trying to get firmware to compile with the Numato HDMI2USB.ucf file, ran into the following issue

Phase 4.2 Initial Placement for Architecture Specific Features
ERROR:Place - ConstraintResolved NO placeable site for
 hdmiMatri_Comp/dvi_rx1/ioclk_buf

ERROR:Place - SIO has over-constrained componet hdmiMatri_Comp/dvi_rx1/ioclk_buf
 to have to placeable sites. Constraints come from driver constraints AND load
 IO constraints
/bin/sh: line 16: 28981 Segmentation fault (core dumped) map -filter "../ise/iseconfig/filter.filter" -intstyle ise -p xc6slx45-csg324-3 -w -logic_opt off -ol high -xe n -t 1 -xt 0 -register_duplication off -r 4 -global_opt off -mt off -ir off -pr b -lc off -power off -o hdmi2usb_map.ncd hdmi2usb.ngd hdmi2usb.pcf
make: *** [map] Error 139

Rohit GSoC mentoring;
- Emailed Rohit about reimbursement for VGA Board hardware parts.
- Rohit has shipped me a board which should arrive at Joel’s tomorrow.
- Weekly mentoring meeting.
- Reviewed Rohit’s weekly summary blog post.
- Reviewed Rohit’s VGA board V2 schematic.
- Reviewed Rohit’s VGA board V2 PCB.

HDMI2USB – Production Board Bring Up – Day 1 (21st July 2014)

mithro — Tue, 22 Jul 2014 11:16:49 +1000

Wrote snippets for the last 3 days.
slidelint work
- Further updated some slidelint site documentation.
- Committed the slidelint nginx configuration.
Got set up in Joel and Rusty’s coworking space.
Explained to Ajit the idea of routing the debug output to multiple locations. Added a diagram to the Debug Design document.
Fixed command line processing in hdl-pretty code. Started down the path of auto-generating help from the customize stuff and then realised I had more important things to do.
Got most of my way through a Python program to generate UCF files from the HDMI2USB Variants spreadsheet.
Xilinx “Compatible” JTAG programmer turned up at Joel’s house. Yet to test to see if it works.

HDMI2USB – Production Board Bring Up – Snippets – Prep Work

mithro — Mon, 21 Jul 2014 12:47:13 +1000

Two weeks ago I decided that I was going to take a week of work and spend it full time working on bringing up the HDMI2USB Production board that I’m working on creating with Numato. Since I’m making all our GSoC students do daily snippets, I thought it would be good for me to do snippets too! Will be good to look back on the week and see what I ended up actually doing.

The following snippets are for the preparation work I did before the week;

Spent a whole bunch of time cleaning up and organising the GitHub issues, including;
- Creating a bunch of tasks and milestone to track the bring up work,
- Writing some documentation on how we use labels in the GitHub issue tracker,
- Organising issues related to GSoC2014. Found a bunch of pull requests from the start of GSoC that hadn’t been merged and looked at merging them.
Created a spreadsheet which compares the production board to the Atlys prototyping board and will be used for tracking all HDMI2USB variants into the future.
Created a document proposing a future directory structure for HDMI2USB firmware to allow multiple boards, interfaces and other features.
Organised with Rohit to get a VGA capture board during this time.
Probably a bunch of other stuff I have totally forgotten.

Friday – 18th July

Finished a bunch of paid work.
Flew to Adelaide. Joel picked me up from the Airport.
Started work on using ‘git-filter-branch’ to extract our PTZ and speaker tracking modules out of the gst-plugins-bad repo into a normal plugins repository while preserving history.
Helped Joel get IPv6 working at his place.
Fixed up IPv6 on home server to allow VMs public access.
Started up the slidelint website VM and got it a working public IPv6 address.
Set up a Ubuntu Precise VM to help test aps-sids flumotion porting work.

Saturday – 19th July

Day off. Spent day playing DOTA 2 and StarCraft 2 with Joel’s mates.

Sunday – 20th

Concentrated on getting slidelint website up and running, targeting announcement at PyCon AU.
Created DNS configuration and deployed it. Also updated the domain tracking spreadsheet.
Rewrote the setup documentation (and ported it to Markdown).
Set up a nginx frontend.
Attempted to get circus-web interface working.
- Turns out it doesn’t work under Python 3 as torandio2 is unmaintained.
- Ported circus-web to sockjs-torando.
- Found the tests don’t work.

Connecting to a Fritz!Box under Linux using vpnc

mithro — Sun, 06 Oct 2013 23:44:04 +1000

I have two Fritz!Box 7390 (one at my place and one at my parents primary residence) and one Fritz!Box 7270 devices (the house they are building). They are pretty reasonable “high end” ADSL routers and a cool feature is they integrated VPN support. I use this functionality to connect the three networks securely together (but that is not what this post is about). This VPN functionality can also be used to connect to your home network while not at home, but information on how to do this from Linux is very sparse (specially if you only want to use FOSS tools to do the connection), so here is how I did it.

Configuring your Fritz!Box

To set up your Fritz!Box you need a configuration file, there is no GUI available in the web interface to support create new VPN configurations. While a number of sites have pre-built .cfg files that look like they should work, my Fritz!Boxes only accept encrypted VPN configuration files (see end of post for more information) and no open source tool exists to encrypt them.

I instead needed to use the FRITZ!Box VPN Connection tool (while it is a Windows program but runs fine under Wine) to create the configuration. It can be downloaded from the AVM website (This is different to the VPN client that AVM also provide). Annoyingly the tool doesn’t just read existing .cfg configuration files, it instead reads it’s own vpnadmin.cfg found in c:/users/your username/Application Data/AVM/FRITZ!VPN and then generates a .cfg file and encrypts it.

Using the FRITZ!Box VPN Connection tool to create a configuration that is compatible with vpnc you must:

Select “Configure VPN for one User” type connection
On the “Select Device” screen, select “iPhone, iPod touch or iPad” option
On the “Enter the user’s email address” screen, despite the admin tool calling this field the user email, just enter a username. I recommend not having any special characters like @ or . in it.
On the “Enter IP address of the User” screen, be careful about what IP address you use (the default should be okay).
- Don’t use the same IP address that the computer uses when connected via wireless/wired. While it seems like a good idea, as the computer would have the same IP address even when remote, it does not work and will mean the device is unable to access the internet when connected to wireless/wired.
- The “Send all data over the VPN tunnel” option on this page does not seem to affect vpnc, it will always route all your data over the vpn connection. See later for how to fix this problem.
On the “Key for the connection” screen enter a password. Copy down the shared secret key, you’ll want it when creating the vpnc config. I recommend also keeping the default shared secret key it generates unless your super paranoid about entropy.

You can check that you have done this correctly in two ways;

The vpnadmin.cfg will have the iphone=1 and xauth_key="your password" options. See below for a partial example;

...
  user {
    nameoremail = "xxxx";
    key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
    ip = 192.168.179.201;
    internetaccess = 0;
    iphone = 1;
    xauth_key = "xxxxx";
  }
...

If you export an unencrypted config file (remember this can’t be loaded onto the Fritz!Box – so make sure you also export an encrypted version!), the remoteid section will have a key field and not a fqdn field, use_xauth will be set to yes and an xauth section will exist. See below for a partial example;

...
  keepalive_ip = 0.0.0.0;
  remoteid {
    key_id = "qnap";
  }
  mode = phase1_mode_aggressive;
  phase1ss = "all/all/all";
  keytype = connkeytype_pre_shared;
  key = "f3e0hcca19ca2a3gaafbI.{1dGe3q8f84";
  cert_do_server_auth = no;
  use_nat_t = yes;
  use_xauth = yes;
  xauth {
    valid = yes;
    username = "qnap";
    passwd = "qnappassword";
  }
  use_cfgmode = yes;
...

Configuring vpnc on your Linux box

The Fritz!Box needs both draft-ietf-ipsec-nat-t-ike-03 support and the client to adopt the server suggested lifetime (which for the Fritz!Box is 3600 seconds). These features where only added to vpnc recently, so make sure your vpnc is newer than subversion revision 511. Both the version in Debian Unstable and any Ubuntu newer than Precise (12.04) have vpnc 0.5.3r512, which is new enough version and you can just apt-get install vpnc. For Fedora 17 x86 systems you can find RPMs at http://www.pabloendres.com/2013/02/27/vpnc-and-fritzbox/

Create the vpnc config in /etc/vpnc/fritzbox.conf using the following as a template (replace the parts in bold/brackets):

IPSec gateway ip address or DNS name of your FritzBox

IKE DH Group dh2
Perfect Forward Secrecy nopfs

IPSec ID [username entered into the "Enter the user's email address" screen]
# "key" from the Fritz!Box VPN configuration
IPSec secret [shared secret key from the "Key for the connection" screen]

NAT Traversal Mode force-natt

Xauth username [username entered into the "Enter the user's email address" screen]
Xauth password [password entered into the "Key for the connection" screen - Not the password use to encrypt the vpnc configuration!]

As this file contains usernames and password, the config file should be owned by root and only readable by the owner.
sudo chown root /etc/vpnc/fritzbox.conf sudo chmod 0600 /etc/vpnc/fritzbox.conf

You should now be able to connect to your home internet using:
sudo vpnc-connect fritzbox

When you are finished, use:
sudo vpnc-disconnect fritzbox

Internet Access while using vpnc

When vpnc connects it will change your default route to go via the tunnel. This will send all your local internet traffic over the VPN connection. If you are setting up the VPN to secure your internet browsing while on a hostile network such as a public WiFi hotspot (such as at a coffee shop) this is what you want.

However, if you just want to be able to access the hosts on your home network, you don’t want this as it will make your internet browsing extraordinarily slow. These routes are set up by /etc/vpn/vpnc-script script, so you will need custom one. Normally this script is provided a lot of information when connecting to other VPN concentrators (such as a Cisco EasyVPN device) but when connecting to the Fritz!Box you only seem to get limited information.

I created the following script in /etc/vpnc/fritzbox-script, marked it as executable (chmod a+x /etc/vpnc/fritzbox-script) and then added “Script /etc/vpnc/fritzbox-script” to my /etc/vpnc/fritzbox.conf file.

#!/bin/sh

IPROUTE=/sbin/ip

case "$reason" in
  pre-init)
    /etc/vpnc/vpnc-script pre-init
    ;;
  connect)
    INTERNAL_IP4_PREFIX=$(echo $INTERNAL_IP4_ADDRESS | sed -e's/\.[0-9]\+$//')
    $IPROUTE link set dev "$TUNDEV" up mtu 1024
    $IPROUTE addr add "$INTERNAL_IP4_ADDRESS/255.255.255.0" peer "$INTERNAL_IP4_ADDRESS" dev "$TUNDEV"
    $IPROUTE route replace "$INTERNAL_IP4_PREFIX.0/255.255.255.0" dev "$TUNDEV"
    $IPROUTE route flush cache
    ;;
  disconnect)
    $IPROUTE link set dev "$TUNDEV" down
    ;;
  *)
    echo "unknown reason '$reason'. Maybe vpnc-script is out of date" 1>&2
    exit 1
    ;;
esac
exit 0

Fritz!Box encrypted VPN configuration files

Fritz!Boxes will only accept VPN configurations which are encrypted. Otherwise when try to import the file you will get the cryptic "Error: Import of the VPN settings failed." as seen below;

To create an encrypted file you must use the FRITZ!Box VPN Connection tool. Then when exporting, select "Save VPN settings in a file under" option, check the "Encrypt VPN settings" check box (the file type should change to .eff), and enter your chosen password twice.

Lightning Timer website

mithro — Fri, 30 Aug 2013 19:04:10 +1000

When doing lightning talks it is very useful to have a timer so that people don’t over time. There use to be a very simple website called lightningtimer.net but it appears to have gone AWOL. I have set up a mirrored the code at lightningtimer.cc and lightningtimer.info

By default it does 5 minutes, but if you can set the hash value to get different times;

http://lightningtimer.cc#480 – 8 minutes
http://lightningtimer.cc#30 – 30 seconds

Code found at https://github.com/simonw/lightningtimer.net

FastComplete, making bash completion fast on remote file systems

mithro — Fri, 16 Dec 2011 14:05:46 +1000

Here at Google we have quite a few remote file systems which contain various tools we use in our day-to-day work. As typing sucks we generally want the tools in our $PATH. When you try to tab complete Bash needs to stat a couple of thousand files and even on fast remote file systems this takes a drastically long time.

I wrote FastComplete as a solution to this problem. The tool creates a local cache of links on your hard drive to everything in your $PATH. It uses a couple of tricks to make sure all the stats remain locally, while still allowing the remote file to change without needing to update the cache. Linux should also keep this information in memory disk cache making tab completion almost instant again. Yay!

You can find FastComplete at https://github.com/mithro/rcfiles/blob/master/bin/fastcomplete It is a stand alone python program which shouldn’t have any non-core dependencies. The usage documentation is as follows;

Fast complete creates a local disk cache of your path.
It’s specifically designed to make bash tab complete run much faster. The correct fix would be to add caching to bash, but it was to hard to do so.

To find out what path fastcomplete is currently using:
> ~tansell/bin/fastcomplete
# Found 3977 commands
export PATH=/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db
To get fastcomplete to rebuild it’s cache:
> ~tansell/bin/fastcomplete --rebuild
# Using path of '/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db'
# Cache /usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a did not exist! Rebuilding....
# Looking in: /home/tansell/bin (execv)
...
# Looking in: /usr/bin (symlink)
...
# Looking in: /home/build/google3/ads/db (execv)
# Found 3977 commands
export PATH=/usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a
To use fastcomplete all the time add the following as the *LAST* line in your ~/.bashrc file. Fastcomplete will echo some output to stderr so you can see what is happening.
# Create a cache of the command
eval `~tansell/bin/fastcomplete $PATH`

Can haz kernel patches?

mithro — Sun, 03 Jan 2010 13:05:37 +1000

As it’s only a couple of weeks till another Linux.conf.au is underway, I thought I better post this draft from the last one! This year LCA is in Wellington, New Zealand, sadly I didn’t get my act together enough to make a proper holiday of it. I’m definitely looking forward to catching up with all the cool hackers at the conference.

One of the best talks I went to Linux.conf.au last year was the Ksplice talk. This is a wonderful tool which allows people to develop “hot patches” so that you never need to reboot again. The developers have done some very cool work which means for 88% of patches a hot fix can be generated automatically. The presenter was a really great speaker too, during the talk he explains some advanced concepts (like hot to fix-up memory structures) I was able to easily understand it all. I can’t watch until I never have to reboot my Linux machines again!

I also attended Rusty‘s tutorial on hacking lguest. Two years ago I submitted a very important patch to lguest which is now included in the kernel. This year I didn’t get as far as I had wanted mostly because I had not gotten enough sleep the night before. I did however submit one patch which was accepted. I guess that makes me a kernel hacker :).

Sadly, Rusty won’t be running another lguest tutorial this year, so it’s unlikely I’ll submit a third patch.

Starhunter, Firefly’s little known older cousin.

mithro — Tue, 26 May 2009 00:05:06 +1000

I have recently been watching and really enjoying a little known series called Starhunter. At first I thought it was an enjoyable rip off of Firefly, so i was surprised to find out that it was made first.

It has so many of the same elements, the quirky ship mechanic, the captain who doesn’t play by the rules with a darker background, raiders, etc. I have no doubt that Joss Whedon stole many the ideas for Firefly from this series and in the process improving on them in many ways. Sadly, like Firefly this series was canceled way before it reached a wide audience.

You can watch the “second season” (Starhunter 2300), which is closer to a spin off rather then a second season (most of the characters have changed), on Youtube. I would highly recommend people who enjoyed Firefly check it out. Since its free, what have you got to loose?

While the first season is available on Joost, it is apparently horrible mangled. Sadly, the DVD’s are not available here in Australia so I’m going I’m going to have to buy them from Amazon.

Programmer art, it’s deadly!

mithro — Wed, 28 Jan 2009 18:33:02 +1000

So one of the other things I worked on at Linux.conf.au was a project Joel Stanley started, “Open Velociraptor Per Child, the Game!”.

The game comes out of the Open Velociraptor Per Child project’s ideas. The OVPC project aims to give every child in the world their very own Velociraptor. The game simulates what would happen if this project was ever to achieve it’s wonderful but lofty goal.

My main contribution was adding support for (very bad) collision detection, scoring and difficult levels. However, what I spent most of my time was spent making bad programmer art, that of the particularly gruesome child death animation. I like to think it was this artwork that meant we won top spot at the Gaming Miniconf competition, beating the “Shave Bdale” game.

You can download the game from the git repository. Have fun!

Compiling for Windows using Cygwin on Linux….

mithro — Tue, 27 Jan 2009 16:22:22 +1000

So for the past week while I have been at the best conference in the world I have been trying to compile tpserver-cpp for Windows. I had done the hard work and gotten it to compile (as documented here, here and here) on Windows previously. However, as I was in Hobart at a Linux conference I didn’t really have access to Windows computer. That was not going to stop me, so I attempted to cross compile the binaries under Linux. This has a number of advantages as it would mean when someone finally gets around to creating a autobuilder, we can produce Windows binaries too.

Ubuntu provides the mingw32 compilers in the repository so I didn’t think it would be all that hard to get working. The problem is that tpserver-cpp does not have a “native” Windows support but cygwin comes to the rescue and provides a compatibly layer. Using cygwin turned out to not be as simple as using mingw32 compiler with the cygwin headers.

I ended up using crosstool to build my own cygwin compiler. I battled for a long while with the fact that Ubuntu now enables “fortify source” by default. This breaks many versions of things like binutils and gcc (which often do naughty things which fortify source does not like). After I figured out how to disable it, I was still was only able to get an ancient version of gcc to compile (3.3.6) which meant I had to fix a lot of problems in the tpserver-cpp code. I guess someone had to do it eventually, but it was annoying that I was forced too.

I then manually downloaded a bunch of cygwin packages to build a tree for the dependencies (such as boost and guile). This was much faster then trying to compile them on my own. Finally, I was able to build tperver-cpp and create a Windows binary! I can confirm it runs fine under Wine and am now getting friends who are still shacked to Windows to test it there.

It sounds much simpler now, but it took me over a week of work to boil it down to these steps. It was like a constant game of wack-a-mole, once I had solved one problem another popped up.

So what now in this area? I want to get a recent version of the compiler working and preferably build all the dependencies ourselves (rather then rely on the cygwin compiled versions). I would ultimately like to see the cygwin compilers being packaged with Ubuntu/Debian in the same way that the mingw32 compilers are. I don’t know if any of that is likely to happen however as I never seem to have enought time. For now I have uploaded a copy of my cross compiler (It needs to be extracted so it is found in /opt/crosstool).

I hope this helps someone!

OSDC & orbital death, better late then never…

mithro — Mon, 26 Jan 2009 14:43:32 +1000

So last year at the end of November I spoke at OSDC. I had been meaning to blog about the conference but never gotten around to doing so. The conference was great, but like previous OSDC’s I came down with something. Sadly, this meant that missed Andrew Tridgell’s talk about the EU-Microsoft agreement. Joel, who had been camped out in my lounge room for the week, said it was really worth listening too. I really wish OSDC recorded all the talks like Linux.conf.au does.

As part of speaking at the conference I had to produce a paper. This paper puts into writing a lot of what I have been talking about. I wasn’t going to post it, but after getting a email out of the blue about the topic, I’ve decided to put a copy here on my blog. I might as well also upload the presentation I gave, but it won’t be very interesting by itself. Both are released under a CC-BY-SA.

If there is one thing that any budding game developer (open source or otherwise) should take away from this talk, it is the following:

When a person is looking for normal software, they have an issue to solve.
When looking for computer games, they are looking to be entertained.

This fundamental difference in mindset should drive every aspect of your game, website and release process.

Edit: How could I forget? A big thanks goes out to Leslie Hawthorn who helped proof read the paper and making it much easier to read!

Mithro rants about stuff

nMigen – a new improved Migen developed by whitequark!

Using “IdentitiesOnly” without key files

TimVideos.us and Google Summer of Code 2016!

TimVideos.us and Google Summer of Code 2016!

TimVideos.us 2016 New Year’s Resolutions

TimVideos.us 2016 New Year’s Resolutions

TimVideos Project

HDMI2USB Project

First V2 “HDMI2USB Production Board” constructed!

HDMI2USB – Production Board Bring Up – Day 8 (28th July 2014)

HDMI2USB – Production Board Bring Up – Day 5, 6 and 7 (25th, 26th and 27th July 2014)

25th July 2014

26th July 2014

27th July 2014

HDMI2USB – Production Board Bring Up – Day 4 (24th July 2014)

HDMI2USB – Production Board Bring Up – Day 3 (23rd July 2014)

Non HDMI2USB stuff

HDMI2USB – Production Board Bring Up – Day 2 (22nd July 2014)

Non-HDMI2USB related stuff

HDMI2USB related stuff

HDMI2USB – Production Board Bring Up – Day 1 (21st July 2014)

HDMI2USB – Production Board Bring Up – Snippets – Prep Work

Friday – 18th July

Saturday – 19th July

Sunday – 20th

Connecting to a Fritz!Box under Linux using vpnc

Configuring your Fritz!Box

Configuring vpnc on your Linux box

Internet Access while using vpnc

Fritz!Box encrypted VPN configuration files

Lightning Timer website

FastComplete, making bash completion fast on remote file systems

Can haz kernel patches?

Starhunter, Firefly’s little known older cousin.

Programmer art, it’s deadly!

Compiling for Windows using Cygwin on Linux….

OSDC & orbital death, better late then never…