Skip to content

HDMI2USB – Production Board Bring Up – Day 4 (24th July 2014)

Generally slow progress today, but some success.

  • Able to successfully load libFPGALink firmware onto Cypress chip and programmed the FPGA using it. Yay!
  • Wrote a test firmware based on http://hamsterworks.co.nz/mediawiki/index.php/DVI-D_Serdes to transmit 720p output from both the TX ports.
    • Images are successfully received on both devices but there is a weird artifact in the image.

 Numato HDMI2USB Prototype driving 2 screens  HDMI2USB weird image artifact

HDMI2USB - Rohit's VGA Capture board

 

  • Set up the streaming system for Carl and PyOhio
    • Tried to figure out where www.timvideos.us was hosted.

HDMI2USB – Production Board Bring Up – Day 3 (23rd July 2014)

  • Looked into if Travis CI has IPv6 connectivity. It appears so!
  • Investigated git server side hooks as a method to run CI on build.timvideos.tv
    • Travis-CI does a git push to build.timvideos.tv which does a make in the post-recieve.
    • Issues;
      • For non-pull requests we could encode the private key using Travis’s encrypted environment variables.
      • What about pull requests? Mean anyone can send arbitrary code to build.timvideos.tv – Maybe use QEmu’s COW support and reboot after each build?
    • Looked a jig for a nice way to write the git hooks.
      • Sent a pull request to fix some documentation.
      • Really designed to run as a “pre-commit” hook locally on a person’s machine rather than a server side hook. Logged an issue to discuss that.
      • Has a lot of nice plugins for Python development already.
      • Would allow us to add hdl-pretty as a commit hook.
    • Alternatives to jig appear to be;
      • https://github.com/icefox/git-hooks – Written in bash
      • https://pypi.python.org/pypi/git-pre-commit-hook – Written in python
  • Got my “fake” Xilinx Platform Cable USB (Model DLC9G) working under Linux – Full instructions can be found at https://github.com/timvideos/HDMI2USB/wiki/Xilinx-Platform-Cable-USB-under-Linux
    • The device was purchased from eBay on the 11th July, costed $37 USD with shipping and arrived at Joel’s house on Monday.
    • First issue was /opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/setup_pcusb didn’t understand I had udev and was trying to install for the ancient hotplug.
      • Fixed by changing line 26 from TP_USE_UDEV=”0″ to TP_USE_UDEV=”1″
    • Next setup_pcusb didn’t tell udev to reload the udev rules. Had to do that with udevadm control –reload-rules
    • Next setup_pcusb didn’t ask me to install fxload, so I needed to install it with apt-get install fxload
    • Next the rules that were installed to /etc/udev/rules.d/xusbdfwu.rules were invalid; they caused the following errors in /var/log/daemon.log
Jul 23 16:40:29 laptop udevd[841]: unknown key 'SYSFS{idVendor}' in /etc/udev/rules.d/xusbdfwu.rules:2
Jul 23 16:40:29 laptop udevd[841]: invalid rule '/etc/udev/rules.d/xusbdfwu.rules:2'
Jul 23 16:46:11 laptop udevd[841]: unknown key 'SYSFS{idVendor}' in /etc/udev/rules.d/xusbdfwu.rules:3
Jul 23 16:46:11 laptop udevd[841]: invalid rule '/etc/udev/rules.d/xusbdfwu.rules:3'
    • This was fixed with;
      • Changing SYSFS to ATTRS
      • Changing BUS to SUBSYSTEM
      • Changing $TEMPNODE to $tempnode
    • The little status light then turned on red! Yay!
    • Was able to do a boundary scan in iMPACT on a Zybo development board after soldering a header onto it.
  • Received the HDMI2USB production boards created by Numato!
    • Started with bunch of stuff to do with LEDs;
      • Logged issues about LEDs not been label intelligently. Such as the “power” LED being labeled “D33″.
      • Researched the DONE net again so I could understand the D2 LED (which should be named D_FPGA_NOT_CONFIGURED or DNCFG for short). Started adding the information to the JTAG/Reset documentation.
      • Figured out why D3 (connected to the Cypress INT1 pin) was only faintly lit.
    • Logged a bunch of issue regarding small silk screen fixes to make future boards easier to understand.
    • Logged an issue about adding some standoffs in the center of the board for mechanical stability.
    • Logged an issue about having a good GND point to connect your probe too.
      • This page describes the two good ways to add a GND test point – http://www.robotroom.com/PCB-Layout-Tips.html
    • Starting researching the 5V rail and if we could remove it totally (thus saving a bunch of stuff). Looks like we can, but needs more investigation.
    • After replacing the JTAG cable was able to use iMPACT to boundary scan and it found Spartan 6 chip!
  • Discovered CEC is 3V3 signal.

Non HDMI2USB stuff

  • Tried to figure out why my home router has decided that it wants to hand out address in the 2001:44b8:31dc:8d01::/64 rather than the 2001:44b8:31dc:8d00::/64 range it use too.
  • Found a bunch of issues with domains served of ns1.mithis.com as the secondary servers where disabled. Root cause was an old version of PowerDNS failing on TCP zone transfers causing domains to become stale and get dropped from the secondaries. Enabled email notification when secondary disables the zones.

 

HDMI2USB – Production Board Bring Up – Day 2 (22nd July 2014)

Non-HDMI2USB related stuff

HDMI2USB related stuff

  • Worked from Joel’s house today to receive the production HDMI2USB boards, they didn’t turn up :(
  • Create a trusty VM on server for Joel to setup HDMI2USB CI on.
  • Investigated origin of files in HDMI2USB code. Current status seems to be;
    • hdl/edid – Written by us (Jahanzeb).
    • hdl/hdmi – Mostly written by Xilinx and released in application notes. Need to figure out the exact licensing of the files (are they MIT? Only for use on Xilinx parts?) and move them to third party.
    • hdl/jpeg_encoder – OpenCores mkjpeg core. Need to move to third party.
    • hdl/usb – Written by us (Jahanzeb).
    • ipcore_dir – Cores generated by Xilinx. Need to figure out the exact licensing of the files (are they MIT? Only for use on Xilinx parts?) and move them to third party.
    • cypress – Confusing mess. We know this and plan to rewrite the firmware using only FOSS tools.
      • Bunch of proprietary binary blobs.
      • Bunch of headers of unknown copyright.
      • Bunch of code written by us.
  • Logged a lot of bugs about warnings Xilinx tools are generating when compiling the firmware.
  • Made generate_ucf.py generate a file for Atlys functionally identical to existing HDMI2USB.ucf file. Was able to successfully build the firmware with generate HDMI2USB.ucf file for the Atlys board.
  • Started trying to get firmware to compile with the Numato HDMI2USB.ucf file, ran into the following issue
Phase 4.2 Initial Placement for Architecture Specific Features
ERROR:Place - ConstraintResolved NO placeable site for
 hdmiMatri_Comp/dvi_rx1/ioclk_buf
ERROR:Place - SIO has over-constrained componet hdmiMatri_Comp/dvi_rx1/ioclk_buf
 to have to placeable sites. Constraints come from driver constraints AND load
 IO constraints
/bin/sh: line 16: 28981 Segmentation fault (core dumped) map -filter "../ise/iseconfig/filter.filter" -intstyle ise -p xc6slx45-csg324-3 -w -logic_opt off -ol high -xe n -t 1 -xt 0 -register_duplication off -r 4 -global_opt off -mt off -ir off -pr b -lc off -power off -o hdmi2usb_map.ncd hdmi2usb.ngd hdmi2usb.pcf
make: *** [map] Error 139
  • Rohit GSoC mentoring;
    • Emailed Rohit about reimbursement for VGA Board hardware parts.
    • Rohit has shipped me a board which should arrive at Joel’s tomorrow.
    • Weekly mentoring meeting.
    • Reviewed Rohit’s weekly summary blog post.
    • Reviewed Rohit’s VGA board V2 schematic.
    • Reviewed Rohit’s VGA board V2 PCB.

HDMI2USB – Production Board Bring Up – Day 1 (21st July 2014)

  • Wrote snippets for the last 3 days.
  • slidelint work
  • Got set up in Joel and Rusty’s coworking space.
  • Explained to Ajit the idea of routing the debug output to multiple locations. Added a diagram to the Debug Design document.
  • Fixed command line processing in hdl-pretty code. Started down the path of auto-generating help from the customize stuff and then realised I had more important things to do.
  • Got most of my way through a Python program to generate UCF files from the HDMI2USB Variants spreadsheet.
  • Xilinx “Compatible” JTAG programmer turned up at Joel’s house. Yet to test to see if it works.

HDMI2USB – Production Board Bring Up – Snippets – Prep Work

Two weeks ago I decided that I was going to take a week of work and spend it full time working on bringing up the HDMI2USB Production board that I’m working on creating with Numato. Since I’m making all our GSoC students do daily snippets, I thought it would be good for me to do snippets too! Will be good to look back on the week and see what I ended up actually doing.

The following snippets are for the preparation work I did before the week;

 

Friday – 18th July

 

Saturday – 19th July

  • Day off. Spent day playing DOTA 2 and StarCraft 2 with Joel’s mates.

 

Sunday – 20th

  • Concentrated on getting slidelint website up and running, targeting announcement at PyCon AU.
  • Created DNS configuration and deployed it. Also updated the domain tracking spreadsheet.
  • Rewrote the setup documentation (and ported it to Markdown).
  • Set up a nginx frontend.
  • Attempted to get circus-web interface working.
    • Turns out it doesn’t work under Python 3 as torandio2 is unmaintained.
    • Ported circus-web to sockjs-torando.
    • Found the tests don’t work.

 

Connecting to a Fritz!Box under Linux using vpnc

I have two Fritz!Box 7390 (one at my place and one at my parents primary residence) and one Fritz!Box 7270 devices (the house they are building). They are pretty reasonable “high end” ADSL routers and a cool feature is they integrated VPN support. I use this functionality to connect the three networks securely together (but that is not what this post is about). This VPN functionality can also be used to connect to your home network while not at home, but information on how to do this from Linux is very sparse (specially if you only want to use FOSS tools to do the connection), so here is how I did it.

Configuring your Fritz!Box

To set up your Fritz!Box you need a configuration file, there is no GUI available in the web interface to support create new VPN configurations. While a number of sites have pre-built .cfg files that look like they should work, my Fritz!Boxes only accept encrypted VPN configuration files (see end of post for more information) and no open source tool exists to encrypt them.

I instead needed to use the FRITZ!Box VPN Connection tool (while it is a Windows program but runs fine under Wine) to create the configuration. It can be downloaded from the AVM website (This is different to the VPN client that AVM also provide). Annoyingly the tool doesn’t just read existing .cfg configuration files, it instead reads it’s own vpnadmin.cfg found in c:/users/your username/Application Data/AVM/FRITZ!VPN and then generates a .cfg file and encrypts it.

Using the FRITZ!Box VPN Connection tool to create a configuration that is compatible with vpnc you must:

  • Select “Configure VPN for one User” type connection
  • On the “Select Device” screen, select “iPhone, iPod touch or iPad” option
  • On the “Enter the user’s email address” screen, despite the admin tool calling this field the user email, just enter a username. I recommend not having any special characters like @ or . in it.
  • On the “Enter IP address of the User” screen, be careful about what IP address you use (the default should be okay).
    • Don’t use the same IP address that the computer uses when connected via wireless/wired. While it seems like a good idea, as the computer would have the same IP address even when remote, it does not work and will mean the device is unable to access the internet when connected to wireless/wired.
    • The “Send all data over the VPN tunnel” option on this page does not seem to affect vpnc, it will always route all your data over the vpn connection. See later for how to fix this problem.
  • On the “Key for the connection” screen enter a password. Copy down the shared secret key, you’ll want it when creating the vpnc config. I recommend also keeping the default shared secret key it generates unless your super paranoid about entropy.

You can check that you have done this correctly in two ways;

  • The vpnadmin.cfg will have the iphone=1 and xauth_key="your password" options. See below for a partial example;
    ...
      user {
        nameoremail = "xxxx";
        key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
        ip = 192.168.179.201;
        internetaccess = 0;
        iphone = 1;
        xauth_key = "xxxxx";
      }
    ...
    

  • If you export an unencrypted config file (remember this can’t be loaded onto the Fritz!Box – so make sure you also export an encrypted version!), the remoteid section will have a key field and not a fqdn field, use_xauth will be set to yes and an xauth section will exist. See below for a partial example;
    ...
      keepalive_ip = 0.0.0.0;
      remoteid {
        key_id = "qnap";
      }
      mode = phase1_mode_aggressive;
      phase1ss = "all/all/all";
      keytype = connkeytype_pre_shared;
      key = "f3e0hcca19ca2a3gaafbI.{1dGe3q8f84";
      cert_do_server_auth = no;
      use_nat_t = yes;
      use_xauth = yes;
      xauth {
        valid = yes;
        username = "qnap";
        passwd = "qnappassword";
      }
      use_cfgmode = yes;
    ...

Configuring vpnc on your Linux box

The Fritz!Box needs both draft-ietf-ipsec-nat-t-ike-03 support and the client to adopt the server suggested lifetime (which for the Fritz!Box is 3600 seconds). These features where only added to vpnc recently, so make sure your vpnc is newer than subversion revision 511. Both the version in Debian Unstable and any Ubuntu newer than Precise (12.04) have vpnc 0.5.3r512, which is new enough version and you can just apt-get install vpnc. For Fedora 17 x86 systems you can find RPMs at http://www.pabloendres.com/2013/02/27/vpnc-and-fritzbox/

Create the vpnc config in /etc/vpnc/fritzbox.conf using the following as a template (replace the parts in bold/brackets):

IPSec gateway ip address or DNS name of your FritzBox

IKE DH Group dh2
Perfect Forward Secrecy nopfs

IPSec ID [username entered into the "Enter the user's email address" screen]
# "key" from the Fritz!Box VPN configuration
IPSec secret [shared secret key from the "Key for the connection" screen]

NAT Traversal Mode force-natt

Xauth username [username entered into the "Enter the user's email address" screen]
Xauth password [password entered into the "Key for the connection" screen - Not the password use to encrypt the vpnc configuration!]

As this file contains usernames and password, the config file should be owned by root and only readable by the owner.

sudo chown root /etc/vpnc/fritzbox.conf
sudo chmod 0600 /etc/vpnc/fritzbox.conf

You should now be able to connect to your home internet using:

sudo vpnc-connect fritzbox

When you are finished, use:

sudo vpnc-disconnect fritzbox

Internet Access while using vpnc

When vpnc connects it will change your default route to go via the tunnel. This will send all your local internet traffic over the VPN connection. If you are setting up the VPN to secure your internet browsing while on a hostile network such as a public WiFi hotspot (such as at a coffee shop) this is what you want.

However, if you just want to be able to access the hosts on your home network, you don’t want this as it will make your internet browsing extraordinarily slow. These routes are set up by /etc/vpn/vpnc-script script, so you will need custom one. Normally this script is provided a lot of information when connecting to other VPN concentrators (such as a Cisco EasyVPN device) but when connecting to the Fritz!Box you only seem to get limited information.

I created the following script in /etc/vpnc/fritzbox-script, marked it as executable (chmod a+x /etc/vpnc/fritzbox-script) and then added “Script /etc/vpnc/fritzbox-script” to my /etc/vpnc/fritzbox.conf file.

#!/bin/sh

IPROUTE=/sbin/ip

case "$reason" in
  pre-init)
    /etc/vpnc/vpnc-script pre-init
    ;;
  connect)
    INTERNAL_IP4_PREFIX=$(echo $INTERNAL_IP4_ADDRESS | sed -e's/\.[0-9]\+$//')
    $IPROUTE link set dev "$TUNDEV" up mtu 1024
    $IPROUTE addr add "$INTERNAL_IP4_ADDRESS/255.255.255.0" peer "$INTERNAL_IP4_ADDRESS" dev "$TUNDEV"
    $IPROUTE route replace "$INTERNAL_IP4_PREFIX.0/255.255.255.0" dev "$TUNDEV"
    $IPROUTE route flush cache
    ;;
  disconnect)
    $IPROUTE link set dev "$TUNDEV" down
    ;;
  *)
    echo "unknown reason '$reason'. Maybe vpnc-script is out of date" 1>&2
    exit 1
    ;;
esac
exit 0

Fritz!Box encrypted VPN configuration files

Fritz!Boxes will only accept VPN configurations which are encrypted. Otherwise when try to import the file you will get the cryptic "Error: Import of the VPN settings failed." as seen below;
Error: Import of the VPN settings failed.

To create an encrypted file you must use the FRITZ!Box VPN Connection tool. Then when exporting, select "Save VPN settings in a file under" option, check the "Encrypt VPN settings" check box (the file type should change to .eff), and enter your chosen password twice.

Export VPN settings

Lightning Timer website

When doing lightning talks it is very useful to have a timer so that people don’t over time. There use to be a very simple website called lightningtimer.net but it appears to have gone AWOL. I have set up a mirrored the code at lightningtimer.cc and lightningtimer.info

By default it does 5 minutes, but if you can set the hash value to get different times;

Code found at https://github.com/simonw/lightningtimer.net

FastComplete, making bash completion fast on remote file systems

Here at Google we have quite a few remote file systems which contain various tools we use in our day-to-day work. As typing sucks we generally want the tools in our $PATH. When you try to tab complete Bash needs to stat a couple of thousand files and even on fast remote file systems this takes a drastically long time.

I wrote FastComplete as a solution to this problem. The tool creates a local cache of links on your hard drive to everything in your $PATH. It uses a couple of tricks to make sure all the stats remain locally, while still allowing the remote file to change without needing to update the cache. Linux should also keep this information in memory disk cache making tab completion almost instant again. Yay!

You can find FastComplete at https://github.com/mithro/rcfiles/blob/master/bin/fastcomplete It is a stand alone python program which shouldn’t have any non-core dependencies. The usage documentation is as follows;

Fast complete creates a local disk cache of your path.
It’s specifically designed to make bash tab complete run much faster. The correct fix would be to add caching to bash, but it was to hard to do so.

To find out what path fastcomplete is currently using:

> ~tansell/bin/fastcomplete
# Found 3977 commands
export PATH=/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db

To get fastcomplete to rebuild it’s cache:

> ~tansell/bin/fastcomplete --rebuild
# Using path of '/home/tansell/bin: ... :/home/build/google3/googledata/validators:/home/build/google3/ads/db'
# Cache /usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a did not exist! Rebuilding....
# Looking in: /home/tansell/bin (execv)
...
# Looking in: /usr/bin (symlink)
...
# Looking in: /home/build/google3/ads/db (execv)
# Found 3977 commands
export PATH=/usr/local/google/users//tansell/tabcache/d7e5fb63454ae33b4a171b6437be904a

To use fastcomplete all the time add the following as the *LAST* line in your ~/.bashrc file. Fastcomplete will echo some output to stderr so you can see what is happening.

# Create a cache of the command
eval `~tansell/bin/fastcomplete $PATH`

Can haz kernel patches?

As it’s only a couple of weeks till another Linux.conf.au is underway, I thought I better post this draft from the last one! This year LCA is in Wellington, New Zealand, sadly I didn’t get my act together enough to make a proper holiday of it. I’m definitely looking forward to catching up with all the cool hackers at the conference.

One of the best talks I went to Linux.conf.au last year was the Ksplice talk. This is a wonderful tool which allows people to develop “hot patches” so that you never need to reboot again. The developers have done some very cool work which means for 88% of patches a hot fix can be generated automatically. The presenter was a really great speaker too, during the talk he explains some advanced concepts (like hot to fix-up memory structures) I was able to easily understand it all. I can’t watch until I never have to reboot my Linux machines again!

I also attended Rusty‘s tutorial on hacking lguest. Two years ago I submitted a very important patch to lguest which is now included in the kernel. This year I didn’t get as far as I had wanted mostly because I had not gotten enough sleep the night before. I did however submit one patch which was accepted. I guess that makes me a kernel hacker :).

Sadly, Rusty won’t be running another lguest tutorial this year, so it’s unlikely I’ll submit a third patch.

Tagged , , , , , , , , ,

Starhunter, Firefly’s little known older cousin.

Tulip - The ship from Starhunter

I have recently been watching and really enjoying a little known series called Starhunter. At first I thought it was an enjoyable rip off of Firefly, so i was surprised to find out that it was made first.

It has so many of the same elements, the quirky ship mechanic, the captain who doesn’t play by the rules with a darker background, raiders, etc. I have no doubt that Joss Whedon stole many the ideas for Firefly from this series and in the process improving on them in many ways. Sadly, like Firefly this series was canceled way before it reached a wide audience.

You can watch the “second season” (Starhunter 2300), which is closer to a spin off  rather then a second season (most of the characters have changed), on Youtube. I would highly recommend people who enjoyed Firefly check it out. Since its free, what have you got to loose?

While the first season is available on Joost, it is apparently horrible mangled. Sadly, the DVD’s are not available here in Australia so I’m going I’m going to have to buy them from Amazon.

Tagged , , ,