Skip to content

{ Author Archives } mithro

nMigen – a new improved Migen developed by whitequark!

As people may know, I’m a big supporter of the Migen and associated LiteX ecosystem. As of today if you are starting a new project today, I would instead recommend that you chose to use nMigen instead.

nMigen, A refreshed Python toolbox for building complex digital hardware is a project that whitequark has been working on for the last couple of years and I’m really liking the new improve syntax and the fact that it interfaces directly to Yosys.

There seems a lot of interesting things happening in the nMigen ecosystem;

2020 05 02 Hardware
Python Comments (0)

Using “IdentitiesOnly” without key files

If you want to restrict the keys that ssh tries when connecting to a server, you need to use the IdentityFile and IdentitiesOnly configuration options in your ssh_config.

However, a couple of the keys I have are auto generated, with the key being loaded directly into ssh-agent and never written to a file on disk. For hopefully obvious reasons, you can’t dump a private key back out of the agent, but it turns out that IdentityFile only needs the public key which you can get.

I ended up using the following script to dump the public keys to files;

# Dump the public keys
for KEY in $(ssh-add -l | sed -e's/[^ ]\+ [^ ]\+ \([^ ]\+\) .*/\1/'); do
  if echo $KEY | grep -q '^/'; then
    continue
  fi
  export KEY_FILE="$HOME/.ssh/agent.$(echo $KEY | sed -e's/[^A-Za-z0-9]/_/g').pub"
  echo "Saving $KEY into $KEY_FILE"
  ssh-add -L | grep $KEY > $KEY_FILE
done

Then I added the following to the ssh_config

IdentitiesOnly true
IdentityFile ~/.ssh/agent.keyname.pub

When running with “ssh -vv” I see the following in the output;

debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/tansell/.ssh/agent.keyb.pub (0x2257da0), explicit
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering ECDSA public key: //home/tansell/.ssh/agent.keyb.pub
debug2: we sent a publickey packet, wait for reply

2016 05 03 Useful Bits Comments (0)

TimVideos.us and Google Summer of Code 2016!

This is a cross post from the HDMI2USB website.

TimVideos.us and Google Summer of Code 2016!

The TimVideos.us project is happy to announce that it has been selected to participate in the Google Summer of Code for 2016 (GSoC). GSoC is a program where students are paid to contribute to selected open source projects over the northern hemisphere summer, flip bits not burgers!

The application period is now open and students have until 25 March 19:00 UTC to apply to work with the TimVideos.us project. The list of proposal ideas includes contributing to the HDMI2USB and things related to the Numato Opsis.

GSoC Logo 2016

Due to the focus on hardware, we are very interested in students who are interested in things like VHDL/Verilog and other HDLs, embedded C programming and operating systems and electronic circuit/PCB design. Some possible ideas include;

If you are interested in applying, your first point of call should be our Google Summer of Code page. Even if you can’t apply to GSoC, you can also help us by forwarding this message to anyone you might think who is interested!

TimVideos Logo

2016 03 15 Hardware
HDMI2USB
Summer of Code
TimVideos.us Comments (0)

TimVideos.us 2016 New Year’s Resolutions

This is a cross post from the HDMI2USB website about my plans for 2016 and the TimVideos project. Last year was an exciting time for my personal projects!

TimVideos.us 2016 New Year’s Resolutions

Hello everyone,

Hope everyone has had an awesome start to 2016 so far. As is tradition in many western countries, I thought I would put together some New Year’s Resolutions and reflect on our progress in 2015. I guess more business minded people might call it a “project roadmap” 🙂

TimVideos Project

In 2015, I decided to focus the TimVideos project on the HDMI2USB project. The three key results of this focus were;

With the success of this focus in 2015, the TimVideos project is going to continue to focus on theHDMI2USB project for 2016 (and I’ll go into more detailed goals shortly).

The TimVideos project has also been mildly successful in collaborating with other open source groups doing things related to video recording and production. In 2016, I hope we can strengthen these bonds and forge new ones. Some specific goals around this include;

HDMI2USB Project

As we are concentrating on the HDMI2USB project, we have some specific goals around that.

HDMI2USB firmware goals;

  • Refactor the HDMI core to allow support a wider range of interfaces, better debugging and addition of more features. A document about the refactor has been started here.
  • Add support for the high-speed GTP transceivers and Mike “Hamster” Field open source DisplayPort core.
  • Get Ethernet support working (on both the Atlys and Opsis boards). The two major Ethernet features are;
    • Ethernet supports identical capture and control feature set to the USB port.
    • Allowing HDMI2USB boards to act has “HDMI over Ethernet extenders”.
  • Support for more hardware;
    • miniSpartan6+
    • Digilent Nexys Video
    • New HDMI2USB designed hardware!
  • Stretch Goals (1)

HDMI2USB hardware goals;

  • (By end of year) Development of a low cost PCI-Express capture card.
  • (By middle 2017) Development of an Opsis V2 based around either an high end Artix-7 or a low end Kintex-7 FPGA.

(1): Stretch goals are things we plan to try and achieve if things go well.

Hope this update give you an idea of what we have planned for 2016! We would love your help making it all possible.

Tim ‘mithro’ Ansell

2016 01 15 HDMI2USB
linux.conf.au
PCB
TimVideos.us Comments (0)

First V2 “HDMI2USB Production Board” constructed!

For the last year and a half, I have been working with Numato Labs to create a “HDMI2USB Production Board” for our HDMI2USB firmware that was originally developed on the Digilent Atlys board. On Friday, they sent me a picture of the first constructed board!

HDMI2USB "Production Board" Version 2

HDMI2USB “Production Board” Version 2

At the end of last year, we decided to abandon our first attempt and start again from scratch, this picture is the result of that work. Some of the reasons we decided to start from scratch was;

  • A “de facto standard” for locking HDMI ports was established and low cost connectors became available. This meant we no longer needed to support both DVI and HDMI connectors, reducing the complexity significantly and solving some persistent issues.
  • Not only did the cost of Spartan 6 parts with high speed “GTP” transceivers drop but our understanding of how to use them increased. This would allow us to create a board which natively supports DisplayPort.
  • The idea streaming via not only USB, but also Ethernet became a stronger possibility, meaning the extra cost of adding ethernet was now worth it.

Overall, our board has the following differences with the Atlys board;

  • Has DDR3 memory instead of DDR2, increasing the memory bandwidth.
  • Has a Spartan S6LX45T with the GTP broken out to DisplayPort headers.
  • Has a PCI-Express style expansion connector instead of the expensive VHDCI connector, allow much cheaper expansion boards.
  • Has all the extra pins on the FX2, increasing the potential USB interface options.
  • Has control over all the HDMI functionality, including hot plug and CEC functionality.
  • Removes parts we don’t need such as the audio, buttons, switches and LEDs.
  • Mounts in any ITX style computer case.
  • Adds UTMI USB (as well as the Cypress FX2)
  • Adds MicroSD connector.
2015 07 05 Hardware
HDMI2USB
TimVideos.us Comments (0)

HDMI2USB – Production Board Bring Up – Day 8 (28th July 2014)

  • Updated the HDMI2USB variant spreadsheet with new Numato board pin out.
  • Successfully generated a HDMI2USB firmware for Numato board with new pin information. That fixes the constraint issue!
  • Successfully generated a HDMI2USB firmware for Numato board with second receive port (RX2) disabled!
    • Was able to view the HDMI2USB test image output on TX2!
    • Was able to capture the HDMI2USB test image via mplayer!
    • Was unable to detect a screen on RX1.
  • Created a “flash.sh” script to use libFPGALink to flash the FPGA and then load the Cypress firmware.
  • Wrote a Python script to interrogate the CDC serial port for status of the firmware.
2014 07 29 HDMI2USB
TimVideos.us Comments (0)

HDMI2USB – Production Board Bring Up – Day 5, 6 and 7 (25th, 26th and 27th July 2014)

25th July 2014

Streaming System Hacking

  • Made the configuration system better and adding a lot of comments to the json file (which isn’t really valid json any more).
  • Made pycon2internal.py slightly better and able to parse PyOhio format (based on the PyCon US format).
  • Added hack to make event2internal.py to use pycon2internal.py at some periods.
  • Set up a bunch of encoders in the Oregon EC2 region.
  • Get access to www.timvideos.us again and deployed the updated website.

 

26th July 2014

  • Went into the Adelaide Hackerspace and created a cable to interface the USB3300 ULPI chip to 2xPMOD headers.
  • Got streaming working for day 1 of PyOhio, it was late as we were disorganised.
  • aps-sids foolishly pointed out a bug in the title of the streaming system, so I taught him about how we actually flumotion for an event.

 

 27th July 2014

  • Did a lot of spreadsheet hacking on the production board pin planning spreadsheet;
    • Added half bank / BUFIO2 regions.
    • Added dedicated clock pin information.
  • Started work on reshuffling the pins to fix the timing issue.
  • Helped aps-sids get Flumotion working on the latest Twisted release. Turned out that someone else had already done most of the work and only a couple very small fixes where needed. aps-sids reported that he was now able to run flumotion on Ubuntu Trusty!

 

2014 07 28 HDMI2USB
TimVideos.us Comments (0)

HDMI2USB – Production Board Bring Up – Day 4 (24th July 2014)

Generally slow progress today, but some success.

  • Able to successfully load libFPGALink firmware onto Cypress chip and programmed the FPGA using it. Yay!
  • Wrote a test firmware based on http://hamsterworks.co.nz/mediawiki/index.php/DVI-D_Serdes to transmit 720p output from both the TX ports.
    • Images are successfully received on both devices but there is a weird artifact in the image.

 Numato HDMI2USB Prototype driving 2 screens  HDMI2USB weird image artifact

HDMI2USB - Rohit's VGA Capture board

 

  • Set up the streaming system for Carl and PyOhio
    • Tried to figure out where www.timvideos.us was hosted.
2014 07 25 HDMI2USB
TimVideos.us Comments (0)

HDMI2USB – Production Board Bring Up – Day 3 (23rd July 2014)

  • Looked into if Travis CI has IPv6 connectivity. It appears so!
  • Investigated git server side hooks as a method to run CI on build.timvideos.tv
    • Travis-CI does a git push to build.timvideos.tv which does a make in the post-recieve.
    • Issues;
      • For non-pull requests we could encode the private key using Travis’s encrypted environment variables.
      • What about pull requests? Mean anyone can send arbitrary code to build.timvideos.tv – Maybe use QEmu’s COW support and reboot after each build?
    • Looked a jig for a nice way to write the git hooks.
      • Sent a pull request to fix some documentation.
      • Really designed to run as a “pre-commit” hook locally on a person’s machine rather than a server side hook. Logged an issue to discuss that.
      • Has a lot of nice plugins for Python development already.
      • Would allow us to add hdl-pretty as a commit hook.
    • Alternatives to jig appear to be;
      • https://github.com/icefox/git-hooks – Written in bash
      • https://pypi.python.org/pypi/git-pre-commit-hook – Written in python
  • Got my “fake” Xilinx Platform Cable USB (Model DLC9G) working under Linux – Full instructions can be found at https://github.com/timvideos/HDMI2USB/wiki/Xilinx-Platform-Cable-USB-under-Linux
    • The device was purchased from eBay on the 11th July, costed $37 USD with shipping and arrived at Joel’s house on Monday.
    • First issue was /opt/Xilinx/14.7/ISE_DS/ISE/bin/lin64/setup_pcusb didn’t understand I had udev and was trying to install for the ancient hotplug.
      • Fixed by changing line 26 from TP_USE_UDEV=”0″ to TP_USE_UDEV=”1″
    • Next setup_pcusb didn’t tell udev to reload the udev rules. Had to do that with udevadm control –reload-rules
    • Next setup_pcusb didn’t ask me to install fxload, so I needed to install it with apt-get install fxload
    • Next the rules that were installed to /etc/udev/rules.d/xusbdfwu.rules were invalid; they caused the following errors in /var/log/daemon.log
Jul 23 16:40:29 laptop udevd[841]: unknown key 'SYSFS{idVendor}' in /etc/udev/rules.d/xusbdfwu.rules:2
Jul 23 16:40:29 laptop udevd[841]: invalid rule '/etc/udev/rules.d/xusbdfwu.rules:2'
Jul 23 16:46:11 laptop udevd[841]: unknown key 'SYSFS{idVendor}' in /etc/udev/rules.d/xusbdfwu.rules:3
Jul 23 16:46:11 laptop udevd[841]: invalid rule '/etc/udev/rules.d/xusbdfwu.rules:3'
    • This was fixed with;
      • Changing SYSFS to ATTRS
      • Changing BUS to SUBSYSTEM
      • Changing $TEMPNODE to $tempnode
    • The little status light then turned on red! Yay!
    • Was able to do a boundary scan in iMPACT on a Zybo development board after soldering a header onto it.
  • Received the HDMI2USB production boards created by Numato!
    • Started with bunch of stuff to do with LEDs;
      • Logged issues about LEDs not been label intelligently. Such as the “power” LED being labeled “D33”.
      • Researched the DONE net again so I could understand the D2 LED (which should be named D_FPGA_NOT_CONFIGURED or DNCFG for short). Started adding the information to the JTAG/Reset documentation.
      • Figured out why D3 (connected to the Cypress INT1 pin) was only faintly lit.
    • Logged a bunch of issue regarding small silk screen fixes to make future boards easier to understand.
    • Logged an issue about adding some standoffs in the center of the board for mechanical stability.
    • Logged an issue about having a good GND point to connect your probe too.
      • This page describes the two good ways to add a GND test point – http://www.robotroom.com/PCB-Layout-Tips.html
    • Starting researching the 5V rail and if we could remove it totally (thus saving a bunch of stuff). Looks like we can, but needs more investigation.
    • After replacing the JTAG cable was able to use iMPACT to boundary scan and it found Spartan 6 chip!
  • Discovered CEC is 3V3 signal.

Non HDMI2USB stuff

  • Tried to figure out why my home router has decided that it wants to hand out address in the 2001:44b8:31dc:8d01::/64 rather than the 2001:44b8:31dc:8d00::/64 range it use too.
  • Found a bunch of issues with domains served of ns1.mithis.com as the secondary servers where disabled. Root cause was an old version of PowerDNS failing on TCP zone transfers causing domains to become stale and get dropped from the secondaries. Enabled email notification when secondary disables the zones.

 

2014 07 24 HDMI2USB
TimVideos.us Comments (0)

HDMI2USB – Production Board Bring Up – Day 2 (22nd July 2014)

Non-HDMI2USB related stuff

HDMI2USB related stuff

  • Worked from Joel’s house today to receive the production HDMI2USB boards, they didn’t turn up 🙁
  • Create a trusty VM on server for Joel to setup HDMI2USB CI on.
  • Investigated origin of files in HDMI2USB code. Current status seems to be;
    • hdl/edid – Written by us (Jahanzeb).
    • hdl/hdmi – Mostly written by Xilinx and released in application notes. Need to figure out the exact licensing of the files (are they MIT? Only for use on Xilinx parts?) and move them to third party.
    • hdl/jpeg_encoder – OpenCores mkjpeg core. Need to move to third party.
    • hdl/usb – Written by us (Jahanzeb).
    • ipcore_dir – Cores generated by Xilinx. Need to figure out the exact licensing of the files (are they MIT? Only for use on Xilinx parts?) and move them to third party.
    • cypress – Confusing mess. We know this and plan to rewrite the firmware using only FOSS tools.
      • Bunch of proprietary binary blobs.
      • Bunch of headers of unknown copyright.
      • Bunch of code written by us.
  • Logged a lot of bugs about warnings Xilinx tools are generating when compiling the firmware.
  • Made generate_ucf.py generate a file for Atlys functionally identical to existing HDMI2USB.ucf file. Was able to successfully build the firmware with generate HDMI2USB.ucf file for the Atlys board.
  • Started trying to get firmware to compile with the Numato HDMI2USB.ucf file, ran into the following issue
Phase 4.2 Initial Placement for Architecture Specific Features
ERROR:Place - ConstraintResolved NO placeable site for
 hdmiMatri_Comp/dvi_rx1/ioclk_buf
ERROR:Place - SIO has over-constrained componet hdmiMatri_Comp/dvi_rx1/ioclk_buf
 to have to placeable sites. Constraints come from driver constraints AND load
 IO constraints
/bin/sh: line 16: 28981 Segmentation fault (core dumped) map -filter "../ise/iseconfig/filter.filter" -intstyle ise -p xc6slx45-csg324-3 -w -logic_opt off -ol high -xe n -t 1 -xt 0 -register_duplication off -r 4 -global_opt off -mt off -ir off -pr b -lc off -power off -o hdmi2usb_map.ncd hdmi2usb.ngd hdmi2usb.pcf
make: *** [map] Error 139
  • Rohit GSoC mentoring;
    • Emailed Rohit about reimbursement for VGA Board hardware parts.
    • Rohit has shipped me a board which should arrive at Joel’s tomorrow.
    • Weekly mentoring meeting.
    • Reviewed Rohit’s weekly summary blog post.
    • Reviewed Rohit’s VGA board V2 schematic.
    • Reviewed Rohit’s VGA board V2 PCB.
2014 07 23 HDMI2USB
TimVideos.us Comments (0)