Using Buildroot for IoT Projects

Buildroot RPi Yocto Logos

Early on in my effort to build a digital dash for my car I quickly realised I needed to run a stripped down version of Linux if I was to have any chance of getting the dash to boot up quickly on the Raspberry Pi. A dash that kept you waiting for tens of seconds or at worst minutes to turn on wouldn’t exactly be the most practical of solutions (especially if you want to avoid speeding fines hehe).

So I was faced with the situation of taking one of the existing distributions available for the RPi and modding it for my own use, or having a crack at making something myself that would do the job. Initially I did start tinkering around with seeing how fast I could get stripped down versions of Raspbian and Arch booting up, which although was interesting did feel as though I was spending a lot of time and effort digging down in the dirt and undoing a lot of work someone else had already spent a lot of time doing. Therefor instead of trying to tear something down to suit my needs, I decided to focus my attention to incrementally building something useful from the ground up. I figured doing it this was would be a great source of learning as I would no doubt begin to run into obstacles other people have already created solutions for, meaning I would appreciate the open source communities hard work even more so.

After a bit of searching I found a couple of promising looking projects to base my dash on (I really didn’t feel like totally reinventing the wheel). Both Buildroot and the Yocto Project share the common goal of creating a complete, easy to customise, embedded Linux system. Both compile Linux from source using cross-compilation (which should cover most performance requirements), are well documented, actively maintained and already used within the embedded systems industry. In fact I had already been exposed to the Yocto project before without even realising when I attended an Intel Edison workshop at the London FabLab, as the Linux image supplied by Intel for use on the Edison is a Yocto build.

Whilst I already had some experience with Yocto I decided to kickstart my digital dash using using Buildroot. Namely in the effort of simplicity as to get started with Buildroot is just so easy. The documentation available is awesome and provides a steady learning curve. I also found a pretty good short book on using Buildroot with the RPi which was neat primer. After a bit of tinkering I had the environment setup in a Ubuntu VM as well as a working filesystem that booted in seconds on my RPi2. I quickly appreciated the number of tools available out of the box in major Linux distributions as theres really not much to do in a basic buildroot build other than logging in and a few system utils to work with.

It was not before long I had my buildroot environment stored under Git and had automated a few of the basic build commands, such as copying over custom apps after a successful build and flashing the filesystem to SD card. All in all it didn’t take a ton of effort to get a simple enough system working that booted up fast, was capable of connecting to the Internet and launched my prototype app on boot. My prototype app is built using QT with PyQT bindings, both of which buildroot supports although not the latest versions.
I would happily recommend Buildroot to anyone who wants to learn more about Linux targeted for embedded devices and IoT projects, it really is a pleasure to work with and the community behind it is very resourceful and helpful.

Looking towards the future for my digital dash there may come a time where I may have to look to migrating towards other projects like Yocto. Buildroot is great and pretty much does everything I need at this point in time, however it does have what I consider to be quite a big no-go – especially in the IoT community. There is no package management system and no binary packages, meaning updates to the system are not possible via packages. If a future release of the Linux kernel fixes a security vulnerability, a full system update would be required which would involve physically removing the SD card and updating the software manually. Best practices for building IoT devices include having an inbuilt mechanism from the word go, allowing for critical updates to be issued that threaten to compromise the device without having to rely on the end user to manually update their device.

All of this is subject to change I guess, Buildroot could one day change how they do things. For now I’m sticking with them as I’m the only one working on the dash project, but being the responsible security guy I am if I open up the project for others to tinker with I would want to switch to another project that supported some form of inbuilt updating.

Anyway it’s about time I get back to finishing rebuilding the engine for the Mk1 so I can actually test the dash in-car for once instead of on my desk!

MegaSquirt 3 Digital Dash

I am pleased to announce and introduce a little side project that I have been working on the past few months (years). The projects initial main focus is to create a HD digital dashboard for the MegaSquirt ECU, to be built using as much COTS (commercial off the shelf) hardware as possible. This means that hopefully with some conscious design planning it should be possible to produce copies fairly easily just by following some documentation and ordering a few bits of the Internet here and there (This is the plan at least, who knows how things will pan out). Theres no reason other ECUs can’t also be supported in the future, however thats the only aftermarket ECU I have at hand and since thats what I run in my cars.

Project Brief

Create a digital dashboard that respects the character and history of the original cluster gauge, whilst also providing access to additional metrics that are critical for any high performance or turbocharged engine.

And so after a bit of hard work this is what I have come up with for now..

Mk1 Escort Digital Dash
The original dash is on the left (front cover removed)

The dash features an 8 inch 1080P LCD display powered by a Raspberry Pi 2 (soon to be RPi 3) running a stripped down build of Linux to allow for fast booting. The dash communicates with the cars MS3 ECU via CAN bus and will include some additional power features that allow it to turn on/off safety via a switched ignition input.

The main motivation for having a digital dash is to take advantage of displaying all the extra sensor data the MS3 ECU utilises and accumulates. There are sensor outputs and ECU calculations that I would be interested in knowing whilst driving, but at the same time I don’t with to clutter the dash with additional gauges and dials as that would spoil the general aesthetics of the interior. Creating a digital dash in the same style as the original dials is a good compromise on having access to the data and not ignoring the originality of the car.

The development and install of the first dashboard will be a prototype version retrofitted into my Mk1 Escort. This type of instrumentation is something that I have wanted for many years, however it wasn’t until recently that I felt I had gained the right knowledge sets and courage to attempt such a project and not forgetting the funds to back. Believe me when I say this has been a long time in the works.

I will post more details on the project in the upcoming weeks but for now just wanted to give a glimpse into some of the more complicated long term projects I’m working on.

Fixing FreeNAS error: Currently unreadable (pending) sectors

I recently received a critical alert email from my FreeNAS box with the following error:

Device: /dev/ada3, Self-Test Log error count increased from 0 to 1
Device: /dev/ada3, 1 Currently unreadable (pending) sectors

Rather glad to know the email alerts I setup is working reliably, but looks as though I might have a few bad sectors on one of my drives.
The following commands resolved the error without resulting in any downtime.

The drive in question was /dev/ada3, so first login to a shell on your FreeNAS box as root and run a SMART long self-test (Replace adaX with your corresponding device).

smartctl -t long /dev/adaX

After the test has finished (It might take a few hours) view the results.

smartctl -a /dev/adaX

From the results remember the sector size and the location of the faulty sector (LBA_of_first_error).
In my case my sector size was 512 and LBA_of_first_error was 3082982872.

To correct the SMART error we will zero out the bad sector(s) on the drive, but first we need to permit access to drive.

sysctl kern.geom.debugflags=16

Now zero the sector stated in self-test results out.
Replace of=/dev/adaX, bs=512, seek=3082982872 with values relevant to your drive.

dd if=/dev/zero of=/dev/ada3 bs=512 count=1 seek=3082982872

Re-run the SMART report command to check if the ‘Current_Pending_Sector’ is now showing 0.

smartctl -a /dev/adaX

To check the ZFS file system integrity run a scrub of the pool, replace poolX with the pool name the drive is under (list pools with ‘zpool list’).

zpool scrub poolX

Finally check the output of the scrub to ensure there are no known data errors.

zpool status -v poolX

3D printing automotive connectors – Part 1

Spread out over the past few months I have been gradually reverse engineering the entire wiring diagram for my Mk1 Escort. This is for when I recreate some of loom to update and surpass Ford’s original specification in order to be more accommodating for my MS3 ECU and larger loads placed upon the cars electrics. In addition to having a more reliable and updated car, I personally benefit from the knowledge gained by having a good understanding of the attitude Ford engineers of the 70’s/80’s had with designed cars. My research has also led me to writing more of my own more in documentation which provides much greater detail than some of the existing diagrams and service manuals currently available.

Recently I have found myself modelling a few connectors I will need for upcoming project work on my Mk1. I believe most of the connectors are proprietary and were produced specifically by Ford, resulting in the connectors being decades out of production and cannot be sourced online easily (or cheaply).

This 12 pin round male connector connects the main 2/6 pod cluster gauges to the main loom inside the dash. Using a set of digital vernier callipers I measured and poked all over the original connectors to document the measurements as best I could. I then fired up Autodesk Fusion 360 for the first time and began modelling after watching a few YouTube videos showing basic parametric modelling techniques.
Being my first ever 3D CAD model and also my first time using Fusion 360, I was very pleased with the final model outcome.
Mk1 Escort connector render

It was 3D printed using white PLA plastic with 100% fill rate on a Ultimaker 2 3D printer. I understand PLA is not ideal material for automotive applications, so I will most likely reprint in ABS plastic when it comes to using the final part.

This was also one of my early 3D prints, and was my first time printing something I had directly modelled myself. The white part to the left is my 3D printed copy and the clear part on the right is the original.
Mk1 Escort dash connectors

The pins fit nice and snug on the copy as they do on the original piece.
Mk1 Escort dash connectors

My latest model is the 8 pin connector which pokes through the firewall for the main engine loom. I will probably have a go at printing this in flexible PLA filament, however I really do not think the material will be up to scratch coping with the harsh environment exposed in the engine bay (high temp, oil, moisture, extreme vibrations etc). So this model will most likely serve as a mould of some kind in order to allow use of a more suitable material (yet TBD).
Mk1 Escort engine loom connector render

I’ll keep posting updates as I continue to model and print new parts and pieces for my Mk1 Escort.

Circus Ponies Notebook is no more

Circus Ponies logo

This afternoon I went on over to Circus Ponies to download a fresh copy of Notebook for a new HackPro of mine, and was presented with this lovely notice.
Thanks for all the fish

Well that was a bit out of the blue, and with no prior notice. No emails, no heads up, no nothing.

Circus Ponies announced today that they are closing up shop, having been acquired by Google. They had been the creators of a fab piece of note taking software called Notebook. It really was a simple bit of kit that did was it was meant to do and nothing more, I used it in conjunction with Owncloud to privately host my project notes which automatically synced between my various computers. Not sure why Google would want their hands on it yet, perhaps to improve their own suite of Apps capable of note taking.

Now begins the joyous task of finding something else that will fit the bill. I had been considering for a while about hosting my own internal wiki to accomplish the same role as Notebook, the only niggle with that idea is I often find myself without a reliable Internet connection (trains) and don’t want to be sat stranded not being able to do work simply because I don’t have the relevant project notes handy.

Anyways It’s not as if the app is crippled without online functionality as it doesn’t really have any. So in the meantime I will continue to use until I can find a suitable replacement or Apple releases an update with breaks compatibility.
That really will be the last nail in the coffin for Notebook.