Avian’s Blog

Barebox is an open source bootloader intended for embedded systems. In the past few weeks I've been spending a lot of time at work polishing its support for Raspberry Pi boards and fixing some rough edges. Some of my patches have already been merged upstream, and I'm hoping the remainder will end up in one the of upcoming releases. I thought I might as well write a short tutorial on how to get it running on a Raspberry Pi 3 with the stock Raspbian Linux distribution.

The Broadcom BCM28xx SoC on Raspberry Pi has a peculiar boot process. First a proprietary binary blob is started on the VideoCore processor. This firmware reads some config files and does some hardware initialization. Finally, it brings up the ARM core, loads a kernel image (or a bootloader, as we'll see later) and jumps to it. When starting the kernel it also passes to it some hardware information, such as a device tree and boot arguments. Raspbian Linux ships with its own patched Linux kernel that tends to be relatively tightly coupled with the VideoCore firmware and the information it provides.

On the other hand, upstream Linux kernels tend to ignore the VideoCore as much as possible. This makes them much easier to use with a bootloader. In fact, a bootloader is required since they can't use the device tree provided by the VideoCore firmware. Unfortunately, upstream kernels also historically have worse support for various peripherals found on Raspberry Pi boards, so sometimes you can't use them. In the rest of this text, I'll be only talking about Raspbian kernels as most of my work focused on making the VideoCore interoperation work correctly for them.

To get started, you'll need a Raspberry Pi 3 board and an SD card with a Raspbian Stretch Lite. You'll also need a USB-to-serial adapter that uses 3.3V levels (I'm using this one) and another computer to connect it to. Connect ground, pin 14 and pin 15 on Raspberry Pi header to ground, RXD and TXD on the serial adapter respectively. We will use this connection to access the Barebox serial console. For the parts where we will interact with the Linux system on the Raspberry Pi you can also connect it to a HDMI monitor and a keyboard. I prefer to work over ssh.

Run the following on the computer to bring up the serial terminal at 115200 baud (adjust the device path as necessary to access your serial adapter):

$ screen /dev/ttyUSB0 115200

It's possible to cross-compile Barebox for ARM on an Intel computer. However, for simplicity we'll just compile Barebox on the Raspberry Pi itself since all the tools are already available there. If you intend to experiment, it's worth setting up either the cross-compiler or two Raspberry Pis: one for compilation and one for testing.

Now log-in to your Raspberry Pi and open a terminal, either through ssh or the monitor and keyboard. Fetch the Barebox source. At the time of writing, not all of my patches (1, 2) are in the upstream repository yet. You can get a patched tree based on the current next branch from my GitHub fork:

$ git clone https://github.com/avian2/barebox.git
$ cd barebox

Update: Barebox v2019.04.0 release should contain everything shown in this blog post.

In the top of the Barebox source distribution, run the following to configure the build for Raspberry Pi:

$ export ARCH=arm
$ make rpi_defconfig

At this point you can also run make menuconfig and poke around. Barebox uses the same configuration and build system as the Linux kernel, so if you've ever compiled Linux the interface should be very familiar. When you are ready to compile, run:

$ make -j4

This should take around a minute on a Raspberry Pi 3. If compilation fails, check if you are missing any tools and install them. You might need to install flex and bison with apt-get.

After a successful compilation the bootloader binaries end up in the images directory. If you didn't modify the default config you should get several builds for different versions of the Raspberry Pi board. Since we're using Raspberry Pi 3, we need barebox-raspberry-pi-3.img. Copy this to the /boot partition:

$ sudo cp images/barebox-raspberry-pi-3.img /boot/barebox.img

Now put the following into /boot/config.txt. What this does is tell the VideoCore firmware to load the Barebox image instead of the Linux kernel and to bring up the serial interface hardware for us:

kernel=barebox.img
enable_uart=1

At this point you should be ready to reboot Raspberry Pi. Do a graceful shutdown with the reboot command. After a few moments you should see the following text appear on the serial terminal on your computer:

barebox 2019.02.0-00345-g0741a17e3 #183 Fri Mar 1 09:44:11 GMT 2019


Board: RaspberryPi Model 3B
bcm2835-gpio 3f200000.gpio@7e200000.of: probed gpiochip-1 with base 0
bcm2835_mci 3f300000.sdhci@7e300000.of: registered as mci0
malloc space: 0x0fe7e5c0 -> 0x1fcfcb7f (size 254.5 MiB)
mci0: detected SD card version 2.0
mci0: registered disk0
environment load /boot/barebox.env: No such file or directory
Maybe you have to create the partition.
running /env/bin/init...

Hit any key to stop autoboot:    3

Press a key to drop into the Hush shell. You'll get a prompt that is very similar to Bash. You can use cd to move around and ls to inspect the filesystem. Barebox uses the concept of filesystem mounts just like Linux. By default, the boot partition on the SD card gets mounted under /boot. The root is a virtual RAM-only filesystem that is not persistent.

The shell supports simple scripting with local and global variables. Barebox calls the shell scripts that get executed on boot the bootloader environment. You can find those under /env. The default environment on Raspberry Pi currently doesn't do much, so if you intend to use Barebox in some application, you will need to do the scripting yourself.

To finally boot the Raspbian system, enter the following commands:

barebox@RaspberryPi Model 3B:/ global linux.bootargs.vc="$global.vc.bootargs"
barebox@RaspberryPi Model 3B:/ bootm -o /vc.dtb /boot/kernel7.img

Loading ARM Linux zImage '/boot/kernel7.img'
Loading devicetree from '/vc.dtb'
commandline: console=ttyAMA0,115200 console=ttyS1,115200n8   8250.nr_uarts=1 bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=tty1 root=PARTUUID=1c5963cc-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
Starting kernel in hyp mode

The global command adds the kernel command-line from VideoCore to the final kernel boot arguments. Barebox saves the command-line that was passed to it from the firmware into the vc.bootargs variable. When booting the kernel, Barebox constructs the final boot arguments by concatenating all global variables that start with linux.bootargs.

The bootm command actually boots the ARMv7 kernel (/boot/kernel7.img) from the SD card using the device tree from the VideoCore. Each time it is started, Barebox saves the VideoCore device tree to its own root filesystem at /vc.dtb. A few moments after issuing the bootm command, the Raspbian system should come up and you should be able to log into it. Depending on whether the serial console has been enabled in the Linux kernel, you might also see kernel boot messages and a log-in prompt in the serial terminal.

You can see that the system was booted from Barebox by inspecting the device tree:

$ cat /sys/firmware/devicetree/base/chosen/barebox-version
barebox-2019.02.0-00345-g0741a17e3

Since we passed both the VideoCore device tree contents and the command-line to the kernel, the system should work as if no bootloader was actually used. This means that settings in cmdline.txt and config.txt, such as device tree overlays and HDMI configuration, should be correctly honored.

In the end, why would you want to use such a bootloader? Increasing reliability and resistance to bad system updates is one reason. For example, Barebox can enable the BCM28xx hardware watchdog early in the boot process. This means that booting a bad kernel is less likely to result in a system that needs a manual reset. A related Barebox feature is the bootchooser which can be used to implement the A/B root partition scheme, so that a failed boot from one root automatically results in the bootloader falling back to the other root. Note however that one thing still missing in Barebox is the support for reading reset-source on Raspberry Pi, so you can't yet distinguish between a power-on reset and a watchdog running out.

As it was pointed out on the Barebox mailing list, this is not perfect. Discussion on the design of the hardware watchdog aside, the bootloader also can't help with VideoCore firmware updates, since the firmware gets started before the bootloader. Also since the device tree and kernel in Raspbian are coupled with the VideoCore firmware, this means that kernel updates in Raspbian can be problematic as well, even with a bootchooser setup. However a step towards a more reliable system, even if not perfect, can still solve some real-life problems.