First Steps with Buildroot May 4, 2020 on tomleb's blog

I am currently working on a personal project which involves building a Linux embedded system on the Raspberry Pi (and eventually other SoC). To build the Linux system, I decided to use buildroot, a “simple, efficient and easy-to-use tool to generate embedded Linux systems through cross-compilation”.

This post will describe how to use buildroot to build a simple C library and a simple golang binary. These are steps that I will need to do for the project, so I figured I better get familiar with it.

Introduction to Buildroot

When building a Linux embedded system, we usually need to cross-compile code because the target architecture is not the same as the host architecture. For example, my current laptop is a Thinkpad T420 and its architecture is x86_64. My target device is a Raspberry Pi 2 Model B which has the ARM architecture armv7l. The armv7l is a 32-bit ARM architecture.

You can use the following command to find out the architecture of a device:

$ uname -m
x86_64

To compile for the target architecture, we have basically two options:

1. Compile on the target machine itself.
2. Compile on a host machine but for the target machine. This is called cross-compiling.

Option 1 is probably the easiest, but it gets burdensome quickly when the target machine is not powerful. On the other hand, option 2 requires a bit more setup, but allows us to use our powerful development machines ¹.

Buildroot is a tool that simplifies the second workflow described. It allows us, among other things, to easily build and use the toolchain to cross-compile for the target machine. Buildroot also comes with lot of configurations built-in to quickly start working on common boards like the Raspberry Pi.

This post is not a full introduction on buildroot. Instead, I highly suggest to follow the blog series Mastering Embedded Linux by George Hilliards. His series show the steps on how to build an embedded Linux system for a Raspberry Pi Zero. Part 3 explains the high level overview of how buildroot works and how to use it.

Initial configuration

You can follow along even if you don’t have a Raspberry Pi handy. For this post, I will not build a complete image but focus on two projects only, in C and in golang.

The first thing we want to do is clone the buildroot repository and checkout the tag 2020.02.1. This makes sure that if you follow the rest of the article, you will have the exact same version I am using.

git clone https://git.buildroot.net/buildroot
cd buildroot/
git checkout 2020.02.1

Then, we select a pre-made configuration for the Raspberry Pi. This configures our toolchain among other things to work for a Raspberry Pi. We also build our toolchain. This might take a while, so in the meantime, you can go read Mastering Embedded Linux if you haven’t.

make raspberrypi2_defconfig
make toolchain

We are now ready to add our first C library as a new package for buildroot.

Adding a C library built with Meson

We will go over the steps necessary to add the libfoo library. This is a small useless library that uses Meson as its build system. The library has two optional dependencies: libxml2 and yaml-0.1. As mentioned in the README, we can pass -Dxml=true and -Dyaml=true to enable xml and yaml features respectively. For buildroot, we will want to add configuration for both of these to easily enable and disable them.

To add a new package, we need to add a new directory in package/ with the name of our package. Buildroot also requires at least two files: Config.h and libfoo.mk².

mkdir package/libfoo/
touch package/libfoo/Config.in
touch package/libfoo/libfoo.mk

Before explaining what the content should be for those two files, we need to add a line in package/Config.in. Let’s add it in the menu Target packages -> Libraries -> Other. This will make our package available to select with make menuconfig.

--- a/package/Config.in
+++ b/package/Config.in
@@ -1745,6 +1745,7 @@ menu "Other"
        source "package/libevdev/Config.in"
        source "package/libevent/Config.in"
        source "package/libffi/Config.in"
+       source "package/libfoo/Config.in"
        source "package/libgee/Config.in"
        source "package/libglib2/Config.in"
        source "package/libglob/Config.in"

The file package/libfoo/Config.in defines options that is used when configuring the embedded system. These options are then used in the Makefiles (*.mk) files to determine what to build and with which features. For each configuration option in package/libfoo/Config.in, we can specify dependencies that will be automatically selected. For more information, look at 17.2.1. Config.in file and the official kconfig language.

We start by putting the configuration option BR2_PACKAGE_LIBFOO for our package in package/libfoo/Config.in. This is the configuration option that determines if our package is selected to be built.

config BR2_PACKAGE_LIBFOO
	bool "libfoo"
	help
	  Useless library that does absolutely nothing

	  https://git.sr.ht/~tomleb/meson_libfoo

The BR2_PACKAGE_LIBFOO option can be used by other packages' Config.in. In our case, we use this variable in the same file to show two other options when libfoo is selected.

We add two new options, BR2_PACKAGE_LIBFOO_XML and BR2_PACKAGE_LIBFOO_YAML that will determine if libfoo will be compiled with xml and yaml support respectively. For example purposes, we set the BR2_PACKAGE_LIBFOO_XML option to be set by default.

if BR2_PACKAGE_LIBFOO

config BR2_PACKAGE_LIBFOO_XML
	bool "libfoo xml"
	default y
	select BR2_PACKAGE_LIBXML2
	help
	  Add xml feature

config BR2_PACKAGE_LIBFOO_YAML
	bool "libfoo yaml"
	select BR2_PACKAGE_LIBYAML
	help
	  Add yaml feature

endif # BR2_PACKAGE_LIBFOO

Notice that for each options, we added a select statements. This describes the dependencies on other options, and those options are automatically selected. For example, if we enable BR2_PACKAGE_LIBFOO_XML, the option BR2_PACKAGE_LIBXML2 will be automatically enabled. You can see the full Config.in file here: package/libfoo/Config.in.

Now that we have defined the configuration options for our libfoo package, let’s describe how the package will be built. This is done in the Makefile file package/libfoo/libfoo.mk.

This file define environment variables to describe how to download the source code, which license the code uses and list the package dependencies. Here, we define that the source code can be found at https://git.sr.ht/~tomleb/meson_libfoo and the ref checked out is 7129ae260cb6767998a63be288e4a7c17f34f4f5. Since this is a library, we set LIBFOO_INSTALL_STAGING = YES. This means that the library will be available to other packages built with buildroot if they need to build against our library. Finally, we put the host-pkgconf dependency because Meson depends on pkg-conf to find the dependencies.

LIBFOO_VERSION = 7129ae260cb6767998a63be288e4a7c17f34f4f5
LIBFOO_SITE = https://git.sr.ht/~tomleb/meson_libfoo
LIBFOO_SITE_METHOD = git
LIBFOO_INSTALL_STAGING = YES
LIBFOO_LICENSE = MIT
LIBFOO_LICENSE_FILES = LICENSE
LIBFOO_DEPENDENCIES = host-pkgconf

Next, we configure the arguments that we will pass to meson build. Remember that libfoo comes with two optional options, one to enable xml, and one to enable yaml. We use the option BR2_PACKAGE_LIBFOO_XML and BR2_PACKAGE_LIBFOO_YAML that we defined in Config.in to determine which options is enabled and disabled.

ifeq ($(BR2_PACKAGE_LIBFOO_XML),y)
LIBFOO_CONF_OPTS += -Dxml=true
LIBFOO_DEPENDENCIES += libxml2
else
LIBFOO_CONF_OPTS += -Dxml=false
endif

ifeq ($(BR2_PACKAGE_LIBFOO_YAML),y)
LIBFOO_CONF_OPTS += -Dyaml=true
LIBFOO_DEPENDENCIES += libyaml
else
LIBFOO_CONF_OPTS += -Dyaml=false
endif

The option LIBFOO_CONF_OPTS contains the flags that will be passed to the meson build command. For example, if BR2_PACKAGE_LIBFOO_XML is enabled and BR2_PACKAGE_LIBFOO_YAML is disabled, then the meson command will become meson build -Dxml=true -Dyaml=false.

The option LIBFOO_DEPENDENCIES contains a list of package dependencies. The items in this dependency list needs to be the name of the package that must be built. For example, when yaml is enabled, then the package libyaml will be built.

Finally, we end the libfoo.mk by telling it how the package must be built. In this case, we specificy that the package must be built with meson.

$(eval $(meson-package))

You can see the whole file package/libfoo/libfoo.mk here: package/libfoo/libfoo.mk.

With these two files written, you can now build your package along with its dependencies with the following command. Note that this might take some time if the meson toolchain was not built previously.

$ make libfoo
...
ninja: Entering directory `output/build/libfoo-custom//build'
[0/1] Installing files.
Installing libfoo.so.1 to output/target/usr/lib
Skipping RPATH fixing

At the end, you can see the file in the directory output/target/usr/lib. You can also verify that the file is built for your target architecture like so. In this case, we see that the file is a shared object for the ARM architecture, as expected!

$ file output/target/usr/lib/libfoo.so.1
output/target/usr/lib/libfoo.so.1: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), statically linked, not stripped

Adding Golang binaries

The second part of my project will require me to build golang binaries and install them in the embedded system. Fortunately, buildroot comes with golang support out of the box. For this example, I will be adding the package libfoo-golang which is also another useless library that comes with two binaries: foo-a and foo-b.

The steps are similar to the libfoo package we have added previously. In fact, only the Makefile differs. Let’s begin by creating the package directory. We will name our package libfoo-golang. The identifier that we will use is LIBFOO_GOLANG.

mkdir package/libfoo-golang/
touch package/libfoo-golang/Config.in
touch package/libfoo-golang/libfoo-golang.mk

We also add our package in the file package/Config.in to make it available when running make menuconfig. Let’s add it in the menu Target packages -> Libraries -> Other.

--- a/package/Config.in
+++ b/package/Config.in
@@ -1745,6 +1745,7 @@ menu "Other"
        source "package/libevdev/Config.in"
        source "package/libevent/Config.in"
        source "package/libffi/Config.in"
+       source "package/libfoo-golang/Config.in"
        source "package/libfoo/Config.in"
        source "package/libgee/Config.in"
        source "package/libglib2/Config.in"

We start by putting the configuration option BR2_PACKAGE_LIBFOO_GOLANG for our package in package/libfoo-golang/Config.in. This is the configuration option that determines if our package is selected to be built. For this example, this option will be used to build the binary foo-a.

config BR2_PACKAGE_LIBFOO_GOLANG
	bool "libfoo-golang"
	depends on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
	help
	  Build the binary foo-a

	  https://git.sr.ht/~tomleb/libfoo-golang

Notice the new Kconfig option depends on. This is similar to select because it is also related to the dependencies of the package. However, depends on only define the dependency and does not select them automatically. This means that in order to be able to enable our package, we first need to manually enable the host golang support. All go packages must depend on BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS option. Fortunately for us, this is enabled by default.

Next, we define another option to build the binary foo-b. This option will only be available if our package is enabled.

if BR2_PACKAGE_LIBFOO_GOLANG

config BR2_PACKAGE_LIBFOO_GOLANG_B
	bool "foo-golang-b"
	help
	  Build the binary foo-b

endif # BR2_PACKAGE_LIBFOO_GOLANG

You can see the whole file package/libfoo-golang/Config.in here: package/libfoo-golang/Config.in.

We can now write the Makefile that will define how to build our golang binaries. Again, this is similar to the previous Makefile, but with some changes specific to golang. We start with the bare minimal to specify how to download the package and what license it has.

################################################################################
#
# libfoo-golang
#
################################################################################

LIBFOO_GOLANG_VERSION = 256a3102d716f131058cdd296f19e412ec170bcf
LIBFOO_GOLANG_SITE = https://git.sr.ht/~tomleb/libfoo-golang
LIBFOO_GOLANG_SITE_METHOD = git
LIBFOO_GOLANG_LICENSE = MIT
LIBFOO_GOLANG_LICENSE_FILES = LICENSE

Next, we specify which binary will be built. By default, we will build the binary found in cmd/foo-a. Thus, we specify the build target cmd/foo-a as well as the binary foo-a that must be installed on the embedded system.

LIBFOO_GOLANG_BUILD_TARGETS += cmd/foo-a
LIBFOO_GOLANG_INSTALL_BINS += foo-a

Next, we conditionally build the binary foo-b based on whether BR2_PACKAGE_LIBFOO_GOLANG_B is enabled or not.

ifeq ($(BR2_PACKAGE_LIBFOO_GOLANG_B),y)
LIBFOO_GOLANG_BUILD_TARGETS += cmd/foo-b
LIBFOO_GOLANG_INSTALL_BINS += foo-b
endif

Finally, we end the file libfoo-golang.mk by telling it how the package must be built. In this case, we specificy that the package must be built with go.

$(eval $(golang-package))

First, for demonstration purposes, let’s use make menuconfig to enable our foo-b binary. We can now build the package foo-golang with the following command. Note that this might take some time if the go toolchain was not built previously.

$ make libfoo-golang
...
/usr/bin/install -D -m 0755 output/build/libfoo-golang-256a3102d716f131058cdd296f19e412ec170bcf/bin/foo-a output/target/usr/bin/foo-a
/usr/bin/install -D -m 0755 output/build/libfoo-golang-256a3102d716f131058cdd296f19e412ec170bcf/bin/foo-b output/target/usr/bin/foo-b

At the end, you can see the files in the directory output/target/usr/bin. You can also verify that the files are built for your target architecture like so.

$ file output/target/usr/bin/foo-a
output/target/usr/bin/foo-a: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, Go BuildID=blGy3odwCd9D77n8KEm4/3K9uKHi4RolfSD8vFAJF/bW5KgH8DgNVfFph5-wPX/cTy_XYOTls0qu5f_l-YU, not stripped
$ file output/target/usr/bin/foo-b
output/target/usr/bin/foo-b: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, Go BuildID=hjDA3WrpI9KBNs1DRWkP/muILpEK7u8jtCkZY6gPw/-PyWQEIv1j8XvIUUSZh8/0rRnB7stjgUq4ARLZvdh, not stripped

Workflow Tips

This post is longer than I wanted it to be already, but let me give you a tip to adding a package while developping it.

You can create the file local.mk and add the following options to make buildroot use a local directory for the package source.

# For libfoo
LIBFOO_OVERRIDE_SRCDIR = <path to libfoo>
# For libfoo-golang
LIBFOO_GOLANG_OVERRIDE_SRCDIR = <path to libfoo-golang>

Conclusion

Whew! In conclusion, we learned how to add a new package to buildroot with meson as a build system. We also learned how to add a golang package that builds multiple binaries.

I highly suggest looking at the buildroot documentation which provides a lot more details and options.

Happy hacking!