Port configure fails on ARM64

On FreeBSD, if you are trying to build a port but it fails at the configure step with a message similar to this:

checking build system type... Invalid configuration `arm64-portbld-freebsd13.2': machine `arm64-portbld' not recognized
configure: error: /bin/sh ./build-aux/config.sub arm64-portbld-freebsd13.2 failed

Here’s a quick-fix that might work for you:

export CONFIGURE_TARGET=arm64-unknown-freebsd13.2
make install

This is similar to passing --host arm64-unknown-freebsd13.2 to the configure script instead of trying to guess it.

FreeBSD on NanoPi R2S

The NanoPi R2S is a nice little ARM board that can serve nicely as a home router or firewall. It is comparable in power to a RaspberrryPi 3 with a 4 cores ARM Cortex-A53 SoC, namely the Rockchip RK3328. It has no video or sound output, only a handful GPIOs compared to a RPi, but it has one great advantage. It has 2 Ethernet ports, one of them is a true Gigabit internal card and the second is also a Gigabit connected through the internal USB3, so it’s a bit slower. On the RPi3, if you want 2 Gigabit Ethernet ports, you need an external card and both ports would run on the USB3, drastically limiting the bandwidth.

They generally come with a heatsink and a fan integrated as the board is quite small compared to a RPi and heats up more easily. People generally use this board with OpenWrt or Ubuntu. In the past, I also used it with Armbian as my home network gateway. People also managed to run it on OpenBSD. But this post focuses on FreeBSD.

ARM64 has been a tier-1 architecture on FreeBSD for more than a year now, which I discussed in this post. I managed to run the RPi 2B+, 3 and 4 easily, but my attempts with the Nanopi R2S proved rather unsuccessful and had their fair share of kernel panics.

Fast forward one year, and we are now at FreeBSD 13.1-p1. So I recently gave it another try, and not only is the whole experience much more stable, the installing process is also much easier and straightforward.

Installation

To install FreeBSD, I used the image provided at PersonalBSD. It’s a custom image based on a patched version of FreeBSD 13. It’s quite customized in fact, and point to its own repo by default. It starts sshd by default, and run both interfaces on DHCP. It has pubkey root login enabled and also has another admin user installed.

The rc.conf has growfs enabled, so the first boot will grow the ROOT UFS partition to the size of your SD card. It’s all GPT. There is a 256MB swap partition, but you can add a file based swap if you want. For instance in /etc/fstab:

md99 none swap sw,file=/.swap,late 0 0

The root partition has the following options enabled:

tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 disabled
tunefs: soft update journaling: (-j)                       disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            2768
tunefs: optimization preference: (-o)                      space

It has soft updates disabled, which means that if the board has a hard reboot, you are in for some fsck nightmare. You can enable softupdates offline when the card is connected to your computer (running FreeBSD) for instance:

tunefs -n /dev/da0p2

I highly recommend to enable soft updates to save you much hassle.

Playing around

By default there is no EEPROM for the MAC addresses so they are randomly generated on each boot. Keep this in mind if you want to use IPv6 and SLAAC. There is also three status LEDs on the board. You can control them from the command line with echo 0|1 > /dev/led/nanopi-r2s:*. The custom image comes with a daemon and rc.local shell lines to setup a default state for the LEDs, but you can easily disable it.

The two interfaces are dwc0 for the Rockchip internal card, and ue0 for the card connected through USB3. Here is a quick test with iperf3 using UDP on both interfaces. Results are given in Mbits/sec.

TX RX
dwc0 765 946
ue0 784 370

 

Running it

But does it fit an use for a full fledged Internet gateway? I ran the Nanopi R2S for some time now as a home network gateway, and while it worked very well most of the time, the R2S would from time to time become completely inaccessible. Although it did not seem to panic as scripts where still running behind the scene. I could not identify what the problem was, but I know that the problem happened when either the Ethernet cable on the LAN interface was disconnected, or when too much traffic arrived on the Ethernet port.

HiFiBerry on Debian ARM64

Long are gone the days of the ten thousands songs Winamp playlist, the modern way of listening to music consists solely of spotify blasting its (not-so-randomly) set of tracks and ads on random variants of laptop/phones/speakers.

But I will forever say no to that.
Spotify is bad for the artists, for you, our planet and it has as much a good impact on musical culture as Facebook has on our social life. Today, the common physical storage available locally (if you don’t depend uniquely on the cloud) has more than enough room to host all the music you’d ever want to listen to. Of course this means that you have to handle your music playlist manually. But that effort will refine your tastes as to what is genuinely good music to your ears, and what is just effective product placement.

To handle my music media center, I always had a RPi stucked to my HiFi system. This RPi would only handle playing music (mpd) and remote control (Cantata, MPDroid, ). This music would be stored remotely on my NAS and accessible in read-only from NFS (yup NFS not SAMBA/CIFS).

The music is played through a RPi HAT, the HiFiBerry DAC+ based on Texas Instruments PCM5102A DAC chip. This was done on a common Raspbian (the only viable solution back then), may be you could do the same easily on the new RaspberryPi OS. I did try to replicate the same setup on FreeBSD but neither the HiFiBerry nor the PCM5102 codec were supported. Attempts to play the music through a USB audio device instead of the HiFiBerry HAT were ultimately unsuccessful probably because of the USB DWC2 host driver. The same attempt on a FreeBSD running RPi4 with PCIe XHCI USB host ran the same USB audio devices perfectly well.

[Debian Diversity Logo -- https://gitlab.com/valessiobrito/artwork (GPLv3 -- https://gitlab.com/valessiobrito/artwork)]

But since I only had a single RPi4 that was already in use, and the fact that RPi4 are currently way overpriced and out-of-stock, I resolved to try a Linux based setup on a RPi3B. But you see, Raspbian is rather restricted by the fact that the RPi platforms ranges from ARMv6 to ARMv8 SoC. That’s 32 to 64-bit, so as a compromise Raspbian runs everything in 32-bit even on ARM64 capable SoC like the RPi3 or better (the new Raspberry Pi OS has some ARM64 images although it’s not widely advertised). The challenge was that it should run Debian ARM64 and play audio through the HiFiBerry DAC+.

The main problem is that the Debian ARM64 image runs a mainline/vanilla Linux kernel. Among other things this version of the kernel does not have support for the HiFiBerry. Instead I resolved to compile my own kernel with the appropriate driver included. Also to avoid the hurdle of porting everything HiFiBerry related and other RPi/BCM goodies to the mainline kernel, I used the Raspberry Pi Foundation kernel source tree.

As a starting point, I used the configuration of the RPi3B kernel from the Debian ARM64 image, which you can find in /boot/config-5.10.0-8-arm64. Crosscompiling the kernel for ARM64 was pretty straight forward:

# Mount the RPi image
mount /dev/sdf1 /mnt/rpi/fat32
mount /dev/sdf2 /mnt/rpi/ext4

# Install and clone the kernel
apt-get install crossbuild-essential-arm64
git clone git://github.com/raspberrypi/linux.git

# Setup some variable for cross-compiling.
cd linux
export KERNEL=kernel8
export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-

# Kernel configuration
# (be sure to select the appropriate platform or the Device Tree Blob won't be compiled)
cp /mnt/rpi/ext4/boot/config-5.10.0-8-arm64 .config
make oldconfig
make menuconfig

# Compile!
make Image modules dtbs -j32

# Install
# (you might have to create the 'overlays' directory in the fat32 partition)
VMLINUZ=vmlinuz-5.10.74+
make INSTALL_MOD_PATH=/mnt/rpi/ext4 modules_install
cp arch/arm64/boot/dts/broadcom/*.dtb  /mnt/rpi/fat32
cp arch/arm64/boot/dts/overlays/*.dtb* /mnt/rpi/fat32/overlays/
cp arch/arm64/boot/Image "/mnt/rpi/fat32/${VMLINUZ}"
cp arch/arm64/boot/Image "/mnt/rpi/ext4/boot/${VMLINUZ}"

# Umount the RPi image and boot it still on the old working kernel.
umount /dev/sdf1
umount /dev/sdf2

# Connect to the RPi and create the initrd.
# This should also adapt the boot config. 
update-initramfs -c -k 5.10.74+

If you want to play with Device Tree overlays for example to use a RPi HAT like the HiFiBerry, you will have to compile the dtoverlay manipulation command from the RPi userland repository. Install libfdt-dev, compile and install libdtovl in helpers/dtoverlay, compile dtoverlay in host_applications/linux/apps/dtoverlay.

You need ConfigFS and the following kernel options for the dtoverlay command to work with dynamic DT:

CONFIG_DTC=y
CONFIG_OF=y
CONFIG_OF_UNITTEST=y
CONFIG_OF_DYNAMIC=y
CONFIG_OF_OVERLAY=y
CONFIG_OF_CONFIGFS=y

Also mount ConfigFS:

mkdir /config
mount -t configfs none /config

The command expects to find the overlays in /boot/overlays but on the Debian ARM64 image they will probably be in /boot/firmware/overlays. I fixed this with a symlink.

Next I had to fiddle a bit with the Device Tree sources. If you want some nice documentation about them, here’s a nice pdf from Freescale that explains a lot.

When I first tried to add the HiFiBerry overlay with dtoverlay hifiberry-dacplus, it could not apply because of “incompatible devices”. This was caused by discrepancies between the ARM and ARM64 DT sources for the RPi. This driver was made for the ARM arch than runs Raspbian, not ARM64. And the ARM arch DTS/DTSI exposes devices differently than ARM64. I added the missing devices on ARM64, recompiled and reinstalled the DTB on the RPi image. After that the DT overlay applied like a charm.

The HiFiBerry DAC+ is now happily playing music on Debian ARM64. The RPi3B runs at ~1.14W on idle and ~1.35W when playing music. I don’t how know much the fat-free low-power kernel config contributes to that, but it seems to run at ~1.25W on Debian’s vanilla kernel and I have some ideas to reduce the consumption even further.