Raspberry Pi 4B (bcm2711) for L4S Build Notes (June 14, 2023)

Help towards building a kernel which is L4S capable and runs on a raspberry pi 4B. The devices are assumed to be used for experimental, wired testing purposes only and not on production networks. This is not an endorsement of L4S but the expected use is for technical evaluations. There is no marking forwarding plane documented here either. That’s on my todo list.

Please read the following disclaimer.

Disclaimer

This site does not make claims on nor endorse L4S generally nor per any specific implementations. The site is provided on an as-is and as-available basis. You acknowledge that you bear all risks involved in using this website or the software and hardware mentioned on this site. The site maintainers make no claims, promises, or guarantees about the accuracy, completeness, or adequacy of the contents of this site, and expressly disclaims liability for errors and omissions in the contents of this site. No warranty of any kind, implied, expressed, or statutory, including but not limited to the warranties of non-infringement of third party rights, title, merchantability, fitness for a particular purpose or freedom from computer virus, is given with respect to the contents of this website or its links to other Internet resources.

Hardware requirements & suggestions

• Raspberry Pi 4B(s) – two or more, 8G RAM is suggested

• Multiport power supply, e.g. min 5V 3A or 15W per port for Rpi, 60W for laptop

• USB A to C cords for Rpi power, 1ft or so

• 64G Micro SD card(s)

• Micro HDMI cable(s)

• HDMI monitor

• USB keyboard and mouse

• USB adapter (or equivalent) for micro SD card

• Optional USB C cord with power switch

• Optional RPi 4B case

• Optional GPS hat for pulse per second (not documented here)

• Optional SD card holder/case

Software

• Debian system for RPI4

• Raspberry Pi imager

• or linux dd

• gzip & gunzip

Other

• bcm2711_defconfig

• Internet access to the Rpi4

• Ethernet switch

• L4S forwarding plane that does marking (not documented here)

Native build with apt (build from source notes here)

Create a full debian bullseye system with 64 bit support, using a 64G micro SD card

1. Download the bullseye image from here or from debian.

2. Use rasberry pi imager to write to a 64G card, it will destroy all existing data

3. Install the card into a raspberry pi 4 with wired ethernet attached

4. Boot card and follow initial setup including creating a user id, and system updates

5. Reboot, set root password with ‘sudo passwd’

6. optionally ‘sudo apt-get install firefox-esr’

7. optionally enable ssh via ‘systemctl start ssh” & ‘systemctl enable ssh’

8. Install build tools

• sudo apt install raspberrypi-kernel-headers build-essential bc git wget bison flex libssl-dev make libncurses-dev

• sudo apt install emacs automake

Build a L4S capable kernel (not optimized)

1. Git clone the kernel per Neal’s doc

• rjmcmahon@raspberrypi:~ $ cd /usr/local/src

• rjmcmahon@raspberrypi:/usr/local/src $ su

• git clone https://github.com/google/bbr.git

2. Checkout l4s branch

• cd bbr

• git checkout l4s-testing-2023-02-23-v3

3. Download bcm2711_defconfig from this site

• copy into .config

• copy into arch/arm64/configs/bcm2711_defconfig (optional)

4. make menuconfig

• check platform as Broadcom BCM2835 family

• Verify Networking support > Networking options > TCP: advanced congestion control

• make -j4 Image modules

• wait awhile (like at 87 minutes)

Install custom kernel

1. make modules_install

2. make install

3. emacs /boot/config.txt

4. kernel=vmlinuz-5.15.72-v8+

5. # uncomment if you get no picture on HDMI for a default "safe" mode

   #hdmi_safe=1

6. or sudo systemctl set-default multi-user.target

7. reboot

Verify TCP Prague and BBRv2

• Uname -r

rjmcmahon@raspberrypi:~ $ uname -r

5.15.72-v8+

• systctl -a | grep avail
  rjmcmahon@raspberrypi:~ $ sudo sysctl -a | grep avail

kernel.random.entropy_avail = 256

kernel.seccomp.actions_avail = kill_process kill_thread trap errno user_notif trace log allow

net.ipv4.tcp_available_congestion_control = reno bbr bbr2 cubic dctcp vegas prague

net.ipv4.tcp_available_ulp =

Configuration for Testing
You can configure the test machine with the following commands.

Enabling Accurate ECN

sysctl net.ipv4.tcp_ecn=3

Enabling BBRv2 CC w/ Accurate ECN

sysctl net.ipv4.tcp_ecn=3 net.ipv4.tcp_congestion_control=bbr2

Enabling TCP Prague CC w/ Accurate ECN

sysctl net.ipv4.tcp_ecn=3 net.ipv4.tcp_congestion_control=prague

Enabling CUBIC CC w/ Accurate ECN

sysctl net.ipv4.tcp_ecn=3 net.ipv4.tcp_congestion_control=cubic

Enabling CUBIC CC w/o Accurate ECN

sysctl net.ipv4.tcp_ecn= net.ipv4.tcp_congestion_control=cubic

Build the latest iperf 2

1. cd /usr/local/src

2. git clone https://git.code.sf.net/p/iperf2/code iperf2-code

3. cd iperf2-code

4. .configure

5. make -j 4

6. make install

Build goresponsiveness tool (git repo)

wget https://go.dev/dl/go1.20.6.linux-arm64.tar.gz

tar -C /usr/local/ -xvf go1.20.6.linux-arm64.tar.gz

tar -C /usr/local -xzf go1.20.6.linux-arm64.tar.gz

export PATH=$PATH:/usr/local/go/bit

type go

go version

cd goresponsiveness/

go mod download

go build networkQuality.go

TCP client stats

• Iperf 2 TCP CCA up stream

Adding a realtime clock

• Order a GPS hat

• Follow these instructions

• Microsecond accurate NTP with a Raspberry Pi and PPS GPS

Example clone with dd (from micro SD to file, use lsblk to find device)

[rjmcmahon@ryzen3950 L4S]$ sudo dd bs=4M if=/dev/sdc of=l4s_kernel conv=fsync

[sudo] password for rjmcmahon:

15226+1 records in

15226+1 records out

63864569856 bytes (64 GB, 59 GiB) copied, 692.6 s, 92.2 MB/s

Example clone with dd (from file to micro SD, use lsblk to find device)

Cross compile for a buildroot small embedded system (not done)

A picture