-
Notifications
You must be signed in to change notification settings - Fork 12
QEMU
This section shows how ath10k can be used with QEMU. Debugging with QEMU can be very useful if the driver is not fully stable since a kernel panic can potentially render the entire system useless. When using QEMU, a faulty system can easily be rebooted and a test can be repeated over and over without having to reboot the computer. Another nice feature with QEMU is that it offers an easy way to debug the kernel with gdb (since it has an integrated gdb server).
Buildroot has a qemu defconfig that can be used as a basis for a custom image. We just need to update it to use a custom kernel and defconfig.
It is recommended to use an as "small" kernel defconfig as possible with buildroot since there is no point in bloating the system with unnecessary stuff that we won't use. So, don't use your .config of your desktop. Instead update the qemu defconfig used by buildroot by default.
It is not recommended to run QEMU with root privileges, so it might be necessary to change the permissions of the USB devices that is going to be connected to the QEMU system.
The easiest way to change the permissions for a specific device is to add an udev rule.
In our case, we want the WSUB6100M to be accessible (read and write) by a non root user.
Insert the WUSB6100M into the host computer and check the dmesg log. Below is an example:
[ 5879.905517] usb 3-2: new high-speed USB device number 4 using xhci_hcd
The log message tells us that it is device number 4 on usb bus 3.
Create an udev rule for WUSB6100M in /etc/udev/rules.d.
In our case the rule will look like this:
/etc/udev/rules.d/51-wusb6100.rules
-----------------------------------
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="13b1", ATTRS{idProduct}=="0042", MODE="0666"
Once the rule is created it is time to test it. In order to test the udev rule, issue the below command:
udevadm test $(udevadm info -q path -n /dev/bus/usb/003/004) 2>&1
The path to the usb device (/dev/bus/usb/003/004 in the above example) can be derived from the dmesg output. 003 is the bus number and 004 is the device number.
Check the output and verify the rule is correct and that no other rule is overriding it. In our case, the rule number (51) is important since we want our rule file to override the default settings for device nodes (we need our custom rule to be executed after 50-udev-default) .
Once we are satisfied with the rules, they should be updated globally:
udevadm control --reload
A ready-to-use buildroot repo is available here:
https://github.com/erstrom/buildroot-ath10k-test.git
branch: ath10k-qemu
It contains all packages etc. needed to setup an ath10k test environment.
First, clone and build the custom buildroot QEMU system:
git clone https://github.com/erstrom/buildroot.git cd buildroot git checkout ath10k-qemu make qemu_ath10k_x86_64_defconfig make
If you want to modify any settings (like changing kernel version etc.),
just run a make menuconfig and change all default settings to suite
your needs.
If you want to use wpa_supplicant to connect to your AP, you would most likely want to create a custom wpa_supplicant.conf file on the target rootfs.
The easiest way to do this is to add the wpa_supplicant.conf file to the rootfs overlay directory:
wpa_passphrase <ssid> <password> > board/qemu/x86_64/rootfs-overlay-ath10k-test/etc/wpa_supplicant.conf
The next step is to run qemu.
Before launching QEMU, make sure the ath10k_usb module is not loaded on the host system. It is recommended to blacklist it if it is present on the host.
Launch QEMU with the below command:
qemu-system-x86_64 -kernel output/images/bzImage \
-initrd output/images/rootfs.cpio \
-usb \
-device usb-ehci,id=ehci \
-device usb-host,vendorid=0x13b1,productid=0x0042 \
-append "console=ttyS0" \
-nographic
vendorid=0x13b1,productid=0x0042 is the vendor and product id of the WUSB6100M.
-device usb-ehci,id=ehci is important in order to make sure QEMU creates a virtual EHCI USB bus (using the -usbdevice host:0x13b1:0x0042 option will result in QEMU creating a UHCI bus instead).
Make sure QEMU has write access to the usb device. See QEMU and USB for more details.
The above qemu call have been added in a wrapper script:
support/scripts/run-qemu-ath10k_usb.sh
Once the virtual system has booted, make sure the WUSB6100M is properly connected (use lsusb or similar check that the usb device is present).
If connected properly, load the kernel module:
modprobe ath10k_usb
Make sure the network device (wlan0 by default) has been created:
ifconfig -a
To connect to an AP:
wpa_supplicant -B -Dnl80211 -iwlan0 -c /etc/wpa_supplicant.conf
The above command assumes you have added your own /etc/wpa_supplicant.conf with the correct setup for your network.
To obtain an IP address with udhcpc (busybox dhcp client):
udhcpc -i wlan0
Do some testing...
This section describes how to run Linux in a QEMU environment with an initrd produced by the ARCH tool mkinitcpio.
mkinitcpio is a tool used to generate initrd cpio archives containing tools copied from the current system. This is a quite big difference when compared to buildroot where all programs in the initrd is built from source.
mkinitcpio is a part of the ARCH Linux distribution and is intended for creating initrd's for ARCH based systems. It is possible that the tool will work with other distributions as well, but this is nothing I have tried.
Use a standard kernel config with ath10k enabled.
It is recommended to install the kernel modules in a dedicated install directory:
cd /path/to/kernel_tree make && \ make modules_install INSTALL_MOD_PATH=/path/to/local/install/dir
The above command will install all modules into /path/to/local/install/dir/lib/modules/<kernel version>.
The modules in this directory will be used later on by mkinitcpio
This section shows how mkinitcpio can be used to generate a simple root file system for QEMU.
A more detailed description about mkinitcpio can be found in the ARCH Linux wiki:
https://wiki.archlinux.org/index.php/mkinitcpio
Generate an initrd image with mkinitcpio using the below invocation.
Make sure mkinitcpio is invoked with fakeroot! Otherwise the generated initrd will not have device nodes etc., setup properly.
fakeroot -- mkinitcpio \ -c /path/to/custom/mkinitcpio.conf \ -g /path/to/initramfs.img \ -k <kernel version> \ -r /path/to/local/install/dir
-g /path/to/initramfs.img is the path to the initrd that will be created.
-k <kernel version> and -r /path/to/local/install/dir will determine from where modules will be copied from. The -r argument must of course match the INSTALL_MOD_PATH variable used when installing the modules. The modules will be copied from /path/to/local/install/dir/lib/modules/<kernel version>
-c /path/to/custom/mkinitcpio.conf is the path to the custom mkinitcpio configuration file. It specifies which tools and modules should be included in the initrd. Below is an example of a configuration file:
mkinitcpio.conf --------------- MODULES="ath10k_usb" BINARIES="find iw wpa_supplicant wget dhcpcd lsusb" HOOKS="base udev autodetect modconf block filesystems keyboard fsck extra-rootfs-overlay"
MODULES lists all modules that will be copied into the initrd. All dependencies will be resolved by mkinitcpio, so there is no need to specify depending modules explicitly.
BINARIES lists all programs that should be copied into the initrd. The programs will be copied from the current running system. Just as for modules, mkinitcpio will resolve all dependencies, so depending libraries don't need to be specified explicitly.
HOOKS lists all mkinitcpio hooks that will be used when generating the initrd. A hook will add some dedicated functionality to the image. base contains basic functionality like init script etc., and is more or less mandatory. udev will add udev support and so on. HOOKS can either be "standard" hooks (those that come with mkinitcpio) or custom made. extra-rootfs-overlay in the above configuration is a custom hook that will populate the rootfs with some tools we need for testing ath10k (more on that later).
Each hook will be executed in the order in which they are added to the space separated HOOKS list. In the above example it means that extra-rootfs-overlay will be executed last. Hence, the files added by this hook can be used to override files added by previous hooks.
Important
The instructions in this section assumes that mkinitcpio has support for base path strip-off in the add_full_dir function. As of 2017-06-06, the mainline mkinitcpio repository (git://git.archlinux.org/mkinitcpio.git) lacks this support (patches have been submitted, but have not yet been integrated). For now on, use repo https://github.com/erstrom/mkinitcpio.git, branch: functions-add_full_dir-v2
Custom initcpio hooks can be added in a sub-directory within the current directory (the directory from where the script is executed) named install ($PWD/install) or in the mkinitcpio system wide directory (/usr/lib/initcpio/install).
All scripts in the install directory will be sourced by mkinitcpio during execution and must follow a standardized format (each script should have its own build function).
Below is the extra-rootfs-overlay hook that was introduced in the previous section:
extra-rootfs-overlay
--------------------
#!/bin/bash
_d_rootfs_overlay="/path/to/rootfs-overlay"
build() {
add_full_dir $_d_rootfs_overlay '*' $_d_rootfs_overlay
}
help() {
cat <<HELPEOF
This hook installs a rootfs overlay from $_d_rootfs_overlay.
HELPEOF
}
# vim: set ft=sh ts=4 sw=4 et:
The build function calls the add_full_dir function with three arguments. The first argument is the path to the rootfs-overlay directory containing the files that should be copied into the image (/path/to/rootfs-overlay)
The second argument is a filter glob (all files will be copied in our case).
The third argument is the strip-off path that will be stripped off from the rootfs overlay path before the files are added into the image. This argument will map /path/to/rootfs-overlay on the host to / in the initrd.
The rootfs-overlay directory should contain at least the following:
- A custom initramfs init script. This is a must since the default init script provided by the base hook will do a switch_root to the "real" rootfs. This is undesired in our case, since we don't have any rootfs to switch to (we will run all our tests directly from the initramfs).
- ath10k firmware.
It is also recommended to add a wpa_supplicant configuration (if wpa_supplicant is to be used in the testing).
Below is the directory layout of my own rootfs-overlay:
rootfs-overlay/
├── init
├── etc
│ ├── init.d
│ │ ├── rcS
│ │ └── S01logging
│ ├── inittab
│ ├── modprobe.d
│ │ └── blacklist.conf
│ └── wpa_supplicant.conf
└── lib
└── firmware
└── ath10k
├── LICENSE.qca_firmware
...
The blacklist.conf file is just an empty file used to override the file added by the modconf hook.
The init script is a custom version of the default init script installed by the base hook.
The most important part of the script is the removal of the switch_root:
original /init
--------------
exec env -i \
"TERM=$TERM" \
/usr/bin/switch_root /new_root $init "$@"
"custom" /init
--------------
exec /sbin/init $*
exec /sbin/init will start the busybox init daemon in the initrd. By default, it will read /etc/inittab and follow the instructions in that file. This file and the init scripts in /etc/init.d can be taken from buildroot.
QEMU can be launched in the same way as in the buildroot case.
Below is an invocation that uses --enable-kvm for faster execution (make sure virtualization instructions are enabled, usually in BIOS):
qemu-system-x86_64 --enable-kvm \
-kernel /path/to/kernel_tree/arch/x86/boot/bzImage \
-initrd /path/to/initramfs.img \
-usb \
-device usb-ehci,id=ehci \
-device usb-host,vendorid=0x13b1,productid=0x0042 \
-append "console=ttyS0" \
-nographic \
-m 1024
-append "console=ttyS0" will append console=ttyS0 to the kernel command line. When combined with -nographic, QEMU will write all output to the current console and no QEMU window will be created. The kernel command line is actually empty by default, so console=ttyS0 will be our only option. Note that we don't need any root= option since we will use the initrd as our only rootfs.
-m 1024 sets the virtual machine memory to 1 Gb (default is 128 Mb, which might be unsufficient)
QEMU has an integrated gdb server, so debugging the kernel with gdb is relatively easy.
Make sure the kernel (and modules) that is going to be debugged is built with CONFIG_DEBUG_INFO=y. CONFIG_FRAME_POINTER=y is also recommended.
Launch QEMU with the -s option in order to enable the gdb server.
Below shows an invocation with -s and -S. -S will halt the QEMU system and wait for the gdb client to start it. This is the recommended approach when debugging with QEMU.
qemu-system-x86_64 --enable-kvm \
-s -S \
-kernel /path/to/kernel_tree/arch/x86/boot/bzImage \
-initrd /path/to/initramfs.img \
-usb \
-device usb-ehci,id=ehci \
-device usb-host,vendorid=0x13b1,productid=0x0042 \
-append "console=ttyS0" \
-nographic \
-m 1024
In another terminal, launch gdb and connect to the QEMU gdb server. By default, QEMU will use TCP port 1234 for the gdb server.
Below is an invocation of gdb that launches a few commands at startup. First, the vmlinux elf file is loaded. Then, a connection to to the QEMU gdb server is established (localhost:1234). A breakpoint is added in start_kernel (the kernel entry point) and the continue command is issued. Since QEMU was launched with -S, it will start executing as soon as continue has been issued and stop in start_kernel.
The next steps are a little bit more confusing. We disconnect from the server just to reconnect again. This is done to circumvent an issue described in the following stack overflow post:
http://stackoverflow.com/questions/11408041/how-to-debug-the-linux-kernel-with-gdb-and-qemu
gdb \
-ex "add-auto-load-safe-path $(pwd)" \
-ex "file /path/to/kernel_tree/vmlinux" \
-ex 'set arch i386:x86-64:intel' \
-ex 'target remote localhost:1234' \
-ex 'break start_kernel' \
-ex 'continue' \
-ex 'disconnect' \
-ex 'set arch i386:x86-64' \
-ex 'target remote localhost:1234'
The stack overflow post is addressing a known (and appearently not fixed) issue with QEMU and gdb.
You can always try without the above (quite cumbersome) procedure, but watch out for the below error message:
Remote 'g' packet reply is too long:
If the above error message appears, please try again with the gdb invocation shown above.
If ath10k is built as modules (the usual case), the .ko file must be loaded into gdb as well.
This is done with the add-symbol-file command. The add-symbol-file command takes the file to load and the address of where the .text section is mapped as arguments.
The location in memory of .text for a kernel module can be obtained from the below sysfs path:
cat /sys/module/<module name>/sections/.text
It is likely that the module gets mapped to the same address every time QEMU is started, so the add-symbol-file may be added directly to the gdb command.
Below shows an invocation of gdb where the symbols from mac80211 are loaded into address 0xffffffffa0000000 and a breakpoint is set in ieee80211_rx_napi
gdb \
-ex "add-auto-load-safe-path $(pwd)" \
-ex "file /path/to/kernel_tree/vmlinux" \
-ex 'set arch i386:x86-64:intel' \
-ex 'target remote localhost:1234' \
-ex 'break start_kernel' \
-ex 'continue' \
-ex 'disconnect' \
-ex 'set arch i386:x86-64' \
-ex 'target remote localhost:1234' \
-ex "add-symbol-file $MAC80211 0xffffffffa0000000" \
-ex 'break ieee80211_rx_napi'
It is obviously very important that the module does in fact gets loaded into address 0xffffffffa0000000 in order for the above command to work, so make sure to read the address of .text before you start debugging.