Bootstrapping the STM32F3 Discovery Board with Nuttx RTOS
Tools of the embedded trade: Make, GCC, Git, and, of course,
XEmacs
Introduction
I recently needed to bootstrap a STM32 based development board, with the intention of taking what I developed on known-working hardware to a custom prototype board that had never run a single byte of code. While my initial test code did not require an RTOS, the production code eventually would so I figured I'd get a jump on the production design by selecting an RTOS up front and throw a shell of some kind on it so I could interact with some basic apps I'd use to debug the new hardware.
My review of the RTOS landscape revealed many offerings but several came with commercial licensing requirements, and the few that didn't were either experiments or effectively abandoned with little or no recent activity. Still others were nothing more than a scheduler and not a complete solution.
I then found Nuttx, which checked all the boxes and at first glance appeared to be well supported, with the maintainer making commits to its Bitbucket repository on an almost daily basis. This article aims to provide a high level overview of my experience getting Nuttx to run on the STM32 F3 Discovery board and is based on detailed notes I took during the process.
Initial Checkout
Even this simple operation proved a little tricky, as the Nuttx project repository uses submodules, i.e. completely independent repositories that are installed as subdirectories in the main Nuttx repository. And while I've been using Git for years, up to this point I had never worked on a project that used submodules.
While learning how they worked I came across a lot of negative comments; so many in fact, that I decided to convert the submodules to subtrees and thus create a single repository, much as I have for every other project I've managed with git. Not long after this, however, I realized that this would make contributing to the project more difficult so I went back to submodules.
I cloned Nuttx with the following command:
$ git clone --recursive https://bitbucket.org/patacongo/nuttx.git nuttx
The --recursive flag tells git to pull not only the top level Nuttx (Core OS) repository but also the board configuration submodule ("config") and architecture specific submodule ("arch"). Then, as I wanted to incorporate some of the applications provided in the application repository "app" I pulled that too via:
$ git clone https://bitbucket.org/patacongo/apps.git apps
Note that the apps directory must be at the same level in the directory hierarchy as the "nuttx" directory. There may be a way to have the apps repository directory live within the nuttx directory, much as the other repositories do, but I took the path of least resistance and used the configuration recommended by the maintainer.
Configuring Nuttx for STM32 F3 Discovery Board
The first thing to do is configure Nuttx. This must occur before the Nuttx specific gnu toolchain called "buildroot" is built. The "config/stm32f3discovery" configuration directory contains two subdirectories (options), nsh and usbnsh.
The usbnsh version is designed to put the console over the USB port labeled on the board as "User" but notes in the configuration say this is not working yet. So nsh was the only option. This puts the console on USART2, which I then connected to a UART->RS232 transceiver module I found on Digikey, and then to my computer via an FTDI based USB->RS232 converter cable that is by default recognized on Linux as a serial device.
To start the configuration process I did the following:
$ cd nuttx/tools $ ./configure.sh stm32f3discovery/nsh
This command does not produce any output unless the board or application name is wrong. The configure script does little more than automate copying of three files to the root of the nuttx directory tree:
- .../stm32f3discovery/Make.def to nuttx/Make.defs
- .../stm32f3discovery/setenv.sh to nuttx/setenv.sh
- .../stm32f3discovery/defconfig to nuttx/.config
Building the Nuttx Toolchain - Buildroot
Nuttx can't use the popular gnu arm embedded toolchain for reasons described in the top level nuttx/README.txt file, the most signficant of which is that the toolchain incorporates C library components and Nuttx provides its own C library so the two would conflict if mated.
Although I was disappointed that the Nuttx project went down this path because I prefer when projects reuse existing tools (imagine if Linus maintained his own version of GCC and glibc to build Linux!) but I can see how this could result in better, more consistent and trouble-free builds over the long haul because everyone would be using the same toolchain.
To start building the toolchain I did:
$ cd buildroot $ cp configs/cortexm4f-eabi-defconfig-4.7.3 .config
and reviewed the output to ensure the configuration selections matched what I saw in the configuration file. The compilation spewed a bunch of warnings like:
conf.c: In function "strip" conf.c:46:2: warning: pointer targets in passing argument 1 of 'strlen' differ in signedness [-Wpointer-sign] l = strlen(p); ^
but it otherwise seemed to complete without errors so I tried to start a build by calling "make" at the top of the source tree. Unfortunately that resulted in the error:
NuttX directory .//../../nuttx does not exist
So I tweaked the .config file and set:
BR2_NUTTX_DIR="$(TOPDIR)/../nuttx"
I restarted the build and it continued this time to download a bunch of stuff off the net, but it once again failed with the error:
configure: error: Building GCC requires GMP 4.2+, MPFR 2.3.1+ and MPC 0.8.0+. Try the --with-gmp, --with-mpfr and/or --with-mpc options to specify their locations. Source code for these libraries can be found at their respective hosting sites as well as at ftp://gcc.gnu.org/pub/gcc/infrastructure/. See also http://gcc.gnu.org/install/prerequisites.html for additional info. If you obtained GMP, MPFR and/or MPC from a vendor distribution package, make sure that you have installed both the libraries and the header files. They may be located in separate packages.
At the time my host was running Ubuntu 14.04, which provides:
- libgmp-dev 5.1.3
- libmpfr-dev 3.1.2
- libmpc-dev 1.0.1
All versions appeared to qualify so I installed them:
# apt-get install libgmp-dev # apt-get install libmpfr-dev # apt-get install libmpc-dev
and then restarted the build. That ran for a while but then I got the error:
configure: error: can not find gperf
I quickly installed that with:
# apt-get install gperf
and restarted make, which finished shortly thereafter. I then cleaned and rebuilt the toolchain to guarantee that I had a reproducible build without errors and to benchmark the compilation time:
$ make clean $ time make real 9m3.362s user 7m33.889s sys 1m17.287s
According to buildroot/README the binaries are placed in buildroot/build_arm_hf but I found them in buildroot/build_arm_hf/staging_dir/bin. I also realized I could delete the build directory buildroot/toolchain_build_arm_hf to save space.
I then changed my environment to find this toolchain. Specifically, I changed the TOOLCHAIN_BIN variable as follows:
$ export TOOLCHAIN_BIN=$HOME/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/bin
I then sourced the setenv.sh file:
$ source setenv.sh
and eventually added this to my .profile since I didn't want to have to call that prior to every build in every terminal.
Note that the Nuttx toolchain appears to be built by default with the hard float ABI enabled. That's a good thing because the F3 is a Cortex M4 core which is equipped with a FPU, but if Nuttx is provisioned for soft float (CONFIG_ARCH_FPU=n) the Nuttx build will fail with errors similar to the following. If you look closely, the error messages point out the incongruity.
AR: stm32_boot.o stm32_spi.o stm32_cxxinitialize.o stm32_autoleds.o stm32_buttons.o stm32_usb.o make[2]: Leaving directory `/home/doug/dev/stm32/rtos/nuttx/configs/stm32f3discovery/src' LD: nuttx arm-nuttx-eabi-ld: error: /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(bpabi.o) uses VFP register arguments, /home/doug/dev/stm32/rtos/nuttx/nuttx does not arm-nuttx-eabi-ld: failed to merge target specific data of file /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(bpabi.o) arm-nuttx-eabi-ld: error: /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(_divdi3.o) uses VFP register arguments, /home/doug/dev/stm32/rtos/nuttx/nuttx does not arm-nuttx-eabi-ld: failed to merge target specific data of file /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(_divdi3.o) arm-nuttx-eabi-ld: error: /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(_udivdi3.o) uses VFP register arguments, /home/doug/dev/stm32/rtos/nuttx/nuttx does not arm-nuttx-eabi-ld: failed to merge target specific data of file /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(_udivdi3.o) arm-nuttx-eabi-ld: error: /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(unwind-arm.o) uses VFP register arguments, /home/doug/dev/stm32/rtos/nuttx/nuttx does not arm-nuttx-eabi-ld: failed to merge target specific data of file /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(unwind-arm.o) arm-nuttx-eabi-ld: error: /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(pr-support.o) uses VFP register arguments, /home/doug/dev/stm32/rtos/nuttx/nuttx does not arm-nuttx-eabi-ld: failed to merge target specific data of file /home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/lib/gcc/arm-nuttx-eabi/4.7.3/libgcc.a(pr-support.o) make[1]: *** [nuttx] Error 1 make[1]: Leaving directory `/home/doug/dev/stm32/rtos/nuttx/arch/arm/src'
I solved this simply by making sure both the toolchain and Nuttx itself were configured for hard float. For what it's worth, this was the output of my toolchain version command when configured for the STM32 F3:
$ arm-nuttx-eabi-gcc -v Using built-in specs. COLLECT_GCC=arm-nuttx-eabi-gcc COLLECT_LTO_WRAPPER=/home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir/libexec/gcc/arm-nuttx-eabi/4.7.3/lto-wrapper Target: arm-nuttx-eabi Configured with: /home/doug/dev/stm32/rtos/buildroot/toolchain_build_arm_hf/gcc-4.7.3/configure --prefix=/home/doug/dev/stm32/rtos/buildroot/build_arm_hf/staging_dir --build=x86_64-pc-elf --host=x86_64-pc-elf --target=arm-nuttx-eabi --enable-languages=c,c++ --disable-__cxa_atexit --disable-libssp --enable-target-optspace --with-gnu-ld --disable-shared --disable-nls --enable-threads --disable-multilib --with-float=hard --with-abi=aapcs-linux --with-arch=armv7-m --with-tune=cortex-m4 --with-mode=thumb Thread model: single gcc version 4.7.3 (GCC)
Building Nuttx
After building the toolchain and altering the PATH to pick them up I had to tweak the Nuttx build configuration. This process uses kconfig and anyone familiar with the Linux kernel configuration process will feel right at home here.
$ cd nuttx $ make menuconfig $ make
With the board configured above using ./configure.sh most of the important stuff including ARM and STM32 chip selection parameters are configured automatically. However, I did need to change the following to get to this point:
- Build Setup → Build Host Platform = Linux
- System Type → Toolchain Selection (Buildroot (Cygwin or Linux))
- Build-Setup → Binary Output Formats → Raw Binary & Intel Hex
- System Type → ARM chip selection (STMicro STM32 F1/F2/F3/F4)
- System Type → STM32 Chip Selection (STM32F303VC)
- System Type → FPU Support (Enabled)
Then I attempted a build (again, from the top level of the nuttx source tree). This produced the elf format (nuttx) binary as well as the Raw (nuttx.bin) and Intel Hex (nuttx.hex) versions. Shortly after I confirmed I could work directly with the elf version I disabled builds of the Hex and Raw formats.
I then called size to gauge the stats of the resulting binary (in this case, the elf format version):
$ arm-nuttx-eabi-size nuttx text data bss dec hex filename 56333 96 2116 58545 e4b1 nuttx
I eventually edited the Makefile to call 'size' on every build because I consider this extremely valuable information. I have always used this in other environments as a last minute sanity check -- once familiar with a given design it's often obvious from these numbers if the build configuration has changed substantially or whether debug is enabled (intentionally or unintentionally).
Incidentally, I considered generating a pull request for this modification but the maintainer said he would not accept it unless I could engineer the modification to avoid the architectures on which the command is not provided or supported by the toolchain. Since I had absolutely no intention of doing that I ultimately dropped the issue. Sadly, this would be a sign of things to come, but I'll address that later.
As you can see, this minimally configured Nuttx binary requires roughly 58K of flash, though my later builds with debug symbols and some extra apps hovered around 85-90K. As the F3 Discovery board comes assembled with the"VC" part which embeds 256K of flash, this leaves roughly 165K for application development. And because the toolchain is not intentionally crippled for licensing purposes like so many commercial toolchains, every bit of that 256K of flash is available.
Note, by the way, that the VC part comes with a mere 40K of SRAM, so while there is a configuration option called:
CONFIG_BOOT_RUNFROMISRAM
it's not usable on this part. For all practical purposes Nuttx must be configured to run from flash via:
CONFIG_BOOT_RUNFROMFLASH=y
This is the default setting. Now you know why.
Accessing the Serial Console over USART2
One of the most critical milestones in bringing up any hardware is communicating with the device over its console. Nuttx uses a bunch of constants to abstract the physical pin assignments used for the console device. The architecture-specific file:
nuttx/arch/arm/src/stm32/chip/stm32f30xxx_pinmap.h
contains the low level pin mapping definitions while the board-specific file:
nuttx/configs/stm32f3discovery/include/board.h
does the high level assignment as shown:
#define GPIO_USART2_RX GPIO_USART2_RX_2 #define GPIO_USART2_TX GPIO_USART2_TX_2
If you look in the pinmap file you will note that there are four different sets of TX and RX pins defined for USART2. If you're not familiar with STM32 or just new to the hardware game entirely you may not realize why this occurs, but it's due to pin multiplexing. Each physical pin can be assigned to multiple functions. In this case Nuttx is configured to use the second function of those pins.
So when using Nuttx on the STM32 F3 the console is set to USART2 and the TX is pin PA2 while RX is pin PA3. I simply built a cable that connected PA2 to the UART → RS232 converter RX pin and PA3 to the converter's TX pin. Then I connected the converter to my Linux host with a USB → RS232 (DB-9) cable and then opened a session to the USB device via screen:
$ screen /dev/ttyUSB0 115200
Incidentally, if you've never used screen before you're really missing out. You'll need it in this application because as I discovered cu or other serial interface utilities would disconnect from the console every time I reset the board, and that's a problem if you're trying to monitor the boot process, which takes a fraction of a second. By the time you try to reconnect another tool the boot is long over. Screen does not fall into this trap, which is why screen is awesome.
Once I got everything hooked up correctly and booted the board with the console connected I received the NuttShell prompt (nsh>). I then entered a question mark (?) followed by return to display the list of available apps. Note that most of these are not applicable to the F3 on the Discovery board so I wound up disabling most of them and saving a few K in the finished binary size.
NuttShell (NSH) nsh> ? help usage: help [-v] [<cmd>] cmp free mkfifo set usleep ? dirname help mh sh xd basename dd hexdump mv sleep break echo kill mw test cat exec ls pwd true cd exit mb rm uname cp false mkdir rmdir unset Builtin Apps: sercon serdis nsh>
Conclusion
All told it took me about three long days to sort out my debug probe and related software, fabricate custom cables, select a convenient source of power (a USB 5V→3.3V converter), select and build the toolchain, configure and build Nuttx and, finally, burn it to the STM32 F3's integrated flash. Unfortunately that was just the beginning of my development efforts.
I found the F3-specific I2C driver so utterly broken I had to rewrite the driver interrupt state machine from scratch. I also was forced to hack up the SPI driver and related (architecture independent) OS interface because the maintainer completely ignored the very possibility of hardware-specific features like CRC generation.
While I was ultimately successful in getting Nuttx to run on my board and could probably use it in production I am contemplating other solutions for several reasons:
Lack of support infrastructure.
The maintainer insists on discussing all things Nuttx in a single group hosted on Yahoo and this is a problem because of the shitty Yahoo UX and the inability to search / filter / subscribe to the group in any meaningful way. In my opinion Nuttx needs an independently managed forums platform and groups created for each architecture or subsystem.
When I suggested as much on the forum I expected the maintainer to naturally endorse this improvement to the project but instead my comments were quickly rebuffed, despite his own admission that the topic is brought up frequently. Coincidence or not, shortly after I concluded my input in that thread I was unsubscribed from the group without notice. Thinking it was yet another Yahoo software glitch I resubscribed and then found I no longer had permissions to post. At that point I took the hint and have not not been back since.
Interestingly, I had several people email me privately afterward asking about some STM32 issues I brought up in the forum. When I told them I was no longer allowed to contribute to the discussion they confirmed the maintainer's attitude problem is well known and he was unlikely to change.
-
The project is written in C but begs for C++.
After working on the codebase I can say the size and nature of the project begs for the use of C++. Not exceptions, templates, and all that esoteric bullshit that cause unacceptable bloat on embedded systems and expose hair-pulling compiler bugs and portability issues, but the simple use of classes and interfaces, which would make it a LOT easier to extend and adapt the codebase, as well as effectively document the inheritance relationships.
I found a lot of duplicated code for the STM32 that is clearly the result of copying entire files to change a few lines of code that differs between processor families, which I believe is the consequence of using the wrong language for the job. I like the simplicity of C on very small projects (and embedded is the only place I still use it), but it doesn't work here.
I also question some of the organizational structure and don't particularly care for the build system which forces all builds to start from the top of the source tree. Every build system I've ever constructed has allowed compilation in any directory containing source files. It sounds like such a minor thing until you experience that convenience.
-
Incomplete support for STM32 and the lack of an STM32 maintainer.
The Nuttx documentation clearly misrepresents the state of support for STM32. Drivers currently claiming to be F3 specific are not in any way complete or fully exploitive of the hardware features.
A review of the F1, F2, F3 and F4 user manuals revealed the register maps for these devices are all different. The differences may be subtle in places where ST shared peripherals but I saw several examples of comments that were completely wrong for the F3, as well as logic illegally writing to bits that are reserved on that part. Not surprisingly those same bits were writeable on other families like the F1/F2. What can I say but the copy/paste problem in the STM32 codebase is real.
The project maintainer stated on the forum that the STM32 architecture has no official maintainer (well, as of early 2016 anyway) and he just plugs in code provided by STM32 developers after a sanity check and coding standards evaluation. Based on that admission I am not surprised at the state of the codebase.
Maintaining a project of this complexity is not practical for one person, or even a few people, which explains why I think it should be forked into architecture-specific sub projects that can be allowed to develop a life of their own. At least then developers wouldn't have to waste time thinking about how design changes might affect ten other architectures, of which they have no knowledge or intention of ever using. Some projects grow too big for their own good, and Nuttx is a one of them.
My advice for anyone attempting to use Nuttx for STM32 is to go into it with eyes open and realize peripheral support is incomplete or at least extremely buggy. It's fine for quickly bootstrapping new hardware as it reduces the number of variables in that process, but for long term support of production systems I'd consider other solutions or expect to spend a lot of time fixing what's broken.