8. Build Process¶
foxBMS uses waf The meta build system
for building binaries and the
documentation.
For detailed information on waf
see waf.io. A short
introcution to waf
is given at
waf.io/apidocs/tutorial.
The more detailed version of how to use waf
is found at
waf.io/book.
8.1. General¶
8.1.1. Where to find the toolchain?¶
The waf
toolchain is located in the directory foxbms\tools
, in the binary
waf
. This archive is automatically unpacked in a directory named
something like waf-{{X}-{{some-hash-value}}
containing the waf
library, where {{X}}
is the dash-separated version number of waf
. It is
unpacked into foxbms\tools
. It is generally assumed that all commands are
run from directory foxbms
. Therefore waf
has to be always called by
python tools\waf some-command
where some-command
is an argument defined
in the wscript
.
Additional build tools are located in foxbms\tools\waftools
. These are
the tooles needed for building the documentation, i.e., doxygen
and
sphinx
.
8.1.2. Where are the build steps described?¶
The build process is described in files named wscript
, that can be found
nearly everywhere inside the different directories inside the foxBMS project.
Later in this documentation this is explained in detail.
8.1.3. General¶
waf
needs always to be run form the top level of the repository and the path
to waf must be given relative to that directory. This path is tools\waf
.
cd path\to\foxbms
python tools\waf {{some-command}}
8.1.4. What commands can be used?¶
To get an overview of support commands run --help
or -h
on the waf
binary:
python tools\waf --help
This is the output in foxBMS version 1.6.8
waf [commands] [options]
Main commands (example: ./waf build -j4)
build : executes the build
build_all : builds all parts of the project (binaries and documentation)
build_libs : executes the build of libs
build_primary : executes the build of primary
build_primary_bare : executes the build of primary_bare
build_secondary : executes the build of secondary
build_secondary_bare : executes the build of secondary_bare
clean : cleans the project
clean_all : cleans all parts of the project
clean_libs : cleans the project libs
clean_primary : cleans the project primary
clean_primary_bare : cleans the project primary_bare
clean_secondary : cleans the project secondary
clean_secondary_bare : cleans the project secondary_bare
configure : configures the project
cpplint : configures cpplint
dist : creates an archive containing the project source code
distcheck : creates tar.bz form the source directory and tries to run a build
distclean : removes build folders and data
doxygen : creates doxygen documentation
doxygen_libs : creates the doxygen documentation of libs
doxygen_primary : creates the doxygen documentation of primary
doxygen_primary_bare : creates the doxygen documentation of primary_bare
doxygen_secondary : creates the doxygen documentation of secondary
doxygen_secondary_bare: creates the doxygen documentation of secondary_bare
flake8 : runs flake8 on the foxBMS repository
list : lists the targets to execute
list_libs : lists the targets to execute for libs
list_primary : lists the targets to execute for primary
list_primary_bare : lists the targets to execute for primary_bare
list_secondary : lists the targets to execute for secondary
list_secondary_bare : lists the targets to execute for secondary_bare
sphinx : creates the sphinx documentation of the project
step : executes tasks in a step-by-step fashion, for debugging
step_libs : executes tasks in a step-by-step fashion, for debugging of libs
step_primary : executes tasks in a step-by-step fashion, for debugging of primary
step_primary_bare : executes tasks in a step-by-step fashion, for debugging of primary_bare
step_secondary : executes tasks in a step-by-step fashion, for debugging of secondary
step_secondary_bare : executes tasks in a step-by-step fashion, for debugging of secondary_bare
Options:
--version show program's version number and exit
-c COLORS, --color=COLORS
whether to use colors (yes/no/auto) [default: auto]
-j JOBS, --jobs=JOBS amount of parallel jobs (8)
-k, --keep continue despite errors (-kk to try harder)
-v, --verbose verbosity level -v -vv or -vvv [default: 0]
--zones=ZONES debugging zones (task_gen, deps, tasks, etc)
-h, --help show this help message and exit
--cpplint-conf-file=CPPLINT_CONF
cpplint configuration file (default: cpplint.yml)
-t TARGET, --target=TARGET
build target: debug (default)/release
-l LIBS, --libs=LIBS name of the library to be used
Configuration options:
Build and installation options:
-p, --progress -p: progress bar; -pp: ide output
Step options:
--files=FILES files to process, by regexp, e.g. "*/main.c,*/test/main.o"
Installation and uninstallation options:
--distcheck-args=ARGS
arguments to pass to distcheck
8.1.5. The configure
command¶
Before building any binaries or documentation is possible, the project needs to
be configured. A successfull configure
command and its ouput is shown below:
python tools\waf configure
(...)
'configure' finished successfully (0.340s)
8.1.6. The build
commands¶
After the project has been configured, a build can be triggered and it is
generally exectued by the build
commands. As foxBMS requries building
variants, one has to use e.g., build_primary
in order to build binaries
for the primary MCU.
python tools\waf build
Waf: Entering directory `.\foxbms\build'
A build variant must be specified, run 'python tools\waf --help'
python tools\waf build_primary
(...)
'build_primary' finished successfully (8.800s)
The possible build
commands, the definition of the and corresponding targets
and the targets itself is listet below:
- Primary MCU
python tools\waf build_primary
The targets are defined at:
foxbms\embedded-software\mcu-primary\src\application\wscript
foxbms\embedded-software\mcu-common\src\engine\wscript
foxbms\embedded-software\mcu-common\src\module\wscript
foxbms\embedded-software\mcu-primary\src\engine\wscript
foxbms\embedded-software\mcu-primary\src\module\wscript
foxbms\embedded-software\mcu-freertos\wscript
foxbms\embedded-software\mcu-hal\STM32F4xx_HAL_Driver\wscript
foxbms\embedded-software\mcu-primary\src\general\wscript
foxbms-application foxbms-common-driver foxbms-common-engine foxbms-common-module foxbms-common-util foxbms-driver foxbms-engine foxbms-module foxbms-os foxbms-stmhal foxbms_primary.elf 'list_primary' finished successfully (0.052s)
- Secondary MCU
python tools\waf build_secondary
The targets are defined at:
foxbms\embedded-software\mcu-secondary\src\application\wscript
foxbms\embedded-software\mcu-common\src\engine\wscript
foxbms\embedded-software\mcu-common\src\module\wscript
foxbms\embedded-software\mcu-secondary\src\engine\wscript
foxbms\embedded-software\mcu-secondary\src\module\wscript
foxbms\embedded-software\mcu-freertos\wscript
foxbms\embedded-software\mcu-hal\STM32F4xx_HAL_Driver\wscript
foxbms\embedded-software\mcu-secondary\src\general\wscript
foxbms-application foxbms-common-driver foxbms-common-engine foxbms-common-module foxbms-common-util foxbms-engine foxbms-os foxbms-stmhal foxbms_secondary.elf 'list_secondary' finished successfully (0.100s)
- General documentation
The general documenation is build by
python tools\waf sphinx
- API documentation
The API documentation is build using the
doxygen_{{variant}}
, thereforepython tools\waf doxygen_primary python tools\waf doxygen_secondary
- Cleaning
It is also possible to clean the binaries and Doxygen documentation. This step is performed by the
clean
command.As seen from
--help
the possibleclean
commands areclean_all
clean_libs
clean_primary
clean_primary_bare
clean_secondary
clean_secondary_bare
distclean
Each command cleans the specified option, except for
distclean
. However it is possible to make a complete clean bydistclean
. Afterdistclean
the entire build directory and all lock files etc. are deleted and the project needs to be configured again. Cleaning the general sphinx documentation alone is currently not supported, but it can be achivied by runningdistclean
.
8.2. Targets¶
As stated above the targets and sub targets of the build process are shown by
list_x
where x
is the specified target. The main target is the
*.elf.unpatched
file. The final targets are build afterwards. After
successfully linking the map file is generated.
These logging files are found in build
and build\{{target}}
.
Additional to the *.elf.unpatched
and *.elf
files a *.hex
and two
*.bin
files of the binary are generated. The *.bin
files are separated
into the flash and the flashheader. The size of each object/binary is written
to a log file.
The targets are build as follows (final targets are filled gray):
![digraph {
rankdir=TB;
graph [fontname = "monospace"];
node [fontname = "monospace"];
edge [fontname = "monospace"];
"foxbms_flash.bin" [style=filled];
"foxbms_flashheader.bin" [style=filled];
"foxbms.hex" [style=filled];
"foxbms.elf" [style=filled];
"foxbms.elf.unpatched" -> "foxbms_flash.bin" [label="objcopy"];
"foxbms.elf.unpatched" -> "foxbms_flashheader.bin.unpatchted" [label="objcopy"];
"foxbms_flash.bin" -> "checksum.yml" [label="tsk_cal_chksum"];
"foxbms_flashheader.bin.unpatchted" -> "checksum.yml" [label="tsk_cal_chksum"];
"checksum.yml" -> "foxbms.elf" [label="objdump"];
"foxbms.elf" -> "foxbms.hex" [label="objcopy"];
"foxbms.elf" -> "foxbms_flashheader.bin" [label="objcopy"];
}](../../_images/graphviz-5a194e28587ade2622df09c3ee5d1544cb55c78a.png)
Fig. 8.1 foxBMS build process targets¶
8.3. Build Process¶
Note
For testing the following explanations it is assumed that
python tools\waf configure
has been run.
This sections gives an overview how the build process is defined. All features
are generally defined in the top wscript
located at
foxbms\wscript
.
The minimum functions that are needed to be defined a build in waf
are:
configure
andbuild
.
As the toolchain needs more targets the following functions need to be
implemented: doxygen
and sphinx
.
Furthermore the following features are needed:
- for calculating the checksum based on the
*.elf.unpatched
file the classtsk_cal_chksum
and for creating the*.elf
file thetsk_wrt_chksum
class and for adding these features the functionadd_chksum_task
, - for stripping the debug symbols in release mode the class
strip
and the functionadd_strip_task
, - for creating a
hex
file from theelf
file the classhexgen
and the functionadd_hexgen_task
, - for generating
bin
files fromelf
files the classestsk_binflashheadergen
,tsk_binflashgen
and the functionadd_bingen_task
and the classtsk_binflashheaderpatch
and the functionadd_patch_bin_task
, - for generating size information of the objects and binaries the class
size
and the functionprocess_sizes
, - for copying the libraries build by
build_libs
into the correct directories the classcopy_libs
and the functionadd_copy_libs
, - for compiling assembler files
*.s
the classSasm
and the functionasm_hook
.
For implementation details see the wscript
itself.
Some of these functionalities require scripts from foxbms\tools
.

8.3.1. General Documentation¶
This build target uses the function def sphinx(bld)
. Since this
definition of a function called sphinx
, it is accepted as command to waf.
This general documentation is generated by running
python tools\waf sphinx
The implementation details of the sphinx
command can be found in
foxbms\tools\waftools\sphinx_build.py
.
8.3.2. Primary and Secondary Binaries and Doxygen Documentation¶
In order to have different build variants, these variants have to be defined.
This is done at the top of the main wscript
at foxbms\wscript
. The
variants have to be defined for the binary build and Doxygen documentation.
from waflib.Build import BuildContext, CleanContext, InstallContext, \
UninstallContext, ListContext, StepContext
for x in variants:
for y in (
BuildContext,
CleanContext,
InstallContext,
UninstallContext,
ListContext,
StepContext
):
name = y.__name__.replace('Context','').lower()
class tmp(y):
if name == 'build':
__doc__ = '''executes the {} of {}'''.format(name, x)
elif name == 'clean':
__doc__ = '''cleans the project {}'''.format(x)
elif name == 'install' or name == 'uninstall':
__doc__ = '''CURRENTLY NOT SUPPORTED:{}s the project {}'''.format(name, x)
elif name == 'list':
__doc__ = '''lists the targets to execute for {}'''.format(x)
elif name == 'step':
__doc__ = '''executes tasks in a step-by-step fashion, for \
debugging of {}'''.format(x)
cmd = name + '_' + x
variant = x
dox = 'doxygen'
class tmp(BuildContext):
__doc__ = '''creates the {} documentation of {}'''.format(dox, x)
cmd = dox + '_' + x
fun = dox
variant = x
In the function build and doxygen the build variant is checked, the the correct sources are selected. If no build variant is specified, an error message is displayed, telling to specify a variant. This is generally implemented something like this:
def build(bld):
import sys
import logging
from waflib import Logs
if not bld.variant:
bld.fatal('A {} variant must be specified, run \'{} {} --help\'\
'.format(bld.cmd, sys.executable, sys.argv[0]))
bld.env.__sw_dir = os.path.normpath('embedded-software')
src_dir = os.path.normpath('mcu-{}'.format(bld.variant))
ldscript = os.path.join(bld.env.__sw_dir, src_dir, 'src', bld.env.ldscript_filename)
For doxygen it is implemented very similar:
def doxygen(bld):
import sys
import logging
from waflib import Logs
if not bld.variant:
bld.fatal('A build variant must be specified, run \'{} {} --help\'\
'.format(sys.executable, sys.argv[0]))
if not bld.env.DOXYGEN:
bld.fatal('Doxygen was not configured. Run \'{} {} --help\'\
'.format(sys.executable, sys.argv[0]))
_docbuilddir = os.path.normpath(bld.bldnode.abspath())
doxygen_conf_dir = os.path.join('documentation', 'doxygen')
os.makedirs(_docbuilddir, exist_ok=True)
conf_file = 'doxygen-{}.conf'.format(bld.variant)
doxygenconf = os.path.join(doxygen_conf_dir, conf_file)
8.4. wscripts¶
As mentioned above, the build process is described in wscript
s, which are
itself valid python scripts.
The top is foxbms\wscript
which defines the functions needed for the
build, e.g., configure
, build
etc.
From the top wscript
the other wscript
s are called recursive by
bld.recurse(..)
.
To get a detailed view on the single build steps, see these files.
8.5. Building and Linking with a Library¶
The toolchain enables to build a library and then links against the library. It is possible to build and link multiple libraries into the binaries.
The wscript
in embedded-software\libs
lists the libraries to be build.
Libraries that should be build have to be listed here. Based on the example
library testlib
it is shown how to include a library in foxBMS.
Note
In fact the libs
directory contains two test libraries
(foxbms-user-lib
and my-foxbms-library
) in order to show how
multiple libraries can be used. The first example shows how to build one
single library and in the second example it is shown, how to build and use
more than one library.
General Setup
The wscript
in embedded-software\libs
lists in the function
bld.recurse(...)
the directories containing the sources for the to be build
library (see line 11 in Listing 8.16).
1 2 3 4 5 6 7 8 9 10 11 | def build(bld):
header_files_src = bld.path.ant_glob('**/*.h')
header_filenames = [n.name for n in header_files_src]
if not len(header_filenames) == len(set(header_filenames)):
duplicates = list(set([x for x in header_filenames
if header_filenames.count(x) > 1]))
err_msg = 'There are headers with the same file name recursively ' \
' inside directory \'{}\':\n' \
'{}'.format(bld.path.abspath(), '\n'.join(duplicates))
bld.fatal(err_msg)
bld.recurse('testlib myfoxbmslibrary')
|
Note
For every additional library that should be build, the directory
containing the library must be added to this line, e.g., if the library
sources are in a directory called advancedalgorithms
this lines needs to look like this:
1 2 def build(bld): bld.recurse('testlib myfoxbmslibrary advancedalgorithms')
The actual build of the library is defined in the wscript
in
embedded-software\libs\testlib
. All source and header files have to be in
the library directory, for this example these are are testlib.c
and
testlib.h
. The library is then build by the wscript
in
embedded-software\libs\testlib
.
Listing 8.18 explained in detail:
All source files that should be build have to be listed in the
srcs
list (see line 5-6.).The name of the library is set to
foxbms-user-lib
(see line 11).Note
For later on further expanding the
advanced-algorithms
example, the library namemy-advanced-algorithm
is assumed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | def build(bld):
header_files_src = bld.path.ant_glob('*.h')
header_files_tar = [os.path.join(bld.env.INCLUDE_DIR_LIBS,
x.path_from(bld.path)) for x in header_files_src]
srcs = ' '.join([
os.path.join('testlib.c')])
includes = os.path.join(bld.bldnode.abspath()) + ' '
includes += ' '.join([bld.path.get_src().abspath()])
bld.stlib(target='foxbms-user-lib',
source=srcs,
includes=includes,
cflags=bld.env.CFLAGS_foxbms,
features=['size', 'copy_libs', 'check_includes'])
bld(features='subst',
source=header_files_src,
target=header_files_tar,
is_copy=True)
|
- The object files (
*.o
) and the library (*.a
) are found inbuild\libs\embedded-software\libs\testlib\
. - The libraries (the
*.a
-files) are copied inbuild\lib\*.a
. When building the default dummy libraries these arebuild\lib\libfoxbms-user-lib.a
andbuild\lib\libmy-foxbms-library.a
. The lib-prefix is generated automatically. This task is generated by thecopy_lib
feature (see line 14). - The headers are copied to
build\include
(see line 15-17).
Warning
The header names for all library headers are checked for uniqueness.
Header files with the same name recursively inside the libs
directory
will lead to a build error. This check needs to be performed, as all
headers get copied to include directory at build\include
.
The Library
The library declaration of super_function(uint8_t a, uint8_t b)
is in testlib.h
:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | #ifndef TESTLIB_H_
#define TESTLIB_H_
/*================== Includes =============================================*/
#include <stdint.h>
/*================== Macros and Definitions ===============================*/
/*================== Constant and Variable Definitions ====================*/
extern uint8_t super_variable;
/*================== Function Prototypes ==================================*/
extern uint16_t super_function(uint8_t a, uint8_t b);
/*================== Function Implementations =============================*/
#endif /* TESTLIB_H_ */
|
The library defines a function super_function(uint8_t a, uint8_t b)
in
testlib.c
:
1 2 3 | uint16_t super_function(uint8_t a, uint8_t b) {
return a + b;
}
|
Building
Build the library (or libraries):
python tools\waf build_libs
Now all libraries are present in
build\libs
and the headers are inbuild\include
.Configure the foxBMS project to work with a library, in the first example it is the
foxbms-user-lib
library.python tools\waf configure --libs=foxbms-user-lib
Note
For including the hypothetical my-advanced-algorithm
library the command
would be:
python tools\waf configure --libs=my-advanced-algorithm
Include the headers needed for the functions in the sources and use the functions as needed.
1 2 3 4 5 6 7 8 9 10 11 12 13
/*================== Includes =============================================*/ /* some other includes */ #include "testlib.h" /*================== Function Prototypes ==================================*/ /*================== Function Implementations =============================*/ int main(void) { uint16_t a = 0; /* Use the function super_function from the library */ a = super_function(2,2); /* other code */ }
Build the foxBMS binary as usual.
python tools\waf build_primary
Building with Multiple Libraries
A project may want to use multiple libraries. For this example the two provided
dummy libraries are assumed (foxbms-user-lib
and my-foxbms-library
).
The project is configured to work with both libraries. The library names are given as command line argument separated by comma (no additional whitespace).
python tools\waf configure --libs=foxbms-user-lib,my-foxbms-library
Build the library (or libraries):
python tools\waf build_libs
Include the headers needed for the functions in the sources and use the functions as needed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
/*================== Includes =============================================*/ /* some other includes */ #include "testlib.h" #include "myfoxbmsalgorithms.h" /*================== Function Prototypes ==================================*/ /*================== Function Implementations =============================*/ int main(void) { uint16_t a = 0; /* Use the function super_function from the library foxbms-user-lib */ a = super_function(2,2); uint16_t b = 0; /* Use the function another_super_function from the library my-foxbms-library */ a = another_super_function(2,2); /* other code */ }
Build the foxBMS binary as usual.
python tools\waf build_primary
8.6. Building the Test¶
Note
The test builds described in this section are not mandatory. They can be used as a simple check that the software architecture is kept (see foxBMS software architecture).
In order to verify that low level drivers (i.e., the drivers in
embedded-software\mcu-common\driver
) do not relay on higher level modules
(e.g., FreeRTOS, database, etc.) two tests are included. These can be build by
python tools\waf configure python tools\waf configure build_primary_bare python tools\waf configure build_secondary_bare