How to run custom scripts as a part of a build?

[chmorgan]

I'd like to use add_custom_command() to generate some files that my project would depend on at compile time.

I can't find any guides on the recommended process for integrating generated files in the build process, nothing is mentioned in any of the zephyr build related docs, google doesn't turn up anything for 'zephyr add_custom_command' or 'zephyr add_custom_target'.

[xingrz]

# Let's say you have a source file `the_generated_file.c` that should be generated at build time

# Firstly, make it being depended by the build system
zephyr_library_sources(${CMAKE_CURRENT_BINARY_DIR}/the_generated_file.c)

# Secondly, declare how to generate this file
add_custom_command(
    OUTPUT  ${CMAKE_CURRENT_BINARY_DIR}/the_generated_file.c
    COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/your_script_to_run.sh
)

set_source_files_properties(
    ${CMAKE_CURRENT_BINARY_DIR}/the_generated_file.c
    PROPERTIES GENERATED TRUE
)

[chmorgan]

@xingrz this looks like it makes sense, let me try it. Is this documented anywhere in the zephyr docs? I can't be the only one generating files as a part of the build.

[xingrz]

It's just some CMake tricks, not Zephyr-specific. (zephyr_library_sources(foo.c) is just a shorthand of something like add_sources(app foo.c))

[chmorgan]

For anyone else reading it should be unnecessary to set_source_file_properties() to set GENERATED, add_custom_command()

From https://cmake.org/cmake/help/latest/command/add_custom_command.html#command:add_custom_command:

OUTPUT
Specify the output files the command is expected to produce. Each output file will be marked with the GENERATED source file property automatically. If the output of the custom command is not actually created as a file on disk it should be marked with the SYMBOLIC source file property.

[wkhadgar]

Just a hint from my experience with generated files, since zephyr also heavily uses this with python scripts, and I got used to adding quite an amount of those "meta" sources that are generated in compile time (trust me they save you A LOT of time if done right).

I got two options that depend on when you want them to be generated, in this case let's assume you have a python script ready:

If you want it to run when CMakeLists loads or reloads, add this to your CMakeLists.txt:

execute_process(
        COMMAND ${CMAKE_COMMAND} -E env ${PYTHON_EXECUTABLE} "path/to/script.py"
        WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)

Otherwise if you'd like it to run before each build:

add_custom_target(script
        COMMAND ${CMAKE_COMMAND} -E env ${PYTHON_EXECUTABLE} "path/to/script.py"
)

add_dependencies(target script)

I have used those for quite a while now, e.g. generating lookup tables for ADC values and converting images to byte arrays for OLED displays. Hope this helps and solves your needs!

[rjmaris]

I have been trying unsuccessfully to get binary files (images, fonts, ...) objcopied to object files and then linked. Here, I thought to have the solution: prepending my cmake code by

zephyr_library_sources(${CMAKE_CURRENT_BINARY_DIR}/the_generated_file.c)

as notified in the first comment posting. But this is only partially true...

Let me explore my code snippet in the project's CmakeLists.txt (where iconfont.bin is 'source code' and iconfont.bin.o is the resulting object file):

add_custom_command(OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/iconfont.bin.o 
    WORKING_DIRECTORY ${PROJECT_SOURCE_DIR}  # this ensures object symbols without prepending path
    COMMAND ${CMAKE_OBJCOPY} 
    ARGS -I binary -O elf32-littlearm iconfont.bin ${CMAKE_CURRENT_BINARY_DIR}/iconfont.bin.o)
target_link_libraries(app PRIVATE ${CMAKE_CURRENT_BINARY_DIR}/iconfont.bin.o)

No object file is created, apparently because no dependency is given that invokes the custom command. But when I prepend the code snippet by the above zephyr_library_sources line, the object file is created, but... with an error: multiple definition of '_binary_iconfont_bin_start'; iconfont.bin.o:(.data+0x0): first defined here.
When I then iterate without the target_link_libraries, no build errors occur!

Thats one issue. The other one is: the object file is associated with the .data-section, not .text, which actually doubles the size of the binary data into both flash and RAM-memory.
Despite the fact that the C-code has an const in extern const unsigned short _binary_iconfont_bin_start;

FYI: ideas to get it as far as above are found e.g. here: https://stackoverflow.com/questions/23000564/cmake-embed-elf-into-executable. The difficulty in the context of zephyr is the fact that mostly, an add_executable() command is part of those examples while Zephyr's standard app is already an executable. To be honest: some stuff makes me confusing. Therefore I second the statement of the initial poster.

BOTTOM LINE:
Part of my difficulties (consuming hours of fiddling) is the fact that I had to realize that an object file is generated only when the C-extern references to the symbols in the object are present and correct. I'd wish to have the capability to force an object linking also when it isn't used. Then the map file can show helpful information.