When CCACHE_BASEDIR variable is set, ccache rewrites absolute paths into
relative paths before computing the hash that identifies the compilation
for all the paths under that directory.
The problem is that those paths are also used when the compiler is
called, so the generated dependency files (*.d) will have a relative
path to the object files, and thus, it will not match our rule for
compiling (we use absolute paths). As dependency files define the
targets this way, any change on its dependencies (e.g. an included
header file) will not re-trigger a build.
For regular modules, adding files to BLOBS is sufficient to create the
corresponding headers.
Application modules are different, as they use a minimal makefile
(makefiles.application.inc.mk) to build, thus application level
variables are not available.
This commit makes Makefile.include pass BLOBS to the application
Makefile as APPLICATION_BLOBS, and application.inc.mk use that variable
as value for BLOBS.
The indirection is necessary so submakefiles (e.g., those visited by
DIRS) do not hard override BLOBS.
By ensuring the PORT auto-detection worked, we can give meaningful
error messages and fail earlier.
This uses ensure_value from makefiles/utils/checks.mk. An include was
added to Makefile.include to make this fuction available to all other
makefiles.
- Decluttered Makefile by moving BOARD_INSUFFICIENT_MEMORY lists from Makefile
to Makefile.ci
- Optimized the list for use of tools:
- One entry per line reduces the number of merge conflicts
- One entry per line allows alphabetical sorting e.g. via Vim's sort command
- Sorted all lists alphabetically
CPU must now be defined by `$(RIOTBOARD)/$(BOARD)/Makefile.features` or
one of its common included Makefile.features file.
The cpu `Makefile.dep` file can now automatically be included when it exists.
Add a 'dependency-debug' and a 'DEPENDENCY_DEBUG=1' option for
'info-boards-supported' to save some variables used when resolving
dependencies.
Print some some 'sorted' variables to simplify comparing the actual value
when the parsing order changed.
This should help tracking changes introduced when refactoring the
dependency parsing.
Rely on creating an intermediate riotbuild.h.in file that is updated on
CFLAGS changes, but then generate 'riotbuild.h' without the comments.
The updated timestamp will still trigger a rebuild but not cause ccache
miss due to the content of the CFLAGS that contains absolute path.
This removes the caching issue due to the absolute path that was
added to `CFLAGS` and so the comment in that file.
This will avoid to load edbg logic and especially adds it to the list of FLASHDEPS: it won't be unnecessarily built when flashing a samr21-xpro/arduino-zero board on IoT-LAB
Do not remove the '-D' and '-U' values from CFLAGS.
This prevents issues where a '-D' could contain a space.
Some values way be duplicated from the 'riotbuild.h' header and the
command line but with the same value so without conflict.
To not put too many things in the command line, the -DMODULE_NAME are
only put in CFLAGS_WITH_MACROS.
Also, as now, the deferred value of CFLAGS is used for 'riotbuild.h',
macros set after the inclusion of `Makefile.include` will be taken into
account.
Old versions of GNU Make (especially on OSX) are a pain to deal
with. This commit introduces a warning when such an old version
is found. The goal is the future the warning will be turned into
an error.
2020.01 is set as the date we remove support.
Complying with version 4.x is easy on OSX by using homebrew, and
on Linux, even Debian stable has the required version.
This adds a new target "archive-check".
Thin archives should be created with relative paths so that archives created
in a build container are useful in the host. This check ensures there are no
absolute paths inside thin archives.
Add make only function to convert strings to lowercase and uppercase.
This can replace the `$(shell echo $(var) | tr 'a-z-' 'A-Z_')` pattern.
Using the 'make' implementation results in being around 100 times faster.
Update the documentation to say it can also be 'ELFFILE'
The first version was setting it to EFLFILE by default but the default
behavior was removed to prevent hiding errors.
It can also be overwritten for application specific needs, like when
using 'riotboot'.
Add a 'buildtest-indocker' that forces executing 'buildtest' for loop
completely inside the container.
It prevents starting one container per compilation wich is slower but
it could hide errors where the host toolchain would be used
It is currently equivalent to `buildtest` but will change when the
`buidtest` handling will be move outside of `BUILD_IN_DOCKER`.
Display an error when executed without BUILD_IN_DOCKER=1.
This changes CFLAGS_WITH_MACROS to be:
* a deferred variable
* not exported so only evaluated when actually used
The value is only used by `Makefile.include` and `makefiles/eclipse.inc.mk`
so not required to export it.
Previously, this was hard-coded to allow one file, hard-coded to be
called "flash file".
This commit allows multiple files to be specified via adding them to the
TEST_EXTRA_FILES variable. All files will be stored in the worker's
application bin directory.
Also, the existence check has been removed, as dwqc bails out on missing
file anyways.
Currently, some targets are serialized before "clean" by conditionally
adding a dependency.
Make does allow ordering using "|" syntax in prerequisites, but for
"clean" that would always trigger the clean target.
By only conditionally setting $(CLEAN) to clean, that can be
circumvented.
The new CLEAN allows any recipe to be run after "clean" by specifying "|
$(CLEAN)" as prerequisite.
Prepare for when boards define `CPU` in `BOARD/Makefile.features`.
Include '$(RIOTCPU)/$(CPU)/Makefile.features' directly when it is
defined. This will allow removing the file inclusion from the
`BOARD/Makefile.features`. The board must then not include it directly.
Transitional change to allow migrating part by parts.
Part of moving CPU/CPU_MODEL definition to Makefile.features to have it
available before Makefile.include.
Update the FEATURES_OPTIONAL meaning to be more in line since
FEATURES_USED is defined. Handle FEATURES_OPTIONAL as a configuration from
the BSP/build that should not be changed anymore after.
`FEATURES_OPTIONAL` are by definition optional so are not supposed to
cause a build to fail.
Only the 'REQUIRED' ones that are not 'PROVIDED' are 'MISSING'.
* Do not change FEATURES_OPTIONAL to remove REQUIRED features
* Prepare for having a different variable for the previous value
* Update dependency resolution/info-build as FEATURES_OPTIONAL cannot be missing
This will allow sharing it between Makefile.include and
makefiles/info-global.inc.mk.
Also some common variables definition can also be moved to here.
Part of moving CPU/CPU_MODEL definition to Makefile.features to have it
available before Makefile.include.
My system (Ubuntu 16.04) creates a backup when opening a file with
`vim`. This backup is called `filename~`. Due to it being a copy of the
file before opening, it also is executable, which is why it is selected
for execution with the `make test` target.
This change makes the `TESTS` macro exclude files ending in `~`. This
way files ending in `~` are not executed with `make test`.
Since e.g. projects like NextCloud are excluding such files also from
their sync [[1]], I think this is fine.
[1]: d7b881feb6/sync-exclude.lst (L3)
This allows querying the build system if there are test available.
Before, one should rely on 'info-debug-variable-TESTS' to print the list
of test files. But was not reliable as sometime the build system printed
messages anyway.
BOARD=esp32-wroom-32 make --silent --no-print-directory \
-C examples/hello-world/ info-debug-variable-TESTS
ESP32_SDK_DIR should be defined as /path/to/esp-idf directory
ESP32_SDK_DIR is set by default to /opt/esp/esp-idf
# empty line here
Now the return code can be trusted.
Include the 'riotboot.mk' file before using FLASHFILE/ELFFILE/HEXFILE
variables. This will allow setting variables to values from riotboot.mk
like `FLASHFILE = $(RIOTBOOT_COMBINED_BIN)` before it is evaluated in
Makefile.include.
It should be included after defining 'BINFILE' for 'riotboot.bin'
handling.
Introduce FLASHFILE variable to start migrating boards to use it.
This is the file that will be used for flashing.
Boards do not currently use it but will migrated in upcoming PRs.
This allows setting other GLOBAL_GOALS by setting it before.
This could be the case for the application `Makefile` or files parsed
from `RIOT_MAKEFILES_GLOBAL_PRE`.
This introduces a new environment variable for a common directory
that holds all output of the build process, such as application or
package binaries. This would also allow to easily redirect output
to any other location, e.g. for out-of-source builds.
riotboot is introduced here and makes use of riotboot_hdr,
which indentifies the images encapsulated as slots.
The slot size and offset is configurable, which makes
slots extendable if needed, e.g. 2 or more slots can be
transparently added.
Co-authored-by: Kaspar Schleiser <kaspar@schleiser.de>
Co-authored-by: Gaëtan Harter <gaetan.harter@fu-berlin.de>
When flash-only was introduced (in #8373), the `flash` rule was
made conditionally dependent on `all` by looking for `flash-only`
in MAKECMDGOALS. This was done to avoid code duplication.
There's a cleaner way, by using canned recipes. When we upgrade the
requirements to gnu make 4, the flash recipe can be defined as ?=.
This rule is not being used, it complicates the makefile and causes
make clean to permform unnecessary actions.
All packages have a Makefile.include, so the rule is not needed anyways.
Also, it is defined with a double colon for no reason.
This allows specifying a list of files that should be parsed by make at
the beginning and at the end of Makefile.include.
It is a generic mechanism to allow specifying system wide configuration:
* Globally overwrite the 'TERMPROG'
* Specify a hard written port / debug_adapter_id for some BOARD values
* Define you own specific targets
* Override default targets
It can include file before and after Makefile.include to allow handling
different configurations.
When building in parallel, clean is done at the same time as the
container starts building.
So the parallel `clean all` handling should also be done for the
`..in-docker-container` target.
Makefile.include contained logic to clone a different RIOT version and
build the application against it. This can be easily achieved by making
a git worktree and overriding RIOTBASE, without adding any complexity
to our makefiles. Therefore, the feature is being removed.
When an application only includes Makefile.include without specifying
RIOTBASE and uses a `Makefile.local` file, RIOTBASE would use a wrong
default value and get the value of the directory where `Makefile.local` is.
The hostname does not provide any additional or necessary information to
the RIOT_VERSION string. On the contrary, some might consider the hostname
as personal information, which should not be exposed unsolicitedly.
Process `Makefile.include` for external modules. It is included after the others
so it could overwrite some of the configuration if wanted.
Process `Makefile.dep` for external modules. It is included before the others so
it could be parsed before setting 'default' values to dependencies.
It fixes issues with the current rule that it is wrong that each
`$(BINDIR)/pkgname.a` is built by going in the package directory:
* `nordic_softdevice_ble.a` is built using `DIRS` and so the
`$(APPLICATION_MODULE).a` target.
* It prevents having packages that are pseudomodules, which may be
required to only use one "library" part of a package.
It also simplifies handling changes in 'ld -r' PR that could produce
objects instead of archives for packages.
Limitation of the current implementation
----------------------------------------
It removes rules being 'file' target and makes them depend on `.PHONY`
targets so always forces re-build.
But having a file target whose file is silently generated by another
target does not trigger a rebuild in Make.
They may have been declared as `order-only` prerequisites but as there are
some edge-cases that may not always work and does not currently add anything,
it was decided to keep them as normal prerequisites until it can be
globally solved.
Now also print warnings if there are missing features or a blacklisted
board when doing 'make flash'
Before, only when doing 'make' or 'make all' the FEATURES_REQUIRED,
BOARD_BLACKLIST/WHITELIST and TOOLCHAINS were checked and a warning was printed.
However as 'flash' triggers 'all' it is a common case to do 'make flash'
directly instead of 'make all flash'. So better also print warnings in
this case.
Assuming `TOOLCHAIN_SUPPORTED` is provided by the board and
`TOOLCHAIN_BLACKLIST` by a module or `pkg`, this outputs the fact that
a toolchain is not supported or blacklisted in a similar manner as
(un-)supported features and boards are.
When downloading the release archive and building an example, the RIOT_VERSION string is not set to "Version 2018.04" but to
Version: UNKNOWN (builddir: /home/me/Downloads/RIOT-2018.04)
This allows sourcing a global VERSION file to manually set it before releases.
Introduce dist/pythonlibs directory to store RIOT python packages.
This directory is exported via PYTHONPATH by the build system to
make it commonly available.
When selecting a directory to build using `-C` with make
`BUILD_IN_DOCKER` will fail, because the `BUILDRELPATH` chooses the path
`make` is executed in, not the path selected by `-C`. This fixes this
bug by replacing `PWD` in the macro's definition with `CURDIR`.
Fixes a regression introduces #9479 which breaks building on macOS.
The introduces `uniq` function is never called, hence the includes
ie. USEMODULE_INCLUDES are not used.
Currently, `USEMODULE_INCLUDES` is deduplicated using a combination of
`tr` and `awk`. The proposed method achieves a deduplication (no
sorting) without the need to shell out by using make internals only.
BUILDDEPS are files / make targets that should be build before compiling.
It can include packages source download, generating headers, modules.
It is the equivalent of `APPDEPS` but not limited to the application.
It cannot be done right now with `APPDEPS` as it is used in `BASELIBS` and
fixing it requires changing mips using it for source files.
They can define PSEUDOMODULES which is used to populate BASELIBS.
It means they must be processed before using BASELIBS immediate value.
This moved the processing of all `Makefile.include` before using `BASELIBS`.
It is also moved before setting `BASELIBS` in `modules.inc.mk to keep the
order logical (it would work anyway thanks to deferred variables evaluation).
This fixes the problems for `make -C tests/unittests`
Build targets were using the immediate value of '$(BASELIBS)' before it was
actually set. bindist.inc.mk should also be processed before as it is adding
`BIN_USEMODULE` to `USEMODULE`.
This fixes use before define problems for:
* `make -C examples/hello-world`
* `make -C examples/bindist`
Include order for board and cpu was
1. cpu include
2. board include
3. board common includes
4. cpu common includes
Its now changed to:
1. board include
2. board common includes
3. cpu include
4. cpu common includes
Verifications:
There are no common headers names between boards and cpus.
Except native that has a 'periph_conf.h' in cpu instead of being in board.
Add support to do flash/reset/term on an IoT-LAB node.
It also allow running test using 'testrunner'.
Configuration variables are:
* `IOTLAB_NODE` which should be set to your node url
* The full url including site to use from your computer `m3-1.grenoble.iot-lab.info`
* The short url when used on the IoT-LAB frontend `m3-1`
* `IOTLAB_EXP_ID` for your experiment id for flash and reset.
By default it tries to use your currently running experiment if you have only one
* `IOTLAB_USER`: is read from `${HOME}/.iotlabrc` as saved by `iotlab-auth`
* It is expected to have run `iotlab-auth` beforehand.
Unittests add libraries in 'BASELIBS' which do not have any rules to be built as
they are built by 'application.inc.mk', packages and the DIRS variable.
So make complains about missing target for the unittests archives.
The fix tells these files are generated when building '$(APPLICATION_MODULE).a'.
The bug was introduced by #8844Fixes#8910
* Add ELFFILE, HEXFILE, print-size targets
$(BINDIR)/$(APPLICATION_MODULE).a target builds libraries in RIOT and in DIRS.
Uses a FORCE phony target dependency to force rebuilding but still let
`make` use the file modification timestamps
This variable can be used to specify the linker which should be used to
link C++ object files. This is needed in order to use g++ for linking
with TOOLCHAIN=llvm.
Set the application module name with APPLICATION_MODULE.
The default module name is "application_$(APPLICATION)".
This should fix name conflicts where an application/test has the same name as
an existing module and so both write to the same archive.
All macros are filtered out of the `CFLAGS` variable and during this
process, the `CFLAGS` string is destroyed. This is due to the fact that
the `RIOT_VERSION` string contains spaces, but the filtering process does
not take the whole string into account (following the first space). This
leads to some parts of the `RIOT_VERSION` string be sill existent in the
`CFLAGS` string.
Instead of exporting the `RIOT_VERSION` string to the `CFLAGS` macro to
just remove it again from there, I exported it to `CFLAGS_WITH_MACROS`.
Fixes#5859
