It turns out that the ID mechanics of docker are even more crazy than
realized before: On Linux (x86_64) they use a different SHA256 when
referring to a locally installed image than when referring to the
same image at dockerhub. On Mac OS (Apple Silicon), the use the repo
SHA256 also when referring to the local image.
Instead of increasing the complexity of the current solution even more
by covering both cases, we now use
`docker.io/riot/riotbuild@sha256:<SHA256_OF_DOCKERHUB_IMAGE>` to refer
to a specific docker image, which hopefully works across systems.
Instead of pulling the image explicitly, we now can rely on docker
to do so automatically if the pinned image is not found locally. As
a result, the knob to disable automatic pulling has been dropped.
Fixes https://github.com/RIOT-OS/RIOT/issues/20853
We had a number of issues being reported that in the end were caused
by building with a version of riotbuild incompatible with the source,
because CI and source were updated in lock-step but users forgot to
run `docker pull` to update theirs.
In addition, checking out a random old release/commit will likely fail,
as older source may contain bugs that did not trigger with older
toolchains. To reliably succeed at building, a matching version of
riotbuild needs to be used for `BUILD_IN_DOCKER=1`.
This just pins the version of riotbuild manually, so it needs to be
updated by hand in lock step with updating the docker container.
In addition, the latest image is pulled automatically on mismatch.
Ideally, this would pull the tagged image. But that fails for unknown
reason when using the documented command for this with both docker and
podman with:
manifest for riot/riotbuild@sha256:f5951bc41dfface6cac869181d703e62cbdd3b7976b0946130a38f2e658000b3 not found: manifest unknown: manifest unknown
Co-authored-by: mguetschow <mikolai.guetschow@tu-dresden.de>
The semantics of `make debug` and `make debugserver` have changed in
the years since the MSP430 integration. This brings the implementation
back into line with the current semantics
- `make debug` now starts both mspdebug and GDB, no need to
run `make debugserver` prior to `make debug` anymore
- `make debug` no longer flashes the target to not waste flash erase
cycles
- GDB mutliarch support is added
- support for selecting a debug adapter by its serial is added
When using podman instead of docker the registry is not set by default.
Docker has a builtin registry default "docker.io". Podman does not have a
default. By specifying the registry explicitly both tools can be used.
This adds a list of variables that should always be passed to docker
since they are commonly set in Makefile/Makefile.include and therefore
can not be checked for their origin.
By default if IoT-LAB Cli Tools V3 is used then BINFILE is used to
flash on IoT-LAB. But BINFILE is not built by default when RIOT_CI_BUILD
is set as a ci optimization.
But since before IOTLAB_NODE was not passed to docker when building it
did not know that it should BUILD BINFILE as well, which led to failures
if doing:
$ IOTLAB_NODE=iotlab-m3.grenoble.iot-lab.info BOARD=iotlab-m3 \
RIOT_CI_BUILD=1 BUILD_IN_DOCKER=1 make -C examples/hello-world/ flash
But if IOTLAB_NODE is passed at is checks for IoT-LAB cli Tools also
happen in the docker container which leads to a make error since those
are not present in docker.
Therefore add BINFILE to BUILD_FILES if RIOT_CI_BUILD is set, but unset
IOTLAB_NODE once INSIDE_DOCKER.
Without passing this through, any crate downloads would go to the
container's image's scratch space, to be re-downloaded for each build,
and with new time stamps that cause compilation to start anew as well.
This will allow the generation of the bin-checksummed file needed by dfu-util flashtool and thus allow to flash the all particle-based board with RUN_IN_DOCKER=1
- Add the new EXTERNAL_BOARD_DIRS variable that can contain a space separated
list of folders containing external boards
- Introduce $(BOARDDIR) as shortcut for $(BOARDSDIR)/$(BOARD)
- Map the existing BOARDSDIR to the new approach
- If BOARDSDIR is provided by the user, it will be added to
EXTERNAL_BOARD_DIRS for backward compatibility. (And a warning is issued
to encourage users migrating to EXTRA_BOARDS.)
- BOARDSDIR is updated after the board is found to "$(BOARDDIR)/..".
- Useful for `include $(BOARDSDIR)/common/external_common/Makefile.dep`
- Provides backward compatibility
Depending on the PROGRAMMER used, the file used for flashing may be
different, like openocd/jlink, so the file being generated when
compiling should be different.
Because of this, when building in docker, the build system needs to know
for which programmer you are building for.
This manifested for 'nordic_softdevice' applications when using 'openocd'
but is not limited to this case.
This adds a new target "archive-check".
Thin archives should be created with relative paths so that archives created
in a build container are useful in the host. This check ensures there are no
absolute paths inside thin archives.
This allows passing other arguments as environment variables in
DOCKER_ENVIRONMENT_CMDLINE and as command line override in
DOCKER_OVERRIDE_CMDLINE without using the automatic detection.
This will allow passing USEMODULE to docker as its value is usually set
in an application Makefile so has its origin changed.
This allows setting other variables that should be exported to the
docker build.
As for other variables, they must still be unmodified before parsing
`docker.inc.mk` to be exported.
This remove executing buildtest `for` loop in docker.
When building completely in docker, 'buildtest' would hide issues when
the host toolchain would be used when doing `make all` directly.
It has the consequence that it now starts a container for each
compilation which is slower.
The previous behavior can be reproduced by using
BUILD_IN_DOCKER=1 make buildtest-indocker
A side effect is also that now `BUILDTEST_MAKE_REDIRECT` would work when
doing `buildtest` with docker.
Add a 'buildtest-indocker' that forces executing 'buildtest' for loop
completely inside the container.
It prevents starting one container per compilation wich is slower but
it could hide errors where the host toolchain would be used
It is currently equivalent to `buildtest` but will change when the
`buidtest` handling will be move outside of `BUILD_IN_DOCKER`.
Display an error when executed without BUILD_IN_DOCKER=1.
This enables "BUILD_IN_DOCKER=1 BOARDS='foo bar' make buildtest".
Without this line, when executed with BUILD_IN_DOCKER, "make buildtest"
would always build all boards.
Introduce FLASHFILE variable to start migrating boards to use it.
This is the file that will be used for flashing.
Boards do not currently use it but will migrated in upcoming PRs.
They are remapped to `$(DOCKER_BUILD_ROOT)/external` if they are not
inside RIOT (usually the case but not for `tests/external_modul_dirs`).
If they are inside 'riotproject' they are currently also remapped to
'external'.
The value of `EXTERNAL_MODULE_DIRS` is then enforced by configuring it on
the command line as the application should not try to set it anymore.
The remapping is done in `external/directory_name` so cannot handle
multiple external directories with the same name.
