17091: USBUS: Add URB support r=benpicco a=bergzand
### Contribution description
This commit adds support for URBs (USB Request/Response Blocks). These
allow for submitting multi-transfer sized buffers with USBUS handling
the individual usbdev xmits. Multiple URBs can be queued at once for a
single endpoint and USBUS will handle them in the order of submission.
OUT endpoint URBs must always consist of a whole number of full-sized
transfers (N x MaxEndpointSize). They will automatically finish after
the endpoint received a transfer less than the endpoint size.
IN endpoints can be arbitrary-sized and do not have to consist of a
whole number of full-sized transmissions. They support a flag to
indicate that the last transfer in the sequence must be less than a full
sized transfer (USBUS_URB_FLAG_AUTO_ZLP) and this adds a zero length
transfer at the end of the transmissions if the last transfer was equal
to the maximum transfer size.
URBs can be cancelled, but if the URB is already being processed it will
be cancelled after the current transmission within the URB is finished.
If it is still in the queue it will immediately be removed from the
queue.
### Testing procedure
- `tests/usbus_cdc_ecm` should still work. Testing one of the usbdev-supported platform should be sufficient here.
### Issues/PRs references
Needs #17064
18148: sys/flash_utils: helpers to store data in flash r=benpicco a=maribu
### Contribution description
This helpers that allow storing, accessing, and working with data in flash that works for both classical Harvard architectures (which do not map flash also into the address space) as well as modern Harvard architectures and von-Neumann architectures.
With this, `examples/default` again runs on the Arduino Uno / Nano. Since this board is still the "entry kit" for many people to embedded hardware, it would be nice to support it with our default example.
### Testing procedure
`examples/default` should run and work on ATmega boards (especially ATmega328P and ATmega32U4 based boards) as well on all other boards now.
### Issues/PRs references
None
Co-authored-by: Koen Zandberg <koen@bergzand.net>
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
This commit adds support for URBs (USB Request/Response Blocks). These
allow for submitting multi-transfer sized buffers with USBUS handling
the individual usbdev xmits. Multiple URBs can be queued at once for a
single endpoint and USBUS will handle them in the order of submission.
OUT endpoint URBs must always consist of a whole number of full-sized
transfers (N x MaxEndpointSize). They will automatically finish after
the endpoint received a transfer less than the endpoint size.
IN endpoints can be arbitrary-sized and do not have to consist of a
whole number of full-sized transmissions. They support a flag to
indicate that the last transfer in the sequence must be less than a full
sized transfer (USBUS_URB_FLAG_AUTO_ZLP) and this adds a zero length
transfer at the end of the transmissions if the last transfer was equal
to the maximum transfer size.
URBs can be cancelled, but if the URB is already being processed it will
be cancelled after the current transmission within the URB is finished.
If it is still in the queue it will immediately be removed from the
queue.
18682: pkg/lwext4: add lightweight implementation of the ext2/3/4 filesystem r=benpicco a=benpicco
Co-authored-by: Benjamin Valentin <benjamin.valentin@ml-pa.com>
Boards with an integrated debugger/programmer that also provides the
serial as UART <--> USB adapter, the TTY serial matches the serial of
the programmer.
This adapts the `serial.inc.mk` to set the `DEBUG_ADAPTER_ID` to the
TTY serial if (and only if) `MOST_RECENT_PORT` *and*
`DEBUG_ADAPTER_ID_IS_TTY_SERIAL` both have a value of `1`. Boards with
an integrated programmer are expected to set
`DEBUG_ADAPTER_ID_IS_TTY_SERIAL` to `1` in their `Makefile.include`.
The previous servo driver didn't provide any benefit over using PWM
directly, as users controlled the servo in terms of PWM duty cycles.
This changes the interface to provide a high level interface that
abstracts the gory PWM details.
In addition, a SAUL layer and auto-initialization is provided.
Co-authored-by: benpicco <benpicco@googlemail.com>
Before the build system used something like
echo "int main(){} void _exit(int n) {(void)n;while(1);}" | LC_ALL=C $(LINK) -xc - -o /dev/null -lc -Wall -Wextra -pedantic <FLAGS_TO_TEST>
to check for flags supported by the compiler (used as linker frontend).
This however no longer works with newlib 4.3.0, which requires the
symbols `_sbrk_r`, `_close_r`, `_lseek_r`, `_read_r`, and `_write_r` to
be present. Instead, now a new file `minimal_linkable.c` was added that
adds all the missing symbols and is used in the linker feature tests
instead.
19057: fuzzing: Add uri_parser setup r=benpicco a=Teufelchen1
Hello!
### Contribution description
This PR is a replacement for PR #18802
In this contribution:
* The variable `AFL_FLAGS` is renamed to `FLAGS_FOR_AFL` because AFL is always complaining that `AFL_FLAGS` is not a valid env var for it. While this is not a bug nor an issue, I found it to be annoying.
* A generic input reader is added to simplify building a test harness
* The usage of this reader is demonstrated by adding a harness for fuzzing the uri_parser
(needs squashing after review)
### Testing procedure
Go to `fuzzing/uri_parser` and run `make all-asan` and `make fuzz` to get some action going.
Also mildly interesting: `./dist/tools/compile_test/compile_like_murdock.py -b native -a fuzzing/uri_parser`
### Issues/PRs references
The original PR #18802 is replaced because the generic input reader is present in both PRs but this PoC harness is much simpler.
19151: examples/gcoap: Fix shell parameter validation r=benpicco a=maribu
### Contribution description
Executing the shell command with an URI-Path that doesn't start with a slash results in an assertion error while composing the client side message. This is suboptimal user experience, so add an explicit check for a valid URI-Path and a dedicated error message.
### Testing procedure
#### In `master`
```
$ make BOARD=microbit-v2 -C examples/gcoap flash term
[...]
2023-01-15 22:23:32,512 # coap get [::1] /.well-known/core
2023-01-15 22:23:32,516 # gcoap_cli: sending msg ID 52272, 23 bytes
2023-01-15 22:23:32,520 # gcoap: response Success, code 2.05, 46 bytes
2023-01-15 22:23:32,524 # </cli/stats>;ct=0;rt="count";obs,</riot/board>
> coap get [::1] foo
2023-01-15 22:23:34,763 # coap get [::1] foo
2023-01-15 22:23:34,763 # 2329
2023-01-15 22:23:34,765 # *** RIOT kernel panic:
2023-01-15 22:23:34,767 # FAILED ASSERTION.
2023-01-15 22:23:34,767 #
2023-01-15 22:23:34,775 # pid | name | state Q | pri | stack ( used) ( free) | base addr | current
2023-01-15 22:23:34,784 # - | isr_stack | - - | - | 512 ( 200) ( 312) | 0x20000000 | 0x200001c8
2023-01-15 22:23:34,793 # 1 | main | running Q | 7 | 1536 ( 1072) ( 464) | 0x200006c0 | 0x2000095c
2023-01-15 22:23:34,802 # 2 | 6lo | bl rx _ | 3 | 1024 ( 328) ( 696) | 0x200036c0 | 0x200039c4
2023-01-15 22:23:34,810 # 3 | ipv6 | bl rx _ | 4 | 1024 ( 460) ( 564) | 0x20001294 | 0x20001574
2023-01-15 22:23:34,819 # 4 | udp | bl rx _ | 5 | 512 ( 300) ( 212) | 0x20003e98 | 0x20003f9c
2023-01-15 22:23:34,828 # 5 | coap | bl anyfl _ | 6 | 1112 ( 704) ( 408) | 0x20000e38 | 0x200011c4
2023-01-15 22:23:34,837 # 6 | nrf802154 | bl anyfl _ | 2 | 896 ( 288) ( 608) | 0x20001a90 | 0x20001d54
2023-01-15 22:23:34,843 # | SUM | | | 6616 ( 3352) ( 3264)
2023-01-15 22:23:34,843 #
2023-01-15 22:23:34,844 # *** halted.
2023-01-15 22:23:34,844 #
```
#### This PR
```
$ make BOARD=microbit-v2 -C examples/gcoap flash term
[...]
make: Entering directory '/home/maribu/Repos/software/RIOT/examples/gcoap'
/home/maribu/Repos/software/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200"
2023-01-15 22:22:27,842 # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
coap get [::1] /.well-known/core
2023-01-15 22:22:40,042 # coap get [::1] /.well-known/core
2023-01-15 22:22:40,046 # gcoap_cli: sending msg ID 25182, 23 bytes
2023-01-15 22:22:40,050 # gcoap: response Success, code 2.05, 46 bytes
2023-01-15 22:22:40,054 # </cli/stats>;ct=0;rt="count";obs,</riot/board>
> coap get [::1] foo
2023-01-15 22:22:43,858 # coap get [::1] foo
2023-01-15 22:22:43,862 # ERROR: URI-Path must start with a "/"
2023-01-15 22:22:43,866 # usage: coap <get|post|put|ping|proxy|info>
```
### Issues/PRs references
None
Co-authored-by: Teufelchen1 <bennet.blischke@haw-hamburg.de>
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
19010: bootloaders/riotboot: add tinyUSB DFU support r=benpicco a=gschorcht
### Contribution description
This PR provides
- the tinyUSB DFU and DFU Runtime support and
- the `riotboot_tinyusb_dfu` bootloader that uses the tinyUSB DFU mode to flash new application images.
~This PR includes PR #18983 for now to be compilable.~
### Testing procedure
1. Use any board that supports the `riotboot´ and `tinyusb_device` features and flash the bootloader first, for example
```
BOARD=nucleo-f767zi make -C bootloaders/riotboot_tinyusb_dfu flash
```
and check that the `riotboot_tinyusb_dfu` bootloader is in DFU mode:
```
dfu-util --list
```
3. Flash a first application using the following command:
```
FEATURES_REQUIRED=riotboot USEMODULE=tinyusb_dfu BOARD=nucleo-f767zi \
make -C tests/saul PROGRAMMER=dfu-util riotboot/flash-slot0
```
and check that the application starts and is seen as upgradable:
```
dfu-util --list
```
4. Restart the node in bootloader DFU mode by:
```
dfu-util -e
```
Flash a second application, for example
```
FEATURES_REQUIRED=riotboot USEMODULE=tinyusb_dfu BOARD=nucleo-f767zi \
make -C tests/shell PROGRAMMER=dfu-util riotboot/flash-slot1
```
and check that the second application starts and is seen as upgradable:
```
dfu-util --list
```
### Issues/PRs references
~Depends on PR #18983~
19149: SECURITY: Describe that declassification is an option r=benpicco a=chrysn
### Contribution description
Our security policy does not contain provisions for the case when what is reported is not what we consider an actual security issue. As it is described now, everything reported through security@ would go through the full treatment, including a point release.
I'm not sure it belongs into the text itself (as it's more about how security reporters interact with the project than internals), but declassification should IMO be backed at least by 3 maintainers, and no strong NACK.
### Issues/PRs references
#19141 followed that procedure after some chat on it on the maintainers channel. (In the discussion, I proposed declassification, with 2.5 people supporting it and one "I was about to, but can we be sure nobody is using it?" voice).
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Co-authored-by: chrysn <chrysn@fsfe.org>
With b30efeeb65 a warning was introduced when using `make term` without
the proper toolchain installed (e.g. when using BUILD_IN_DOCKER, but
`term` outside of the docker). This removes this warning
19050: boards/common/cc26xx cc13xx: clean up and fix flash configs r=benpicco a=maribu
### Contribution description
- Add support for XDS110 debugger via `OPENOCD_DEBUG_ADAPTER=xds110`
- Clean up OpenOCD configs in `boards/common/cc26xx_cc13xx`
- No longer hardcode the debugger to xds110, but use `OPENOCD_DEBUG_ADATER ?= xds110`
- Add support for cc13x0, cc13x2, cc26x0
- `boards/cc2650*`: drop custom OpenOCD config in favor of shared one
- add variables needed to support flashing with `PROGRAMMER=jlink`
- allow specifying a custom OpenOCD command to bring the device to a halt state, as the default `reset halt` (which causes a second reset) is causing issues with the ICEPick JTAG routers in the CC26xx - CC13xx devices
- Use `halt` instead of `reset halt` for CC26xx / CC13xx boards in OpenOCD to avoid issues in flashing
### Testing procedure
```
make BOARD=cc2650-launchpad -C examples/default flash
```
Should now work. The same should still work for other cc26xx cc13xx boards.
### Issues/PRs references
Partially fixes: https://github.com/RIOT-OS/RIOT/issues/18750
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
The Makefile function `version_is_greater_or_equal` is used to check if
a version of GNU Make is at least the required one. However, it has
the built-in assumption the version numbers have to format x.y.z,
but Alpine Linux currently ships GNU Make 4.4. This results in
`$(call _pad_number,3,)` which runs `printf '$03d' ''` in the shell,
which is not valid.
This fixes the issue by making `_pad_number` more robust by fall back to
printing `0` with the given padding, if the number given to print is
empty.
Typically, OpenOCD is already performing a reset on connect. A
`reset halt` to bring the target to a `halt` state for flashing will
result in the device going through a second reset cycle. This can be
problematic with some device, such as the CC26xx MCUs. For these
devices, an `OPENOCD_CMD_RESET_HALT := -c 'halt'` will avoid the second
reset that is causing the issues.
This adds the configuration to allow choosing the XDS110 used in
cc13xx-launchpad and cc26xx-launchpad boards via the
`OPENOCD_DEBUG_ADAPTER` variable.
Update the list of target triples to match
`makefiles/arch/riscv.inc.mk`. This fixes compilation with toolchains
other than the obsolete toolchain that uses the incorrect
`riscv-none-embed` triple.
19030: tests/periph_timer_short_relative_set: improve test r=benpicco a=maribu
### Contribution description
Reduce the number lines to output by only testing for intervals 0..15 to speed up the test.
In addition, run each test case 128 repetitions (it is still faster than before) to give some confidence the short relative set actually succeeded.
### Testing procedure
The test application should consistently fail or succeed, rather than occasionally passing.
### Issues/PRs references
None
19085: makefiles/tests/tests.inc.mk: fix test/available target r=benpicco a=maribu
### Contribution description
`dist/tools/compile_and_test_for_board/compile_and_test_for_board.py` relies on `make test/available` to check if a test if available. However, this so far did not take `TEST_ON_CI_BLACKLIST` and `TEST_ON_CI_WHITELIST` into account, resulting in tests being executed for boards which they are not available. This should fix the issue.
### Testing procedure
#### Expected to fail
```
$ make BOARD=nrf52840dk -C tests/gcoap_fileserver test/available
$ make BOARD=microbit -C tests/log_color test/available
```
(On `master`, they succeed, but fail in this PR.)
#### Expected to succeed
```
$ make BOARD=native -C tests/gcoap_fileserver test/available
$ make BOARD=nrf52840dk -C tests/pkg_edhoc_c test/available
$ make BOARD=nrf52840dk -C tests/log_color test/available
```
(Succeed in both `master` and this PR.)
### Issues/PRs references
None
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
dist/tools/compile_and_test_for_board/compile_and_test_for_board.py
relies on `make test/available` to check if a test if available.
However, this so far did not take `TEST_ON_CI_BLACKLIST` and
`TEST_ON_CI_WHITELIST` into account, resulting in tests being executed
for boards which they are not available. This should fix the issue.
19074: cpu/esp8266: build the SDK bootloader from source r=benpicco a=gschorcht
### Contribution description
This PR is a takeover of PR #17043, which is rebased to the current master and includes some corrections that became necessary after rebasing.
**Copied from description of PR #17043:**
We had four versions of pre-built bootloaders for the esp8266 with different settings of logging and color logging. These bootloaders were manually built from the SDK and shipped with RIOT-OS source code. However there are more settings that affect the bootloader build that are relevant to the app or final board that uses this bootloader. In particular, flash size and flash speed is important for the bootloader to be able to load an app from a large partition table at the fastest speed supported by the board layout and flash chip.
Another example is the UART baudrate of the logging output from the bootloader. The boot ROM will normally start at a baud rate of 74880 (depending on the crystal installed), so it might make sense to keep the UART output at the same speed so we can debug boot modes and bootloader with the same terminal.
This patch builds the `bootloader.bin` file from the ESP8266 SDK source code. The code is built as a module (`esp8266_bootloader`) which at the moment doesn't generate any object code for the application and only produces a `bootloader.bin` file set to the `BOOTLOADER_BIN` make variable for the `esptool.inc.mk` to flash.
The code needs to be compiled and linked with custom rules defined in the module's Makefile since the `bootloader.bin` is its own separate application.
The `BOOTLOADER_BIN` variable is changed from a path relative to the `$(RIOTCPU)/$(CPU)/bin/` directory to be full path. This makes it easier for applications or board to provide their own bootloader binary if needed.
As a result of building the bootloader from source we fixed the issue of having a large partition table.
### Testing procedure
Use following command to flash the application with STDIO UART baudrate of 115200 baud.
```
BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
Connect with a terminal programm of your choice (unfortunatly `picocom` and `socat` don't support a baudrate close to 74880), for example:
```
python -m serial.tools.miniterm /dev/ttyUSB0 74880
```
On reset, the `esp8266-esp-12x` node shows the ROM bootloader log output
```
ets Jan 8 2013,rst cause:2, boot mode:(3,7)
load 0x40100000, len 6152, room 16
tail 8
chksum 0x6f
load 0x3ffe8008, len 24, room 0
tail 8
chksum 0x86
load 0x3ffe8020, len 3408, room 0
tail 0
chksum 0x79
```
as well as the second-stage bootloader built by this PR (`ESP-IDF v3.1-51-g913a06a9ac3`) at 74880 baudrate.
```
I (42) boot: ESP-IDF v3.1-51-g913a06a9ac3 2nd stage bootloader
I (42) boot: compile time 11:25:03
I (42) boot: SPI Speed : 26.7MHz
...
I (151) boot: Loaded app from partition at offset 0x10000
```
The application output is seen as garbage since the `esp8266-esp-12x` uses 115200 as baurate by default.
To see all output at a baudrate of 74880 baud, you can use the following command:
```
CFLAGS='-DSTDIO_UART_BAUDRATE=74880' BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
If the application is built without options, the ROOM bootloader output will be 74880 baud and the second stage bootloader and application output will be 115200 baud.
### Issues/PRs references
Fixes issue #16402
Co-authored-by: iosabi <iosabi@protonmail.com>
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Instead of using a fixed position of the image file in the flash, the variable `FLASHFILE_POS` is used which allows to override the default position of the image in the flash at 0x10000.
We had four versions of pre-built bootloaders for the esp8266 with
different settings of logging and color logging. These bootloaders were
manually built from the SDK and shipped with RIOT-OS source code.
However there are more settings that affect the bootloader build that
are relevant to the app or final board that uses this bootloader. In
particular, flash size and flash speed is important for the bootloader
to be able to load an app from a large partition table at the fastest
speed supported by the board layout and flash chip.
Another example is the UART baudrate of the logging output from the
bootloader. The boot ROM will normally start at a baud rate of 74880
(depending on the crystal installed), so it might make sense to keep
the UART output at the same speed so we can debug boot modes and
bootloader with the same terminal.
This patch builds the bootloader.bin file from the ESP8266 SDK source
code. The code is built as a module (esp8266_bootloader) which at the
moment doesn't generate any object code for the application and only
produces a bootloader.bin file set to the BOOTLOADER_BIN make variable
for the esptool.inc.mk to flash.
The code needs to be compiled and linked with custom rules defined in
the module's Makefile since the bootloader.bin is its own separate
application.
The `BOOTLOADER_BIN` variable is changed from a path relative to the
`$(RIOTCPU)/$(CPU)/bin/` directory to be full path. This makes it easier
for applications or board to provide their own bootloader binary if
needed.
As a result of building the bootloader from source we fixed the issue of
having a large partition table. Fixes#16402.
Allow overriding the shell command used to auto-detect the TTY of a
board with `MOST_RECENT_PORT=1` via the `TTY_SELECT_CMD` variable.
The use case is to also detect Arduino Mega 2560 clones with cheap
USB UART bridges (for which the filter command may yield false
positives) while preferring genuine Arduino Mega 2560 boards (if
found) over the clones (as the filter for genuine boards does not yield
false positives).
Boards that are shipped with a DFU bootloader define the `dfu-util` configuration in their `Makefile.include`. However, when `riotboot_dfu` is used as the DFU bootloader, the board's `dfu-util` configuration must be overridden by the configuration as required by `riotboot_dfu` to use it to flash applications. Therefore, all `dfu-util` configurations are defined as overridable in the board's `Makefile.include` and the configuration as required by `riotboot_dfu` is included before the board's `Makefile.include`.
When trying to compile `examples/hello-world` on my 465 MHz Mendocino
I got
/home/benpicco/dev/RIOT/makefiles/toolchain/gnu.inc.mk:29: extraneous text after 'ifneq' directive
Turns out the version of `make` on Archlinux32 is newer than that in
Ubuntu 22.04, which somehow ignored the extra parenthesis.
Turns out this (t)rusty old machine is still good for writing patches :D
Hooking into the existing wrappers for `malloc()`, `calloc()`,
`realloc()`, and `free()`, the new (pseudo) module `malloc_tracing`
prints out the calls to the given functions, the program counter of
the caller, as well as the return result.
The intent is to aid debugging double-frees, invalid frees, or memory
leaks.
Modern binutils complain about segments with RWX permissions. While
this is indeed a bad habit, RIOT ignores segments permissions anyway.
(We do have a `mpu_noexec_ram` module to disable execution of all of
RAM, which would do so regardless of the segment permission.) So for
now, we can safely just disable the warning.
This fixes:
/usr/lib/gcc/arm-none-eabi/12.2.0/../../../../arm-none-eabi/bin/ld: warning: /home/maribu/Repos/software/RIOT/examples/default/bin/nucleo-f767zi/default.elf has a LOAD segment with RWX permissions
collect2: error: ld returned 1 exit status
The test for the requirement of disabling wchar_t size warnings assumed
that $(CC) is used for linking, instead of $(LINK). With GCC $(LINK)
and $(CC) happen to be (in most cases) identical, but with LLVM they
are not. This results in issues with compiling with LLVM.
GCC on some platforms does need an executable stack to generate
trampoline code, but use of nested functions is not allowed in
RIOT's code base anyway.
This allows including C headers from C++. It sadly reduced the
diagnostics on C++ code as well, were there warning may make sense as
unintended side effect. We may be able to drop that later on, when more
C APIs are properly wrapped in native C++ APIs, so that C headers do no
longer need to be compatible with C++ compilers.
In an ideal world everyone would just install `gdb-multiarch` and be
happy. However, some MCUs need magic GDB versions sprinkled with
unicorn-stardust-Espressif-patches...
Since there is little reason to have `$(target)-gdb` installed in
addition to `gdb-multiarch` if `gdb-multiarch` would work fine, let's
assume the user wants to use `$(target)-gdb` when present over
`gdb-multiarch`.
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Add USE_MODULE += "stdio_uart_onlcr" to enable it.
This is named after the "onlcr" stty flag, which does the same thing.
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
Model the LoRaWAN integration to GNRC's netif command (ifconfig) as
submodule of it, namely `shell_cmd_gnrc_netif_lorawan`.
This should fix a regression introduced by
https://github.com/RIOT-OS/RIOT/pull/18355
When using podman instead of docker the registry is not set by default.
Docker has a builtin registry default "docker.io". Podman does not have a
default. By specifying the registry explicitly both tools can be used.
