There seems to be a consensus to go for a more ELF centric approach
whenever possible. J-Link seems to work fine with ELF files, so let's
avoid the extra step to create a bin file from the ELF file prior to
flashing.
While testing examples/micropython I notice that the default of miniterm.py is actually miniterm.
To simplify user setups, this checks for miniterm.py first then falls back to miniterm.
19374: makefiles/boards/stm32: fix DFU_USB_ID handling r=benpicco a=gschorcht
### Contribution description
This PR fixes the still existing problem that an application can't be flashed to a STM32 board if it uses `riotboot_dfu` with default VID/PID (1209:7d02).
In PR #18964 item 1, the problem was already described that an application can't be flashed on a board that is using `riotboot_dfu`. Using for example
```python
FEATURES_REQUIRED+=riotboot USEMODULE+=usbus_dfu make -C examples/saul BOARD=nucleo-f767zi \
PROGRAMMER=dfu-util all riotboot/flash-slot0
```
always leads to error
```python
/home/gs/src/RIOT-Xtensa-ESP.esp-idf-4.4/makefiles/boards/stm32.inc.mk:28: *** DFU_USB_ID is not set. Stop.
/home/gs/src/RIOT-Xtensa-ESP.esp-idf-4.4/makefiles/boot/riotboot.mk:87: recipe for target 'riotboot/bootloader/binfile' failed
```
even if `DFU_USB_ID` variable is set as described in documentation.
```python
FEATURES_REQUIRED+=riotboot USEMODULE+=usbus_dfu make -C examples/saul BOARD=nucleo-f767zi \
PROGRAMMER=dfu-util DFU_USB_ID=1209:7d02 all riotboot/flash-slot0
```
The reason is that the variable `DFU_USB_ID` isn't exported and the check 8dc8bf3567/makefiles/boards/stm32.inc.mk (L27-L29) sees an empty `DFU_USB_ID` variable here. It prevents to use `dfu-util` event though the following `dfu-util.mk` will generate a default value for this variable from the `USB_VID` and `USB_PID` variables if necessary.
Commit 6a76b94c6e0ae97bc743cc90d0811e691a35869e of PR #18964 tried to fix this problem but wasn't merged for any reason.
To fix this problem, the check is completely removed. If a board such as `weact-f4x1cx` uses a DFU boorloader and requires a certain VID/PID combination, board's makefile is responsible to set `DFU_USB_ID` variable.
### Testing procedure
It is not necessary to use a real boad, checking the compilation process is sufficient.
1. Using default VID/PID as described in documentation:
```python
FEATURES_REQUIRED+=riotboot USEMODULE+=usbus_dfu make -C examples/saul BOARD=nucleo-f767zi \
PROGRAMMER=dfu-util all riotboot/flash-slot0
```
can't be compiled without this PR but calls `dfu-util` correctly with this PR using the default VID/PID:
```python
dfu-util --device 1209:7d02 --alt 0 --download examples/saul/bin/nucleo-f767zi/riotboot_files/slot0.1678440536.bin
```
2. Using a VID/PID as described in documentation:
```python
FEATURES_REQUIRED+=riotboot USEMODULE+=usbus_dfu make -C examples/saul BOARD=nucleo-f767zi \
DFU_USB_ID=1209:affe PROGRAMMER=dfu-util all riotboot/flash-slot0
```
can't be compiled without this PR but calls `dfu-util` correctly with this PR using the default VID/PID:
```python
dfu-util --device 1209:affe --alt 0 --download examples/saul/bin/nucleo-f767zi/riotboot_files/slot0.1678440536.bin
```
3. Compiling a board with DFU bootloader
```python
make -C examples/saul flash BOARD=weact-f411ce
```
should still call dfu-util with correct VID/PID:
```python
dfu-util --device 0483:df11 --alt 0 --download /home/gs/src/RIOT-Xtensa-ESP.esp-idf-4.4/examples/saul/bin/weact-f411ce/saul_example.bin --dfuse-address 0x8000000:leave
```
### Issues/PRs references
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
The variable `DFU_USB_ID` is used by `dfu-util` to specify the device to be flashed. The STM32 make system prevents `dfu-util` from being used as programmer if this variable is not set, although `dfu-util.mk` generates a default value for this variable from the `USB_VID` and `USB_PID` variables if necessary. Therefore, checking the `DFU_USB_ID` variable is removed here. If a board requires a specific combination of VID/PID for `dfu_util`, it is responsible for setting the `DFU_USB_ID` variable.
Whenever the module of a peripheral driver, i.e., periph_* should be used, the corresponding entry in the
FEATURES_REQUIRED should be added. Conflicts between these modules are
only checked when this entry is present.
17091: USBUS: Add URB support r=benpicco a=bergzand
### Contribution description
This commit adds support for URBs (USB Request/Response Blocks). These
allow for submitting multi-transfer sized buffers with USBUS handling
the individual usbdev xmits. Multiple URBs can be queued at once for a
single endpoint and USBUS will handle them in the order of submission.
OUT endpoint URBs must always consist of a whole number of full-sized
transfers (N x MaxEndpointSize). They will automatically finish after
the endpoint received a transfer less than the endpoint size.
IN endpoints can be arbitrary-sized and do not have to consist of a
whole number of full-sized transmissions. They support a flag to
indicate that the last transfer in the sequence must be less than a full
sized transfer (USBUS_URB_FLAG_AUTO_ZLP) and this adds a zero length
transfer at the end of the transmissions if the last transfer was equal
to the maximum transfer size.
URBs can be cancelled, but if the URB is already being processed it will
be cancelled after the current transmission within the URB is finished.
If it is still in the queue it will immediately be removed from the
queue.
### Testing procedure
- `tests/usbus_cdc_ecm` should still work. Testing one of the usbdev-supported platform should be sufficient here.
### Issues/PRs references
Needs #17064
18148: sys/flash_utils: helpers to store data in flash r=benpicco a=maribu
### Contribution description
This helpers that allow storing, accessing, and working with data in flash that works for both classical Harvard architectures (which do not map flash also into the address space) as well as modern Harvard architectures and von-Neumann architectures.
With this, `examples/default` again runs on the Arduino Uno / Nano. Since this board is still the "entry kit" for many people to embedded hardware, it would be nice to support it with our default example.
### Testing procedure
`examples/default` should run and work on ATmega boards (especially ATmega328P and ATmega32U4 based boards) as well on all other boards now.
### Issues/PRs references
None
Co-authored-by: Koen Zandberg <koen@bergzand.net>
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
This commit adds support for URBs (USB Request/Response Blocks). These
allow for submitting multi-transfer sized buffers with USBUS handling
the individual usbdev xmits. Multiple URBs can be queued at once for a
single endpoint and USBUS will handle them in the order of submission.
OUT endpoint URBs must always consist of a whole number of full-sized
transfers (N x MaxEndpointSize). They will automatically finish after
the endpoint received a transfer less than the endpoint size.
IN endpoints can be arbitrary-sized and do not have to consist of a
whole number of full-sized transmissions. They support a flag to
indicate that the last transfer in the sequence must be less than a full
sized transfer (USBUS_URB_FLAG_AUTO_ZLP) and this adds a zero length
transfer at the end of the transmissions if the last transfer was equal
to the maximum transfer size.
URBs can be cancelled, but if the URB is already being processed it will
be cancelled after the current transmission within the URB is finished.
If it is still in the queue it will immediately be removed from the
queue.
18682: pkg/lwext4: add lightweight implementation of the ext2/3/4 filesystem r=benpicco a=benpicco
Co-authored-by: Benjamin Valentin <benjamin.valentin@ml-pa.com>
Boards with an integrated debugger/programmer that also provides the
serial as UART <--> USB adapter, the TTY serial matches the serial of
the programmer.
This adapts the `serial.inc.mk` to set the `DEBUG_ADAPTER_ID` to the
TTY serial if (and only if) `MOST_RECENT_PORT` *and*
`DEBUG_ADAPTER_ID_IS_TTY_SERIAL` both have a value of `1`. Boards with
an integrated programmer are expected to set
`DEBUG_ADAPTER_ID_IS_TTY_SERIAL` to `1` in their `Makefile.include`.
The previous servo driver didn't provide any benefit over using PWM
directly, as users controlled the servo in terms of PWM duty cycles.
This changes the interface to provide a high level interface that
abstracts the gory PWM details.
In addition, a SAUL layer and auto-initialization is provided.
Co-authored-by: benpicco <benpicco@googlemail.com>
Before the build system used something like
echo "int main(){} void _exit(int n) {(void)n;while(1);}" | LC_ALL=C $(LINK) -xc - -o /dev/null -lc -Wall -Wextra -pedantic <FLAGS_TO_TEST>
to check for flags supported by the compiler (used as linker frontend).
This however no longer works with newlib 4.3.0, which requires the
symbols `_sbrk_r`, `_close_r`, `_lseek_r`, `_read_r`, and `_write_r` to
be present. Instead, now a new file `minimal_linkable.c` was added that
adds all the missing symbols and is used in the linker feature tests
instead.
19057: fuzzing: Add uri_parser setup r=benpicco a=Teufelchen1
Hello!
### Contribution description
This PR is a replacement for PR #18802
In this contribution:
* The variable `AFL_FLAGS` is renamed to `FLAGS_FOR_AFL` because AFL is always complaining that `AFL_FLAGS` is not a valid env var for it. While this is not a bug nor an issue, I found it to be annoying.
* A generic input reader is added to simplify building a test harness
* The usage of this reader is demonstrated by adding a harness for fuzzing the uri_parser
(needs squashing after review)
### Testing procedure
Go to `fuzzing/uri_parser` and run `make all-asan` and `make fuzz` to get some action going.
Also mildly interesting: `./dist/tools/compile_test/compile_like_murdock.py -b native -a fuzzing/uri_parser`
### Issues/PRs references
The original PR #18802 is replaced because the generic input reader is present in both PRs but this PoC harness is much simpler.
19151: examples/gcoap: Fix shell parameter validation r=benpicco a=maribu
### Contribution description
Executing the shell command with an URI-Path that doesn't start with a slash results in an assertion error while composing the client side message. This is suboptimal user experience, so add an explicit check for a valid URI-Path and a dedicated error message.
### Testing procedure
#### In `master`
```
$ make BOARD=microbit-v2 -C examples/gcoap flash term
[...]
2023-01-15 22:23:32,512 # coap get [::1] /.well-known/core
2023-01-15 22:23:32,516 # gcoap_cli: sending msg ID 52272, 23 bytes
2023-01-15 22:23:32,520 # gcoap: response Success, code 2.05, 46 bytes
2023-01-15 22:23:32,524 # </cli/stats>;ct=0;rt="count";obs,</riot/board>
> coap get [::1] foo
2023-01-15 22:23:34,763 # coap get [::1] foo
2023-01-15 22:23:34,763 # 2329
2023-01-15 22:23:34,765 # *** RIOT kernel panic:
2023-01-15 22:23:34,767 # FAILED ASSERTION.
2023-01-15 22:23:34,767 #
2023-01-15 22:23:34,775 # pid | name | state Q | pri | stack ( used) ( free) | base addr | current
2023-01-15 22:23:34,784 # - | isr_stack | - - | - | 512 ( 200) ( 312) | 0x20000000 | 0x200001c8
2023-01-15 22:23:34,793 # 1 | main | running Q | 7 | 1536 ( 1072) ( 464) | 0x200006c0 | 0x2000095c
2023-01-15 22:23:34,802 # 2 | 6lo | bl rx _ | 3 | 1024 ( 328) ( 696) | 0x200036c0 | 0x200039c4
2023-01-15 22:23:34,810 # 3 | ipv6 | bl rx _ | 4 | 1024 ( 460) ( 564) | 0x20001294 | 0x20001574
2023-01-15 22:23:34,819 # 4 | udp | bl rx _ | 5 | 512 ( 300) ( 212) | 0x20003e98 | 0x20003f9c
2023-01-15 22:23:34,828 # 5 | coap | bl anyfl _ | 6 | 1112 ( 704) ( 408) | 0x20000e38 | 0x200011c4
2023-01-15 22:23:34,837 # 6 | nrf802154 | bl anyfl _ | 2 | 896 ( 288) ( 608) | 0x20001a90 | 0x20001d54
2023-01-15 22:23:34,843 # | SUM | | | 6616 ( 3352) ( 3264)
2023-01-15 22:23:34,843 #
2023-01-15 22:23:34,844 # *** halted.
2023-01-15 22:23:34,844 #
```
#### This PR
```
$ make BOARD=microbit-v2 -C examples/gcoap flash term
[...]
make: Entering directory '/home/maribu/Repos/software/RIOT/examples/gcoap'
/home/maribu/Repos/software/RIOT/dist/tools/pyterm/pyterm -p "/dev/ttyACM0" -b "115200"
2023-01-15 22:22:27,842 # Connect to serial port /dev/ttyACM0
Welcome to pyterm!
Type '/exit' to exit.
coap get [::1] /.well-known/core
2023-01-15 22:22:40,042 # coap get [::1] /.well-known/core
2023-01-15 22:22:40,046 # gcoap_cli: sending msg ID 25182, 23 bytes
2023-01-15 22:22:40,050 # gcoap: response Success, code 2.05, 46 bytes
2023-01-15 22:22:40,054 # </cli/stats>;ct=0;rt="count";obs,</riot/board>
> coap get [::1] foo
2023-01-15 22:22:43,858 # coap get [::1] foo
2023-01-15 22:22:43,862 # ERROR: URI-Path must start with a "/"
2023-01-15 22:22:43,866 # usage: coap <get|post|put|ping|proxy|info>
```
### Issues/PRs references
None
Co-authored-by: Teufelchen1 <bennet.blischke@haw-hamburg.de>
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
19010: bootloaders/riotboot: add tinyUSB DFU support r=benpicco a=gschorcht
### Contribution description
This PR provides
- the tinyUSB DFU and DFU Runtime support and
- the `riotboot_tinyusb_dfu` bootloader that uses the tinyUSB DFU mode to flash new application images.
~This PR includes PR #18983 for now to be compilable.~
### Testing procedure
1. Use any board that supports the `riotboot´ and `tinyusb_device` features and flash the bootloader first, for example
```
BOARD=nucleo-f767zi make -C bootloaders/riotboot_tinyusb_dfu flash
```
and check that the `riotboot_tinyusb_dfu` bootloader is in DFU mode:
```
dfu-util --list
```
3. Flash a first application using the following command:
```
FEATURES_REQUIRED=riotboot USEMODULE=tinyusb_dfu BOARD=nucleo-f767zi \
make -C tests/saul PROGRAMMER=dfu-util riotboot/flash-slot0
```
and check that the application starts and is seen as upgradable:
```
dfu-util --list
```
4. Restart the node in bootloader DFU mode by:
```
dfu-util -e
```
Flash a second application, for example
```
FEATURES_REQUIRED=riotboot USEMODULE=tinyusb_dfu BOARD=nucleo-f767zi \
make -C tests/shell PROGRAMMER=dfu-util riotboot/flash-slot1
```
and check that the second application starts and is seen as upgradable:
```
dfu-util --list
```
### Issues/PRs references
~Depends on PR #18983~
19149: SECURITY: Describe that declassification is an option r=benpicco a=chrysn
### Contribution description
Our security policy does not contain provisions for the case when what is reported is not what we consider an actual security issue. As it is described now, everything reported through security@ would go through the full treatment, including a point release.
I'm not sure it belongs into the text itself (as it's more about how security reporters interact with the project than internals), but declassification should IMO be backed at least by 3 maintainers, and no strong NACK.
### Issues/PRs references
#19141 followed that procedure after some chat on it on the maintainers channel. (In the discussion, I proposed declassification, with 2.5 people supporting it and one "I was about to, but can we be sure nobody is using it?" voice).
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Co-authored-by: chrysn <chrysn@fsfe.org>
With b30efeeb65 a warning was introduced when using `make term` without
the proper toolchain installed (e.g. when using BUILD_IN_DOCKER, but
`term` outside of the docker). This removes this warning
19050: boards/common/cc26xx cc13xx: clean up and fix flash configs r=benpicco a=maribu
### Contribution description
- Add support for XDS110 debugger via `OPENOCD_DEBUG_ADAPTER=xds110`
- Clean up OpenOCD configs in `boards/common/cc26xx_cc13xx`
- No longer hardcode the debugger to xds110, but use `OPENOCD_DEBUG_ADATER ?= xds110`
- Add support for cc13x0, cc13x2, cc26x0
- `boards/cc2650*`: drop custom OpenOCD config in favor of shared one
- add variables needed to support flashing with `PROGRAMMER=jlink`
- allow specifying a custom OpenOCD command to bring the device to a halt state, as the default `reset halt` (which causes a second reset) is causing issues with the ICEPick JTAG routers in the CC26xx - CC13xx devices
- Use `halt` instead of `reset halt` for CC26xx / CC13xx boards in OpenOCD to avoid issues in flashing
### Testing procedure
```
make BOARD=cc2650-launchpad -C examples/default flash
```
Should now work. The same should still work for other cc26xx cc13xx boards.
### Issues/PRs references
Partially fixes: https://github.com/RIOT-OS/RIOT/issues/18750
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
The Makefile function `version_is_greater_or_equal` is used to check if
a version of GNU Make is at least the required one. However, it has
the built-in assumption the version numbers have to format x.y.z,
but Alpine Linux currently ships GNU Make 4.4. This results in
`$(call _pad_number,3,)` which runs `printf '$03d' ''` in the shell,
which is not valid.
This fixes the issue by making `_pad_number` more robust by fall back to
printing `0` with the given padding, if the number given to print is
empty.
Typically, OpenOCD is already performing a reset on connect. A
`reset halt` to bring the target to a `halt` state for flashing will
result in the device going through a second reset cycle. This can be
problematic with some device, such as the CC26xx MCUs. For these
devices, an `OPENOCD_CMD_RESET_HALT := -c 'halt'` will avoid the second
reset that is causing the issues.
This adds the configuration to allow choosing the XDS110 used in
cc13xx-launchpad and cc26xx-launchpad boards via the
`OPENOCD_DEBUG_ADAPTER` variable.
Update the list of target triples to match
`makefiles/arch/riscv.inc.mk`. This fixes compilation with toolchains
other than the obsolete toolchain that uses the incorrect
`riscv-none-embed` triple.
19030: tests/periph_timer_short_relative_set: improve test r=benpicco a=maribu
### Contribution description
Reduce the number lines to output by only testing for intervals 0..15 to speed up the test.
In addition, run each test case 128 repetitions (it is still faster than before) to give some confidence the short relative set actually succeeded.
### Testing procedure
The test application should consistently fail or succeed, rather than occasionally passing.
### Issues/PRs references
None
19085: makefiles/tests/tests.inc.mk: fix test/available target r=benpicco a=maribu
### Contribution description
`dist/tools/compile_and_test_for_board/compile_and_test_for_board.py` relies on `make test/available` to check if a test if available. However, this so far did not take `TEST_ON_CI_BLACKLIST` and `TEST_ON_CI_WHITELIST` into account, resulting in tests being executed for boards which they are not available. This should fix the issue.
### Testing procedure
#### Expected to fail
```
$ make BOARD=nrf52840dk -C tests/gcoap_fileserver test/available
$ make BOARD=microbit -C tests/log_color test/available
```
(On `master`, they succeed, but fail in this PR.)
#### Expected to succeed
```
$ make BOARD=native -C tests/gcoap_fileserver test/available
$ make BOARD=nrf52840dk -C tests/pkg_edhoc_c test/available
$ make BOARD=nrf52840dk -C tests/log_color test/available
```
(Succeed in both `master` and this PR.)
### Issues/PRs references
None
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
