19106: core/lib: Add macros/utils.h header r=aabadie a=maribu
### Contribution description
The macros CONCAT(), MIN(), and MAX() are defined over and over again in RIOT's code base. This de-duplicates the code by moving the macros to a common place.
### Testing procedure
Generated binaries don't change, as this only a de-duplication of macros that doesn't change their definition.
### Issues/PRs references
None
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
The macros CONCAT(), MIN(), and MAX() are defined over and over again in
RIOT's code base. This de-duplicates the code by moving the macros to a
common place.
18632: tests/thread_float: do not overload slow MCUs with IRQs r=kaspar030 a=maribu
### Contribution description
If the regular context switches are triggered too fast, slow MCUs will be able to spent little time on actually progressing in the test. This will scale the IRQ rate with the CPU clock as a crude way too keep load within limits.
### Testing procedure
The unit test should now pass on the Microduino CoreRF
```
$ make BOARD=microduino-corerf AVRDUDE_PROGRAMMER=dragon_jtag -C tests/thread_float flash test
make: Entering directory '/home/maribu/Repos/software/RIOT/tests/thread_float'
Building application "tests_thread_float" for "microduino-corerf" with MCU "atmega128rfa1".
[...]
text data bss dec hex filename
12834 520 3003 16357 3fe5 /home/maribu/Repos/software/RIOT/tests/thread_float/bin/microduino-corerf/tests_thread_float.elf
avrdude -c dragon_jtag -p m128rfa1 -U flash:w:/home/maribu/Repos/software/RIOT/tests/thread_float/bin/microduino-corerf/tests_thread_float.hex
[...]
Welcome to pyterm!
Type '/exit' to exit.
READY
s
START
main(): This is RIOT! (Version: 2022.10-devel-858-g18566-tests/thread_float)
THREADS CREATED
Context switch every 3125 µs
{ "threads": [{ "name": "idle", "stack_size": 192, "stack_used": 88 }]}
{ "threads": [{ "name": "main", "stack_size": 640, "stack_used": 220 }]}
THREAD t1 start
THREAD t2 start
THREAD t3 start
t1: 141.443770
t3: 141.466810
t1: 141.443770
t3: 141.466810
t1: 141.443770
t3: 141.466810
t1: 141.443770
t3: 141.466810
t1: 141.443770
t3: 141.466810
t1: 141.443770
t3: 141.466810
t1: 141.443770
make: Leaving directory '/home/maribu/Repos/software/RIOT/tests/thread_float'
```
(~~Note: The idle thread exiting is something that should never occur. I guess the culprit may be `cpu_switch_context_exit()` messing things up when the main thread exits. But that is not directly related to what this PR aims to fix. Adding a `thread_sleep()` at the end of `main()` does indeed prevent the idle thread from exiting.~~
Update: That's expected. The idle thread stats are printed on exit of the main thread, the idle thread does not actually exit.)
### Issues/PRs references
Fixes https://github.com/RIOT-OS/RIOT/issues/16908 maybe?
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
19030: tests/periph_timer_short_relative_set: improve test r=benpicco a=maribu
### Contribution description
Reduce the number lines to output by only testing for intervals 0..15 to speed up the test.
In addition, run each test case 128 repetitions (it is still faster than before) to give some confidence the short relative set actually succeeded.
### Testing procedure
The test application should consistently fail or succeed, rather than occasionally passing.
### Issues/PRs references
None
19085: makefiles/tests/tests.inc.mk: fix test/available target r=benpicco a=maribu
### Contribution description
`dist/tools/compile_and_test_for_board/compile_and_test_for_board.py` relies on `make test/available` to check if a test if available. However, this so far did not take `TEST_ON_CI_BLACKLIST` and `TEST_ON_CI_WHITELIST` into account, resulting in tests being executed for boards which they are not available. This should fix the issue.
### Testing procedure
#### Expected to fail
```
$ make BOARD=nrf52840dk -C tests/gcoap_fileserver test/available
$ make BOARD=microbit -C tests/log_color test/available
```
(On `master`, they succeed, but fail in this PR.)
#### Expected to succeed
```
$ make BOARD=native -C tests/gcoap_fileserver test/available
$ make BOARD=nrf52840dk -C tests/pkg_edhoc_c test/available
$ make BOARD=nrf52840dk -C tests/log_color test/available
```
(Succeed in both `master` and this PR.)
### Issues/PRs references
None
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
18950: tests/unittests: add unit tests for core_mbox r=benpicco a=maribu
### Contribution description
As the title says
### Testing procedure
The test cases are run on `native` by Murdock anyway.
### Issues/PRs references
Split out of https://github.com/RIOT-OS/RIOT/pull/18949
Co-authored-by: Marian Buschsieweke <marian.buschsieweke@ovgu.de>
If the regular context switches are triggered too fast, slow MCUs
will be able to spent little time on actually progressing in the
test. This will scale the IRQ rate with the CPU clock as a crude way
too keep load within limits.
19037: sys/usb, pkg/tinyusb: move USB board reset from highlevel STDIO to CDC ACM r=dylad a=gschorcht
### Contribution description
The USB board reset function `usb_board_reset_coding_cb` can be used on any CDC-ACM interface, even if the CDC ACM interface is not used as high-level STDIO. Therefore, this PR provides the following changes:
- The call of the board reset function `usb_board_reset_coding_cb` from USBUS stack has been moved from the STDIO CDC ACM implementation to the CDC ACM implementation and is thus a feature of any USBUS CDC ACM interface which does not necessarily have to be used as highlevel STDIO.
- The call of the board reset function `usb_board_reset_coding_cb` from tinyUSB stack been moved from module `tinyusb_stdio_cdc_acm` to module `tinyusb_contrib` and is compiled in if the `tinyusb_class_cdc` module is used together the `tinyusb_device` module. Thus, it is now a feature of the tinyUSB CDC ACM interface, which does not necessarily have to be used as highlevel STDIO.
- The `usb_board_reset` module defines the `usb_board_reset_in_bootloader` function as a weak symbol to be used when reset in bootloader if no real implementation of this function is compiled in and the `riotboot_reset` module is not used. It only prints an error message that the reset in bootloader is not supported. This is necessary if the module `usb_board_reset` is used to be able to restart the board with an application via a USB CDC ACM interface, but the board's bootloader does not support the reset in bootloader feature.
- A test application has been added that either uses the highlevel STDIO `stdio_acm_cdc` or creates a CDC-ACM interface to enable board resets via USB. If the `usbus_dfu` module is used, it also initializes the DFU interface to be able to work together with the `riotboot_dfu` bootloader.
### Testing procedure
1. Use a board with a bootloader that supports the reset in bootloader via USB, but don't use the highlevel STDIO to check that it works with `usbus_cdc_acm`, for example:
```python
USEMODULE=stdio_uart BOARD=arduino-mkr1000 make -C tests/usb_board_reset flash
```
After reset in application with command
```python
stty -F /dev/ttyACM0 raw ispeed 600 ospeed 600 cs8 -cstopb ignpar eol 255 eof 255
```
command `dmesg` should give an output like the following with RIOT's test VID/PID:
```python
dmesg
[1745182.057403] usb 1-4.1.2: new full-speed USB device number 69 using xhci_hcd
[1745182.160386] usb 1-4.1.2: New USB device found, idVendor=1209, idProduct=7d01, bcdDevice= 1.00
[1745182.160390] usb 1-4.1.2: New USB device strings: Mfr=3, Product=2, SerialNumber=4
[1745182.160392] usb 1-4.1.2: Product: arduino-mkr1000
[1745182.160393] usb 1-4.1.2: Manufacturer: RIOT-os.org
[1745182.160395] usb 1-4.1.2: SerialNumber: 6B6C2CA5229020D8
[1745182.170982] cdc_acm 1-4.1.2:1.0: ttyACM0: USB ACM device
```
After reset in bootloader with command
```python
stty -F /dev/ttyACM0 raw ispeed 1200 ospeed 1200 cs8 -cstopb ignpar eol 255 eof 255
```
command `dmesg` should give an output like the following with vendor VID/PID:
```python
[1746220.443792] usb 1-4.1.2: new full-speed USB device number 70 using xhci_hcd
[1746220.544705] usb 1-4.1.2: New USB device found, idVendor=2341, idProduct=024e, bcdDevice= 2.00
[1746220.544708] usb 1-4.1.2: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[1746220.553471] cdc_acm 1-4.1.2:1.0: ttyACM0: USB ACM device
```
2. Test the same as in 1., but this time use the highlevel STDIO to check that there is no regression and it still works with `stdio_cdc_acm`, for example:
```python
BOARD=arduino-mkr1000 make -C tests/usb_board_reset flash
```
3. Use a board that supports `riotboot_dfu` but doesn't use the highlevel STDIO and flash the `riotboot_dfu` bootloader, for example:
```python
BOARD=stm32f429i-disc1 make -C bootloaders/riotboot_dfu flash term
```
Once the bootloader is flashed, command `dfu-util --list` should give something like the following:
```python
Found DFU: [1209:7d02] ver=0100, devnum=14, cfg=1, intf=0, path="1-2", alt=1, name="RIOT-OS Slot 1", serial="6591620BCB270283"
Found DFU: [1209:7d02] ver=0100, devnum=14, cfg=1, intf=0, path="1-2", alt=0, name="RIOT-OS Slot 0", serial="6591620BCB270283"
```
If the output gives only
```python
Found Runtime: [1209:7d00] ver=0100, devnum=123, cfg=1, intf=0, path="1-2", alt=0, name="RIOT-OS bootloader", serial="6591620BCB270283"
```
an application is already running in DFU Runtime mode. Use `dfu-util -e` to restart it in bootloader DFU mode.
Then flash the test application, for example:
```python
FEATURES_REQUIRED=riotboot USEMODULE='usbus_dfu riotboot_reset' \
BOARD=stm32f429i-disc1 make -C tests/usbus_board_reset PROGRAMMER=dfu-util riotboot/flash-slot0
```
Once the test application is flashed, command `dfu-util --list` should give:
```python
Found Runtime: [1209:7d00] ver=0100, devnum=123, cfg=1, intf=0, path="1-2", alt=0, name="RIOT-OS bootloader", serial="6591620BCB270283"
```
Now, use command
```python
stty -F /dev/ttyACM1 raw ispeed 600 ospeed 600 cs8 -cstopb ignpar eol 255 eof 255
```
to restart the board in application. Command `dfu-util --list` should give again the following:
```python
Found Runtime: [1209:7d00] ver=0100, devnum=123, cfg=1, intf=0, path="1-2", alt=0, name="RIOT-OS bootloader", serial="6591620BCB270283"
```
That is, the application is running in DFU Runtime mode. Then use command
```python
stty -F /dev/ttyACM1 raw ispeed 1200 ospeed 1200 cs8 -cstopb ignpar eol 255 eof 255
```
to restart the board in bootloader DFU mode. Command `dfu-util --list` should now give the following:
```python
Found DFU: [1209:7d02] ver=0100, devnum=50, cfg=1, intf=0, path="1-2", alt=1, name="RIOT-OS Slot 1", serial="7D156425A950A8EB"
Found DFU: [1209:7d02] ver=0100, devnum=50, cfg=1, intf=0, path="1-2", alt=0, name="RIOT-OS Slot 0", serial="7D156425A950A8EB"
```
That is, the bootloader is in DFU mode and another application can be flash.
4. After a hard reset of the board under 3., try the commands `reboot` and `bootloader`.
5. To check the same for tinyUSB, use the existing tinyUSB application with a CDC ACM interface and add module `usb_board_reset`, for example:
```python
USEMODULE=usb_board_reset BOARD=stm32f429i-disc1 make -C tests/pkg_tinyusb_cdc_msc flash term
```
After flashing, it should be possible to restart the application with command:
```python
stty -F /dev/ttyACM1 raw ispeed 600 ospeed 600 cs8 -cstopb ignpar eol 255 eof 255
```
When using command
```python
stty -F /dev/ttyACM1 raw ispeed 1200 ospeed 1200 cs8 -cstopb ignpar eol 255 eof 255
```
the following error message should be shown in terminal
```python
[cdc-acm] reset in bootloader is not supported
```
### Issues/PRs references
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
The test application either uses the USBUS highlevel STDIO module `stdio_acm_cdc` or it creates a CDC ACM interface to enable board reset via USBUS CDC ACM. If the `usbus_dfu` module is used together with the `riotboot_dfu` bootloader, it also initializes the USBUS DFU Runtime interface.
Reduce the number lines to output by only testing for intervals 0..15
to speed up the test.
In addition, run each test case 128 repetitions (it is still faster
than before) to give some confidence the short relative set actually
succeeded.
To synchronize communication via shared memory between ISR context and
thread context it is a common misconception that `volatile` is
sufficient. This is however is not the cause and the cause of many
subtle data race bugs. This fixes the issue.
The MPU based stack guard is very unpleased by the stack overflow
happening during the test. The increase in stack size makes the MPU
stack guard happy again.
Previously, the test vectors were encoded into the python test scripts,
converted to base64, and send over to the device under test via stdio.
The application sent back the output after converting it to base64
first, which was read back in by the test script and decoded. Finally,
the test script compared the result with the expected result.
This made the test complex, slow and, flanky, as stdio on interfaces
such as UART has a high bit error rate and some quirks (e.g. the EDBG
UART bridge e.g. in the samr21-xpro dropping bytes when bursts of more
than 64 bytes at a time are send).
This basically rewrites the test to embed the test vectors in the
firmware and do the comparison on the devices. This fixes test failures
on the samr21-xpro, the nRF52840-DK and likely many others. Also, it
is now fast.
It turns out that the fix for the missing terminating newline is not
robust. This hopefully fixes the issue and resolves the following
error message:
In file included from /home/maribu/Repos/software/RIOT/tests/pkg_emlearn/main.c:25:
/home/maribu/Repos/software/RIOT/tests/pkg_emlearn/model.h:7221:36: error: stray '\' in program
7221 | /* fix for no newline at eof */\n
| ^
/home/maribu/Repos/software/RIOT/tests/pkg_emlearn/model.h:7221:38: error: expected ';' before '_Alignas'
7221 | /* fix for no newline at eof */\n
| ^
| ;
The test (at least locally) fails on the long shell line detection in
`master`, as the EDBG UART adapter drops chars when more than 64 bytes
are send at a time. This works around the issue:
- The line buffer in the test is reduced to 60 bytes, so that
overflowing it becomes possible with sending less than 64 bytes.
- The test script is adapted to exceed the shell buffer size by one
byte only (due to linefeed char), rather than significantly.
- Sending more than 64 bytes would result in the linefeed being
dropped by the EDBG adapter and the test failing
Finally, the shell buffer is no longer allocated on the stack and,
hence, the main stack size could be reduced a bit. The test still
passes on the Nucleo-F767ZI which is notorious in failing on tight
stacks due to the MPU stack guard - so the stack size reduction is
expected to work for all boards.
