Log outputs generated by binary ESP32 SDK libraries are mapped to the ESP32's log module which supports colored and tagged log outpus. Tagged log outputs from SDK libraries are handled accordingly.
The implementation of `log_module` for ESP32 was changed from functions to a macro-based implementation to be able to use the bunch of macros for colored and tagget log output generation.
Renames crypto functions of ESP32 SDK in vendor code to resolve the conflicts between `wpa-supplicant` crypto functions and RIOT's `crypto` and `hashes` modules.
To resolve the conflicts between `wpa-supplicant` crypto functions (part of the ESP32 SDK) and RIOT's `crypto` and `hashes` modules, the crypto function headers from ESP32 SDK are added to vendor code and the crypto functions are renamed using the prefix `wpa_`.
A number of tests insist that the number of thread priority levels is 16. However, when using the WiFi interface, a number of high priority threads are required to handle the WiFi hardware. In this case, the number of thread priority levels must be 32. Solves the problem of tests `tests/shell`.
Removes the dependency of the module riot_freertos from module xtimer. This avoids that xtimer is used even if it is not really needed which in turn occupies the first timer device and tests/periph_timer fails.
ESP32 log output was always tagged with additional information by default. The tag consists the type of the log message, the system time in ms, and the module or function in which the log message is generated. By introducing module `esp_log_tagged`, these additional information are disabled by default and can be enabled by using module `esp_log_tagged`.
Log module of ESP32 supports colored log outputs when module `esp_log_color` is enabled. The generation of colored log outputs is realized by a extending the bunch of macros with an additional letter indicating the type of log message,
For the implementation of the colored log output, two versions of the bootloader are introduced, one version with colored log output and one version without colors.
If `SPI_CS_UNDEF` is given as the `cs` parameter, CS pin must not be handled by the driver. Furthermore, if `cont` parameter is true, CS pin must not be disabled at the end of one transfer.
The ADC2 controller of the ESP32 is used by the WiFi module. The GPIOs connected to the ADC2 controller cannot be used as ADC channels if the WiFi module is enabled. This is clarified by a note in the documentation.
Thin archives also cause a boot loop when using the flash module.
To prevent further surprises, disable thin archives unconditionally
until the cause for this behaviour is known.
For a yet unknown reason, both esp8266 and esp32 get stuck in a
reboot loop when thin archives are used.
As a workaround, disable thin archives for now if esp_wifi is used.
fixes#12258
* CPU files should already have 'CPU' defined by the board.
* Do not conditionally define CPU as it is not needed.
This is part of cleanup prior to moving the CPU/CPU_MODEL to
Makefile.features.
Instead of having a send buffer as member `esp_wifi` netdev, a local variable is used now as send buffer. This avoids the need for a locking mechanism and reduces the risk of deadlocks.
Receive call back function `_esp_wifi_rx_cb` is called from WiFi hardware driver with a pointer to a frame buffer that is allocated in the WiFi hardware driver. This frame buffer was freed immediately after copying its content to a single local receive buffer of the `esp_wifi` netdev. The local receive buffer remained occupied until the protocol stack had processed it. Further incoming packets were dropped. However, very often a number of subsequent WiFi frames are received at the same time before the first one is processed completely. Having the single local receive buffer to hold only one received frame, led to a number of lost packets, even at low network load. Therefore, a ringbuffer of rx_buf elements was introduced which doesn't store the frames directly but only references to the frame buffers allocated in WiFi hardware driver. Since there is enough memory to hold several frames, the frames buffers allocated in WiFi hardware driver aren't freed immediatly any longer but are kept until the frame is processed by the protocol stack. This results in a much less loss rate of packets.
Events of different type can be pending at the same time. Therefore it is not possible to use ascending identifiers for the presence of a pending event. Rather, each event type has to be represented by one bit. Thes bits ORed identify all types of pending events. In the esp_wifi_isr function all pending events are then handled in one call. Otherwise, some events might be lost.
Use the -gz option to compress ELF sections containing DWARF information.
This saves around 50% of disk space, without any side effects.
See https://gcc.gnu.org/onlinedocs/gcc-9.2.0/gcc/Debugging-Options.html#Debugging-Options
for more infomation on this option.
Some platforms have an outdated toolchain that does not support -gz so
the flag is blacklisted there. Even then, the results are quite impressive.
I used @cladmi's `buildtest` branch (https://github.com/cladmi/RIOT/tree/wip/du/buildtest)
with this change and compiled the `examples/default` application:
```
$ BUILD_IN_DOCKER=1 DOCKER="sudo docker" make -C examples/default buildtest-indocker
```
The size was obtained with:
```
$ find output -name "*.bin.bindirsize" -type f -exec tail -n1 '{}' \; | cut -f 1 | awk '{s+=$1} END {printf "%.0f", s}'
```
Results:
- Vanilla: 10328112 KB (~10GB).
- with -gz: 4982788 KB (~5GB).
This was inspired by #8496.
Module `newlib` is now used by default. Therefore, the separation of initialization of ctors and the newlibc is not needed any longer. Instead of calling `do_global_ctors` and `_init` separately, `__libc_init_array` is called. Explicit function `do_global_ctors` is removed.
While deleting multiple sectors in flash, interrupts were disabled over the whole time. Thus, deleting the entire flash led to the triggering of the watchdog timer and thus to a restart. Therefore, the interrupts and the cache are disabled only for the time of deleting a single sector. The same problem occurred for read and write large data sets.
Initializing the stdio file descriptors in global reent structure with newlib fake stdio file descriptors led to the problem that newlib stdio functions printf and puts were not working since they can't operate on these fake stdio file descriptors. Therefore, this initialization was removed. Now, the real stdio file descriptors as created automatically by newlib are used. Specific functions `printf`, `puts`, `getchar`and `putchar` are not required any longer and are removed now.
Modules newlib and newlib_syscalls_default are now used by default. Conditional compilations for MODULE_NEWLIB_SYSCALLS_DEFAULT as well as alternative code are removed completely.
printf and puts used ets_printf before. Unfortunately, ets_printf adds an additional \r for each \n which is not consistent with other RIOT platforms. As a result some automatic tests failed. Therefore, both functions write now character-wise directly to the UART interface.
Using a mutex for critical section handling with portENTER_CRITICAL and portEXIT_CRITICAL does not work for RIOT, as this function can also be called in the interrupt context. Therefore, the given mutex is not used. Instead, the basic default FreeRTOS mechanism for critical sections is used by simply disabling interrupts. Since context switches for the ESP32 are also based on interrupts, there is no possibility that another thread will enter the critical section once the interrupts are disabled.
The option value length of Ethernet addresses can be more than 6 byte in lwIP. Therefore, the max_len parameter is check to be greater than or equal to ETHERNET_ADDR_LEN.
Module esp_idf_heap is enabled in cpu/esp32/Makefile.dep depending on other modules. Since cpu/esp32/Makefile.dep is read after cpu/esp32/Makefile.include, the conditional definition of the linker options for the wrapper functions had to be moved from cpu/esp32/Makefile.include to cpu/esp32/Makefile.dep.
If module esp_idf_heap is used, the memory management functions _malloc_r, _realloc_r, _calloc_r and _free_r have to be overridden by wrapper functions to use the heap_* functions of module _esp_idf_heap. However, this can lead to multiple symbol errors for these functions for some applications. To solve this symbol conflict, _malloc_r, _realloc_r, _calloc_r and _free_r functions are renamed to __wrap_* and the linker options are extended by -Wl,-wrap option when module esp_idf_heap is used.
When standard C libraries are added to BASELIBS to group them together with all other modules, there are multiple definitions for the putchar function. The one that is defined writing to the UART as standard output and the one that is provided by the standard C libraries. To solve this symbol conflict, putchar and getchar functions that use the UART as standard output/input are renamed to __wrap_putchar and __wrap_getchar and the linker options are extended by -Wl,-wrap option.
When linking an application, symbol pthread_setcancelstate is not known in standard C libraries, even if the pthread module is linked. This is because the pthread module is grouped with all other modules, but not with the default C libraries when they are added to LINK_FLAGS. Therefore, standard C libraries have to be added also to BASELIBS to group them with all other modules.
Fixes the problem that the compilation of an applications can throw unknown symbol errors for functions that aren't use at all. Thus, it is possible to remove the warning for unknown symbols and the compilation can abort if there are real unknown symbols.
Although it isn't explicitly specified in API, gpio_read should return the last written output value for output ports. Since the handling of inputs and outputs is strictly separated by several registers in ESP32, gpio_read returned always the initial value of the input register. Therefore, a case distinction had to make. While for input ports the real value has to be read from the input register, the last written value for the output port has to be read from the output register.
Rational: the periph_common module is required by (most) other periph drivers
and also during startup of the CPU/MCU to run periph_init. The latter is only
required if other periph drivers are used, hence periph_common should be a
depency of periph_* modules and *not* of the CPU/MCU. This PR fixes that
by making periph_common a depency of periph_* and removing the explicit
include in the CPU/MCU implementation.
PREFLASHER/PREFFLAGS/FLASHDEPS are evaluated by the main Makefile.include.
Their value does not need to be exported.
Testing
-------
`git diff --word-diff` only reports `export` being removed.
`git show --stat` reports `16 insertions(+), 16 deletions(-)`
Which is the same amount as lines that where matching
`export[[:blank::]]\+VARIABLE` plus the newline that is said to have
changed.
FLASHER and FFLAGS are evaluated by the main Makefile.include or by file
included by it. Their value does not need to be exported.
This will also prevent evaluating 'PORT' for FFLAGS when not needed.
Testing
-------
`git diff --word-diff` only reports `export` being removed.
`git show --stat` reports `84 insertions(+), 84 deletions(-)`
Which is the same amount as lines that where matching
`export[[:blank::]]\+VARIABLE`.
UART devices are now configured using static array in header files instead of static variables in implementation to be able to define UART_NUMOF using the size of the array instead of a variable.
SPI devices are now configured using static array in header files instead of static variables in implementation to be able to define SPI_NUMOF using the size of the array instead of a variable.
I2C devices are now configured using static array in header files instead of static variables in implementation to be able to define I2C_NUMOF using the size of the array instead of a variable.
DAC pins are now configured using static arrays in header files instead of static variables in implementation to be able to define DAC_NUMOF using the size of these arrays instead of a variable.
ADC pins are now configured using static arrays in header files instead of static variables in implementation to be able to define ADC_NUMOF using the size of these arrays instead of a variable.
Functions that are used by ADC and DAC peripherals are moved to a new submodule periph_adc_ctrl. This is necessary to compile separate submodules for ADC and DAC.
`top_of_stack` isn't aligned down to the previous 16 byte aligned address. Furthermore, `top_of_stack` as well as `XT_CP_SIZE` are used unaligned in `cpu/esp_common/vendor/xtensa/portasm.S` in the address computation for the coprocessor save area, .
Aligning pointer `p` down to the previous 16 byte aligned address results in a wrong address of the coprocessor save area during the initialization of the thread context. This leads to wrong values and wrong positions of these values in the coprocessor save area in inital thread context.
Since ESP8266 doesn't have a coprocessor, this bug affects only ESP32.
During the flash step esptool.py gives the following warning:
WARNING: Flash size arguments in megabits like '16m' are deprecated.
Please use the equivalent size '2MB'.
Megabit arguments may be removed in a future release.
esptool.py v2.7-dev
This patch replaces '16m' with '2MB' to enable future compatibility.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Fix of #11354: Function '_write_r' of ESP32's newlibc does not write the output of function 'write(STDIO_FILENO, ...)' to the UART interface. To fix this problem, module 'newlib_syscalls_default' is now used by default. Function '_write_r' of module 'newlib_syscalls_default' uses 'stdio_write' which in turn uses 'uart_write' if module 'stdio_uart' is used which is now the default case for ESP32.
xSemaphoreTakeRecursive() returned before the fix: pdFALSE(equal to pdFAIL) when the call was successful in obtaining the semaphore
and pdTRUE(equal to pdPASS) when the call did not successfully obtain the semaphore.
According to freertos documentation:
"pdPASS Returned only if the call to xSemaphoreTakeRecursive() was successful in obtaining the semaphore"
"pdFAIL Returned if the call to xSemaphoreTakeRecursive() did not successfully obtain the semaphore."
Fixed it to return the correct value.
xSemaphoreTake() returned before the fix: pdFALSE(equal to pdFAIL) when the call was successful in obtaining the semaphore
and pdTRUE(equal to pdPASS) when the call did not successfully obtain the semaphore.
According to freertos documentation:
"pdPASS Returned only if the call to xSemaphoreTake() was successful in obtaining the semaphore"
"pdFAIL Returned if the call to xSemaphoreTake() did not successfully obtain the semaphore."
Fixed it to return the correct value.
The GPIO for RX has to be initialized as input before the GPIO for TX can be initialized as output. Otherwise it could lead to creash if RX GPIO was used as output before.
Function uart_set_baudrate which is only used internally was made static and renamed to _uart_set_baudrate to indicate that it is an internal function. Furthermore, an additional waiting for flushed TX FIFO added. The reconfiguration is now handled as critical section.
An additional _ for static symbols has been added by mistake and should be removed. This will make future merging with the reimplementation of ESP8266 easier.
The build system contains several instances of
INCLUDES += -I$(RIOTBASE)/sys/posix/include
This is bypassing the module management system, by directly accesing
headers without depending on a module. The module is the posix module.
That line is also added when one of the posix_* modules is requested.
According to the docs, the posix module provides headers only, but in
reality there is also inet.c.
This patch:
- Moves `inet.c` into `posix_inet`, leaving `posix` as a headers-only
module.
- Rename `posix` as `posix_headers` to make it clear the module only
includes headers.
- Makes `posix_*` modules depend on `posix_headers`, thus removing the
explicit `INCLUDES+=...` in `sys/Makefile.include`.
- Ocurrences of `INCLUDES+=...` are replaced by an explicit dependency
on `posix_headers`.
The default macros GPIO_PIN and GPIO_UNDEF do not have to be overridden. The GPIO_PIN macro definition was even wrong for 40 GPIOs without splitting into ports, even if that did not lead to erroneous behavior.
Fixes sporadic blocking of the wifi thread in esp_wifi_recv_cb function under heavy network load conditions when frames are coming in faster than they can be processed. Since esp_wifi_recv_cb function is not executed in interrupt context, the msg_send function used for ISR event can block when the message queue is full. With this change esp_wifi can be flooded with icmpv6 packets of maximum size without any problems over hours.
RX callback function should be register when WiFi has been connected to AP successfully and should be unregistered when WiFi disconnects from AP. Therefore, esp_wifi_internal_reg_rxcb is called now in event handler on event SYSTEM_EVENT_STA_CONNECTED. It is reset now on event SYSTEM_EVENT_STA_DISCONNECTED.
Before function wifi_connect is executed, starting the WiFi driver should have been finished. This is indicated by the WiFi driver by sending event SYSTEM_EVENT_STA_START. Function wifi_connect is moved therefore to the event handler for SYSTEM_EVENT_STA_START.
The size of received and transmitted frames was stored in an uint8_t, which did not allow to process frames larger than 255 octets. However, WiFi has an MTU of 1500 octets.
Adds a memset function `system_secure_memset` which cannot be optimized out by the compiler. It uses the libsodium approach of weak symbols. Function system_secure_memset calls the standard memset. Calling an empty function declared with weak attribute after the memset call, prevents the compiler to optimize it out. The overhead is only one function call.
Although ESP32 has four SPI controllers, only two of them can be effectively used (HSP and VSPI). The third one (FSPI) is used for external memory such as flash and PSRAM and can not be used for peripherals. FSPI is therefore removed from the API. In addition, the SPI0_DEV and SPI1_DEV configuration parameters are renamed SPI0_CTRL and SPI1_CTRL to better describe what they define and to avoid confusion with SPI_DEV (0) and SPI_DEV (1).
Although ESP32 has four SPI controllers, only two of them can be effectively used (HSP and VSPI). The third one (FSPI) is used for external memory such as flash and PSRAM and can not be used for peripherals. FSPI is therefore removed from the API. In addition, the SPI0_DEV and SPI1_DEV configuration parameters are renamed SPI0_CTRL and SPI1_CTRL to better describe what they define and to avoid confusion with SPI_DEV (0) and SPI_DEV (1).
cpu/esp32/include/periph_cpu.h overrides the default definition of adc_res_t from periph/adc with a definition which contains only four resolution, two new resolutions and two resolutions defined by the default definition of adc_res_t. This gives compilation errors if an application uses other resolutions. According to the documentation, adc_sample should return -1 if the resolution is not supported. All other CPUs override adc_res_t either to add new resolutions or to mark resolutions as unsupported. But they all allow to use them at the interface. Therefore, esp32 overrides now the definition of adc_res_t with all resolutions that are defined by the default definition of adc_res_t and new platform specific resolutions. It returns -1 if a resolution is used in adc_sample that is not supported.
This commit adds the ESP32 vendor libraries for WLAN to the BASELIBS variable. This avoids having to define an additional archive group in the LINKGFLAGS variable which contains these vendor libraries and again RIOT module archive files with the symbols that are refered by these vendor libraries.
Some ESP32 boards (like my SparkFun ESP32 Thing) have a main clock
crystal that runs at 26MHz, not 40MHz. RIOT appears to assume 40MHz.
The mismatch causes the UART to not sync properly, resulting in
garbage written to the terminal instead of log output.
I’ve added:
* A new board configuration constant ESP32_XTAL_FREQ that defaults
to 40, but can be overridden by a board def or at build time to
force a specific value (i.e. 26).
* Some code spliced into system_clk_init() to check this constant and
call rtc_clk_init() to set the correct frequency.
* A copy of the rtf_clk_init() function from the ESP-IDF sources.
Fixes#10272
The problem seemed to be a pipelining problem of write and read instructions when swapping the context. An isync instruction when returning from a context switch solves the potential pipelining problem.
Reason for the problem was that tast_exit function in thread_arch.c tried to release the thread a second time although it was already released in sched_task_exit. A simple check whether the thread is already released (sched_active_thread == NULL) solved the problem.
Xtensa newlib version requires pthread_setcancelstate as symbol. Therefore, the module pthread was always used, which in turn requires the module xtimer. The xtimer module, however, uses TIMER_DEV(0). Therefore, tests/timers failed for TIMER_DEV(0).
