19074: cpu/esp8266: build the SDK bootloader from source r=benpicco a=gschorcht
### Contribution description
This PR is a takeover of PR #17043, which is rebased to the current master and includes some corrections that became necessary after rebasing.
**Copied from description of PR #17043:**
We had four versions of pre-built bootloaders for the esp8266 with different settings of logging and color logging. These bootloaders were manually built from the SDK and shipped with RIOT-OS source code. However there are more settings that affect the bootloader build that are relevant to the app or final board that uses this bootloader. In particular, flash size and flash speed is important for the bootloader to be able to load an app from a large partition table at the fastest speed supported by the board layout and flash chip.
Another example is the UART baudrate of the logging output from the bootloader. The boot ROM will normally start at a baud rate of 74880 (depending on the crystal installed), so it might make sense to keep the UART output at the same speed so we can debug boot modes and bootloader with the same terminal.
This patch builds the `bootloader.bin` file from the ESP8266 SDK source code. The code is built as a module (`esp8266_bootloader`) which at the moment doesn't generate any object code for the application and only produces a `bootloader.bin` file set to the `BOOTLOADER_BIN` make variable for the `esptool.inc.mk` to flash.
The code needs to be compiled and linked with custom rules defined in the module's Makefile since the `bootloader.bin` is its own separate application.
The `BOOTLOADER_BIN` variable is changed from a path relative to the `$(RIOTCPU)/$(CPU)/bin/` directory to be full path. This makes it easier for applications or board to provide their own bootloader binary if needed.
As a result of building the bootloader from source we fixed the issue of having a large partition table.
### Testing procedure
Use following command to flash the application with STDIO UART baudrate of 115200 baud.
```
BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
Connect with a terminal programm of your choice (unfortunatly `picocom` and `socat` don't support a baudrate close to 74880), for example:
```
python -m serial.tools.miniterm /dev/ttyUSB0 74880
```
On reset, the `esp8266-esp-12x` node shows the ROM bootloader log output
```
ets Jan 8 2013,rst cause:2, boot mode:(3,7)
load 0x40100000, len 6152, room 16
tail 8
chksum 0x6f
load 0x3ffe8008, len 24, room 0
tail 8
chksum 0x86
load 0x3ffe8020, len 3408, room 0
tail 0
chksum 0x79
```
as well as the second-stage bootloader built by this PR (`ESP-IDF v3.1-51-g913a06a9ac3`) at 74880 baudrate.
```
I (42) boot: ESP-IDF v3.1-51-g913a06a9ac3 2nd stage bootloader
I (42) boot: compile time 11:25:03
I (42) boot: SPI Speed : 26.7MHz
...
I (151) boot: Loaded app from partition at offset 0x10000
```
The application output is seen as garbage since the `esp8266-esp-12x` uses 115200 as baurate by default.
To see all output at a baudrate of 74880 baud, you can use the following command:
```
CFLAGS='-DSTDIO_UART_BAUDRATE=74880' BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
If the application is built without options, the ROOM bootloader output will be 74880 baud and the second stage bootloader and application output will be 115200 baud.
### Issues/PRs references
Fixes issue #16402
Co-authored-by: iosabi <iosabi@protonmail.com>
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Instead of using a fixed position of the image file in the flash, the variable `FLASHFILE_POS` is used which allows to override the default position of the image in the flash at 0x10000.
The former FLASH_MODE_{DOUT,DIO,QOUT,QIO} defines are replaced by the corresponding CONFIG_FLASHMODE_{DOUT,DIO,QOUT,QIO} and CONFIG_ESPTOOLPY_FLASHMODE_{DOUT,DIO,QOUT,QIO} as used by the ESP-IDF. This is also needed for the migration of defining flash mode in Kconfig.
The directory `dist/tools/esptool` already contains a couple of ESP tools and not only esptool.py. As the location for a couple of ESP related tools, it is more clear to call it `esptools` instead of `esptool`.
The vendor binary libraries of ESP-IDF are provided as a separate GIT repository. These libraries are defined as separate package for two reasons: 1. RIOT packages don't support to clone GIT repositories recursively; 2. ESP-IDF pulls a lot of other GIT repositories that are not needed when it is cloned recursively.
RIOT-OS uses part of Espressif ESP8266 RTOS SDK to build support for
this CPU. The SDK includes some vendor-provided closed source
pre-compiled libraries that we need to modify to adapt to RIOT-OS
usage. This library modifications was done once and uploaded to a fork
of the vendor repository and was provided as an environment variable.
This patch changes two things:
1. It installs the SDK as a RIOT PKG from the new pkg/esp8266_sdk
directory instead of requiring the user to download it separately.
2. It performs the library modifications (symbol renames) on the pkg
Makefile removing the need to use a fork with the modifications applied
and simplifying the SDK update and future modifications.
This change sets the SDK package version (git SHA) to the same one that
our fork was using as a parent in the vendor repository, meaning that
the output libraries are exactly the same as before.
Tested with
```
ESP8266_RTOS_SDK_DIR=/dev/null USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
and verified that the program works. The boot message now includes:
```
ESP8266-RTOS-SDK Version v3.1-51-g913a06a9
```
confirming the SDK version used.
`/dev/null` in the test is just to make sure that no evaluation of
`ESP8266_RTOS_SDK_DIR` in make is affected by the environment variable
value which would be set to the SDK for people who followed the set up
instructions before this change.
Tested the checkout size:
```bash
$ du -hs build/pkg/esp8266_sdk/
124M build/pkg/esp8266_sdk/
```
If esp_idf_heap is not used, implement calloc through a custom wrapper
function on top of malloc to add overflow detection, which is not
present in the newlib forks with xtensa support yet.
This is an implementation of the ESP32 SoftAP mode using the
`esp_wifi_ap` pseudomodule.
Signed-off-by: Jean Pierre Dudey <jeandudey@hotmail.com>
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Flashing an ESP board first requires the creation of a flash image from the ELF file. This is realized in the `preflash` target. However, the `preflash` target only depends on the variable `BUILD_BEFORE_FLASH` but on the ELF file. Therefore, the variable `BUILD_BEFORE_FLASH` must be set to the ELF file to ensure that when using multiple make processes, the compilation of the ELF file is completed before the flash image is created.
