19074: cpu/esp8266: build the SDK bootloader from source r=benpicco a=gschorcht
### Contribution description
This PR is a takeover of PR #17043, which is rebased to the current master and includes some corrections that became necessary after rebasing.
**Copied from description of PR #17043:**
We had four versions of pre-built bootloaders for the esp8266 with different settings of logging and color logging. These bootloaders were manually built from the SDK and shipped with RIOT-OS source code. However there are more settings that affect the bootloader build that are relevant to the app or final board that uses this bootloader. In particular, flash size and flash speed is important for the bootloader to be able to load an app from a large partition table at the fastest speed supported by the board layout and flash chip.
Another example is the UART baudrate of the logging output from the bootloader. The boot ROM will normally start at a baud rate of 74880 (depending on the crystal installed), so it might make sense to keep the UART output at the same speed so we can debug boot modes and bootloader with the same terminal.
This patch builds the `bootloader.bin` file from the ESP8266 SDK source code. The code is built as a module (`esp8266_bootloader`) which at the moment doesn't generate any object code for the application and only produces a `bootloader.bin` file set to the `BOOTLOADER_BIN` make variable for the `esptool.inc.mk` to flash.
The code needs to be compiled and linked with custom rules defined in the module's Makefile since the `bootloader.bin` is its own separate application.
The `BOOTLOADER_BIN` variable is changed from a path relative to the `$(RIOTCPU)/$(CPU)/bin/` directory to be full path. This makes it easier for applications or board to provide their own bootloader binary if needed.
As a result of building the bootloader from source we fixed the issue of having a large partition table.
### Testing procedure
Use following command to flash the application with STDIO UART baudrate of 115200 baud.
```
BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
Connect with a terminal programm of your choice (unfortunatly `picocom` and `socat` don't support a baudrate close to 74880), for example:
```
python -m serial.tools.miniterm /dev/ttyUSB0 74880
```
On reset, the `esp8266-esp-12x` node shows the ROM bootloader log output
```
ets Jan 8 2013,rst cause:2, boot mode:(3,7)
load 0x40100000, len 6152, room 16
tail 8
chksum 0x6f
load 0x3ffe8008, len 24, room 0
tail 8
chksum 0x86
load 0x3ffe8020, len 3408, room 0
tail 0
chksum 0x79
```
as well as the second-stage bootloader built by this PR (`ESP-IDF v3.1-51-g913a06a9ac3`) at 74880 baudrate.
```
I (42) boot: ESP-IDF v3.1-51-g913a06a9ac3 2nd stage bootloader
I (42) boot: compile time 11:25:03
I (42) boot: SPI Speed : 26.7MHz
...
I (151) boot: Loaded app from partition at offset 0x10000
```
The application output is seen as garbage since the `esp8266-esp-12x` uses 115200 as baurate by default.
To see all output at a baudrate of 74880 baud, you can use the following command:
```
CFLAGS='-DSTDIO_UART_BAUDRATE=74880' BAUD=74880 USEMODULE=esp_log_startup make -C tests/shell BOARD=esp8266-esp-12x flash
```
If the application is built without options, the ROOM bootloader output will be 74880 baud and the second stage bootloader and application output will be 115200 baud.
### Issues/PRs references
Fixes issue #16402
Co-authored-by: iosabi <iosabi@protonmail.com>
Co-authored-by: Gunar Schorcht <gunar@schorcht.net>
Instead of using a fixed position of the image file in the flash, the variable `FLASHFILE_POS` is used which allows to override the default position of the image in the flash at 0x10000.
We had four versions of pre-built bootloaders for the esp8266 with
different settings of logging and color logging. These bootloaders were
manually built from the SDK and shipped with RIOT-OS source code.
However there are more settings that affect the bootloader build that
are relevant to the app or final board that uses this bootloader. In
particular, flash size and flash speed is important for the bootloader
to be able to load an app from a large partition table at the fastest
speed supported by the board layout and flash chip.
Another example is the UART baudrate of the logging output from the
bootloader. The boot ROM will normally start at a baud rate of 74880
(depending on the crystal installed), so it might make sense to keep
the UART output at the same speed so we can debug boot modes and
bootloader with the same terminal.
This patch builds the bootloader.bin file from the ESP8266 SDK source
code. The code is built as a module (esp8266_bootloader) which at the
moment doesn't generate any object code for the application and only
produces a bootloader.bin file set to the BOOTLOADER_BIN make variable
for the esptool.inc.mk to flash.
The code needs to be compiled and linked with custom rules defined in
the module's Makefile since the bootloader.bin is its own separate
application.
The `BOOTLOADER_BIN` variable is changed from a path relative to the
`$(RIOTCPU)/$(CPU)/bin/` directory to be full path. This makes it easier
for applications or board to provide their own bootloader binary if
needed.
As a result of building the bootloader from source we fixed the issue of
having a large partition table. Fixes#16402.
If Quad SPI modes qout or qio are set by variable FLASH_MODE, esptool.py has to be called with parameter `--flash_mode dio` so that the first stage bootloader is always using Dual SPI mode.
The directory `dist/tools/esptool` already contains a couple of ESP tools and not only esptool.py. As the location for a couple of ESP related tools, it is more clear to call it `esptools` instead of `esptool`.
The partition table of the device in the esp8266 and esp32 based boards
was set to a default table with one "factory" partition with exactly
the size of the compiled firmware. This is problematic if we want to
update the device on the field.
This patch allows to set the `PARTITION_TABLE_CSV` variable from the
Makefile to a .csv file with a custom partition table, for example this
could be set to a partition table with two ota entries, or with a single
factory entry but of a known fixed size.
