RIOT/pkg/openwsn/doc.txt

/**
 @defgroup pkg_openwsn   OpenWSN network stack
 @ingroup  pkg
 @ingroup  net
 @brief    Provides a RIOT adaption of the OpenWSN network stack
 @see      https://github.com/openwsn-berkeley/openwsn-fw
 @experimental

 # OpenWSN RIOT Port

 This implementation integrates the [OpenWSN](https://github.com/openwsn-berkeley/openwsn-fw)
 full network stack (UDP, IPv6 (6LoWPAN), RPL, 6TiSCH) into RIOT.

 It can be used instead of GNRC on supported 802.15.4 radios, and compared to
 GNRC, provides a full 6TiSCH implementation. It supports RIOT's
 @ref net_sock API, so it can be used as a drop-in replacement for application
 built on top of @ref net_sock.

 This port provides a new RIOT "board" to the OpenWSN software. In this way
 RIOT's hardware abstraction connects to OpenWSN's interfaces.

 The simple scheduling mechanism in OpenWSN is run in a RIOT thread with
 second highest priority after the radio thread (THREAD_PRIORITY_MAIN - 4).

 The current port of OpenWSN currently needs a root node that works along an
 external tool that performs routing and handles join procedure:
 [Openvisualizer](https://github.com/openwsn-berkeley/openvisualizer)


 ## Joining a network

 The first thing a new mote will need to do is to find a network. On boot it
 will actively be listening for enhanced beacons. Once a beacon is received it
 will adjust its timers drift and synchronize with the network. Re-synchronization
 will be happening constantly to compensate for oscillator and timer drifts.

 Once synchronized the node will need to join the network. OpenWSN uses CoJP
 [constrained join protocol](https://datatracker.ietf.org/doc/draft-ietf-6tisch-minimal-security/)
 an the stack itself only handles Join Requests. The JRC (join registrar/coordinator,
 a central entity) is not running on the root node, but alongside it, in the
 `OpenVisualizer` external tool. This functionality is now optional but can be
 included through the `openwsn_cjoin` module.

 Once joined the device has the required keys to start listening to DIS (DODAG
 Information Solicitation) messages and to send DIO (DODAG Information Object)
 requests. Once it knows about the topology of the network it is able to
 send packets.

 OpenWSN uses source routing. This means that unless the recipient of a packet
 is one of the parents in the RPL tree the packet will have to go up the tree
 to the root node. But in OpenWSN RPL implementation the node does not know
 how to route, instead it is `OpenVisualizer` which generates an SRH (Source
 Routing Header), attaches to the incoming packet and sends it down the tree.

 ## Hardware abstraction implementation

 Following, details about the implementation of selected hardware modules.

 ### sctimer

 The `sctimer` ("single compare timer") in OpenWSN is the lowest timer
 abstraction which is used by the higher layer timer module `opentimers`. In
 the end it is responsible for scheduling on the MAC layer. To enable low power
 energy modes, this timer usually uses the RTC (real time clock) or RTT (real
 time timer) module.

 This port has two possible implementations or sctimer, one on top of
 periph_rtt and another on top of ztimer, `sctimer_rtt` and `sctimer_ztimer`
 respectively. If possible `ztimer` should be preferred, being a virtual
 timer it will allow RIOT applications/modules to use the low level RTT timer.
 But ztimer (any virtual timer) has some overhead which can be costly
 depending on the different platforms used, specially when `openserial`
 is required. More on this in openserial and Known Issues.

 #### sctimer_ztimer

 In order to get the most portable code, this implementation uses
 ztimer and defines a new `ztimer_clock` (`ZTIMER_32768`) that operates
 at 32768Khz to have a resolution of ~30usec/tick (same as OpenWSN).

 When available `ZTIMER_32768` will be built on top of `periph_rtt` to
 get low power capabilities. If not it will be built on top of a
 regular timer. In either case it will be shifted up if the base
 frequency is lower than 32768Hz or frac if higher.

 When next interrupt to schedule is already late, current time,
 implementations in OpenWSN directly trigger a hardware interrupt.
 Until able to trigger sw isr directly a callback is set 0 ticks in
 the future, which internally will be set to `now + RTT_MIN_OFFSET`.

 #### sctimer_rtt

 In order to reduce overhead this implementation uses bare RTT. It
 expects a RTT running at 32768Hz to have a resolution of ~30usec/tick
 (same as OpenWSN). If `RTT_FREQUENCY` is lower than 32768Hz then a
 simple time-division mechanism will be used to speed up the clock.
 This only works if `RTT_FREQUENCY` is 32768Hz/2.

 When next interrupt to schedule is already late, current time,
 Implementations in OpenWSN directly trigger a hardware interrupt.
 Until able to trigger sw isr directly a callback is set
 `RTT_MIN_OFFSET` ticks in the future.

 ### radio

 The radio adaptation maps to RIOT's @ref netdev API or on
 @ref drivers_ieee802154_hal API. The later is preferred but not yet
 supported by all boards.

 Hardware MAC layer features such as CSMA/CA, ACK handling and retransmissions
 are handled by OpenWSN, so the radio driver must support disabling AUTOACK
 and CSMA handling by the hardware. Frame filtering must as well be disabled.

 OpenWSN needs to be notified when a frame reception/transmission starts and
 when it ends. Without these synchronization can still work but in a deteriorated
 way.

 OpenWSN expects to recover crc information on every received frame even if it
 will simply drop frames with invalid crc. The stack can function correctly if
 radio drivers automatically drop frames with an invalid crc (i.e. the stack
 doesn't get notified about these frames).

 #### ieee802154_hal

 The radio drivers should support the following caps:

     - `IEEE802154_CAP_IRQ_RX_START`
     - `IEEE802154_CAP_IRQ_TX_START`

 For CRC error handling:

     - `IEEE802154_CAP_IRQ_CRC_ERROR`

 #### netdev

 The radio adaptation preloads the buffer so `NETOPT_PRELOADING` must be
 supported.

 The radio drivers should support the following netdev events:

     - `NETDEV_EVENT_RX_STARTED`
     - `NETDEV_EVENT_TX_STARTED`
     - `NETDEV_EVENT_RX_COMPLETE`
     - `NETDEV_EVENT_TX_COMPLETE`

 ### uart

 In RIOT, the first configured uart device is mapped to STDIO in most cases.
 In OpenWSN however, the `openserial` tool uses uart to feed external software
 running on a host computer such as
 [Openvisualizer](https://github.com/openwsn-berkeley/openvisualizer).
 To enable use of these tools, an uart adaptation is provided.

 This is provided through the `openwsn_serial` (`openserial`) module. It
 will use the next available uart that is not used by STDIO
 (checking STDIO_UART_DEV). When multiple uart are available STDIO and
 `openserial` can be used in parallel. If `stdio_null` is used then `openserial`
 will use `STDIO_UART_DEV`, otherwise it will use the next available uart.
 e.g. If `STDIO_UART_DEV = UART_DEV(1)` `OPENWSN_UART_DEV  = UART_DEV(0)` if
 there are uarts.

 OpenWSN uart abstraction makes use of tx hardware interrupts to execute a
 previously registered callback after every byte is sent out. These interrupts
 are currently not defined in RIOT in a generic way, so instead a timer is set
 to fire shortly after a byte is written.

 It uses `ztimer` to set the timer since it's already pulled in as a dependency.

 ### Openserial

 As was mentioned before any OpenWSN network will require a root node which is
 connected to an `OpenVisualizer` instance running on a host computer.
 Interaction between `OpenVisualizer` and the root-node is done over serial. As
 OpenWSN uses source routing, this means that ultimately all network traffic
 must go from the root node to `OpenVisualizer` and back.

 OpenSerial uses software flow control (XonXoff) to turn off serial activity
 while time critical TSCH operation are ongoing. But software flow control
 can cause issues since delays in the serial pipe (either because of a remote
 connection, buffers, etc..) can lead to bytes being transmitted when one side
 is not yet ready.

 Serial data is transmitted as High-Level Data Link Control (HDLC) frames.
 Since network traffic is tunneled through the serial pipe, in order to
 to have a stable connection these packets must not be lost. Packets can be
 lost in multiple ways but at the end it reduces to bytes being overridden in
 the uart reception buffer.

 1. The last byte in an HDCL frame is received
 2. Serial pipe delays hindering XonXoff operation
 3. Interrupts are masked or higher/equal ISR are running

 1. When the last byte of a frame is received some parsing and handling of the
    frame occur. This takes some time and during that time OpenSerial can't
    handle more incoming bytes. This in practice limits the baudrate depending
    on the CPU's speed.

 2. This can occur when a debugger/virtual-port might act as a
    buffer between the mote and the host, or when there is a latency in the
    connection (for example tcp connection to iotlab).

 3. Since uart reception is interrupted based if the uart ISR is not serviced
    for too long then bytes start overriding each other in the reception
    buffer. In an application where only OpenWSN is running then there are 3
    functions/operations that run in ISR or with disabled ISR.

  (a) `opentimers_timer_callback` (OpenWSN timer abstraction callback)
  (b) `sctimer_setCompare` and `sctimer_readCounter`
  (c) `ztimer_handler`

 (a) and (b) are closely related since `opentimers_timer_callback` will
  itself call `sctimer_setCompare` and `sctimer_readCounter`. And this itself
  will depend on `ztimer_set` and `ztimer_now` or `rtt_get_counter` or
  `rtt_set_alarm` execution time.

 (c) is also related to (a) (b) since `ztimer_handler` will also call
 `opentimers_timer_callback` as well as the underlying `rtt` functions.

 Since a 115200 baudrate means ~1byte every 10us, none of the above can take
 longer than that or bytes could be lost at that baudrate.

 Because of the above mentioned reasons, it is preferable to use `sctimer_rtt`
 for the root node or "border router" on an OpenWSN network, since this reduces
 the likeliness of packets being lost. For non root nodes, OpenSerial only
 provides debugging information so no special care needs to be taken.

 It is also recommended that the root node should act as a border router
 running only the OpenWSN stack to avoid other threads/ISR disrupting serial
 reception.

 ## Tested Platforms and Pin configurations

 So far, this has been successfully tested on `iotlab-m3`,`nucleo-f103` and
 `samr21-xpro`, all based on at86rf23x radios. These radios make use of
 the `radio_netdev` adaptation.

 ### Synchronization

 To join a network a node must first receive EB (enhanced beacons). Once an EB
 is received the node will be synchronized with the network. Synchronization
 times are not deterministic, they depend on the following:

    `SLOTFRAME_LENGTH*SLOTDURATION*P_CHANNEL*P_EB`

 `SLOTFRAME_LENGTH` in OpenWSN is 101, and this port uses 20ms as the slotOffset
 duration. `P_EB` specifies the a probability for a node to transmit an EB.
 By default it's 10%, that means that on average it will take 10 tries before an
 EB is transmitted. `P_CHANNEL` is the probability for the transmitter's and
 receiver channel to match. If channel hopping is disabled this means that the
 average worst case scenario is `101*20ms*10 ~= 20s`, so 20s for synchronization
 to take place. `EB_PORTION` can be changed to increase the likelihood of EB to be
 sent. This can also be achieved by reducing `SLOTFRAME_LENGTH`, but the later can
 have an impact on the MSF (Minimal Scheduling Function). If too few cells are
 available this could increase the likelihood of collisions.

 On the other hand if channel hopping is enabled then the joining node picks a
 random channel to start listening on. The transmitter also picks a random
 channel to start transmitting on and then follows a channel hopping template.
 This would take on average ~8 tries for it to hit the correct channel for the
 first time, and then it would hit it every 16 hops, on average this could lead
 synchronization times of around `320s`.

 If nodes are having trouble in staying synchronized increasing `P_EB` by
 reducing the value of `EB_PORTION` can also be done. Note that `EB_PORTION`
 and `SLOTFRAME_LENGTH` are not configurable by default so need to be overridden
 with `CFLAGS`.

 See [Tune parameters in OpenWSN](https://openwsn.atlassian.net/wiki/spaces/OW/pages/132055073/Tune+parameters+in+OpenWSN)
 for more details.

 ### Timing

 Timing is essential for OpenWSN to work properly. For optimal results most
 parameters in `board_info.h` should be measured for the specific hardware used.
 OpenWSN has done that for most of their boards. These values can not be taken
 directly from OpenWSN since they do not necessarily use the same TIMER's or
 clock speeds.

 For more details on those parameters refer to:
 https://openwsn.atlassian.net/wiki/spaces/OW/pages/688251/State+Machine

 Since all these parameters are HW dependent, it also means that hybrid
 networks (different type of underlying hardware) might desynchronize often,
 or not manage to keep in sync at all.

 Print messages during TSCH operation should be avoided since these can disrupt
 TSCH timings.

 ### ledpins & debugpins

 The OpenWSN software provides different hooks all over the stack to toggle
 different LEDs as well as debug pins to examine state and scheduling of a node.
 Default configuration files are provided for both. The LED configuration maps to
 RIOTs `LEDX_PIN` definitions, if available. On Nucleo boards LED0 line is
 shared SPI, so is not used.

 The default configuration can be overwritten by setting `OPENWSN_LEDS_DEFAULT`
 in the form of `leds_config_t`. The debugpins work similarly by setting
 `OPENWSN_DEBUGPINS_BOARD` in the form of `debugpins_config_t`.

 The default configuration maps to OpenWSN reference hardware `openmote-b`.

 ## Optional Modules and Features

 The following modules are optional and can be disabled if not needed, or to
 lower the stack footprint.

  - `openwsn_cjoin`: this enabled the use of Constrained Join Protocol (CoJP)
  - `openwsn_6lo_fragmentation`: this enable 6LoWPAN fragmentation
  - `openwsn_iee802154e_security`: enable link layer security
  - `openwsn_adaptive_msf`: allow the MSF algorithm to dynamically remove and
    allocate slots

 ## Testing and debugging

 List of some items which are helpful to explore the functionality of OpenWSN:

 - LED pins and debug pins as mentioned above in combination with a logic analyzer.
 The expected behavior is described in:
 [OpenWSN wiki](https://openwsn.atlassian.net/wiki/spaces/OW/pages/688257/Schedules).

 - The provided test application provides a UDP client and server. If the UDP
 server is able to receive packets, the mechanism is considered to work correctly.
 You should also be able to ping the device from your host. See
 `tests/pkg/openwsn/README.md` for more details.

 - To speed up synchronization and make sniffing easier you can disable channel
 hopping by setting (`CFLAGS=-DIEEE802154E_SINGLE_CHANNEL=26`).

 - To sniff the packets either use a 802.15.4 capable board and follow at:
 https://github.com/RIOT-OS/applications/blob/master/sniffer/tools/README.md.
 Alternatively use a Raspberry Pi with an external radio such as Openlabs and
 incorporate Linux WPAN tools. In addition to that, there's also other
 hardware such as the ATUSB IEEE 802.15.4 USB Adapter which can directly be
 used on your Linux computer with WPAN tools installed. If you conduct your
 experiments on the IoT-LAB testbed you might want to use a:
 [sniffer profile](https://www.iot-lab.info/tutorials/radio-sniffer).

 - To explore the channel hopping mechanism there are rather expensive
 multi-channel sniffers such as the BeamLogic 802.15.4 Site Analyzer that can
 sniff all channels simultaneously. Alternatively you can set up multiple
 separate sniffer devices locally or make use of the `sniffer_aggregator` on
 the IoT-LAB testbed.

 - To test Openserial on a given platform the target `make openv-serial` can
 be used on a BOARD flashed with `tests/pkg/openwsn` (`USEMODULE=openwsn_serial`)
 must be included as well. The following output should appear:

    ```
    Test Setup:
    ------------------
    Iterations:    100
    Packet length: 100
    Echo timeout:    2

    Test Progress:

      [####################################]  100%

    Test Statistics:
    ------------------
    Pkts send:      100
    Echo success:   100
    Echo timeout:     0
    Echo corrupted:   0
    ```

 The test should be considered passing if success rate is > 98%.

 ## Known Issues

 The following errors might be visible when using `openwsn_serial`:

 - `[OPENSERIAL] wrong CRC in input Buffer`

 Since a timer is set to simulate a uart transmit interrupt, it can happen that
 the interrupt is missed if another interrupt occurs during that time, this
 seems to lead to the input buffer missing a byte and so CRC fails. More details
 where given in the `openserial` section.

 - `[IEEE802154E] wdDataDuration overflows while at state 19 in slotOffset 0`

 This error can show up when the radio starts receiving (receives the SFD) and
 therefore triggers a `NETDEV_RX_STARTED` but then no `NETDEV_TX_STARTED`
 event follows. This happens when packets with invalid CRC are received.
 netdev currently silently drops these packets without notifying upper layers.
 But this does not affect the stack operation, so they can be ignored.

 - `[IEEE802154E] large timeCorr.: -18 ticks (code loc. 0)`

 Most crystals used to clock the RTT will drift even those with a very similarly
 drift (10-30ppm). It's is normal then for motes adjust their timerCorr as long
 as it stays within the above mentioned margins and if motes are able to stay
 synchronized over the long run. If there aren't then maybe board_info.h
 parameters require tuning for the specific platform.

- `[JRC:ERROR] Type-error in conversion of 5N=ex`

This errors happen when a node tries to rejoin the network. This error is only
associated to a log print, so can be ignored.

- `[coap:WARNING] coapRcBadRequest(reason=OSCOAP unprotect failed: oscoapError(reason=Replay protection failed))`

The join procedure uses a replay window. If a node had already joined the network
and for some reason attempts to rejoin again, then the replay windows will need
to expire for it's join request to be accepted.

The following errors are platform specific.

 - samr21-xpro issues:
      - The serial debugger hinders Openserial operation, an ftdi device must
        be used.

 - sam0 issues:
      - sam0 requires 180us busy loops every time an alarm is set or the
        counter is read. Because of the later only `sctimer_rtt` can be used
        and the max. tested baudrate for openserial is of 19200 bauds.

 - iotlab-m3 issues:
      - openserial does not work reliably over 57600 bauds or when using
        sctimer_ztimer.

Other errors:

- missed characters over stdio

TSCH state machine disable occurs in IRQ context and disables IRQ during time
critical sections. This can cause bytes sent over stdio to be missed.

 ## Todos

 - `sctimer` to trigger an ISR immediately using software interrupts.
 - The UART wrapper uses ztimer to fake an interrupt after one byte
   has been sent. This should also be done with software interrupts.

 ##  Future Steps

 It would be desirable to achieve an extraction of the MAC layer.

 */