Merge pull request #18190 from krzysztof-cabaj/doc-tests

tests/doc: some additions and reorganization of doc page

tests/README.md

@@ -1,14 +1,90 @@
-Running and creating tests                        {#running-and-creating-tests}
+# Running and creating tests                        {#running-and-creating-tests}
-==========================
 
 There are a number of tests included in RIOT. They are located in the
 [tests folder](https://github.com/RIOT-OS/RIOT/tree/master/tests). These tests
 allow basic functionality to be verified as well as provide an example of
 usage.
 
+# Running automated tests
 
-Implementing automated tests
+Some tests can be performed automatically. The test automation scripts are
-----------------------------
+defined in the `<test_application>/tests/` folder. They are written in python
+and interact through the serial (typically UART) with the test application code running on a
+board to do the validation. It is recommended to flash the board with the
+test just before running it because some platforms cannot be reset while
+testing.
+
+## Running single test
+
+From the test application directory run:
+
+    BOARD=<board_of_your_choice> make flash test
+
+
+An automated way of knowing if a test is available is to execute the
+'test/available' target from the test application directory.
+It executes without error if tests run by 'make test' are present.
+
+    make test/available
+
+## Running all test for particular board
+
+If you would like execute all tests for given board, you could use dedicated
+script `compile_and_test_for_board.py`
+
+Go to main RIOT directory and execute command:
+
+    ./dist/tools/compile_and_test_for_board/compile_and_test_for_board.py . <board_of_your_choice> --with-test-only --jobs=4
+
+More details concerning other available parameters provided by this tool can be found in
+[README.md file](https://github.com/RIOT-OS/RIOT/tree/master/dist/tools/compile_and_test_for_board)
+and directly in [compile_and_test_for_board.py](https://github.com/RIOT-OS/RIOT/tree/master/dist/tools/compile_and_test_for_board/compile_and_test_for_board.py) script.
+
+## Running tests that require a preliminary manual configuration
+
+Some tests need active monitoring or manual setup steps but still have some
+automated scripts. The test automation scripts are defined in the
+`<test_application>/tests-with-config/` folder.
+For running them, follow the setup or analysis documentation and use the
+`test-with-config` target.
+
+## Running tests that require root privileges
+
+Some tests require root privileges to launch their automated script. In this
+case, the test automation scripts are defined in the
+`<test_application>/tests-as-root/` folder.
+For running them, follow the setup or analysis documentation and use the
+`test-as-root` target.
+
+## Cleaning intermediate files
+
+After test execution intermediate files are not automatically deleted.
+Execution of multiple tests, especially all for particular board could generate
+many files. For example, after execution of all test for stm32f469i-disco board
+(more than 230 tests) around 7.5 GB of intermediate files are created.
+
+There are few methods for cleaning intermediate files.
+
+If you would like to clean intermediate file only for particular board you should
+go to main RIOT directory and execute one from these commands:
+
+    ./dist/tools/compile_and_test_for_board/compile_and_test_for_board.py . <board_of_your_choice> --compile-targets clean
+
+or
+
+    make BOARD=<board_of_your_choice> clean
+
+
+If you would like to clean intermediate files for all boards go to main RIOT
+directory and use this command.
+
+@warning This command cleans all local files, for example, pkg downloads and
+locally generared docs.
+
+
+    make distclean
+
+# Implementing automated tests
 
 The goal is to be able to run all tests in a sequential way for as many targets
 as possible.
@@ -34,32 +110,7 @@ script and use `test_utils_interactive_sync();` function (e.g.:
 you can disable `test_utils_interactive_sync_shell` module in the application
 `Makefile`: `DISABLE_MODULE += test_utils_interactive_sync_shell`.
 
-
+## Automated Tests Guidelines
-Running automated tests
------------------------
-
-Some tests can be performed automatically. The test automation scripts are
-defined in the `<test_application>/tests/` folder. They are written in python
-and interact through the uart with the test application code running on a
-board to do the validation. It is recommended to flash the board with the
-test just before running it because some platforms cannot be reset while
-testing.
-
-From the test application directory run:
-
-    BOARD=<board_of_your_choice> make flash test
-
-
-An automated way of knowing if a test is available is to execute the
-'test/available' target from the test application directory.
-It executes without error if tests run by 'make test' are present.
-
-    make test/available
-
-
-Automated Tests Guidelines
---------------------------
-
 
 When using `pexpect` `$` is useless for matching the end of a line, instead use
 `\r\n`([pexpect end-of-line](https://pexpect.readthedocs.io/en/stable/overview.html#find-the-end-of-line-cr-lf-conventions)).
@@ -93,8 +144,7 @@ pattern like `\r\n`, `\s`, `\)`, etc..
     child.expect(r'some string: (\d+) ,')
 ~~~
 
-Use expect() instead of assert()
+## Use expect() instead of assert()
---------------------------------
 
 In order to make a test application functional in all cases, use `expect()`
 instead of `assert()`. The former works like the latter, but will still be
@@ -104,8 +154,7 @@ be compiled with that flag.  Otherwise, the application would force compiling
 all tested code with assertions enabled.
 `expect()` is defined in the header `test_utils/expect.h`.
 
-Interaction through the uart
+## Interaction through the uart
-----------------------------
 
 Tests implemented with `testrunner` use the `cleanterm` target that
 provides an interaction without adding extra text output or input handling.
@@ -117,22 +166,3 @@ The expected behavior is verified with the test in `tests/test_tools`.
 Tests cannot rely on having on all boards and terminal programs:
 * unbuffered input
 * allowing sending special characters like `ctrl+c/ctrl+d`
-
-
-Running tests that require a preliminary manual configuration
--------------------------------------------------------------
-
-Some tests need active monitoring or manual setup steps but still have some
-automated scripts. The test automation scripts are defined in the
-`<test_application>/tests-with-config/` folder.
-For running them, follow the setup or analysis documentation and use the
-`test-with-config` target.
-
-Running tests that require root privileges
-------------------------------------------
-
-Some tests require root privileges to launch their automated script. In this
-case, the test automation scripts are defined in the
-`<test_application>/tests-as-root/` folder.
-For running them, follow the setup or analysis documentation and use the
-`test-as-root` target.