Diff: mbed-os/TEST_APPS/readme.md

Revision:

0:8fdf9a60065b

--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/mbed-os/TEST_APPS/readme.md	Wed Oct 10 00:33:53 2018 +0000
@@ -0,0 +1,103 @@
+## Running Icetea tests located under mbed-os
+
+### Structure
+
+Mbed OS has a folder called `TEST_APPS` that contains everything related to Icetea testing.
+
+There are currently three folders:
+
+- `device` - contains all the different test applications you can flash to your board.
+- `icetea_plugins` - contains plugins that are being used by some of the testcases, needed for the test execution.
+- `testcases` - contains Icetea testcases written in Python.
+
+The testcases depend on test applications.
+
+### Preparing your work environment
+
+#### Prerequisites
+
+You need Icetea and version 1.8.0 or higher of Mbed CLI installed.
+
+#### Selecting the network interface to use
+
+Depending on the device, there might be a default network interface type defined in `mbed-os/targets/targets.json`, which you can use to locate a default test-config file.
+
+If the default network interface type is not defined or you want to use a different interface than the default, you need to provide a test-config file to the mbed test with `--test-config` option.
+
+The test-config file contains the necessary information for the test application. There are some test-config files located under `mbed-os/tools/test_configs`.
+
+Devices that have their network drivers residing inside `mbed-os` can use generic `test_configs`, such as `HeapBlockDeviceAndEthernetInterface.json` and `HeapBlockDeviceAndWifiInterface.json`. Otherwise, you need to use a device-specific test-config.
+
+### Running the tests
+
+Now that you have selected the interface, you can run the Icetea tests from the `mbed-os` root on your command-line by running the following command:
+
+`mbed test -m <target> -t <toolchain> --icetea`
+
+This command compiles the OS, compiles the test applications, creates a test suite and then starts running the tests.
+
+If you only want to run some specific tests, you can use the `-n` option. You can choose multiple tests by separating them with a comma (`,`):
+
+`mbed test -m <target> -t <toolchain> --icetea -n test1,test2`
+
+#### Running the tests with a specific test-config
+
+Some devices may offer multiple network interfaces to operate with. For example, `UBLOX_EVK_ODIN_W2` offers ethernet and Wi-Fi capabilities.
+
+The tests can be run for either one of those using already existing test-config -files.
+
+To run the tests with the Wi-Fi interface:
+
+`mbed test -m UBLOX_EVK_ODIN_W2 -t <toolchain> --icetea --test-config tools/test_configs/HeapBlockDeviceAndWifiInterface.json`
+
+To run the tests with the ethernet interface:
+
+`mbed test -m UBLOX_EVK_ODIN_W2 -t <toolchain> --icetea --test-config tools/test_configs/HeapBlockDeviceAndEthernetInterface.json`
+
+#### Providing Wi-Fi access point information
+
+If you are using the Wi-Fi interface for running the tests, you need to also provide information about the used access point.
+
+The information can be provided in the used test-config file.
+
+Example of access point information:
+
+```
+    "target_overrides": {
+        "*": {
+            "target.network-default-interface-type": "WIFI",
+            "nsapi.default-wifi-ssid": "\"ssid\"",
+            "nsapi.default-wifi-password": "\"password\"",
+            "nsapi.default-wifi-security": "WPA_WPA2"
+        }
+    }
+```
+
+### Test results
+
+Icetea prints the results from the test run to the command-line, and the final result looks similar to this:
+
+```
++--------------------------------+---------+-------------+-------------+-----------+----------+
+| Testcase                       | Verdict | Fail Reason | Skip Reason | platforms | duration |
++--------------------------------+---------+-------------+-------------+-----------+----------+
+| test_cmdline                   |   pass  |             |             |    K64F   |  8.555   |
+| UDPSOCKET_BIND_PORT            |   pass  |             |             |    K64F   |  19.614  |
+| TCPSOCKET_BIND_PORT            |   pass  |             |             |    K64F   |  15.852  |
+| TCPSERVER_ACCEPT               |   pass  |             |             |    K64F   |  41.629  |
+| TCPSOCKET_ECHOTEST_BURST_SHORT |   pass  |             |             |    K64F   |  19.926  |
++--------------------------------+---------+-------------+-------------+-----------+----------+
++---------------+----------------+
+|    Summary    |                |
++---------------+----------------+
+| Final Verdict |      PASS      |
+|     count     |       5        |
+|    passrate   |    100.00 %    |
+|      pass     |       5        |
+|    Duration   | 0:01:45.576000 |
++---------------+----------------+
+```
+
+You can also find the results from the tests in the `mbed-os/log` folder.
+
+You probably want to add the log folder to your `.mbedignore` file to prevent issues with build commands becoming too long over the time.