Greg Steiert / pegasus_dev

Development mbed library for MAX32630FTHR

Dependents:   blinky_max32630fthr

docs/TESTING.md@3:1198227e6421, 2016-12-16 (annotated)

Committer:
switches
Date:
Fri Dec 16 16:27:57 2016 +0000
Revision:
3:1198227e6421
Parent:
0:5c4d7b2438d3 
Changed ADC scale for MAX32625 platforms to 1.2V full scale to match MAX32630 platforms

Who changed what in which revision?

UserRevisionLine numberNew contents of line
switches 0:5c4d7b2438d3 1 # Mbed SDK automated test suite
switches 0:5c4d7b2438d3 2 ## Introduction
switches 0:5c4d7b2438d3 3
switches 0:5c4d7b2438d3 4 Test suit allows users to run locally on their machines Mbed SDK’s tests included in Mbed SDK repository. It also allows users to create their own tests and for example add new tests to test set as they progress with their project. If test is generic enough it could be included into official Mbed SDK test pool just do it via normal pull request!
switches 0:5c4d7b2438d3 5
switches 0:5c4d7b2438d3 6 Each test is supervised by python script called “host test” which will at least Test suite is using build script API to compile and build test source together with required by test libraries like CMSIS, Mbed, Ethernet, USB etc.
switches 0:5c4d7b2438d3 7
switches 0:5c4d7b2438d3 8 ## What is host test?
switches 0:5c4d7b2438d3 9 Test suite supports test supervisor concept. This concept is realized by separate Python script called ```host test```. Host tests can be found in ```mbed/tools/host_tests/``` directory. Note: In newer mbed versions (mbed OS) host tests will be separate library.
switches 0:5c4d7b2438d3 10
switches 0:5c4d7b2438d3 11 Host test script is executed in parallel with test runner to monitor test execution. Basic host test just monitors device's default serial port for test results returned by test runner. Simple tests will print test result on serial port. In other cases host tests can for example judge by test results returned by test runner if test passed or failed. It all depends on test itself.
switches 0:5c4d7b2438d3 12
switches 0:5c4d7b2438d3 13 In some cases host test can be TCP server echoing packets from test runner and judging packet loss. In other cases it can just check if values returned from accelerometer are actually valid (sane).
switches 0:5c4d7b2438d3 14
switches 0:5c4d7b2438d3 15 ## Test suite core: singletest.py script
switches 0:5c4d7b2438d3 16
switches 0:5c4d7b2438d3 17 ```singletest.py``` script located in ```mbed/tools/``` is a test suite script which allows users to compile, build tests and test runners (also supports CppUTest unit test library). Script also is responsible for test execution on devices selected by configuration files.
switches 0:5c4d7b2438d3 18
switches 0:5c4d7b2438d3 19 ### Parameters of singletest.py
switches 0:5c4d7b2438d3 20
switches 0:5c4d7b2438d3 21 Test execution script ```singletest.py``` is a fairly powerful tool to run tests for mbed SDK platform. It is flexible and allows users to configure test execution process and define which mbed platforms will be tested.
switches 0:5c4d7b2438d3 22
switches 0:5c4d7b2438d3 23 By specifying external configuration files (in JSON format) you can gain flexibility and prepare many different test scenarios. Just pass configuration file names to your script and run it.
switches 0:5c4d7b2438d3 24
switches 0:5c4d7b2438d3 25 #### MUTs Specification
switches 0:5c4d7b2438d3 26 You can easily configure your MUTs (Mbed Under Test) by creating configuration file with MUTs description.
switches 0:5c4d7b2438d3 27 Note: This configuration file must be in [JSON format](http://www.w3schools.com/json/).
switches 0:5c4d7b2438d3 28 Note: Unfortunately JSON format is not allowing you to have comments inside JSON code.
switches 0:5c4d7b2438d3 29
switches 0:5c4d7b2438d3 30 Let’s see some example and let's try to configure small "test farm" with three devices connected to your host computer. In this example no peripherals (like SD card or EEPROM) are connected to our Mbed boards. We will use three platforms in this example:
switches 0:5c4d7b2438d3 31 * [NXP LPC1768](https://mbed.org/platforms/mbed-LPC1768) board.
switches 0:5c4d7b2438d3 32 * \[Freescale KL25Z](https://mbed.org/platforms/KL25Z) board and
switches 0:5c4d7b2438d3 33 * [STMicro Nucleo F103RB](https://mbed.org/platforms/ST-Nucleo-F103RB) board.
switches 0:5c4d7b2438d3 34 After connecting boards to our host machine (PC) we can check which serial ports and disks they occupy. For our example let's assume that:
switches 0:5c4d7b2438d3 35 * ```LPC1768``` serial port is on ```COM4``` and disk drive is ```J:```.
switches 0:5c4d7b2438d3 36 * ```KL25Z``` serial port is on ```COM39``` and disk drive is ```E:```.
switches 0:5c4d7b2438d3 37 * ```NUCLEO_F103RB``` serial port is on ```COM11``` and disk drive is ```I:```.
switches 0:5c4d7b2438d3 38 If you are working under Linux your port and disk could look like /dev/ttyACM5 and /media/usb5.
switches 0:5c4d7b2438d3 39
switches 0:5c4d7b2438d3 40 This information is needed to create ```muts_all.json``` configuration file. You can create it in ```mbed/tools/``` directory:
switches 0:5c4d7b2438d3 41 ```
switches 0:5c4d7b2438d3 42 $ touch muts_all.json
switches 0:5c4d7b2438d3 43 ```
switches 0:5c4d7b2438d3 44
switches 0:5c4d7b2438d3 45 Its name will be passed to ```singletest.py``` script after ```-M``` (MUTs specification) switch. Let’s see how this file's content would look like in our example below:
switches 0:5c4d7b2438d3 46 ```json
switches 0:5c4d7b2438d3 47 {
switches 0:5c4d7b2438d3 48 "1" : {"mcu": "LPC1768",
switches 0:5c4d7b2438d3 49 "port":"COM4",
switches 0:5c4d7b2438d3 50 "disk":"J:\\",
switches 0:5c4d7b2438d3 51 "peripherals": []
switches 0:5c4d7b2438d3 52 },
switches 0:5c4d7b2438d3 53
switches 0:5c4d7b2438d3 54 "2" : {"mcu": "KL25Z",
switches 0:5c4d7b2438d3 55 "port":"COM39",
switches 0:5c4d7b2438d3 56 "disk":"E:\\",
switches 0:5c4d7b2438d3 57 "peripherals": []
switches 0:5c4d7b2438d3 58 },
switches 0:5c4d7b2438d3 59
switches 0:5c4d7b2438d3 60 "3" : {"mcu": "NUCLEO_F103RB",
switches 0:5c4d7b2438d3 61 "port":"COM11",
switches 0:5c4d7b2438d3 62 "disk":"I:\\",
switches 0:5c4d7b2438d3 63 "peripherals": []
switches 0:5c4d7b2438d3 64 }
switches 0:5c4d7b2438d3 65 }
switches 0:5c4d7b2438d3 66 ```
switches 0:5c4d7b2438d3 67
switches 0:5c4d7b2438d3 68 Note: We will leave field ```peripherals``` empty for the sake of this example. We will explain it later. All you need to do now is to properly fill fields ```mcu```, ```port``` and ```disk```.
switches 0:5c4d7b2438d3 69
switches 0:5c4d7b2438d3 70 Note: Please make sure files muts_all.json and test_spec.json are in tools/ directory. We will assume in this example they are.
switches 0:5c4d7b2438d3 71 Where to find ```mcu``` names? You can use option ```-S``` of ```build.py``` script (in ```mbed/tools/``` directory) to check all supported off-line MCUs names.
switches 0:5c4d7b2438d3 72
switches 0:5c4d7b2438d3 73 Note: If you update mbed device firmware or even disconnect / reconnect mbed device you may find that serial port / disk configuration changed. You need to update configuration file accordingly or you will face connection problems and obviously tests will not run.
switches 0:5c4d7b2438d3 74
switches 0:5c4d7b2438d3 75 #### Peripherals testing
switches 0:5c4d7b2438d3 76 When using MUTs configuration file (switch ```-M```) you can define in MUTs JSON file peripherals connected to your device:
switches 0:5c4d7b2438d3 77 ```json
switches 0:5c4d7b2438d3 78 {
switches 0:5c4d7b2438d3 79 "1" : {"mcu" : "KL25Z",
switches 0:5c4d7b2438d3 80 "port" : "COM39",
switches 0:5c4d7b2438d3 81 "disk" : "E:\\",
switches 0:5c4d7b2438d3 82 "peripherals" : ["SD", "24LC256"]}
switches 0:5c4d7b2438d3 83 }
switches 0:5c4d7b2438d3 84 ```
switches 0:5c4d7b2438d3 85 You can force test suite to run only common tests (switch ```-C```) or only peripheral tests (switch ```-P```).
switches 0:5c4d7b2438d3 86 ```
switches 0:5c4d7b2438d3 87 $ python singletest.py -i test_spec.json -M muts_all.json -C
switches 0:5c4d7b2438d3 88 ```
switches 0:5c4d7b2438d3 89 will not include tests for SD card and EEPROM 24LC256.
switches 0:5c4d7b2438d3 90 ```
switches 0:5c4d7b2438d3 91 $ python singletest.py -i test_spec.json -M muts_all.json -P
switches 0:5c4d7b2438d3 92 ```
switches 0:5c4d7b2438d3 93 will only run tests bind to SD card and EEPROM 24LC256.
switches 0:5c4d7b2438d3 94
switches 0:5c4d7b2438d3 95 Note: option ```-P``` is useful for example in cases when you have same platform and different shields you want to test. No need to test common part all the time (timers, RTC, RTOS etc.). You can force to test peripherals only on some devices and for example only common tests on other devices.
switches 0:5c4d7b2438d3 96
switches 0:5c4d7b2438d3 97 #### Additional MUTs configuration file settings
switches 0:5c4d7b2438d3 98 You can add extra information to each MUT configuration. In particular you can specify which flashing (binary copy method) should be used, how to reset target and for example set reset timeout (used to delay test execution just after reset).
switches 0:5c4d7b2438d3 99
switches 0:5c4d7b2438d3 100 muts_all.json:
switches 0:5c4d7b2438d3 101 ```json
switches 0:5c4d7b2438d3 102 {
switches 0:5c4d7b2438d3 103 "1" : {"mcu" : "LPC1768",
switches 0:5c4d7b2438d3 104 "port" : "COM77",
switches 0:5c4d7b2438d3 105 "disk" : "G:\\",
switches 0:5c4d7b2438d3 106 "peripherals" : ["TMP102", "digital_loop", "port_loop", "analog_loop", "SD"]},
switches 0:5c4d7b2438d3 107
switches 0:5c4d7b2438d3 108 "2" : {"mcu" : "KL25Z",
switches 0:5c4d7b2438d3 109 "port" : "COM89",
switches 0:5c4d7b2438d3 110 "disk" : "F:\\",
switches 0:5c4d7b2438d3 111 "peripherals" : ["SD", "24LC256", "KL25Z"],
switches 0:5c4d7b2438d3 112 "copy_method" : "copy",
switches 0:5c4d7b2438d3 113 "reset_type" : "default",
switches 0:5c4d7b2438d3 114 "reset_tout" : "2"},
switches 0:5c4d7b2438d3 115
switches 0:5c4d7b2438d3 116 "3" : {"mcu" : "LPC11U24",
switches 0:5c4d7b2438d3 117 "port" : "COM76",
switches 0:5c4d7b2438d3 118 "disk" : "E:\\",
switches 0:5c4d7b2438d3 119 "peripherals" : []}
switches 0:5c4d7b2438d3 120 }
switches 0:5c4d7b2438d3 121 ```
switches 0:5c4d7b2438d3 122 Please note that for MUT no. 2 few extra parameters were defined: ```copy_method```, ```reset_type``` and ```reset_tout```. Using this extra options you can tell test suite more about MUT you are using. This will allow you to be more flexible in terms of how you configure and use your MUTs.
switches 0:5c4d7b2438d3 123
switches 0:5c4d7b2438d3 124 * ```copy_method``` - STRING - tells test suite which binary copy method should be used.
switches 0:5c4d7b2438d3 125 You may notice that ```singletest.py``` command line help contains description about:
switches 0:5c4d7b2438d3 126 * Option ```-c``` (in MUTs file called ```copy_method```) with available copy methods supported by test suite plugin system.
switches 0:5c4d7b2438d3 127 * Option ```-r``` (in MUTs file called reset_type) with available reset methods supported by test suite plugin system.
switches 0:5c4d7b2438d3 128 * ```reset_type``` - STRING - some boards may require special reset handling, for example vendor specific command must be executed to reset device.
switches 0:5c4d7b2438d3 129 * ```reset_tout``` - INTEGER - extra timeout just after device is reseted. May be used to wait for few seconds so device may finish booting, flashing data internally etc.
switches 0:5c4d7b2438d3 130
switches 0:5c4d7b2438d3 131 Part of help listing for singletest.py:
switches 0:5c4d7b2438d3 132 ```
switches 0:5c4d7b2438d3 133 -c COPY_METHOD, --copy-method=COPY_METHOD
switches 0:5c4d7b2438d3 134 Select binary copy (flash) method. Default is Python's
switches 0:5c4d7b2438d3 135 shutil.copy() method. Plugin support: copy, cp,
switches 0:5c4d7b2438d3 136 default, eACommander, eACommander-usb, xcopy
switches 0:5c4d7b2438d3 137 -r MUT_RESET_TYPE, --reset-type=MUT_RESET_TYPE
switches 0:5c4d7b2438d3 138 Extra reset method used to reset MUT by host test
switches 0:5c4d7b2438d3 139 script. Plugin support: default, eACommander,
switches 0:5c4d7b2438d3 140 eACommander-usb
switches 0:5c4d7b2438d3 141 ```
switches 0:5c4d7b2438d3 142
switches 0:5c4d7b2438d3 143 ----
switches 0:5c4d7b2438d3 144
switches 0:5c4d7b2438d3 145 Now we've already defined how our devices are connected to our host PC. We can continue and define which of this MUTs will be tested and which compilers we will use to compile and build Mbed SDK and tests. To do so we need to create test specification file (let's call it ```test_spec.json```) and put inside our configuration file information about which MUTs we actually want to test. We will pass this file's name to ```singletest.py``` script using ```-i``` switch.
switches 0:5c4d7b2438d3 146
switches 0:5c4d7b2438d3 147 Below we can see how sample ```test_spec.json``` file content could look like. (I've included all possible toolchains, we will change it in a moment):
switches 0:5c4d7b2438d3 148 ```json
switches 0:5c4d7b2438d3 149 {
switches 0:5c4d7b2438d3 150 "targets": {
switches 0:5c4d7b2438d3 151 "LPC1768" : ["ARM", "uARM", "GCC_ARM", "GCC_CR", "IAR"],
switches 0:5c4d7b2438d3 152 "KL25Z" : ["ARM", "GCC_ARM"],
switches 0:5c4d7b2438d3 153 "NUCLEO_F103RB" : ["ARM", "uARM"]
switches 0:5c4d7b2438d3 154 }
switches 0:5c4d7b2438d3 155 }
switches 0:5c4d7b2438d3 156 ```
switches 0:5c4d7b2438d3 157 Above example configuration will force tests for LPC1768, KL25Z, NUCLEO_F103RB platforms and:
switches 0:5c4d7b2438d3 158
switches 0:5c4d7b2438d3 159 * Compilers: ```ARM```, ```uARM```, ```GCC_ARM```, ```GCC_CR``` and ```IAR``` will be used to compile tests for NXP's ```LPC1768```.
switches 0:5c4d7b2438d3 160 * Compilers: ```ARM``` and ```GCC_ARM``` will be used for Freescales' ```KL25Z``` platform.
switches 0:5c4d7b2438d3 161 * Compilers: ```ARM``` and ```uARM``` will be used for STMicro's ```NUCLEO_F103RB``` platform.
switches 0:5c4d7b2438d3 162
switches 0:5c4d7b2438d3 163 For our example purposes let's assume we only have Keil ARM compiler, so let's change configuration in ```test_spec.json``` file and reduce number of compiler to those we actually have:
switches 0:5c4d7b2438d3 164 ```json
switches 0:5c4d7b2438d3 165 {
switches 0:5c4d7b2438d3 166 "targets": {
switches 0:5c4d7b2438d3 167 "LPC1768" : ["ARM", "uARM"],
switches 0:5c4d7b2438d3 168 "KL25Z" : ["ARM"],
switches 0:5c4d7b2438d3 169 "NUCLEO_F103RB" : ["ARM", "uARM"]
switches 0:5c4d7b2438d3 170 }
switches 0:5c4d7b2438d3 171 }
switches 0:5c4d7b2438d3 172 ```
switches 0:5c4d7b2438d3 173 #### Run your tests
switches 0:5c4d7b2438d3 174
switches 0:5c4d7b2438d3 175 After you configure all your MUTs and compilers you are ready to run tests. Make sure your devices are connected and your configuration files reflect your current configuration (serial ports, devices). Go to tools directory in your mbed location.
switches 0:5c4d7b2438d3 176 ```
switches 0:5c4d7b2438d3 177 $ cd tools/
switches 0:5c4d7b2438d3 178 ```
switches 0:5c4d7b2438d3 179 and execute test suite script.
switches 0:5c4d7b2438d3 180 ```
switches 0:5c4d7b2438d3 181 $ python singletest.py -i test_spec.json -M muts_all.json
switches 0:5c4d7b2438d3 182 ```
switches 0:5c4d7b2438d3 183 To check your configuration before test execution please use ```--config``` switch:
switches 0:5c4d7b2438d3 184 ```
switches 0:5c4d7b2438d3 185 $ python singletest.py -i test_spec.json -M muts_all.json --config
switches 0:5c4d7b2438d3 186 MUTs configuration in m.json:
switches 0:5c4d7b2438d3 187 +-------+-------------+---------------+------+-------+
switches 0:5c4d7b2438d3 188 | index | peripherals | mcu | disk | port |
switches 0:5c4d7b2438d3 189 +-------+-------------+---------------+------+-------+
switches 0:5c4d7b2438d3 190 | 1 | | LPC1768 | J:\ | COM4 |
switches 0:5c4d7b2438d3 191 | 3 | | NUCLEO_F103RB | I:\ | COM11 |
switches 0:5c4d7b2438d3 192 | 2 | | KL25Z | E:\ | COM39 |
switches 0:5c4d7b2438d3 193 +-------+-------------+---------------+------+-------+
switches 0:5c4d7b2438d3 194
switches 0:5c4d7b2438d3 195 Test specification in t.json:
switches 0:5c4d7b2438d3 196 +---------------+-----+------+
switches 0:5c4d7b2438d3 197 | mcu | ARM | uARM |
switches 0:5c4d7b2438d3 198 +---------------+-----+------+
switches 0:5c4d7b2438d3 199 | NUCLEO_F103RB | Yes | Yes |
switches 0:5c4d7b2438d3 200 | KL25Z | Yes | - |
switches 0:5c4d7b2438d3 201 | LPC1768 | Yes | Yes |
switches 0:5c4d7b2438d3 202 +---------------+-----+------+
switches 0:5c4d7b2438d3 203 ```
switches 0:5c4d7b2438d3 204 It should help you localize basic problems with configuration files and toolchain configuration.
switches 0:5c4d7b2438d3 205 Note: Configurations with issues will be marked with ```*``` sign.
switches 0:5c4d7b2438d3 206
switches 0:5c4d7b2438d3 207 Having multiple configuration files allows you to manage your test scenarios in more flexible manner. You can:
switches 0:5c4d7b2438d3 208
switches 0:5c4d7b2438d3 209 * Set up all platforms and toolchains used during testing.
switches 0:5c4d7b2438d3 210 * Define (using script's ```-n``` switch) which tests you want to run during testing.
switches 0:5c4d7b2438d3 211 * Just run regression (all tests). Regression is default setting for test script.
switches 0:5c4d7b2438d3 212
switches 0:5c4d7b2438d3 213 You can also force ```singletest.py``` script to:
switches 0:5c4d7b2438d3 214 * Run only peripherals' tests (switch ```-P```) or
switches 0:5c4d7b2438d3 215 * Just skip peripherals' tests (switch ```-C```).
switches 0:5c4d7b2438d3 216 * Build mbed SDK, libraries and corresponding tests with multiple cores, just use ```-j X``` option where ```X``` is number of cores you want to use for compilation.
switches 0:5c4d7b2438d3 217 ```
switches 0:5c4d7b2438d3 218 $ python singletest.py -i test_spec.json -M muts_all.json -j 8
switches 0:5c4d7b2438d3 219 ```
switches 0:5c4d7b2438d3 220 * Print test cases console output using ```-V``` option.
switches 0:5c4d7b2438d3 221 * Only build mbed SDK, tests and dependant libraries with switch ```-O```:
switches 0:5c4d7b2438d3 222 ```
switches 0:5c4d7b2438d3 223 $ python singletest.py -i test_spec.json -M muts_all.json -j 8 -O
switches 0:5c4d7b2438d3 224 ```
switches 0:5c4d7b2438d3 225 * Execute each test case multiple times with ```--global-loops X``` option, where ```X``` number of repeats. Additionally use option ```-W``` to continue repeating test cases execution only if they continue to fail.
switches 0:5c4d7b2438d3 226 ```
switches 0:5c4d7b2438d3 227 $ python singletest.py -i test_spec.json -M muts_all.json --global-loops 3 -W
switches 0:5c4d7b2438d3 228 ```
switches 0:5c4d7b2438d3 229 * Option ```--loops``` can be used to overwrite global loop count and redefine loop count for particular tests. Define test loops as ```TEST_ID=X``` where ```X``` is integer and separate loops count definitions by comma if necessary. E.g. ```TEST_1=5,TEST_2=20,TEST_3=2```.
switches 0:5c4d7b2438d3 230 ```
switches 0:5c4d7b2438d3 231 $ python singletest.py -i test_spec.json -M muts_all.json RTOS_1=10,RTOS_2=5
switches 0:5c4d7b2438d3 232 ```
switches 0:5c4d7b2438d3 233 This will execute test ```RTOS_1``` ten (10) times and test ```RTOS_2``` five (5) times.
switches 0:5c4d7b2438d3 234 * Force non default copy method. Note that mbed platforms can be flashed with just binary drag&drop. We simply copy file onto mbed's disk and interface chip flashes target MCU with given binary. Force non standard (Python specific) copy method by using option ```-c COPY_METHOD``` where ```COPY_METHOD``` can be shell, command line copy command like: ```cp```, ```copy````, ```xcopy``` etc. Make sure those commands are available from command line!
switches 0:5c4d7b2438d3 235 ```
switches 0:5c4d7b2438d3 236 $ python singletest.py -i test_spec.json -M muts_all.json -c cp
switches 0:5c4d7b2438d3 237 ```
switches 0:5c4d7b2438d3 238 * Run only selected tests. You can select which tests should be executed when you run test suite. Use ```-n``` switch to define tests by their ids you want to execute. Use comma to separate test ids:
switches 0:5c4d7b2438d3 239 ```
switches 0:5c4d7b2438d3 240 $ python singletest.py -i test_spec.json -M muts_all.json -n RTOS_1,RTOS_2,RTOS_3,MBED_10,MBED_16,MBED_11
switches 0:5c4d7b2438d3 241 ```
switches 0:5c4d7b2438d3 242 * Set common output binary name for all tests. In some cases you would like to have the same name for all tests. You can use switch ```--firmware-name``` to specify (without extension) build script output binary name.
switches 0:5c4d7b2438d3 243 In below example we would like to have all test binaries called ```firmware.bin`` (or with other extension like .elf, .hex depending on target accepted format).
switches 0:5c4d7b2438d3 244 ```
switches 0:5c4d7b2438d3 245 $ python singletest.py -i test_spec.json -M muts_all.json --firmware-name firmware
switches 0:5c4d7b2438d3 246 ```
switches 0:5c4d7b2438d3 247 * Where to find test list? Tests are defined in file ```tests.py``` in ```mbed/tools/``` directory. ```singletest.py``` uses test metadata in ```tests.py``` to resolve libraries dependencies and build tests for proper platforms and peripherals. Option ```-R``` can be used to get test names and direct path and test configuration.
switches 0:5c4d7b2438d3 248 ```
switches 0:5c4d7b2438d3 249 $ python singletest.py -R
switches 0:5c4d7b2438d3 250 +-------------+-----------+---------------------------------------+--------------+-------------------+----------+--------------------------------------------------------+
switches 0:5c4d7b2438d3 251 | id | automated | description | peripherals | host_test | duration | source_dir |
switches 0:5c4d7b2438d3 252 +-------------+-----------+---------------------------------------+--------------+-------------------+----------+--------------------------------------------------------+
switches 0:5c4d7b2438d3 253 | MBED_1 | False | I2C SRF08 | SRF08 | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\i2c_SRF08 |
switches 0:5c4d7b2438d3 254 | MBED_10 | True | Hello World | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\hello |
switches 0:5c4d7b2438d3 255 | MBED_11 | True | Ticker Int | - | host_test | 20 | C:\Work\mbed\libraries\tests\mbed\ticker |
switches 0:5c4d7b2438d3 256 | MBED_12 | True | C++ | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\cpp |
switches 0:5c4d7b2438d3 257 | MBED_13 | False | Heap & Stack | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\heap_and_stack |
switches 0:5c4d7b2438d3 258 | MBED_14 | False | Serial Interrupt | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\serial_interrupt |
switches 0:5c4d7b2438d3 259 | MBED_15 | False | RPC | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\rpc |
switches 0:5c4d7b2438d3 260 | MBED_16 | True | RTC | - | host_test | 15 | C:\Work\mbed\libraries\tests\mbed\rtc |
switches 0:5c4d7b2438d3 261 | MBED_17 | False | Serial Interrupt 2 | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\serial_interrupt_2 |
switches 0:5c4d7b2438d3 262 | MBED_18 | False | Local FS Directory | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\dir |
switches 0:5c4d7b2438d3 263 ...
switches 0:5c4d7b2438d3 264 ```
switches 0:5c4d7b2438d3 265 Note: you can filter tests by ```id``` column, just use ```-f``` option and give test name or regular expression:
switches 0:5c4d7b2438d3 266 ```
switches 0:5c4d7b2438d3 267 $ python singletest.py -R -f RTOS
switches 0:5c4d7b2438d3 268 +--------------+-----------+-------------------------+-------------+-----------+----------+---------------------------------------------------+
switches 0:5c4d7b2438d3 269 | id | automated | description | peripherals | host_test | duration | source_dir |
switches 0:5c4d7b2438d3 270 +--------------+-----------+-------------------------+-------------+-----------+----------+---------------------------------------------------+
switches 0:5c4d7b2438d3 271 | CMSIS_RTOS_1 | False | Basic | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\basic |
switches 0:5c4d7b2438d3 272 | CMSIS_RTOS_2 | False | Mutex | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\mutex |
switches 0:5c4d7b2438d3 273 | CMSIS_RTOS_3 | False | Semaphore | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\semaphore |
switches 0:5c4d7b2438d3 274 | CMSIS_RTOS_4 | False | Signals | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\signals |
switches 0:5c4d7b2438d3 275 | CMSIS_RTOS_5 | False | Queue | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\queue |
switches 0:5c4d7b2438d3 276 | CMSIS_RTOS_6 | False | Mail | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\mail |
switches 0:5c4d7b2438d3 277 | CMSIS_RTOS_7 | False | Timer | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\timer |
switches 0:5c4d7b2438d3 278 | CMSIS_RTOS_8 | False | ISR | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\isr |
switches 0:5c4d7b2438d3 279 | RTOS_1 | True | Basic thread | - | host_test | 15 | C:\Work\mbed\libraries\tests\rtos\mbed\basic |
switches 0:5c4d7b2438d3 280 | RTOS_2 | True | Mutex resource lock | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\mbed\mutex |
switches 0:5c4d7b2438d3 281 | RTOS_3 | True | Semaphore resource lock | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\mbed\semaphore |
switches 0:5c4d7b2438d3 282 | RTOS_4 | True | Signals messaging | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\signals |
switches 0:5c4d7b2438d3 283 | RTOS_5 | True | Queue messaging | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\queue |
switches 0:5c4d7b2438d3 284 | RTOS_6 | True | Mail messaging | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\mail |
switches 0:5c4d7b2438d3 285 | RTOS_7 | True | Timer | - | host_test | 15 | C:\Work\mbed\libraries\tests\rtos\mbed\timer |
switches 0:5c4d7b2438d3 286 | RTOS_8 | True | ISR (Queue) | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\isr |
switches 0:5c4d7b2438d3 287 | RTOS_9 | True | SD File write-read | SD | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\file |
switches 0:5c4d7b2438d3 288 +--------------+-----------+-------------------------+-------------+-----------+----------+---------------------------------------------------+
switches 0:5c4d7b2438d3 289 ```
switches 0:5c4d7b2438d3 290
switches 0:5c4d7b2438d3 291 * Shuffle your tests. We strongly encourage you to shuffle your test order each time you execute test suite script.
switches 0:5c4d7b2438d3 292 Rationale: It is probable that tests executed in one particular order will pass and in other will fail. To shuffle your tests’ order please use option ```-u``` (or ```--shuffle```):
switches 0:5c4d7b2438d3 293 ```
switches 0:5c4d7b2438d3 294 $ python singletest.py -i test_spec.json -M muts_all.json --shuffle
switches 0:5c4d7b2438d3 295 ```
switches 0:5c4d7b2438d3 296 Above command will force test script to randomly generate shuffle seed and shuffle test order execution. Note: Shuffle seed is float in ```[0.0, 1.0)```.
switches 0:5c4d7b2438d3 297
switches 0:5c4d7b2438d3 298 You can always recreate particular test order by forcing shuffle (```-u``` or ```--shuffle``` switch) and passing shuffle seed back to test suite using ```--shuffle-seed``` switch:
switches 0:5c4d7b2438d3 299 ```
switches 0:5c4d7b2438d3 300 $ python singletest.py -i test_spec.json -M muts_all.json --shuffle --shuffle-seed 0.4041028336
switches 0:5c4d7b2438d3 301 ```
switches 0:5c4d7b2438d3 302 Note: You can also supply your own randomly generated shuffle seed to drive particular test execution order scenarios. Just make sure shuffle seed is float in ```[0.0, 1.0)```.
switches 0:5c4d7b2438d3 303 You can find test shuffle seed in test summary:
switches 0:5c4d7b2438d3 304 ```
switches 0:5c4d7b2438d3 305 ...
switches 0:5c4d7b2438d3 306 | OK | LPC1768 | ARM | MBED_A9 | Serial Echo at 115200 | 2.84 | 10 | 1/1 |
switches 0:5c4d7b2438d3 307 +--------+---------+-----------+-----------+-----------------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 308 Result: 1 FAIL / 22 OK
switches 0:5c4d7b2438d3 309 Shuffle Seed: 0.4041028336
switches 0:5c4d7b2438d3 310
switches 0:5c4d7b2438d3 311 Completed in 234.85 sec
switches 0:5c4d7b2438d3 312 ```
switches 0:5c4d7b2438d3 313
switches 0:5c4d7b2438d3 314 ### Example of device configuration (one device connected to host computer)
switches 0:5c4d7b2438d3 315
switches 0:5c4d7b2438d3 316 This example will show you how to configure single device, run general tests or only peripheral tests. We will also show some real test result examples.
switches 0:5c4d7b2438d3 317
switches 0:5c4d7b2438d3 318 1. We will test only one board STMIcro Nucleo ```F334R8``` board connected to our PC (port ```COM46``` and disk is ```E:```).
switches 0:5c4d7b2438d3 319 2. We will also connect EEPROM ```24LC256``` to SDA, SCL pins of our Nucleo board and define 24LC256 peripheral to make sure our test suite will run all available tests for ```24LC256```.
switches 0:5c4d7b2438d3 320
switches 0:5c4d7b2438d3 321 Let's configure our one MUT and set uARM as the only compiler we will use to compiler Mbed SDK and tests.
switches 0:5c4d7b2438d3 322 We also need to create two configuration files ```muts_all.json``` and ```test_spec.json``` to pass our small testbed configuration to test script.
switches 0:5c4d7b2438d3 323
switches 0:5c4d7b2438d3 324 muts_all.json:
switches 0:5c4d7b2438d3 325 ```json
switches 0:5c4d7b2438d3 326 {
switches 0:5c4d7b2438d3 327 "1" : {
switches 0:5c4d7b2438d3 328 "mcu": "NUCLEO_F334R8",
switches 0:5c4d7b2438d3 329 "port":"COM46",
switches 0:5c4d7b2438d3 330 "disk":"E:\\",
switches 0:5c4d7b2438d3 331 "peripherals": ["24LC256"]
switches 0:5c4d7b2438d3 332 }
switches 0:5c4d7b2438d3 333 }
switches 0:5c4d7b2438d3 334 ```
switches 0:5c4d7b2438d3 335 Note: By defining ```"peripherals": ["24LC256"]``` we are passing to test suite information that this particular board has EEPROM 24LC256 connected to our board.
switches 0:5c4d7b2438d3 336
switches 0:5c4d7b2438d3 337 test_spec.json:
switches 0:5c4d7b2438d3 338 ```json
switches 0:5c4d7b2438d3 339 {
switches 0:5c4d7b2438d3 340 "targets": {
switches 0:5c4d7b2438d3 341 "NUCLEO_F334R8" : ["uARM"]
switches 0:5c4d7b2438d3 342 }
switches 0:5c4d7b2438d3 343 }
switches 0:5c4d7b2438d3 344 ```
switches 0:5c4d7b2438d3 345 Note:
switches 0:5c4d7b2438d3 346 * Please make sure device is connected before we will start running tests.
switches 0:5c4d7b2438d3 347 * Please make sure files ```muts_all.json``` and ```test_spec.json``` are in ```mbed/tools/``` directory.
switches 0:5c4d7b2438d3 348 Now you can call test suite and execute tests:
switches 0:5c4d7b2438d3 349 ```
switches 0:5c4d7b2438d3 350 $ python singletest.py -i test_spec.json -M muts_all.json
switches 0:5c4d7b2438d3 351 ...
switches 0:5c4d7b2438d3 352 Test summary:
switches 0:5c4d7b2438d3 353 +--------+---------------+-----------+-----------+---------------------------------+--------------------+---------------+
switches 0:5c4d7b2438d3 354 | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) |
switches 0:5c4d7b2438d3 355 +--------+---------------+-----------+-----------+---------------------------------+--------------------+---------------+
switches 0:5c4d7b2438d3 356 | OK | NUCLEO_F334R8 | uARM | MBED_A25 | I2C EEPROM line read/write test | 12.41 | 15 |
switches 0:5c4d7b2438d3 357 | OK | NUCLEO_F334R8 | uARM | MBED_A1 | Basic | 3.42 | 10 |
switches 0:5c4d7b2438d3 358 | OK | NUCLEO_F334R8 | uARM | EXAMPLE_1 | /dev/null | 3.42 | 10 |
switches 0:5c4d7b2438d3 359 | OK | NUCLEO_F334R8 | uARM | MBED_24 | Timeout Int us | 11.47 | 15 |
switches 0:5c4d7b2438d3 360 | OK | NUCLEO_F334R8 | uARM | MBED_25 | Time us | 11.43 | 15 |
switches 0:5c4d7b2438d3 361 | OK | NUCLEO_F334R8 | uARM | MBED_26 | Integer constant division | 3.37 | 10 |
switches 0:5c4d7b2438d3 362 | OK | NUCLEO_F334R8 | uARM | MBED_23 | Ticker Int us | 12.43 | 15 |
switches 0:5c4d7b2438d3 363 | OK | NUCLEO_F334R8 | uARM | MBED_A19 | I2C EEPROM read/write test | 11.42 | 15 |
switches 0:5c4d7b2438d3 364 | OK | NUCLEO_F334R8 | uARM | MBED_11 | Ticker Int | 12.43 | 20 |
switches 0:5c4d7b2438d3 365 | OK | NUCLEO_F334R8 | uARM | MBED_10 | Hello World | 2.42 | 10 |
switches 0:5c4d7b2438d3 366 | OK | NUCLEO_F334R8 | uARM | MBED_12 | C++ | 3.42 | 10 |
switches 0:5c4d7b2438d3 367 | OK | NUCLEO_F334R8 | uARM | MBED_16 | RTC | 4.76 | 15 |
switches 0:5c4d7b2438d3 368 | UNDEF | NUCLEO_F334R8 | uARM | MBED_2 | stdio | 20.42 | 20 |
switches 0:5c4d7b2438d3 369 | UNDEF | NUCLEO_F334R8 | uARM | MBED_A9 | Serial Echo at 115200 | 10.37 | 10 |
switches 0:5c4d7b2438d3 370 +--------+---------------+-----------+-----------+---------------------------------+--------------------+---------------+
switches 0:5c4d7b2438d3 371 Result: 2 UNDEF / 12 OK
switches 0:5c4d7b2438d3 372
switches 0:5c4d7b2438d3 373 Completed in 160 sec
switches 0:5c4d7b2438d3 374 ```
switches 0:5c4d7b2438d3 375
switches 0:5c4d7b2438d3 376 If we want to get additional test summary with results in separate columns please use option ```-t```.
switches 0:5c4d7b2438d3 377 ```
switches 0:5c4d7b2438d3 378 $ python singletest.py -i test_spec.json -M muts_all.json -t
switches 0:5c4d7b2438d3 379 ...
switches 0:5c4d7b2438d3 380 Test summary:
switches 0:5c4d7b2438d3 381 +---------------+-----------+---------------------------------+-------+
switches 0:5c4d7b2438d3 382 | Target | Test ID | Test Description | uARM |
switches 0:5c4d7b2438d3 383 +---------------+-----------+---------------------------------+-------+
switches 0:5c4d7b2438d3 384 | NUCLEO_F334R8 | EXAMPLE_1 | /dev/null | OK |
switches 0:5c4d7b2438d3 385 | NUCLEO_F334R8 | MBED_10 | Hello World | OK |
switches 0:5c4d7b2438d3 386 | NUCLEO_F334R8 | MBED_11 | Ticker Int | OK |
switches 0:5c4d7b2438d3 387 | NUCLEO_F334R8 | MBED_12 | C++ | OK |
switches 0:5c4d7b2438d3 388 | NUCLEO_F334R8 | MBED_16 | RTC | OK |
switches 0:5c4d7b2438d3 389 | NUCLEO_F334R8 | MBED_2 | stdio | UNDEF |
switches 0:5c4d7b2438d3 390 | NUCLEO_F334R8 | MBED_23 | Ticker Int us | OK |
switches 0:5c4d7b2438d3 391 | NUCLEO_F334R8 | MBED_24 | Timeout Int us | OK |
switches 0:5c4d7b2438d3 392 | NUCLEO_F334R8 | MBED_25 | Time us | OK |
switches 0:5c4d7b2438d3 393 | NUCLEO_F334R8 | MBED_26 | Integer constant division | OK |
switches 0:5c4d7b2438d3 394 | NUCLEO_F334R8 | MBED_A1 | Basic | OK |
switches 0:5c4d7b2438d3 395 | NUCLEO_F334R8 | MBED_A19 | I2C EEPROM read/write test | OK |
switches 0:5c4d7b2438d3 396 | NUCLEO_F334R8 | MBED_A25 | I2C EEPROM line read/write test | OK |
switches 0:5c4d7b2438d3 397 | NUCLEO_F334R8 | MBED_A9 | Serial Echo at 115200 | UNDEF |
switches 0:5c4d7b2438d3 398 +---------------+-----------+---------------------------------+-------+
switches 0:5c4d7b2438d3 399 ```
switches 0:5c4d7b2438d3 400 ----
switches 0:5c4d7b2438d3 401 Please do not forget you can combine few options together to get result you want. For example you want to repeat few tests multiple number of times, shuffle test ids execution order and select only tests which are critical for you at this point. You can do it using switch -n, --global-loops with --loops and --shuffle:
switches 0:5c4d7b2438d3 402
switches 0:5c4d7b2438d3 403 Execute above command to:
switches 0:5c4d7b2438d3 404
switches 0:5c4d7b2438d3 405 * Run only tests: ```RTOS_1```, ```RTOS_2```, ```RTOS_3```, ```MBED_10```, ```MBED_16```, ```MBED_11```.
switches 0:5c4d7b2438d3 406 * Shuffle test execution order. Note tests in loops will not be shuffled.
switches 0:5c4d7b2438d3 407 * Set global loop count to 3 - each test will repeated 3 times.
switches 0:5c4d7b2438d3 408 * Overwrite global loop count (set above to 3) and:
switches 0:5c4d7b2438d3 409 * Force to loop test RTOS_1 to execute 3 times.
switches 0:5c4d7b2438d3 410 * Force to loop test RTOS_2 to execute 4 times.
switches 0:5c4d7b2438d3 411 * Force to loop test RTOS_3 to execute 5 times.
switches 0:5c4d7b2438d3 412 * Force to loop test MBED_11 to execute 5 times.
switches 0:5c4d7b2438d3 413
switches 0:5c4d7b2438d3 414 ```
switches 0:5c4d7b2438d3 415 $ python singletest.py -i test_spec.json -M muts_all.json -n RTOS_1,RTOS_2,RTOS_3,MBED_10,MBED_16,MBED_11 --shuffle --global-loops 3 --loops RTOS_1=3,RTOS_2=4,RTOS_3=5,MBED_11=5
switches 0:5c4d7b2438d3 416 ```
switches 0:5c4d7b2438d3 417
switches 0:5c4d7b2438d3 418 # CppUTest unit test library support
switches 0:5c4d7b2438d3 419 ## CppUTest in Mbed SDK testing introduction
switches 0:5c4d7b2438d3 420 [CppUTest](http://cpputest.github.io/) is a C / C++ based unit xUnit test framework for unit testing and for test-driving your code. It is written in C++ but is used in C and C++ projects and frequently used in embedded systems but it works for any C / C++ project.
switches 0:5c4d7b2438d3 421
switches 0:5c4d7b2438d3 422 Mbed SDK test suite supports writing tests using CppUTest. All you need to do it to provide CppUTest sources and includes with Mbed SDK port. This is already done for you so all you need to do it to get proper sources in your project directory.
switches 0:5c4d7b2438d3 423 CppUTest’s core design principles are:
switches 0:5c4d7b2438d3 424 * Simple in design and simple in use.
switches 0:5c4d7b2438d3 425 * Portable to old and new platforms.
switches 0:5c4d7b2438d3 426 * Build with Test-driven Development in mind.
switches 0:5c4d7b2438d3 427
switches 0:5c4d7b2438d3 428 ## From where you can get more help about CppUTest library and unit testing
switches 0:5c4d7b2438d3 429 • You can read [CppUTest manual](http://cpputest.github.io/manual.html)
switches 0:5c4d7b2438d3 430 * [CppUTest forum](https://groups.google.com/forum/?fromgroups#!forum/cpputest)
switches 0:5c4d7b2438d3 431 * [CppUTest on GitHub](https://github.com/cpputest/cpputest)
switches 0:5c4d7b2438d3 432 * Finally, if you think unit testing is new concept for you, you can have a grasp of it on Wikipedia pages about [unit testing](http://en.wikipedia.org/wiki/Unit_testing) and continue from there.
switches 0:5c4d7b2438d3 433
switches 0:5c4d7b2438d3 434 ## How to add CppUTest to your current Mbed SDK installation
switches 0:5c4d7b2438d3 435
switches 0:5c4d7b2438d3 436 ### Do I need CppUTest port for Mbed SDK?
switches 0:5c4d7b2438d3 437 Yes, you do. If you want to use CppUTest with Mbed SDK you need to have CppUTest version with ARMCC compiler (only ARM flavor for now) port and Mbed SDK console port (if you want to have output on serial port). All is already prepared by Mbed engineers and you can get it for example here: http://mbed.org/users/rgrover1/code/CppUTest/
switches 0:5c4d7b2438d3 438
switches 0:5c4d7b2438d3 439 ### Prerequisites
switches 0:5c4d7b2438d3 440 * Installed [git client](http://git-scm.com/downloads/).
switches 0:5c4d7b2438d3 441 * Installed [Mercurial client](http://mercurial.selenic.com/).
switches 0:5c4d7b2438d3 442
switches 0:5c4d7b2438d3 443 ### How / where to install
switches 0:5c4d7b2438d3 444 We want to create directory structure similar to one below:
switches 0:5c4d7b2438d3 445 ```
switches 0:5c4d7b2438d3 446 \your_project_directory
switches 0:5c4d7b2438d3 447 │
switches 0:5c4d7b2438d3 448 ├───cpputest
switches 0:5c4d7b2438d3 449 │ ├───include
switches 0:5c4d7b2438d3 450 │ └───src
switches 0:5c4d7b2438d3 451 └───mbed
switches 0:5c4d7b2438d3 452 ├───libraries
switches 0:5c4d7b2438d3 453 ├───travis
switches 0:5c4d7b2438d3 454 └───tools
switches 0:5c4d7b2438d3 455 ```
switches 0:5c4d7b2438d3 456
switches 0:5c4d7b2438d3 457 Please go to directory with your project. For example it could be c:\Projects\Project.
switches 0:5c4d7b2438d3 458 ```
switches 0:5c4d7b2438d3 459 $ cd c:\Projects\Project
switches 0:5c4d7b2438d3 460 ```
switches 0:5c4d7b2438d3 461 If your project directory already has your mbed SDK repository included just execute below command (Mercurial console client). It should download CppUTest with Mbed SDK port.
switches 0:5c4d7b2438d3 462 ```
switches 0:5c4d7b2438d3 463 $ hg clone https://mbed.org/users/rgrover1/code/cpputest/
switches 0:5c4d7b2438d3 464 ```
switches 0:5c4d7b2438d3 465
switches 0:5c4d7b2438d3 466 You should see something like this after you execute Mercurial clone command:
switches 0:5c4d7b2438d3 467 ```
switches 0:5c4d7b2438d3 468 $ hg clone https://mbed.org/users/rgrover1/code/cpputest/
switches 0:5c4d7b2438d3 469 destination directory: cpputest
switches 0:5c4d7b2438d3 470 requesting all changes
switches 0:5c4d7b2438d3 471 adding changesets
switches 0:5c4d7b2438d3 472 adding manifests
switches 0:5c4d7b2438d3 473 adding file changes
switches 0:5c4d7b2438d3 474 added 3 changesets with 69 changes to 42 files
switches 0:5c4d7b2438d3 475 updating to branch default
switches 0:5c4d7b2438d3 476 41 files updated, 0 files merged, 0 files removed, 0 files unresolved
switches 0:5c4d7b2438d3 477 ```
switches 0:5c4d7b2438d3 478
switches 0:5c4d7b2438d3 479 Confirm your project structure. It should look more or less like this:
switches 0:5c4d7b2438d3 480 ```
switches 0:5c4d7b2438d3 481 $ ls
switches 0:5c4d7b2438d3 482 cpputest mbed
switches 0:5c4d7b2438d3 483 ```
switches 0:5c4d7b2438d3 484 From now on CppUTest is in correct path. Each time you want to compile unit tests for CppUTest build script will always look for CppUTest library in the same directory where mbed library is.
switches 0:5c4d7b2438d3 485
switches 0:5c4d7b2438d3 486 ## New off-line mbed SDK project with CppUTest support
switches 0:5c4d7b2438d3 487
switches 0:5c4d7b2438d3 488 If you are creating new mbed SDK project and you want to use CppUTest with it you need to download both mbed SDK and CppUTest with mbed port to the same directory. You can do it like this:
switches 0:5c4d7b2438d3 489 ```
switches 0:5c4d7b2438d3 490 $ cd c:\Projects\Project
switches 0:5c4d7b2438d3 491 $ git clone https://github.com/mbedmicro/mbed.git
switches 0:5c4d7b2438d3 492 $ hg clone https://mbed.org/users/rgrover1/code/cpputest/
switches 0:5c4d7b2438d3 493 ```
switches 0:5c4d7b2438d3 494
switches 0:5c4d7b2438d3 495 After above three steps you should have proper directory structure. All you need to do now is to configure your ```mbed_settings.py``` in ```mbed``` directory. Please refer to mbed SDK build script documentation for details.
switches 0:5c4d7b2438d3 496
switches 0:5c4d7b2438d3 497 ## CppUTest with mbed port
switches 0:5c4d7b2438d3 498 To make sure you actualy have CppUTest library with mbed SDK port you can go to CppUTest ```armcc``` platform directory:
switches 0:5c4d7b2438d3 499 ```
switches 0:5c4d7b2438d3 500 $ cd c:/Projects/Project/cpputest/src/Platforms/armcc/
switches 0:5c4d7b2438d3 501 ```
switches 0:5c4d7b2438d3 502 And open file ```UtestPlatform.cpp```.
switches 0:5c4d7b2438d3 503
switches 0:5c4d7b2438d3 504 You should find part of code responsible for porting console on default serial port of the mbed device:
switches 0:5c4d7b2438d3 505 ```c++
switches 0:5c4d7b2438d3 506 #include "Serial.h"
switches 0:5c4d7b2438d3 507 using namespace mbed;
switches 0:5c4d7b2438d3 508
switches 0:5c4d7b2438d3 509 int PlatformSpecificPutchar(int c)
switches 0:5c4d7b2438d3 510 {
switches 0:5c4d7b2438d3 511 /* Please modify this block for test results to be reported as
switches 0:5c4d7b2438d3 512 * console output. The following is a sample implementation using a
switches 0:5c4d7b2438d3 513 * Serial object connected to the console. */
switches 0:5c4d7b2438d3 514 #define NEED_TEST_REPORT_AS_CONSOLE_OUTPUT 1
switches 0:5c4d7b2438d3 515 #if NEED_TEST_REPORT_AS_CONSOLE_OUTPUT
switches 0:5c4d7b2438d3 516 extern Serial console;
switches 0:5c4d7b2438d3 517
switches 0:5c4d7b2438d3 518 #define NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE 1
switches 0:5c4d7b2438d3 519 #if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE
switches 0:5c4d7b2438d3 520 /* CppUTest emits \n line terminators in its reports; some terminals
switches 0:5c4d7b2438d3 521 * need the linefeed (\r) in addition. */
switches 0:5c4d7b2438d3 522 if (c == '\n') {
switches 0:5c4d7b2438d3 523 console.putc('\r');
switches 0:5c4d7b2438d3 524 }
switches 0:5c4d7b2438d3 525 #endif /* #if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE */
switches 0:5c4d7b2438d3 526
switches 0:5c4d7b2438d3 527 return (console.putc(c));
switches 0:5c4d7b2438d3 528 #else /* NEED_TEST_REPORT_AS_CONSOLE_OUTPUT */
switches 0:5c4d7b2438d3 529 return (0);
switches 0:5c4d7b2438d3 530 #endif /* NEED_TEST_REPORT_AS_CONSOLE_OUTPUT */
switches 0:5c4d7b2438d3 531 }
switches 0:5c4d7b2438d3 532 ```
switches 0:5c4d7b2438d3 533
switches 0:5c4d7b2438d3 534 You can find cpputest UT test runner main function in mbed sources: ```c:/Projects/Project/mbed/libraries/tests/utest/testrunner/testrunner.cpp```. Test runner code (in ```testrunner.cpp```) only defined console object and executes all unit tests:
switches 0:5c4d7b2438d3 535 ```c++
switches 0:5c4d7b2438d3 536 #include "CommandLineTestRunner.h"
switches 0:5c4d7b2438d3 537 #include <stdio.h>
switches 0:5c4d7b2438d3 538 #include "mbed.h"
switches 0:5c4d7b2438d3 539 #include "testrunner.h"
switches 0:5c4d7b2438d3 540 #include "test_env.h"
switches 0:5c4d7b2438d3 541
switches 0:5c4d7b2438d3 542 /**
switches 0:5c4d7b2438d3 543 Object 'mbed_cpputest_console' is used to show prints on console.
switches 0:5c4d7b2438d3 544 It is declared in \cpputest\src\Platforms\armcc\UtestPlatform.cpp
switches 0:5c4d7b2438d3 545 */
switches 0:5c4d7b2438d3 546 Serial mbed_cpputest_console(STDIO_UART_TX, STDIO_UART_RX);
switches 0:5c4d7b2438d3 547
switches 0:5c4d7b2438d3 548 int main(int ac, char** av) {
switches 0:5c4d7b2438d3 549 MBED_HOSTTEST_TIMEOUT(20);
switches 0:5c4d7b2438d3 550 MBED_HOSTTEST_SELECT(default_auto);
switches 0:5c4d7b2438d3 551 MBED_HOSTTEST_DESCRIPTION(Unit test);
switches 0:5c4d7b2438d3 552 MBED_HOSTTEST_START("UT");
switches 0:5c4d7b2438d3 553
switches 0:5c4d7b2438d3 554 unsigned failureCount = 0;
switches 0:5c4d7b2438d3 555 {
switches 0:5c4d7b2438d3 556 // Some compilers may not pass ac, av so we need to supply them ourselves
switches 0:5c4d7b2438d3 557 int ac = 2;
switches 0:5c4d7b2438d3 558 char* av[] = {__FILE__, "-v"};
switches 0:5c4d7b2438d3 559 failureCount = CommandLineTestRunner::RunAllTests(ac, av);
switches 0:5c4d7b2438d3 560 }
switches 0:5c4d7b2438d3 561
switches 0:5c4d7b2438d3 562 MBED_HOSTTEST_RESULT(failureCount == 0);
switches 0:5c4d7b2438d3 563 return failureCount;
switches 0:5c4d7b2438d3 564 }
switches 0:5c4d7b2438d3 565 ```
switches 0:5c4d7b2438d3 566
switches 0:5c4d7b2438d3 567 ## Unit test location
switches 0:5c4d7b2438d3 568 Unit tests source code is located in below directory: ```c:/Projects/Project/mbed/libraries/tests/utest/```
switches 0:5c4d7b2438d3 569
switches 0:5c4d7b2438d3 570 Each sub directory except testrunner contains compilable unit test source files with test groups and test cases. You can see utest structure below. Please note this is just example and in the future this directory will contain many sub directories with unit tests.
switches 0:5c4d7b2438d3 571 ```
switches 0:5c4d7b2438d3 572 $ c:\Projects\Project\mbed\libraries\tests\utest> tree
switches 0:5c4d7b2438d3 573 utest
switches 0:5c4d7b2438d3 574 ├───basic
switches 0:5c4d7b2438d3 575 ├───semihost_fs
switches 0:5c4d7b2438d3 576 └───testrunner
switches 0:5c4d7b2438d3 577 ```
switches 0:5c4d7b2438d3 578
switches 0:5c4d7b2438d3 579 ## Define unit tests in mbed SDK test suite structure
switches 0:5c4d7b2438d3 580 All tests defined in test suite are described in ```mbed/tools/tests.py``` file. This file stores data structure ```TESTS``` which is a list of simple structures describing each test. Below you can find example of ```TESTS``` structure which is configuring one of the unit tests.
switches 0:5c4d7b2438d3 581 ```
switches 0:5c4d7b2438d3 582 .
switches 0:5c4d7b2438d3 583 .
switches 0:5c4d7b2438d3 584 .
switches 0:5c4d7b2438d3 585 {
switches 0:5c4d7b2438d3 586 "id": "UT_2", "description": "Semihost file system",
switches 0:5c4d7b2438d3 587 "source_dir": join(TEST_DIR, "utest", "file"),
switches 0:5c4d7b2438d3 588 "dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
switches 0:5c4d7b2438d3 589 "automated": False,
switches 0:5c4d7b2438d3 590 "mcu": ["LPC1768", "LPC2368", "LPC11U24"]
switches 0:5c4d7b2438d3 591 },
switches 0:5c4d7b2438d3 592 .
switches 0:5c4d7b2438d3 593 .
switches 0:5c4d7b2438d3 594 .
switches 0:5c4d7b2438d3 595 ```
switches 0:5c4d7b2438d3 596 Note: In dependency section we've added library ```CPPUTEST_LIBRARY``` which is pointing build script to CppUTest library with mbed port. This is a must for unit tests to be compiled with CppUTest library.
switches 0:5c4d7b2438d3 597
switches 0:5c4d7b2438d3 598 ### Tests are now divided into two types:
switches 0:5c4d7b2438d3 599 #### 'Hello world' tests
switches 0:5c4d7b2438d3 600 First type of test cases we call 'hello world' tests. They do not dependent on CppUTest library and are monolithic programs usually composed of one main function. You can find this tests in below example directories:
switches 0:5c4d7b2438d3 601
switches 0:5c4d7b2438d3 602 * ```mbed/libraries/tests/mbed/```
switches 0:5c4d7b2438d3 603 * ```mbed/libraries/tests/net/```
switches 0:5c4d7b2438d3 604 * ```mbed/libraries/tests/rtos/```
switches 0:5c4d7b2438d3 605 * ```mbed/libraries/tests/usb/```
switches 0:5c4d7b2438d3 606
switches 0:5c4d7b2438d3 607 Usually ‘hello world’ test cases are using ```test_env.cpp``` and ```test_env.h``` files which implement simple test framework used to communicate with host test and help test framework instrument your tests.
switches 0:5c4d7b2438d3 608
switches 0:5c4d7b2438d3 609 Below you can see listing of ```test_env.h``` file which contains simple macro definitions used to communicate (via serial port printouts) between test case (on hardware) and host test script (on host computer).
switches 0:5c4d7b2438d3 610 Each use case should print on console basic information like:
switches 0:5c4d7b2438d3 611 * Default test case timeout.
switches 0:5c4d7b2438d3 612 * Which host test should be used to supervise test case execution.
switches 0:5c4d7b2438d3 613 * Test description and test case ID (short identifier).
switches 0:5c4d7b2438d3 614
switches 0:5c4d7b2438d3 615 ```c++
switches 0:5c4d7b2438d3 616 .
switches 0:5c4d7b2438d3 617 .
switches 0:5c4d7b2438d3 618 .
switches 0:5c4d7b2438d3 619 // Test result related notification functions
switches 0:5c4d7b2438d3 620 void notify_start();
switches 0:5c4d7b2438d3 621 void notify_completion(bool success);
switches 0:5c4d7b2438d3 622 bool notify_completion_str(bool success, char* buffer);
switches 0:5c4d7b2438d3 623 void notify_performance_coefficient(const char* measurement_name, const int value);
switches 0:5c4d7b2438d3 624 void notify_performance_coefficient(const char* measurement_name, const unsigned int value);
switches 0:5c4d7b2438d3 625 void notify_performance_coefficient(const char* measurement_name, const double value);
switches 0:5c4d7b2438d3 626
switches 0:5c4d7b2438d3 627 // Host test auto-detection API
switches 0:5c4d7b2438d3 628 void notify_host_test_name(const char *host_test);
switches 0:5c4d7b2438d3 629 void notify_timeout(int timeout);
switches 0:5c4d7b2438d3 630 void notify_test_id(const char *test_id);
switches 0:5c4d7b2438d3 631 void notify_test_description(const char *description);
switches 0:5c4d7b2438d3 632
switches 0:5c4d7b2438d3 633 // Host test auto-detection API
switches 0:5c4d7b2438d3 634 #define MBED_HOSTTEST_START(TESTID) notify_test_id(TESTID); notify_start()
switches 0:5c4d7b2438d3 635 #define MBED_HOSTTEST_SELECT(NAME) notify_host_test_name(#NAME)
switches 0:5c4d7b2438d3 636 #define MBED_HOSTTEST_TIMEOUT(SECONDS) notify_timeout(SECONDS)
switches 0:5c4d7b2438d3 637 #define MBED_HOSTTEST_DESCRIPTION(DESC) notify_test_description(#DESC)
switches 0:5c4d7b2438d3 638 #define MBED_HOSTTEST_RESULT(RESULT) notify_completion(RESULT)
switches 0:5c4d7b2438d3 639
switches 0:5c4d7b2438d3 640 /**
switches 0:5c4d7b2438d3 641 Test auto-detection preamble example:
switches 0:5c4d7b2438d3 642 main() {
switches 0:5c4d7b2438d3 643 MBED_HOSTTEST_TIMEOUT(10);
switches 0:5c4d7b2438d3 644 MBED_HOSTTEST_SELECT( host_test );
switches 0:5c4d7b2438d3 645 MBED_HOSTTEST_DESCRIPTION(Hello World);
switches 0:5c4d7b2438d3 646 MBED_HOSTTEST_START("MBED_10");
switches 0:5c4d7b2438d3 647 // Proper 'host_test.py' should take over supervising of this test
switches 0:5c4d7b2438d3 648
switches 0:5c4d7b2438d3 649 // Test code
switches 0:5c4d7b2438d3 650 bool result = ...;
switches 0:5c4d7b2438d3 651
switches 0:5c4d7b2438d3 652 MBED_HOSTTEST_RESULT(result);
switches 0:5c4d7b2438d3 653 }
switches 0:5c4d7b2438d3 654 */
switches 0:5c4d7b2438d3 655 .
switches 0:5c4d7b2438d3 656 .
switches 0:5c4d7b2438d3 657 .
switches 0:5c4d7b2438d3 658 ```
switches 0:5c4d7b2438d3 659
switches 0:5c4d7b2438d3 660 Example of 'hello world' test:
switches 0:5c4d7b2438d3 661 ```c++
switches 0:5c4d7b2438d3 662 #include "mbed.h"
switches 0:5c4d7b2438d3 663 #include "test_env.h"
switches 0:5c4d7b2438d3 664
switches 0:5c4d7b2438d3 665 #define CUSTOM_TIME 1256729737
switches 0:5c4d7b2438d3 666
switches 0:5c4d7b2438d3 667 int main() {
switches 0:5c4d7b2438d3 668 MBED_HOSTTEST_TIMEOUT(20);
switches 0:5c4d7b2438d3 669 MBED_HOSTTEST_SELECT(rtc_auto);
switches 0:5c4d7b2438d3 670 MBED_HOSTTEST_DESCRIPTION(RTC);
switches 0:5c4d7b2438d3 671 MBED_HOSTTEST_START("MBED_16");
switches 0:5c4d7b2438d3 672
switches 0:5c4d7b2438d3 673 char buffer[32] = {0};
switches 0:5c4d7b2438d3 674 set_time(CUSTOM_TIME); // Set RTC time to Wed, 28 Oct 2009 11:35:37
switches 0:5c4d7b2438d3 675 while(1) {
switches 0:5c4d7b2438d3 676 time_t seconds = time(NULL);
switches 0:5c4d7b2438d3 677 strftime(buffer, 32, "%Y-%m-%d %H:%M:%S %p", localtime(&seconds));
switches 0:5c4d7b2438d3 678 printf("MBED: [%ld] [%s]\r\n", seconds, buffer);
switches 0:5c4d7b2438d3 679 wait(1);
switches 0:5c4d7b2438d3 680 }
switches 0:5c4d7b2438d3 681 }
switches 0:5c4d7b2438d3 682 ```
switches 0:5c4d7b2438d3 683
switches 0:5c4d7b2438d3 684 #### 'Unit test' test cases
switches 0:5c4d7b2438d3 685 Second group of tests are unit tests. They are using CppUTest library and require you to write ```TEST_GROUP```s and ```TEST```s in your test files. Test suite will add test runner sources to your test automatically so you can concentrate on writing tests.
switches 0:5c4d7b2438d3 686
switches 0:5c4d7b2438d3 687 Example of unit test:
switches 0:5c4d7b2438d3 688 ```c++
switches 0:5c4d7b2438d3 689 #include "TestHarness.h"
switches 0:5c4d7b2438d3 690 #include <utility>
switches 0:5c4d7b2438d3 691 #include "mbed.h"
switches 0:5c4d7b2438d3 692
switches 0:5c4d7b2438d3 693 TEST_GROUP(BusOut_mask)
switches 0:5c4d7b2438d3 694 {
switches 0:5c4d7b2438d3 695 };
switches 0:5c4d7b2438d3 696
switches 0:5c4d7b2438d3 697 TEST(BusOut_mask, led_1_2_3)
switches 0:5c4d7b2438d3 698 {
switches 0:5c4d7b2438d3 699 BusOut bus_data(LED1, LED2, LED3);
switches 0:5c4d7b2438d3 700 CHECK_EQUAL(0x07, bus_data.mask());
switches 0:5c4d7b2438d3 701 }
switches 0:5c4d7b2438d3 702
switches 0:5c4d7b2438d3 703 TEST(BusOut_mask, led_nc_nc_nc_nc)
switches 0:5c4d7b2438d3 704 {
switches 0:5c4d7b2438d3 705 BusOut bus_data(NC, NC, NC, NC);
switches 0:5c4d7b2438d3 706 CHECK_EQUAL(0x00, bus_data.mask());
switches 0:5c4d7b2438d3 707 }
switches 0:5c4d7b2438d3 708
switches 0:5c4d7b2438d3 709 TEST(BusOut_mask, led_1_2_3_nc_nc)
switches 0:5c4d7b2438d3 710 {
switches 0:5c4d7b2438d3 711 BusOut bus_data(LED1, LED2, LED3, NC, NC);
switches 0:5c4d7b2438d3 712 CHECK_EQUAL(0x07, bus_data.mask());
switches 0:5c4d7b2438d3 713 }
switches 0:5c4d7b2438d3 714
switches 0:5c4d7b2438d3 715 TEST(BusOut_mask, led_1_nc_2_nc_nc_3)
switches 0:5c4d7b2438d3 716 {
switches 0:5c4d7b2438d3 717 BusOut bus_data(LED1, NC, LED2, NC, NC, LED3);
switches 0:5c4d7b2438d3 718 CHECK_EQUAL(0x25, bus_data.mask());
switches 0:5c4d7b2438d3 719 }
switches 0:5c4d7b2438d3 720 ```
switches 0:5c4d7b2438d3 721
switches 0:5c4d7b2438d3 722 ## Example
switches 0:5c4d7b2438d3 723 In below example we will run two example unit tests that are now available. tests ```UT_1``` and ```UT_2``` are unit tests used for now only to check if mbed SDK works with CppUTest library and if tests are being executed. In future number of unit tests will increase, nothing is also stopping you from writing and executing your own unit tests!
switches 0:5c4d7b2438d3 724
switches 0:5c4d7b2438d3 725 ### Example configuration
switches 0:5c4d7b2438d3 726 By default unit tests ```UT_1``` and ```UT_2``` are not automated - simply test suite will ignore them. Also we do not want to create dependency to CppUTest library each time someone executes automation.
switches 0:5c4d7b2438d3 727
switches 0:5c4d7b2438d3 728 Note: To compile ```UT_1``` and ```UT_2``` tests CppUTest library described above, installation is needed and not all users wish to add UT libs to their project. Also new to mbed users may find it difficult. This is why unit testing is an extra feature active only after you deliberately install and enable needed components.
switches 0:5c4d7b2438d3 729
switches 0:5c4d7b2438d3 730 Bellow snippet shows how to modify 'automated' flag so test suite will consider unit tests ```UT_1``` and ```UT_2``` as part of "automated test portfolio". Just change flag 'automated' from ```False``` to ```True```.
switches 0:5c4d7b2438d3 731
switches 0:5c4d7b2438d3 732 ```tests.py``` listing related to ```UT_1``` and ```UT_2```:
switches 0:5c4d7b2438d3 733 ```python
switches 0:5c4d7b2438d3 734 .
switches 0:5c4d7b2438d3 735 .
switches 0:5c4d7b2438d3 736 .
switches 0:5c4d7b2438d3 737 # CPPUTEST Library provides Unit testing Framework
switches 0:5c4d7b2438d3 738 #
switches 0:5c4d7b2438d3 739 # To write TESTs and TEST_GROUPs please add CPPUTEST_LIBRARY to 'dependencies'
switches 0:5c4d7b2438d3 740 #
switches 0:5c4d7b2438d3 741 # This will also include:
switches 0:5c4d7b2438d3 742 # 1. test runner - main function with call to CommandLineTestRunner::RunAllTests(ac, av)
switches 0:5c4d7b2438d3 743 # 2. Serial console object to print test result on serial port console
switches 0:5c4d7b2438d3 744 #
switches 0:5c4d7b2438d3 745
switches 0:5c4d7b2438d3 746 # Unit testing with cpputest library
switches 0:5c4d7b2438d3 747 {
switches 0:5c4d7b2438d3 748 "id": "UT_1", "description": "Basic",
switches 0:5c4d7b2438d3 749 "source_dir": join(TEST_DIR, "utest", "basic"),
switches 0:5c4d7b2438d3 750 "dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
switches 0:5c4d7b2438d3 751 "automated": True,
switches 0:5c4d7b2438d3 752 },
switches 0:5c4d7b2438d3 753 {
switches 0:5c4d7b2438d3 754 "id": "UT_2", "description": "Semihost file system",
switches 0:5c4d7b2438d3 755 "source_dir": join(TEST_DIR, "utest", "semihost_fs"),
switches 0:5c4d7b2438d3 756 "dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
switches 0:5c4d7b2438d3 757 "automated": True,
switches 0:5c4d7b2438d3 758 "mcu": ["LPC1768", "LPC2368", "LPC11U24"]
switches 0:5c4d7b2438d3 759 },
switches 0:5c4d7b2438d3 760 .
switches 0:5c4d7b2438d3 761 .
switches 0:5c4d7b2438d3 762 .
switches 0:5c4d7b2438d3 763 ```
switches 0:5c4d7b2438d3 764
switches 0:5c4d7b2438d3 765 ### Execute tests
switches 0:5c4d7b2438d3 766 In my test I will use common [LPC1768](http://developer.mbed.org/platforms/mbed-LPC1768/) mbed-enabled board because unit test ```UT_2``` is checking semi-host functionality which is available on this board and handful of others.
switches 0:5c4d7b2438d3 767
switches 0:5c4d7b2438d3 768 Configure your ```test_spec.json``` and ```muts_all.json``` files (refer to test suite build script and automation description) and set mbed disk and serial port.
switches 0:5c4d7b2438d3 769
switches 0:5c4d7b2438d3 770 ```
switches 0:5c4d7b2438d3 771 $ singletest.py -i test_spec.json -M muts_all.json -n UT_1,UT_2 -V
switches 0:5c4d7b2438d3 772 Building library CMSIS (LPC1768, ARM)
switches 0:5c4d7b2438d3 773 Building library MBED (LPC1768, ARM)
switches 0:5c4d7b2438d3 774 Building library CPPUTEST (LPC1768, ARM)
switches 0:5c4d7b2438d3 775 Building project BASIC (LPC1768, ARM)
switches 0:5c4d7b2438d3 776 Executing 'python host_test.py -p COM77 -d E:\ -t 10'
switches 0:5c4d7b2438d3 777 Test::Output::Start
switches 0:5c4d7b2438d3 778 Host test instrumentation on port: "COM77" and disk: "E:\"
switches 0:5c4d7b2438d3 779 TEST(FirstTestGroup, FirstTest) - 0 ms
switches 0:5c4d7b2438d3 780
switches 0:5c4d7b2438d3 781 OK (1 tests, 1 ran, 3 checks, 0 ignored, 0 filtered out, 3 ms)
switches 0:5c4d7b2438d3 782
switches 0:5c4d7b2438d3 783 {{success}}
switches 0:5c4d7b2438d3 784 {{end}}
switches 0:5c4d7b2438d3 785 Test::Output::Finish
switches 0:5c4d7b2438d3 786 TargetTest::LPC1768::ARM::UT_1::Basic [OK] in 2.43 of 10 sec
switches 0:5c4d7b2438d3 787 Building library CPPUTEST (LPC1768, ARM)
switches 0:5c4d7b2438d3 788 Building project SEMIHOST_FS (LPC1768, ARM)
switches 0:5c4d7b2438d3 789 Executing 'python host_test.py -p COM77 -d E:\ -t 10'
switches 0:5c4d7b2438d3 790 Test::Output::Start
switches 0:5c4d7b2438d3 791 Host test instrumentation on port: "COM77" and disk: "E:\"
switches 0:5c4d7b2438d3 792 TEST(FirstTestGroup, FirstTest) - 9 ms
switches 0:5c4d7b2438d3 793
switches 0:5c4d7b2438d3 794 OK (1 tests, 1 ran, 10 checks, 0 ignored, 0 filtered out, 10 ms)
switches 0:5c4d7b2438d3 795
switches 0:5c4d7b2438d3 796 {{success}}
switches 0:5c4d7b2438d3 797 {{end}}
switches 0:5c4d7b2438d3 798 Test::Output::Finish
switches 0:5c4d7b2438d3 799 TargetTest::LPC1768::ARM::UT_2::Semihost file system [OK] in 2.43 of 10 sec
switches 0:5c4d7b2438d3 800 Test summary:
switches 0:5c4d7b2438d3 801 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 802 | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
switches 0:5c4d7b2438d3 803 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 804 | OK | LPC1768 | ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 805 | OK | LPC1768 | ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 806 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 807 Result: 2 OK
switches 0:5c4d7b2438d3 808
switches 0:5c4d7b2438d3 809 Completed in 12.02 sec
switches 0:5c4d7b2438d3 810 ```
switches 0:5c4d7b2438d3 811
switches 0:5c4d7b2438d3 812 You can compile unit tests using various number of supported compilers, below just few examples with working solutions:
switches 0:5c4d7b2438d3 813 ```
switches 0:5c4d7b2438d3 814 Test summary:
switches 0:5c4d7b2438d3 815 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 816 | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
switches 0:5c4d7b2438d3 817 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 818 | OK | LPC1768 | ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 819 | OK | LPC1768 | ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 820 | OK | LPC1768 | uARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 821 | OK | LPC1768 | uARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 822 | OK | LPC1768 | GCC_ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 823 | OK | LPC1768 | GCC_ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 824 | OK | LPC1768 | GCC_CR | UT_1 | Basic | 3.44 | 10 | 1/1 |
switches 0:5c4d7b2438d3 825 | OK | LPC1768 | GCC_CR | UT_2 | Semihost file system | 3.43 | 10 | 1/1 |
switches 0:5c4d7b2438d3 826 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
switches 0:5c4d7b2438d3 827 Result: 8 OK
switches 0:5c4d7b2438d3 828
switches 0:5c4d7b2438d3 829 Completed in 55.85 sec
switches 0:5c4d7b2438d3 830 ```