Nitin Shukla / LED_Demo

nkjnm

Dependencies:   MAX44000 nexpaq_mdk

Fork of LED_Demo by Maxim nexpaq

mbd_os/docs/TESTING.md@1:55a6170b404f, 2016-09-17 (annotated)

Committer:
nexpaq
Date:
Sat Sep 17 16:32:05 2016 +0000
Revision:
1:55a6170b404f
checking in for sharing

Who changed what in which revision?

UserRevisionLine numberNew contents of line
nexpaq 1:55a6170b404f 1 # Mbed SDK automated test suite
nexpaq 1:55a6170b404f 2 ## Introduction
nexpaq 1:55a6170b404f 3
nexpaq 1:55a6170b404f 4 Test suit allows users to run locally on their machines Mbed SDK’s tests included in Mbed SDK repository. It also allows users to create their own tests and for example add new tests to test set as they progress with their project. If test is generic enough it could be included into official Mbed SDK test pool just do it via normal pull request!
nexpaq 1:55a6170b404f 5
nexpaq 1:55a6170b404f 6 Each test is supervised by python script called “host test” which will at least Test suite is using build script API to compile and build test source together with required by test libraries like CMSIS, Mbed, Ethernet, USB etc.
nexpaq 1:55a6170b404f 7
nexpaq 1:55a6170b404f 8 ## What is host test?
nexpaq 1:55a6170b404f 9 Test suite supports test supervisor concept. This concept is realized by separate Python script called ```host test```. Host tests can be found in ```mbed/tools/host_tests/``` directory. Note: In newer mbed versions (mbed OS) host tests will be separate library.
nexpaq 1:55a6170b404f 10
nexpaq 1:55a6170b404f 11 Host test script is executed in parallel with test runner to monitor test execution. Basic host test just monitors device's default serial port for test results returned by test runner. Simple tests will print test result on serial port. In other cases host tests can for example judge by test results returned by test runner if test passed or failed. It all depends on test itself.
nexpaq 1:55a6170b404f 12
nexpaq 1:55a6170b404f 13 In some cases host test can be TCP server echoing packets from test runner and judging packet loss. In other cases it can just check if values returned from accelerometer are actually valid (sane).
nexpaq 1:55a6170b404f 14
nexpaq 1:55a6170b404f 15 ## Test suite core: singletest.py script
nexpaq 1:55a6170b404f 16
nexpaq 1:55a6170b404f 17 ```singletest.py``` script located in ```mbed/tools/``` is a test suite script which allows users to compile, build tests and test runners (also supports CppUTest unit test library). Script also is responsible for test execution on devices selected by configuration files.
nexpaq 1:55a6170b404f 18
nexpaq 1:55a6170b404f 19 ### Parameters of singletest.py
nexpaq 1:55a6170b404f 20
nexpaq 1:55a6170b404f 21 Test execution script ```singletest.py``` is a fairly powerful tool to run tests for mbed SDK platform. It is flexible and allows users to configure test execution process and define which mbed platforms will be tested.
nexpaq 1:55a6170b404f 22
nexpaq 1:55a6170b404f 23 By specifying external configuration files (in JSON format) you can gain flexibility and prepare many different test scenarios. Just pass configuration file names to your script and run it.
nexpaq 1:55a6170b404f 24
nexpaq 1:55a6170b404f 25 #### MUTs Specification
nexpaq 1:55a6170b404f 26 You can easily configure your MUTs (Mbed Under Test) by creating configuration file with MUTs description.
nexpaq 1:55a6170b404f 27 Note: This configuration file must be in [JSON format](http://www.w3schools.com/json/).
nexpaq 1:55a6170b404f 28 Note: Unfortunately JSON format is not allowing you to have comments inside JSON code.
nexpaq 1:55a6170b404f 29
nexpaq 1:55a6170b404f 30 Let’s see some example and let's try to configure small "test farm" with three devices connected to your host computer. In this example no peripherals (like SD card or EEPROM) are connected to our Mbed boards. We will use three platforms in this example:
nexpaq 1:55a6170b404f 31 * [NXP LPC1768](https://mbed.org/platforms/mbed-LPC1768) board.
nexpaq 1:55a6170b404f 32 * \[Freescale KL25Z](https://mbed.org/platforms/KL25Z) board and
nexpaq 1:55a6170b404f 33 * [STMicro Nucleo F103RB](https://mbed.org/platforms/ST-Nucleo-F103RB) board.
nexpaq 1:55a6170b404f 34 After connecting boards to our host machine (PC) we can check which serial ports and disks they occupy. For our example let's assume that:
nexpaq 1:55a6170b404f 35 * ```LPC1768``` serial port is on ```COM4``` and disk drive is ```J:```.
nexpaq 1:55a6170b404f 36 * ```KL25Z``` serial port is on ```COM39``` and disk drive is ```E:```.
nexpaq 1:55a6170b404f 37 * ```NUCLEO_F103RB``` serial port is on ```COM11``` and disk drive is ```I:```.
nexpaq 1:55a6170b404f 38 If you are working under Linux your port and disk could look like /dev/ttyACM5 and /media/usb5.
nexpaq 1:55a6170b404f 39
nexpaq 1:55a6170b404f 40 This information is needed to create ```muts_all.json``` configuration file. You can create it in ```mbed/tools/``` directory:
nexpaq 1:55a6170b404f 41 ```
nexpaq 1:55a6170b404f 42 $ touch muts_all.json
nexpaq 1:55a6170b404f 43 ```
nexpaq 1:55a6170b404f 44
nexpaq 1:55a6170b404f 45 Its name will be passed to ```singletest.py``` script after ```-M``` (MUTs specification) switch. Let’s see how this file's content would look like in our example below:
nexpaq 1:55a6170b404f 46 ```json
nexpaq 1:55a6170b404f 47 {
nexpaq 1:55a6170b404f 48 "1" : {"mcu": "LPC1768",
nexpaq 1:55a6170b404f 49 "port":"COM4",
nexpaq 1:55a6170b404f 50 "disk":"J:\\",
nexpaq 1:55a6170b404f 51 "peripherals": []
nexpaq 1:55a6170b404f 52 },
nexpaq 1:55a6170b404f 53
nexpaq 1:55a6170b404f 54 "2" : {"mcu": "KL25Z",
nexpaq 1:55a6170b404f 55 "port":"COM39",
nexpaq 1:55a6170b404f 56 "disk":"E:\\",
nexpaq 1:55a6170b404f 57 "peripherals": []
nexpaq 1:55a6170b404f 58 },
nexpaq 1:55a6170b404f 59
nexpaq 1:55a6170b404f 60 "3" : {"mcu": "NUCLEO_F103RB",
nexpaq 1:55a6170b404f 61 "port":"COM11",
nexpaq 1:55a6170b404f 62 "disk":"I:\\",
nexpaq 1:55a6170b404f 63 "peripherals": []
nexpaq 1:55a6170b404f 64 }
nexpaq 1:55a6170b404f 65 }
nexpaq 1:55a6170b404f 66 ```
nexpaq 1:55a6170b404f 67
nexpaq 1:55a6170b404f 68 Note: We will leave field ```peripherals``` empty for the sake of this example. We will explain it later. All you need to do now is to properly fill fields ```mcu```, ```port``` and ```disk```.
nexpaq 1:55a6170b404f 69
nexpaq 1:55a6170b404f 70 Note: Please make sure files muts_all.json and test_spec.json are in tools/ directory. We will assume in this example they are.
nexpaq 1:55a6170b404f 71 Where to find ```mcu``` names? You can use option ```-S``` of ```build.py``` script (in ```mbed/tools/``` directory) to check all supported off-line MCUs names.
nexpaq 1:55a6170b404f 72
nexpaq 1:55a6170b404f 73 Note: If you update mbed device firmware or even disconnect / reconnect mbed device you may find that serial port / disk configuration changed. You need to update configuration file accordingly or you will face connection problems and obviously tests will not run.
nexpaq 1:55a6170b404f 74
nexpaq 1:55a6170b404f 75 #### Peripherals testing
nexpaq 1:55a6170b404f 76 When using MUTs configuration file (switch ```-M```) you can define in MUTs JSON file peripherals connected to your device:
nexpaq 1:55a6170b404f 77 ```json
nexpaq 1:55a6170b404f 78 {
nexpaq 1:55a6170b404f 79 "1" : {"mcu" : "KL25Z",
nexpaq 1:55a6170b404f 80 "port" : "COM39",
nexpaq 1:55a6170b404f 81 "disk" : "E:\\",
nexpaq 1:55a6170b404f 82 "peripherals" : ["SD", "24LC256"]}
nexpaq 1:55a6170b404f 83 }
nexpaq 1:55a6170b404f 84 ```
nexpaq 1:55a6170b404f 85 You can force test suite to run only common tests (switch ```-C```) or only peripheral tests (switch ```-P```).
nexpaq 1:55a6170b404f 86 ```
nexpaq 1:55a6170b404f 87 $ python singletest.py -i test_spec.json -M muts_all.json -C
nexpaq 1:55a6170b404f 88 ```
nexpaq 1:55a6170b404f 89 will not include tests for SD card and EEPROM 24LC256.
nexpaq 1:55a6170b404f 90 ```
nexpaq 1:55a6170b404f 91 $ python singletest.py -i test_spec.json -M muts_all.json -P
nexpaq 1:55a6170b404f 92 ```
nexpaq 1:55a6170b404f 93 will only run tests bind to SD card and EEPROM 24LC256.
nexpaq 1:55a6170b404f 94
nexpaq 1:55a6170b404f 95 Note: option ```-P``` is useful for example in cases when you have same platform and different shields you want to test. No need to test common part all the time (timers, RTC, RTOS etc.). You can force to test peripherals only on some devices and for example only common tests on other devices.
nexpaq 1:55a6170b404f 96
nexpaq 1:55a6170b404f 97 #### Additional MUTs configuration file settings
nexpaq 1:55a6170b404f 98 You can add extra information to each MUT configuration. In particular you can specify which flashing (binary copy method) should be used, how to reset target and for example set reset timeout (used to delay test execution just after reset).
nexpaq 1:55a6170b404f 99
nexpaq 1:55a6170b404f 100 muts_all.json:
nexpaq 1:55a6170b404f 101 ```json
nexpaq 1:55a6170b404f 102 {
nexpaq 1:55a6170b404f 103 "1" : {"mcu" : "LPC1768",
nexpaq 1:55a6170b404f 104 "port" : "COM77",
nexpaq 1:55a6170b404f 105 "disk" : "G:\\",
nexpaq 1:55a6170b404f 106 "peripherals" : ["TMP102", "digital_loop", "port_loop", "analog_loop", "SD"]},
nexpaq 1:55a6170b404f 107
nexpaq 1:55a6170b404f 108 "2" : {"mcu" : "KL25Z",
nexpaq 1:55a6170b404f 109 "port" : "COM89",
nexpaq 1:55a6170b404f 110 "disk" : "F:\\",
nexpaq 1:55a6170b404f 111 "peripherals" : ["SD", "24LC256", "KL25Z"],
nexpaq 1:55a6170b404f 112 "copy_method" : "copy",
nexpaq 1:55a6170b404f 113 "reset_type" : "default",
nexpaq 1:55a6170b404f 114 "reset_tout" : "2"},
nexpaq 1:55a6170b404f 115
nexpaq 1:55a6170b404f 116 "3" : {"mcu" : "LPC11U24",
nexpaq 1:55a6170b404f 117 "port" : "COM76",
nexpaq 1:55a6170b404f 118 "disk" : "E:\\",
nexpaq 1:55a6170b404f 119 "peripherals" : []}
nexpaq 1:55a6170b404f 120 }
nexpaq 1:55a6170b404f 121 ```
nexpaq 1:55a6170b404f 122 Please note that for MUT no. 2 few extra parameters were defined: ```copy_method```, ```reset_type``` and ```reset_tout```. Using this extra options you can tell test suite more about MUT you are using. This will allow you to be more flexible in terms of how you configure and use your MUTs.
nexpaq 1:55a6170b404f 123
nexpaq 1:55a6170b404f 124 * ```copy_method``` - STRING - tells test suite which binary copy method should be used.
nexpaq 1:55a6170b404f 125 You may notice that ```singletest.py``` command line help contains description about:
nexpaq 1:55a6170b404f 126 * Option ```-c``` (in MUTs file called ```copy_method```) with available copy methods supported by test suite plugin system.
nexpaq 1:55a6170b404f 127 * Option ```-r``` (in MUTs file called reset_type) with available reset methods supported by test suite plugin system.
nexpaq 1:55a6170b404f 128 * ```reset_type``` - STRING - some boards may require special reset handling, for example vendor specific command must be executed to reset device.
nexpaq 1:55a6170b404f 129 * ```reset_tout``` - INTEGER - extra timeout just after device is reseted. May be used to wait for few seconds so device may finish booting, flashing data internally etc.
nexpaq 1:55a6170b404f 130
nexpaq 1:55a6170b404f 131 Part of help listing for singletest.py:
nexpaq 1:55a6170b404f 132 ```
nexpaq 1:55a6170b404f 133 -c COPY_METHOD, --copy-method=COPY_METHOD
nexpaq 1:55a6170b404f 134 Select binary copy (flash) method. Default is Python's
nexpaq 1:55a6170b404f 135 shutil.copy() method. Plugin support: copy, cp,
nexpaq 1:55a6170b404f 136 default, eACommander, eACommander-usb, xcopy
nexpaq 1:55a6170b404f 137 -r MUT_RESET_TYPE, --reset-type=MUT_RESET_TYPE
nexpaq 1:55a6170b404f 138 Extra reset method used to reset MUT by host test
nexpaq 1:55a6170b404f 139 script. Plugin support: default, eACommander,
nexpaq 1:55a6170b404f 140 eACommander-usb
nexpaq 1:55a6170b404f 141 ```
nexpaq 1:55a6170b404f 142
nexpaq 1:55a6170b404f 143 ----
nexpaq 1:55a6170b404f 144
nexpaq 1:55a6170b404f 145 Now we've already defined how our devices are connected to our host PC. We can continue and define which of this MUTs will be tested and which compilers we will use to compile and build Mbed SDK and tests. To do so we need to create test specification file (let's call it ```test_spec.json```) and put inside our configuration file information about which MUTs we actually want to test. We will pass this file's name to ```singletest.py``` script using ```-i``` switch.
nexpaq 1:55a6170b404f 146
nexpaq 1:55a6170b404f 147 Below we can see how sample ```test_spec.json``` file content could look like. (I've included all possible toolchains, we will change it in a moment):
nexpaq 1:55a6170b404f 148 ```json
nexpaq 1:55a6170b404f 149 {
nexpaq 1:55a6170b404f 150 "targets": {
nexpaq 1:55a6170b404f 151 "LPC1768" : ["ARM", "uARM", "GCC_ARM", "GCC_CR", "IAR"],
nexpaq 1:55a6170b404f 152 "KL25Z" : ["ARM", "GCC_ARM"],
nexpaq 1:55a6170b404f 153 "NUCLEO_F103RB" : ["ARM", "uARM"]
nexpaq 1:55a6170b404f 154 }
nexpaq 1:55a6170b404f 155 }
nexpaq 1:55a6170b404f 156 ```
nexpaq 1:55a6170b404f 157 Above example configuration will force tests for LPC1768, KL25Z, NUCLEO_F103RB platforms and:
nexpaq 1:55a6170b404f 158
nexpaq 1:55a6170b404f 159 * Compilers: ```ARM```, ```uARM```, ```GCC_ARM```, ```GCC_CR``` and ```IAR``` will be used to compile tests for NXP's ```LPC1768```.
nexpaq 1:55a6170b404f 160 * Compilers: ```ARM``` and ```GCC_ARM``` will be used for Freescales' ```KL25Z``` platform.
nexpaq 1:55a6170b404f 161 * Compilers: ```ARM``` and ```uARM``` will be used for STMicro's ```NUCLEO_F103RB``` platform.
nexpaq 1:55a6170b404f 162
nexpaq 1:55a6170b404f 163 For our example purposes let's assume we only have Keil ARM compiler, so let's change configuration in ```test_spec.json``` file and reduce number of compiler to those we actually have:
nexpaq 1:55a6170b404f 164 ```json
nexpaq 1:55a6170b404f 165 {
nexpaq 1:55a6170b404f 166 "targets": {
nexpaq 1:55a6170b404f 167 "LPC1768" : ["ARM", "uARM"],
nexpaq 1:55a6170b404f 168 "KL25Z" : ["ARM"],
nexpaq 1:55a6170b404f 169 "NUCLEO_F103RB" : ["ARM", "uARM"]
nexpaq 1:55a6170b404f 170 }
nexpaq 1:55a6170b404f 171 }
nexpaq 1:55a6170b404f 172 ```
nexpaq 1:55a6170b404f 173 #### Run your tests
nexpaq 1:55a6170b404f 174
nexpaq 1:55a6170b404f 175 After you configure all your MUTs and compilers you are ready to run tests. Make sure your devices are connected and your configuration files reflect your current configuration (serial ports, devices). Go to tools directory in your mbed location.
nexpaq 1:55a6170b404f 176 ```
nexpaq 1:55a6170b404f 177 $ cd tools/
nexpaq 1:55a6170b404f 178 ```
nexpaq 1:55a6170b404f 179 and execute test suite script.
nexpaq 1:55a6170b404f 180 ```
nexpaq 1:55a6170b404f 181 $ python singletest.py -i test_spec.json -M muts_all.json
nexpaq 1:55a6170b404f 182 ```
nexpaq 1:55a6170b404f 183 To check your configuration before test execution please use ```--config``` switch:
nexpaq 1:55a6170b404f 184 ```
nexpaq 1:55a6170b404f 185 $ python singletest.py -i test_spec.json -M muts_all.json --config
nexpaq 1:55a6170b404f 186 MUTs configuration in m.json:
nexpaq 1:55a6170b404f 187 +-------+-------------+---------------+------+-------+
nexpaq 1:55a6170b404f 188 | index | peripherals | mcu | disk | port |
nexpaq 1:55a6170b404f 189 +-------+-------------+---------------+------+-------+
nexpaq 1:55a6170b404f 190 | 1 | | LPC1768 | J:\ | COM4 |
nexpaq 1:55a6170b404f 191 | 3 | | NUCLEO_F103RB | I:\ | COM11 |
nexpaq 1:55a6170b404f 192 | 2 | | KL25Z | E:\ | COM39 |
nexpaq 1:55a6170b404f 193 +-------+-------------+---------------+------+-------+
nexpaq 1:55a6170b404f 194
nexpaq 1:55a6170b404f 195 Test specification in t.json:
nexpaq 1:55a6170b404f 196 +---------------+-----+------+
nexpaq 1:55a6170b404f 197 | mcu | ARM | uARM |
nexpaq 1:55a6170b404f 198 +---------------+-----+------+
nexpaq 1:55a6170b404f 199 | NUCLEO_F103RB | Yes | Yes |
nexpaq 1:55a6170b404f 200 | KL25Z | Yes | - |
nexpaq 1:55a6170b404f 201 | LPC1768 | Yes | Yes |
nexpaq 1:55a6170b404f 202 +---------------+-----+------+
nexpaq 1:55a6170b404f 203 ```
nexpaq 1:55a6170b404f 204 It should help you localize basic problems with configuration files and toolchain configuration.
nexpaq 1:55a6170b404f 205 Note: Configurations with issues will be marked with ```*``` sign.
nexpaq 1:55a6170b404f 206
nexpaq 1:55a6170b404f 207 Having multiple configuration files allows you to manage your test scenarios in more flexible manner. You can:
nexpaq 1:55a6170b404f 208
nexpaq 1:55a6170b404f 209 * Set up all platforms and toolchains used during testing.
nexpaq 1:55a6170b404f 210 * Define (using script's ```-n``` switch) which tests you want to run during testing.
nexpaq 1:55a6170b404f 211 * Just run regression (all tests). Regression is default setting for test script.
nexpaq 1:55a6170b404f 212
nexpaq 1:55a6170b404f 213 You can also force ```singletest.py``` script to:
nexpaq 1:55a6170b404f 214 * Run only peripherals' tests (switch ```-P```) or
nexpaq 1:55a6170b404f 215 * Just skip peripherals' tests (switch ```-C```).
nexpaq 1:55a6170b404f 216 * Build mbed SDK, libraries and corresponding tests with multiple cores, just use ```-j X``` option where ```X``` is number of cores you want to use for compilation.
nexpaq 1:55a6170b404f 217 ```
nexpaq 1:55a6170b404f 218 $ python singletest.py -i test_spec.json -M muts_all.json -j 8
nexpaq 1:55a6170b404f 219 ```
nexpaq 1:55a6170b404f 220 * Print test cases console output using ```-V``` option.
nexpaq 1:55a6170b404f 221 * Only build mbed SDK, tests and dependant libraries with switch ```-O```:
nexpaq 1:55a6170b404f 222 ```
nexpaq 1:55a6170b404f 223 $ python singletest.py -i test_spec.json -M muts_all.json -j 8 -O
nexpaq 1:55a6170b404f 224 ```
nexpaq 1:55a6170b404f 225 * Execute each test case multiple times with ```--global-loops X``` option, where ```X``` number of repeats. Additionally use option ```-W``` to continue repeating test cases execution only if they continue to fail.
nexpaq 1:55a6170b404f 226 ```
nexpaq 1:55a6170b404f 227 $ python singletest.py -i test_spec.json -M muts_all.json --global-loops 3 -W
nexpaq 1:55a6170b404f 228 ```
nexpaq 1:55a6170b404f 229 * Option ```--loops``` can be used to overwrite global loop count and redefine loop count for particular tests. Define test loops as ```TEST_ID=X``` where ```X``` is integer and separate loops count definitions by comma if necessary. E.g. ```TEST_1=5,TEST_2=20,TEST_3=2```.
nexpaq 1:55a6170b404f 230 ```
nexpaq 1:55a6170b404f 231 $ python singletest.py -i test_spec.json -M muts_all.json RTOS_1=10,RTOS_2=5
nexpaq 1:55a6170b404f 232 ```
nexpaq 1:55a6170b404f 233 This will execute test ```RTOS_1``` ten (10) times and test ```RTOS_2``` five (5) times.
nexpaq 1:55a6170b404f 234 * Force non default copy method. Note that mbed platforms can be flashed with just binary drag&drop. We simply copy file onto mbed's disk and interface chip flashes target MCU with given binary. Force non standard (Python specific) copy method by using option ```-c COPY_METHOD``` where ```COPY_METHOD``` can be shell, command line copy command like: ```cp```, ```copy````, ```xcopy``` etc. Make sure those commands are available from command line!
nexpaq 1:55a6170b404f 235 ```
nexpaq 1:55a6170b404f 236 $ python singletest.py -i test_spec.json -M muts_all.json -c cp
nexpaq 1:55a6170b404f 237 ```
nexpaq 1:55a6170b404f 238 * Run only selected tests. You can select which tests should be executed when you run test suite. Use ```-n``` switch to define tests by their ids you want to execute. Use comma to separate test ids:
nexpaq 1:55a6170b404f 239 ```
nexpaq 1:55a6170b404f 240 $ python singletest.py -i test_spec.json -M muts_all.json -n RTOS_1,RTOS_2,RTOS_3,MBED_10,MBED_16,MBED_11
nexpaq 1:55a6170b404f 241 ```
nexpaq 1:55a6170b404f 242 * Set common output binary name for all tests. In some cases you would like to have the same name for all tests. You can use switch ```--firmware-name``` to specify (without extension) build script output binary name.
nexpaq 1:55a6170b404f 243 In below example we would like to have all test binaries called ```firmware.bin`` (or with other extension like .elf, .hex depending on target accepted format).
nexpaq 1:55a6170b404f 244 ```
nexpaq 1:55a6170b404f 245 $ python singletest.py -i test_spec.json -M muts_all.json --firmware-name firmware
nexpaq 1:55a6170b404f 246 ```
nexpaq 1:55a6170b404f 247 * Where to find test list? Tests are defined in file ```tests.py``` in ```mbed/tools/``` directory. ```singletest.py``` uses test metadata in ```tests.py``` to resolve libraries dependencies and build tests for proper platforms and peripherals. Option ```-R``` can be used to get test names and direct path and test configuration.
nexpaq 1:55a6170b404f 248 ```
nexpaq 1:55a6170b404f 249 $ python singletest.py -R
nexpaq 1:55a6170b404f 250 +-------------+-----------+---------------------------------------+--------------+-------------------+----------+--------------------------------------------------------+
nexpaq 1:55a6170b404f 251 | id | automated | description | peripherals | host_test | duration | source_dir |
nexpaq 1:55a6170b404f 252 +-------------+-----------+---------------------------------------+--------------+-------------------+----------+--------------------------------------------------------+
nexpaq 1:55a6170b404f 253 | MBED_1 | False | I2C SRF08 | SRF08 | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\i2c_SRF08 |
nexpaq 1:55a6170b404f 254 | MBED_10 | True | Hello World | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\hello |
nexpaq 1:55a6170b404f 255 | MBED_11 | True | Ticker Int | - | host_test | 20 | C:\Work\mbed\libraries\tests\mbed\ticker |
nexpaq 1:55a6170b404f 256 | MBED_12 | True | C++ | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\cpp |
nexpaq 1:55a6170b404f 257 | MBED_13 | False | Heap & Stack | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\heap_and_stack |
nexpaq 1:55a6170b404f 258 | MBED_14 | False | Serial Interrupt | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\serial_interrupt |
nexpaq 1:55a6170b404f 259 | MBED_15 | False | RPC | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\rpc |
nexpaq 1:55a6170b404f 260 | MBED_16 | True | RTC | - | host_test | 15 | C:\Work\mbed\libraries\tests\mbed\rtc |
nexpaq 1:55a6170b404f 261 | MBED_17 | False | Serial Interrupt 2 | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\serial_interrupt_2 |
nexpaq 1:55a6170b404f 262 | MBED_18 | False | Local FS Directory | - | host_test | 10 | C:\Work\mbed\libraries\tests\mbed\dir |
nexpaq 1:55a6170b404f 263 ...
nexpaq 1:55a6170b404f 264 ```
nexpaq 1:55a6170b404f 265 Note: you can filter tests by ```id``` column, just use ```-f``` option and give test name or regular expression:
nexpaq 1:55a6170b404f 266 ```
nexpaq 1:55a6170b404f 267 $ python singletest.py -R -f RTOS
nexpaq 1:55a6170b404f 268 +--------------+-----------+-------------------------+-------------+-----------+----------+---------------------------------------------------+
nexpaq 1:55a6170b404f 269 | id | automated | description | peripherals | host_test | duration | source_dir |
nexpaq 1:55a6170b404f 270 +--------------+-----------+-------------------------+-------------+-----------+----------+---------------------------------------------------+
nexpaq 1:55a6170b404f 271 | CMSIS_RTOS_1 | False | Basic | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\basic |
nexpaq 1:55a6170b404f 272 | CMSIS_RTOS_2 | False | Mutex | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\mutex |
nexpaq 1:55a6170b404f 273 | CMSIS_RTOS_3 | False | Semaphore | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\semaphore |
nexpaq 1:55a6170b404f 274 | CMSIS_RTOS_4 | False | Signals | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\signals |
nexpaq 1:55a6170b404f 275 | CMSIS_RTOS_5 | False | Queue | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\queue |
nexpaq 1:55a6170b404f 276 | CMSIS_RTOS_6 | False | Mail | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\cmsis\mail |
nexpaq 1:55a6170b404f 277 | CMSIS_RTOS_7 | False | Timer | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\timer |
nexpaq 1:55a6170b404f 278 | CMSIS_RTOS_8 | False | ISR | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\cmsis\isr |
nexpaq 1:55a6170b404f 279 | RTOS_1 | True | Basic thread | - | host_test | 15 | C:\Work\mbed\libraries\tests\rtos\mbed\basic |
nexpaq 1:55a6170b404f 280 | RTOS_2 | True | Mutex resource lock | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\mbed\mutex |
nexpaq 1:55a6170b404f 281 | RTOS_3 | True | Semaphore resource lock | - | host_test | 20 | C:\Work\mbed\libraries\tests\rtos\mbed\semaphore |
nexpaq 1:55a6170b404f 282 | RTOS_4 | True | Signals messaging | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\signals |
nexpaq 1:55a6170b404f 283 | RTOS_5 | True | Queue messaging | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\queue |
nexpaq 1:55a6170b404f 284 | RTOS_6 | True | Mail messaging | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\mail |
nexpaq 1:55a6170b404f 285 | RTOS_7 | True | Timer | - | host_test | 15 | C:\Work\mbed\libraries\tests\rtos\mbed\timer |
nexpaq 1:55a6170b404f 286 | RTOS_8 | True | ISR (Queue) | - | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\isr |
nexpaq 1:55a6170b404f 287 | RTOS_9 | True | SD File write-read | SD | host_test | 10 | C:\Work\mbed\libraries\tests\rtos\mbed\file |
nexpaq 1:55a6170b404f 288 +--------------+-----------+-------------------------+-------------+-----------+----------+---------------------------------------------------+
nexpaq 1:55a6170b404f 289 ```
nexpaq 1:55a6170b404f 290
nexpaq 1:55a6170b404f 291 * Shuffle your tests. We strongly encourage you to shuffle your test order each time you execute test suite script.
nexpaq 1:55a6170b404f 292 Rationale: It is probable that tests executed in one particular order will pass and in other will fail. To shuffle your tests’ order please use option ```-u``` (or ```--shuffle```):
nexpaq 1:55a6170b404f 293 ```
nexpaq 1:55a6170b404f 294 $ python singletest.py -i test_spec.json -M muts_all.json --shuffle
nexpaq 1:55a6170b404f 295 ```
nexpaq 1:55a6170b404f 296 Above command will force test script to randomly generate shuffle seed and shuffle test order execution. Note: Shuffle seed is float in ```[0.0, 1.0)```.
nexpaq 1:55a6170b404f 297
nexpaq 1:55a6170b404f 298 You can always recreate particular test order by forcing shuffle (```-u``` or ```--shuffle``` switch) and passing shuffle seed back to test suite using ```--shuffle-seed``` switch:
nexpaq 1:55a6170b404f 299 ```
nexpaq 1:55a6170b404f 300 $ python singletest.py -i test_spec.json -M muts_all.json --shuffle --shuffle-seed 0.4041028336
nexpaq 1:55a6170b404f 301 ```
nexpaq 1:55a6170b404f 302 Note: You can also supply your own randomly generated shuffle seed to drive particular test execution order scenarios. Just make sure shuffle seed is float in ```[0.0, 1.0)```.
nexpaq 1:55a6170b404f 303 You can find test shuffle seed in test summary:
nexpaq 1:55a6170b404f 304 ```
nexpaq 1:55a6170b404f 305 ...
nexpaq 1:55a6170b404f 306 | OK | LPC1768 | ARM | MBED_A9 | Serial Echo at 115200 | 2.84 | 10 | 1/1 |
nexpaq 1:55a6170b404f 307 +--------+---------+-----------+-----------+-----------------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 308 Result: 1 FAIL / 22 OK
nexpaq 1:55a6170b404f 309 Shuffle Seed: 0.4041028336
nexpaq 1:55a6170b404f 310
nexpaq 1:55a6170b404f 311 Completed in 234.85 sec
nexpaq 1:55a6170b404f 312 ```
nexpaq 1:55a6170b404f 313
nexpaq 1:55a6170b404f 314 ### Example of device configuration (one device connected to host computer)
nexpaq 1:55a6170b404f 315
nexpaq 1:55a6170b404f 316 This example will show you how to configure single device, run general tests or only peripheral tests. We will also show some real test result examples.
nexpaq 1:55a6170b404f 317
nexpaq 1:55a6170b404f 318 1. We will test only one board STMIcro Nucleo ```F334R8``` board connected to our PC (port ```COM46``` and disk is ```E:```).
nexpaq 1:55a6170b404f 319 2. We will also connect EEPROM ```24LC256``` to SDA, SCL pins of our Nucleo board and define 24LC256 peripheral to make sure our test suite will run all available tests for ```24LC256```.
nexpaq 1:55a6170b404f 320
nexpaq 1:55a6170b404f 321 Let's configure our one MUT and set uARM as the only compiler we will use to compiler Mbed SDK and tests.
nexpaq 1:55a6170b404f 322 We also need to create two configuration files ```muts_all.json``` and ```test_spec.json``` to pass our small testbed configuration to test script.
nexpaq 1:55a6170b404f 323
nexpaq 1:55a6170b404f 324 muts_all.json:
nexpaq 1:55a6170b404f 325 ```json
nexpaq 1:55a6170b404f 326 {
nexpaq 1:55a6170b404f 327 "1" : {
nexpaq 1:55a6170b404f 328 "mcu": "NUCLEO_F334R8",
nexpaq 1:55a6170b404f 329 "port":"COM46",
nexpaq 1:55a6170b404f 330 "disk":"E:\\",
nexpaq 1:55a6170b404f 331 "peripherals": ["24LC256"]
nexpaq 1:55a6170b404f 332 }
nexpaq 1:55a6170b404f 333 }
nexpaq 1:55a6170b404f 334 ```
nexpaq 1:55a6170b404f 335 Note: By defining ```"peripherals": ["24LC256"]``` we are passing to test suite information that this particular board has EEPROM 24LC256 connected to our board.
nexpaq 1:55a6170b404f 336
nexpaq 1:55a6170b404f 337 test_spec.json:
nexpaq 1:55a6170b404f 338 ```json
nexpaq 1:55a6170b404f 339 {
nexpaq 1:55a6170b404f 340 "targets": {
nexpaq 1:55a6170b404f 341 "NUCLEO_F334R8" : ["uARM"]
nexpaq 1:55a6170b404f 342 }
nexpaq 1:55a6170b404f 343 }
nexpaq 1:55a6170b404f 344 ```
nexpaq 1:55a6170b404f 345 Note:
nexpaq 1:55a6170b404f 346 * Please make sure device is connected before we will start running tests.
nexpaq 1:55a6170b404f 347 * Please make sure files ```muts_all.json``` and ```test_spec.json``` are in ```mbed/tools/``` directory.
nexpaq 1:55a6170b404f 348 Now you can call test suite and execute tests:
nexpaq 1:55a6170b404f 349 ```
nexpaq 1:55a6170b404f 350 $ python singletest.py -i test_spec.json -M muts_all.json
nexpaq 1:55a6170b404f 351 ...
nexpaq 1:55a6170b404f 352 Test summary:
nexpaq 1:55a6170b404f 353 +--------+---------------+-----------+-----------+---------------------------------+--------------------+---------------+
nexpaq 1:55a6170b404f 354 | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) |
nexpaq 1:55a6170b404f 355 +--------+---------------+-----------+-----------+---------------------------------+--------------------+---------------+
nexpaq 1:55a6170b404f 356 | OK | NUCLEO_F334R8 | uARM | MBED_A25 | I2C EEPROM line read/write test | 12.41 | 15 |
nexpaq 1:55a6170b404f 357 | OK | NUCLEO_F334R8 | uARM | MBED_A1 | Basic | 3.42 | 10 |
nexpaq 1:55a6170b404f 358 | OK | NUCLEO_F334R8 | uARM | EXAMPLE_1 | /dev/null | 3.42 | 10 |
nexpaq 1:55a6170b404f 359 | OK | NUCLEO_F334R8 | uARM | MBED_24 | Timeout Int us | 11.47 | 15 |
nexpaq 1:55a6170b404f 360 | OK | NUCLEO_F334R8 | uARM | MBED_25 | Time us | 11.43 | 15 |
nexpaq 1:55a6170b404f 361 | OK | NUCLEO_F334R8 | uARM | MBED_26 | Integer constant division | 3.37 | 10 |
nexpaq 1:55a6170b404f 362 | OK | NUCLEO_F334R8 | uARM | MBED_23 | Ticker Int us | 12.43 | 15 |
nexpaq 1:55a6170b404f 363 | OK | NUCLEO_F334R8 | uARM | MBED_A19 | I2C EEPROM read/write test | 11.42 | 15 |
nexpaq 1:55a6170b404f 364 | OK | NUCLEO_F334R8 | uARM | MBED_11 | Ticker Int | 12.43 | 20 |
nexpaq 1:55a6170b404f 365 | OK | NUCLEO_F334R8 | uARM | MBED_10 | Hello World | 2.42 | 10 |
nexpaq 1:55a6170b404f 366 | OK | NUCLEO_F334R8 | uARM | MBED_12 | C++ | 3.42 | 10 |
nexpaq 1:55a6170b404f 367 | OK | NUCLEO_F334R8 | uARM | MBED_16 | RTC | 4.76 | 15 |
nexpaq 1:55a6170b404f 368 | UNDEF | NUCLEO_F334R8 | uARM | MBED_2 | stdio | 20.42 | 20 |
nexpaq 1:55a6170b404f 369 | UNDEF | NUCLEO_F334R8 | uARM | MBED_A9 | Serial Echo at 115200 | 10.37 | 10 |
nexpaq 1:55a6170b404f 370 +--------+---------------+-----------+-----------+---------------------------------+--------------------+---------------+
nexpaq 1:55a6170b404f 371 Result: 2 UNDEF / 12 OK
nexpaq 1:55a6170b404f 372
nexpaq 1:55a6170b404f 373 Completed in 160 sec
nexpaq 1:55a6170b404f 374 ```
nexpaq 1:55a6170b404f 375
nexpaq 1:55a6170b404f 376 If we want to get additional test summary with results in separate columns please use option ```-t```.
nexpaq 1:55a6170b404f 377 ```
nexpaq 1:55a6170b404f 378 $ python singletest.py -i test_spec.json -M muts_all.json -t
nexpaq 1:55a6170b404f 379 ...
nexpaq 1:55a6170b404f 380 Test summary:
nexpaq 1:55a6170b404f 381 +---------------+-----------+---------------------------------+-------+
nexpaq 1:55a6170b404f 382 | Target | Test ID | Test Description | uARM |
nexpaq 1:55a6170b404f 383 +---------------+-----------+---------------------------------+-------+
nexpaq 1:55a6170b404f 384 | NUCLEO_F334R8 | EXAMPLE_1 | /dev/null | OK |
nexpaq 1:55a6170b404f 385 | NUCLEO_F334R8 | MBED_10 | Hello World | OK |
nexpaq 1:55a6170b404f 386 | NUCLEO_F334R8 | MBED_11 | Ticker Int | OK |
nexpaq 1:55a6170b404f 387 | NUCLEO_F334R8 | MBED_12 | C++ | OK |
nexpaq 1:55a6170b404f 388 | NUCLEO_F334R8 | MBED_16 | RTC | OK |
nexpaq 1:55a6170b404f 389 | NUCLEO_F334R8 | MBED_2 | stdio | UNDEF |
nexpaq 1:55a6170b404f 390 | NUCLEO_F334R8 | MBED_23 | Ticker Int us | OK |
nexpaq 1:55a6170b404f 391 | NUCLEO_F334R8 | MBED_24 | Timeout Int us | OK |
nexpaq 1:55a6170b404f 392 | NUCLEO_F334R8 | MBED_25 | Time us | OK |
nexpaq 1:55a6170b404f 393 | NUCLEO_F334R8 | MBED_26 | Integer constant division | OK |
nexpaq 1:55a6170b404f 394 | NUCLEO_F334R8 | MBED_A1 | Basic | OK |
nexpaq 1:55a6170b404f 395 | NUCLEO_F334R8 | MBED_A19 | I2C EEPROM read/write test | OK |
nexpaq 1:55a6170b404f 396 | NUCLEO_F334R8 | MBED_A25 | I2C EEPROM line read/write test | OK |
nexpaq 1:55a6170b404f 397 | NUCLEO_F334R8 | MBED_A9 | Serial Echo at 115200 | UNDEF |
nexpaq 1:55a6170b404f 398 +---------------+-----------+---------------------------------+-------+
nexpaq 1:55a6170b404f 399 ```
nexpaq 1:55a6170b404f 400 ----
nexpaq 1:55a6170b404f 401 Please do not forget you can combine few options together to get result you want. For example you want to repeat few tests multiple number of times, shuffle test ids execution order and select only tests which are critical for you at this point. You can do it using switch -n, --global-loops with --loops and --shuffle:
nexpaq 1:55a6170b404f 402
nexpaq 1:55a6170b404f 403 Execute above command to:
nexpaq 1:55a6170b404f 404
nexpaq 1:55a6170b404f 405 * Run only tests: ```RTOS_1```, ```RTOS_2```, ```RTOS_3```, ```MBED_10```, ```MBED_16```, ```MBED_11```.
nexpaq 1:55a6170b404f 406 * Shuffle test execution order. Note tests in loops will not be shuffled.
nexpaq 1:55a6170b404f 407 * Set global loop count to 3 - each test will repeated 3 times.
nexpaq 1:55a6170b404f 408 * Overwrite global loop count (set above to 3) and:
nexpaq 1:55a6170b404f 409 * Force to loop test RTOS_1 to execute 3 times.
nexpaq 1:55a6170b404f 410 * Force to loop test RTOS_2 to execute 4 times.
nexpaq 1:55a6170b404f 411 * Force to loop test RTOS_3 to execute 5 times.
nexpaq 1:55a6170b404f 412 * Force to loop test MBED_11 to execute 5 times.
nexpaq 1:55a6170b404f 413
nexpaq 1:55a6170b404f 414 ```
nexpaq 1:55a6170b404f 415 $ python singletest.py -i test_spec.json -M muts_all.json -n RTOS_1,RTOS_2,RTOS_3,MBED_10,MBED_16,MBED_11 --shuffle --global-loops 3 --loops RTOS_1=3,RTOS_2=4,RTOS_3=5,MBED_11=5
nexpaq 1:55a6170b404f 416 ```
nexpaq 1:55a6170b404f 417
nexpaq 1:55a6170b404f 418 # CppUTest unit test library support
nexpaq 1:55a6170b404f 419 ## CppUTest in Mbed SDK testing introduction
nexpaq 1:55a6170b404f 420 [CppUTest](http://cpputest.github.io/) is a C / C++ based unit xUnit test framework for unit testing and for test-driving your code. It is written in C++ but is used in C and C++ projects and frequently used in embedded systems but it works for any C / C++ project.
nexpaq 1:55a6170b404f 421
nexpaq 1:55a6170b404f 422 Mbed SDK test suite supports writing tests using CppUTest. All you need to do it to provide CppUTest sources and includes with Mbed SDK port. This is already done for you so all you need to do it to get proper sources in your project directory.
nexpaq 1:55a6170b404f 423 CppUTest’s core design principles are:
nexpaq 1:55a6170b404f 424 * Simple in design and simple in use.
nexpaq 1:55a6170b404f 425 * Portable to old and new platforms.
nexpaq 1:55a6170b404f 426 * Build with Test-driven Development in mind.
nexpaq 1:55a6170b404f 427
nexpaq 1:55a6170b404f 428 ## From where you can get more help about CppUTest library and unit testing
nexpaq 1:55a6170b404f 429 • You can read [CppUTest manual](http://cpputest.github.io/manual.html)
nexpaq 1:55a6170b404f 430 * [CppUTest forum](https://groups.google.com/forum/?fromgroups#!forum/cpputest)
nexpaq 1:55a6170b404f 431 * [CppUTest on GitHub](https://github.com/cpputest/cpputest)
nexpaq 1:55a6170b404f 432 * Finally, if you think unit testing is new concept for you, you can have a grasp of it on Wikipedia pages about [unit testing](http://en.wikipedia.org/wiki/Unit_testing) and continue from there.
nexpaq 1:55a6170b404f 433
nexpaq 1:55a6170b404f 434 ## How to add CppUTest to your current Mbed SDK installation
nexpaq 1:55a6170b404f 435
nexpaq 1:55a6170b404f 436 ### Do I need CppUTest port for Mbed SDK?
nexpaq 1:55a6170b404f 437 Yes, you do. If you want to use CppUTest with Mbed SDK you need to have CppUTest version with ARMCC compiler (only ARM flavor for now) port and Mbed SDK console port (if you want to have output on serial port). All is already prepared by Mbed engineers and you can get it for example here: http://mbed.org/users/rgrover1/code/CppUTest/
nexpaq 1:55a6170b404f 438
nexpaq 1:55a6170b404f 439 ### Prerequisites
nexpaq 1:55a6170b404f 440 * Installed [git client](http://git-scm.com/downloads/).
nexpaq 1:55a6170b404f 441 * Installed [Mercurial client](http://mercurial.selenic.com/).
nexpaq 1:55a6170b404f 442
nexpaq 1:55a6170b404f 443 ### How / where to install
nexpaq 1:55a6170b404f 444 We want to create directory structure similar to one below:
nexpaq 1:55a6170b404f 445 ```
nexpaq 1:55a6170b404f 446 \your_project_directory
nexpaq 1:55a6170b404f 447 │
nexpaq 1:55a6170b404f 448 ├───cpputest
nexpaq 1:55a6170b404f 449 │ ├───include
nexpaq 1:55a6170b404f 450 │ └───src
nexpaq 1:55a6170b404f 451 └───mbed
nexpaq 1:55a6170b404f 452 ├───libraries
nexpaq 1:55a6170b404f 453 ├───travis
nexpaq 1:55a6170b404f 454 └───tools
nexpaq 1:55a6170b404f 455 ```
nexpaq 1:55a6170b404f 456
nexpaq 1:55a6170b404f 457 Please go to directory with your project. For example it could be c:\Projects\Project.
nexpaq 1:55a6170b404f 458 ```
nexpaq 1:55a6170b404f 459 $ cd c:\Projects\Project
nexpaq 1:55a6170b404f 460 ```
nexpaq 1:55a6170b404f 461 If your project directory already has your mbed SDK repository included just execute below command (Mercurial console client). It should download CppUTest with Mbed SDK port.
nexpaq 1:55a6170b404f 462 ```
nexpaq 1:55a6170b404f 463 $ hg clone https://mbed.org/users/rgrover1/code/cpputest/
nexpaq 1:55a6170b404f 464 ```
nexpaq 1:55a6170b404f 465
nexpaq 1:55a6170b404f 466 You should see something like this after you execute Mercurial clone command:
nexpaq 1:55a6170b404f 467 ```
nexpaq 1:55a6170b404f 468 $ hg clone https://mbed.org/users/rgrover1/code/cpputest/
nexpaq 1:55a6170b404f 469 destination directory: cpputest
nexpaq 1:55a6170b404f 470 requesting all changes
nexpaq 1:55a6170b404f 471 adding changesets
nexpaq 1:55a6170b404f 472 adding manifests
nexpaq 1:55a6170b404f 473 adding file changes
nexpaq 1:55a6170b404f 474 added 3 changesets with 69 changes to 42 files
nexpaq 1:55a6170b404f 475 updating to branch default
nexpaq 1:55a6170b404f 476 41 files updated, 0 files merged, 0 files removed, 0 files unresolved
nexpaq 1:55a6170b404f 477 ```
nexpaq 1:55a6170b404f 478
nexpaq 1:55a6170b404f 479 Confirm your project structure. It should look more or less like this:
nexpaq 1:55a6170b404f 480 ```
nexpaq 1:55a6170b404f 481 $ ls
nexpaq 1:55a6170b404f 482 cpputest mbed
nexpaq 1:55a6170b404f 483 ```
nexpaq 1:55a6170b404f 484 From now on CppUTest is in correct path. Each time you want to compile unit tests for CppUTest build script will always look for CppUTest library in the same directory where mbed library is.
nexpaq 1:55a6170b404f 485
nexpaq 1:55a6170b404f 486 ## New off-line mbed SDK project with CppUTest support
nexpaq 1:55a6170b404f 487
nexpaq 1:55a6170b404f 488 If you are creating new mbed SDK project and you want to use CppUTest with it you need to download both mbed SDK and CppUTest with mbed port to the same directory. You can do it like this:
nexpaq 1:55a6170b404f 489 ```
nexpaq 1:55a6170b404f 490 $ cd c:\Projects\Project
nexpaq 1:55a6170b404f 491 $ git clone https://github.com/mbedmicro/mbed.git
nexpaq 1:55a6170b404f 492 $ hg clone https://mbed.org/users/rgrover1/code/cpputest/
nexpaq 1:55a6170b404f 493 ```
nexpaq 1:55a6170b404f 494
nexpaq 1:55a6170b404f 495 After above three steps you should have proper directory structure. All you need to do now is to configure your ```mbed_settings.py``` in ```mbed``` directory. Please refer to mbed SDK build script documentation for details.
nexpaq 1:55a6170b404f 496
nexpaq 1:55a6170b404f 497 ## CppUTest with mbed port
nexpaq 1:55a6170b404f 498 To make sure you actualy have CppUTest library with mbed SDK port you can go to CppUTest ```armcc``` platform directory:
nexpaq 1:55a6170b404f 499 ```
nexpaq 1:55a6170b404f 500 $ cd c:/Projects/Project/cpputest/src/Platforms/armcc/
nexpaq 1:55a6170b404f 501 ```
nexpaq 1:55a6170b404f 502 And open file ```UtestPlatform.cpp```.
nexpaq 1:55a6170b404f 503
nexpaq 1:55a6170b404f 504 You should find part of code responsible for porting console on default serial port of the mbed device:
nexpaq 1:55a6170b404f 505 ```c++
nexpaq 1:55a6170b404f 506 #include "Serial.h"
nexpaq 1:55a6170b404f 507 using namespace mbed;
nexpaq 1:55a6170b404f 508
nexpaq 1:55a6170b404f 509 int PlatformSpecificPutchar(int c)
nexpaq 1:55a6170b404f 510 {
nexpaq 1:55a6170b404f 511 /* Please modify this block for test results to be reported as
nexpaq 1:55a6170b404f 512 * console output. The following is a sample implementation using a
nexpaq 1:55a6170b404f 513 * Serial object connected to the console. */
nexpaq 1:55a6170b404f 514 #define NEED_TEST_REPORT_AS_CONSOLE_OUTPUT 1
nexpaq 1:55a6170b404f 515 #if NEED_TEST_REPORT_AS_CONSOLE_OUTPUT
nexpaq 1:55a6170b404f 516 extern Serial console;
nexpaq 1:55a6170b404f 517
nexpaq 1:55a6170b404f 518 #define NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE 1
nexpaq 1:55a6170b404f 519 #if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE
nexpaq 1:55a6170b404f 520 /* CppUTest emits \n line terminators in its reports; some terminals
nexpaq 1:55a6170b404f 521 * need the linefeed (\r) in addition. */
nexpaq 1:55a6170b404f 522 if (c == '\n') {
nexpaq 1:55a6170b404f 523 console.putc('\r');
nexpaq 1:55a6170b404f 524 }
nexpaq 1:55a6170b404f 525 #endif /* #if NEED_LINE_FEED_IN_ADDITION_TO_NEWLINE */
nexpaq 1:55a6170b404f 526
nexpaq 1:55a6170b404f 527 return (console.putc(c));
nexpaq 1:55a6170b404f 528 #else /* NEED_TEST_REPORT_AS_CONSOLE_OUTPUT */
nexpaq 1:55a6170b404f 529 return (0);
nexpaq 1:55a6170b404f 530 #endif /* NEED_TEST_REPORT_AS_CONSOLE_OUTPUT */
nexpaq 1:55a6170b404f 531 }
nexpaq 1:55a6170b404f 532 ```
nexpaq 1:55a6170b404f 533
nexpaq 1:55a6170b404f 534 You can find cpputest UT test runner main function in mbed sources: ```c:/Projects/Project/mbed/libraries/tests/utest/testrunner/testrunner.cpp```. Test runner code (in ```testrunner.cpp```) only defined console object and executes all unit tests:
nexpaq 1:55a6170b404f 535 ```c++
nexpaq 1:55a6170b404f 536 #include "CommandLineTestRunner.h"
nexpaq 1:55a6170b404f 537 #include <stdio.h>
nexpaq 1:55a6170b404f 538 #include "mbed.h"
nexpaq 1:55a6170b404f 539 #include "testrunner.h"
nexpaq 1:55a6170b404f 540 #include "test_env.h"
nexpaq 1:55a6170b404f 541
nexpaq 1:55a6170b404f 542 /**
nexpaq 1:55a6170b404f 543 Object 'mbed_cpputest_console' is used to show prints on console.
nexpaq 1:55a6170b404f 544 It is declared in \cpputest\src\Platforms\armcc\UtestPlatform.cpp
nexpaq 1:55a6170b404f 545 */
nexpaq 1:55a6170b404f 546 Serial mbed_cpputest_console(STDIO_UART_TX, STDIO_UART_RX);
nexpaq 1:55a6170b404f 547
nexpaq 1:55a6170b404f 548 int main(int ac, char** av) {
nexpaq 1:55a6170b404f 549 MBED_HOSTTEST_TIMEOUT(20);
nexpaq 1:55a6170b404f 550 MBED_HOSTTEST_SELECT(default_auto);
nexpaq 1:55a6170b404f 551 MBED_HOSTTEST_DESCRIPTION(Unit test);
nexpaq 1:55a6170b404f 552 MBED_HOSTTEST_START("UT");
nexpaq 1:55a6170b404f 553
nexpaq 1:55a6170b404f 554 unsigned failureCount = 0;
nexpaq 1:55a6170b404f 555 {
nexpaq 1:55a6170b404f 556 // Some compilers may not pass ac, av so we need to supply them ourselves
nexpaq 1:55a6170b404f 557 int ac = 2;
nexpaq 1:55a6170b404f 558 char* av[] = {__FILE__, "-v"};
nexpaq 1:55a6170b404f 559 failureCount = CommandLineTestRunner::RunAllTests(ac, av);
nexpaq 1:55a6170b404f 560 }
nexpaq 1:55a6170b404f 561
nexpaq 1:55a6170b404f 562 MBED_HOSTTEST_RESULT(failureCount == 0);
nexpaq 1:55a6170b404f 563 return failureCount;
nexpaq 1:55a6170b404f 564 }
nexpaq 1:55a6170b404f 565 ```
nexpaq 1:55a6170b404f 566
nexpaq 1:55a6170b404f 567 ## Unit test location
nexpaq 1:55a6170b404f 568 Unit tests source code is located in below directory: ```c:/Projects/Project/mbed/libraries/tests/utest/```
nexpaq 1:55a6170b404f 569
nexpaq 1:55a6170b404f 570 Each sub directory except testrunner contains compilable unit test source files with test groups and test cases. You can see utest structure below. Please note this is just example and in the future this directory will contain many sub directories with unit tests.
nexpaq 1:55a6170b404f 571 ```
nexpaq 1:55a6170b404f 572 $ c:\Projects\Project\mbed\libraries\tests\utest> tree
nexpaq 1:55a6170b404f 573 utest
nexpaq 1:55a6170b404f 574 ├───basic
nexpaq 1:55a6170b404f 575 ├───semihost_fs
nexpaq 1:55a6170b404f 576 └───testrunner
nexpaq 1:55a6170b404f 577 ```
nexpaq 1:55a6170b404f 578
nexpaq 1:55a6170b404f 579 ## Define unit tests in mbed SDK test suite structure
nexpaq 1:55a6170b404f 580 All tests defined in test suite are described in ```mbed/tools/tests.py``` file. This file stores data structure ```TESTS``` which is a list of simple structures describing each test. Below you can find example of ```TESTS``` structure which is configuring one of the unit tests.
nexpaq 1:55a6170b404f 581 ```
nexpaq 1:55a6170b404f 582 .
nexpaq 1:55a6170b404f 583 .
nexpaq 1:55a6170b404f 584 .
nexpaq 1:55a6170b404f 585 {
nexpaq 1:55a6170b404f 586 "id": "UT_2", "description": "Semihost file system",
nexpaq 1:55a6170b404f 587 "source_dir": join(TEST_DIR, "utest", "file"),
nexpaq 1:55a6170b404f 588 "dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
nexpaq 1:55a6170b404f 589 "automated": False,
nexpaq 1:55a6170b404f 590 "mcu": ["LPC1768", "LPC2368", "LPC11U24"]
nexpaq 1:55a6170b404f 591 },
nexpaq 1:55a6170b404f 592 .
nexpaq 1:55a6170b404f 593 .
nexpaq 1:55a6170b404f 594 .
nexpaq 1:55a6170b404f 595 ```
nexpaq 1:55a6170b404f 596 Note: In dependency section we've added library ```CPPUTEST_LIBRARY``` which is pointing build script to CppUTest library with mbed port. This is a must for unit tests to be compiled with CppUTest library.
nexpaq 1:55a6170b404f 597
nexpaq 1:55a6170b404f 598 ### Tests are now divided into two types:
nexpaq 1:55a6170b404f 599 #### 'Hello world' tests
nexpaq 1:55a6170b404f 600 First type of test cases we call 'hello world' tests. They do not dependent on CppUTest library and are monolithic programs usually composed of one main function. You can find this tests in below example directories:
nexpaq 1:55a6170b404f 601
nexpaq 1:55a6170b404f 602 * ```mbed/libraries/tests/mbed/```
nexpaq 1:55a6170b404f 603 * ```mbed/libraries/tests/net/```
nexpaq 1:55a6170b404f 604 * ```mbed/libraries/tests/rtos/```
nexpaq 1:55a6170b404f 605 * ```mbed/libraries/tests/usb/```
nexpaq 1:55a6170b404f 606
nexpaq 1:55a6170b404f 607 Usually ‘hello world’ test cases are using ```test_env.cpp``` and ```test_env.h``` files which implement simple test framework used to communicate with host test and help test framework instrument your tests.
nexpaq 1:55a6170b404f 608
nexpaq 1:55a6170b404f 609 Below you can see listing of ```test_env.h``` file which contains simple macro definitions used to communicate (via serial port printouts) between test case (on hardware) and host test script (on host computer).
nexpaq 1:55a6170b404f 610 Each use case should print on console basic information like:
nexpaq 1:55a6170b404f 611 * Default test case timeout.
nexpaq 1:55a6170b404f 612 * Which host test should be used to supervise test case execution.
nexpaq 1:55a6170b404f 613 * Test description and test case ID (short identifier).
nexpaq 1:55a6170b404f 614
nexpaq 1:55a6170b404f 615 ```c++
nexpaq 1:55a6170b404f 616 .
nexpaq 1:55a6170b404f 617 .
nexpaq 1:55a6170b404f 618 .
nexpaq 1:55a6170b404f 619 // Test result related notification functions
nexpaq 1:55a6170b404f 620 void notify_start();
nexpaq 1:55a6170b404f 621 void notify_completion(bool success);
nexpaq 1:55a6170b404f 622 bool notify_completion_str(bool success, char* buffer);
nexpaq 1:55a6170b404f 623 void notify_performance_coefficient(const char* measurement_name, const int value);
nexpaq 1:55a6170b404f 624 void notify_performance_coefficient(const char* measurement_name, const unsigned int value);
nexpaq 1:55a6170b404f 625 void notify_performance_coefficient(const char* measurement_name, const double value);
nexpaq 1:55a6170b404f 626
nexpaq 1:55a6170b404f 627 // Host test auto-detection API
nexpaq 1:55a6170b404f 628 void notify_host_test_name(const char *host_test);
nexpaq 1:55a6170b404f 629 void notify_timeout(int timeout);
nexpaq 1:55a6170b404f 630 void notify_test_id(const char *test_id);
nexpaq 1:55a6170b404f 631 void notify_test_description(const char *description);
nexpaq 1:55a6170b404f 632
nexpaq 1:55a6170b404f 633 // Host test auto-detection API
nexpaq 1:55a6170b404f 634 #define MBED_HOSTTEST_START(TESTID) notify_test_id(TESTID); notify_start()
nexpaq 1:55a6170b404f 635 #define MBED_HOSTTEST_SELECT(NAME) notify_host_test_name(#NAME)
nexpaq 1:55a6170b404f 636 #define MBED_HOSTTEST_TIMEOUT(SECONDS) notify_timeout(SECONDS)
nexpaq 1:55a6170b404f 637 #define MBED_HOSTTEST_DESCRIPTION(DESC) notify_test_description(#DESC)
nexpaq 1:55a6170b404f 638 #define MBED_HOSTTEST_RESULT(RESULT) notify_completion(RESULT)
nexpaq 1:55a6170b404f 639
nexpaq 1:55a6170b404f 640 /**
nexpaq 1:55a6170b404f 641 Test auto-detection preamble example:
nexpaq 1:55a6170b404f 642 main() {
nexpaq 1:55a6170b404f 643 MBED_HOSTTEST_TIMEOUT(10);
nexpaq 1:55a6170b404f 644 MBED_HOSTTEST_SELECT( host_test );
nexpaq 1:55a6170b404f 645 MBED_HOSTTEST_DESCRIPTION(Hello World);
nexpaq 1:55a6170b404f 646 MBED_HOSTTEST_START("MBED_10");
nexpaq 1:55a6170b404f 647 // Proper 'host_test.py' should take over supervising of this test
nexpaq 1:55a6170b404f 648
nexpaq 1:55a6170b404f 649 // Test code
nexpaq 1:55a6170b404f 650 bool result = ...;
nexpaq 1:55a6170b404f 651
nexpaq 1:55a6170b404f 652 MBED_HOSTTEST_RESULT(result);
nexpaq 1:55a6170b404f 653 }
nexpaq 1:55a6170b404f 654 */
nexpaq 1:55a6170b404f 655 .
nexpaq 1:55a6170b404f 656 .
nexpaq 1:55a6170b404f 657 .
nexpaq 1:55a6170b404f 658 ```
nexpaq 1:55a6170b404f 659
nexpaq 1:55a6170b404f 660 Example of 'hello world' test:
nexpaq 1:55a6170b404f 661 ```c++
nexpaq 1:55a6170b404f 662 #include "mbed.h"
nexpaq 1:55a6170b404f 663 #include "test_env.h"
nexpaq 1:55a6170b404f 664
nexpaq 1:55a6170b404f 665 #define CUSTOM_TIME 1256729737
nexpaq 1:55a6170b404f 666
nexpaq 1:55a6170b404f 667 int main() {
nexpaq 1:55a6170b404f 668 MBED_HOSTTEST_TIMEOUT(20);
nexpaq 1:55a6170b404f 669 MBED_HOSTTEST_SELECT(rtc_auto);
nexpaq 1:55a6170b404f 670 MBED_HOSTTEST_DESCRIPTION(RTC);
nexpaq 1:55a6170b404f 671 MBED_HOSTTEST_START("MBED_16");
nexpaq 1:55a6170b404f 672
nexpaq 1:55a6170b404f 673 char buffer[32] = {0};
nexpaq 1:55a6170b404f 674 set_time(CUSTOM_TIME); // Set RTC time to Wed, 28 Oct 2009 11:35:37
nexpaq 1:55a6170b404f 675 while(1) {
nexpaq 1:55a6170b404f 676 time_t seconds = time(NULL);
nexpaq 1:55a6170b404f 677 strftime(buffer, 32, "%Y-%m-%d %H:%M:%S %p", localtime(&seconds));
nexpaq 1:55a6170b404f 678 printf("MBED: [%ld] [%s]\r\n", seconds, buffer);
nexpaq 1:55a6170b404f 679 wait(1);
nexpaq 1:55a6170b404f 680 }
nexpaq 1:55a6170b404f 681 }
nexpaq 1:55a6170b404f 682 ```
nexpaq 1:55a6170b404f 683
nexpaq 1:55a6170b404f 684 #### 'Unit test' test cases
nexpaq 1:55a6170b404f 685 Second group of tests are unit tests. They are using CppUTest library and require you to write ```TEST_GROUP```s and ```TEST```s in your test files. Test suite will add test runner sources to your test automatically so you can concentrate on writing tests.
nexpaq 1:55a6170b404f 686
nexpaq 1:55a6170b404f 687 Example of unit test:
nexpaq 1:55a6170b404f 688 ```c++
nexpaq 1:55a6170b404f 689 #include "TestHarness.h"
nexpaq 1:55a6170b404f 690 #include <utility>
nexpaq 1:55a6170b404f 691 #include "mbed.h"
nexpaq 1:55a6170b404f 692
nexpaq 1:55a6170b404f 693 TEST_GROUP(BusOut_mask)
nexpaq 1:55a6170b404f 694 {
nexpaq 1:55a6170b404f 695 };
nexpaq 1:55a6170b404f 696
nexpaq 1:55a6170b404f 697 TEST(BusOut_mask, led_1_2_3)
nexpaq 1:55a6170b404f 698 {
nexpaq 1:55a6170b404f 699 BusOut bus_data(LED1, LED2, LED3);
nexpaq 1:55a6170b404f 700 CHECK_EQUAL(0x07, bus_data.mask());
nexpaq 1:55a6170b404f 701 }
nexpaq 1:55a6170b404f 702
nexpaq 1:55a6170b404f 703 TEST(BusOut_mask, led_nc_nc_nc_nc)
nexpaq 1:55a6170b404f 704 {
nexpaq 1:55a6170b404f 705 BusOut bus_data(NC, NC, NC, NC);
nexpaq 1:55a6170b404f 706 CHECK_EQUAL(0x00, bus_data.mask());
nexpaq 1:55a6170b404f 707 }
nexpaq 1:55a6170b404f 708
nexpaq 1:55a6170b404f 709 TEST(BusOut_mask, led_1_2_3_nc_nc)
nexpaq 1:55a6170b404f 710 {
nexpaq 1:55a6170b404f 711 BusOut bus_data(LED1, LED2, LED3, NC, NC);
nexpaq 1:55a6170b404f 712 CHECK_EQUAL(0x07, bus_data.mask());
nexpaq 1:55a6170b404f 713 }
nexpaq 1:55a6170b404f 714
nexpaq 1:55a6170b404f 715 TEST(BusOut_mask, led_1_nc_2_nc_nc_3)
nexpaq 1:55a6170b404f 716 {
nexpaq 1:55a6170b404f 717 BusOut bus_data(LED1, NC, LED2, NC, NC, LED3);
nexpaq 1:55a6170b404f 718 CHECK_EQUAL(0x25, bus_data.mask());
nexpaq 1:55a6170b404f 719 }
nexpaq 1:55a6170b404f 720 ```
nexpaq 1:55a6170b404f 721
nexpaq 1:55a6170b404f 722 ## Example
nexpaq 1:55a6170b404f 723 In below example we will run two example unit tests that are now available. tests ```UT_1``` and ```UT_2``` are unit tests used for now only to check if mbed SDK works with CppUTest library and if tests are being executed. In future number of unit tests will increase, nothing is also stopping you from writing and executing your own unit tests!
nexpaq 1:55a6170b404f 724
nexpaq 1:55a6170b404f 725 ### Example configuration
nexpaq 1:55a6170b404f 726 By default unit tests ```UT_1``` and ```UT_2``` are not automated - simply test suite will ignore them. Also we do not want to create dependency to CppUTest library each time someone executes automation.
nexpaq 1:55a6170b404f 727
nexpaq 1:55a6170b404f 728 Note: To compile ```UT_1``` and ```UT_2``` tests CppUTest library described above, installation is needed and not all users wish to add UT libs to their project. Also new to mbed users may find it difficult. This is why unit testing is an extra feature active only after you deliberately install and enable needed components.
nexpaq 1:55a6170b404f 729
nexpaq 1:55a6170b404f 730 Bellow snippet shows how to modify 'automated' flag so test suite will consider unit tests ```UT_1``` and ```UT_2``` as part of "automated test portfolio". Just change flag 'automated' from ```False``` to ```True```.
nexpaq 1:55a6170b404f 731
nexpaq 1:55a6170b404f 732 ```tests.py``` listing related to ```UT_1``` and ```UT_2```:
nexpaq 1:55a6170b404f 733 ```python
nexpaq 1:55a6170b404f 734 .
nexpaq 1:55a6170b404f 735 .
nexpaq 1:55a6170b404f 736 .
nexpaq 1:55a6170b404f 737 # CPPUTEST Library provides Unit testing Framework
nexpaq 1:55a6170b404f 738 #
nexpaq 1:55a6170b404f 739 # To write TESTs and TEST_GROUPs please add CPPUTEST_LIBRARY to 'dependencies'
nexpaq 1:55a6170b404f 740 #
nexpaq 1:55a6170b404f 741 # This will also include:
nexpaq 1:55a6170b404f 742 # 1. test runner - main function with call to CommandLineTestRunner::RunAllTests(ac, av)
nexpaq 1:55a6170b404f 743 # 2. Serial console object to print test result on serial port console
nexpaq 1:55a6170b404f 744 #
nexpaq 1:55a6170b404f 745
nexpaq 1:55a6170b404f 746 # Unit testing with cpputest library
nexpaq 1:55a6170b404f 747 {
nexpaq 1:55a6170b404f 748 "id": "UT_1", "description": "Basic",
nexpaq 1:55a6170b404f 749 "source_dir": join(TEST_DIR, "utest", "basic"),
nexpaq 1:55a6170b404f 750 "dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
nexpaq 1:55a6170b404f 751 "automated": True,
nexpaq 1:55a6170b404f 752 },
nexpaq 1:55a6170b404f 753 {
nexpaq 1:55a6170b404f 754 "id": "UT_2", "description": "Semihost file system",
nexpaq 1:55a6170b404f 755 "source_dir": join(TEST_DIR, "utest", "semihost_fs"),
nexpaq 1:55a6170b404f 756 "dependencies": [MBED_LIBRARIES, TEST_MBED_LIB, CPPUTEST_LIBRARY],
nexpaq 1:55a6170b404f 757 "automated": True,
nexpaq 1:55a6170b404f 758 "mcu": ["LPC1768", "LPC2368", "LPC11U24"]
nexpaq 1:55a6170b404f 759 },
nexpaq 1:55a6170b404f 760 .
nexpaq 1:55a6170b404f 761 .
nexpaq 1:55a6170b404f 762 .
nexpaq 1:55a6170b404f 763 ```
nexpaq 1:55a6170b404f 764
nexpaq 1:55a6170b404f 765 ### Execute tests
nexpaq 1:55a6170b404f 766 In my test I will use common [LPC1768](http://developer.mbed.org/platforms/mbed-LPC1768/) mbed-enabled board because unit test ```UT_2``` is checking semi-host functionality which is available on this board and handful of others.
nexpaq 1:55a6170b404f 767
nexpaq 1:55a6170b404f 768 Configure your ```test_spec.json``` and ```muts_all.json``` files (refer to test suite build script and automation description) and set mbed disk and serial port.
nexpaq 1:55a6170b404f 769
nexpaq 1:55a6170b404f 770 ```
nexpaq 1:55a6170b404f 771 $ singletest.py -i test_spec.json -M muts_all.json -n UT_1,UT_2 -V
nexpaq 1:55a6170b404f 772 Building library CMSIS (LPC1768, ARM)
nexpaq 1:55a6170b404f 773 Building library MBED (LPC1768, ARM)
nexpaq 1:55a6170b404f 774 Building library CPPUTEST (LPC1768, ARM)
nexpaq 1:55a6170b404f 775 Building project BASIC (LPC1768, ARM)
nexpaq 1:55a6170b404f 776 Executing 'python host_test.py -p COM77 -d E:\ -t 10'
nexpaq 1:55a6170b404f 777 Test::Output::Start
nexpaq 1:55a6170b404f 778 Host test instrumentation on port: "COM77" and disk: "E:\"
nexpaq 1:55a6170b404f 779 TEST(FirstTestGroup, FirstTest) - 0 ms
nexpaq 1:55a6170b404f 780
nexpaq 1:55a6170b404f 781 OK (1 tests, 1 ran, 3 checks, 0 ignored, 0 filtered out, 3 ms)
nexpaq 1:55a6170b404f 782
nexpaq 1:55a6170b404f 783 {{success}}
nexpaq 1:55a6170b404f 784 {{end}}
nexpaq 1:55a6170b404f 785 Test::Output::Finish
nexpaq 1:55a6170b404f 786 TargetTest::LPC1768::ARM::UT_1::Basic [OK] in 2.43 of 10 sec
nexpaq 1:55a6170b404f 787 Building library CPPUTEST (LPC1768, ARM)
nexpaq 1:55a6170b404f 788 Building project SEMIHOST_FS (LPC1768, ARM)
nexpaq 1:55a6170b404f 789 Executing 'python host_test.py -p COM77 -d E:\ -t 10'
nexpaq 1:55a6170b404f 790 Test::Output::Start
nexpaq 1:55a6170b404f 791 Host test instrumentation on port: "COM77" and disk: "E:\"
nexpaq 1:55a6170b404f 792 TEST(FirstTestGroup, FirstTest) - 9 ms
nexpaq 1:55a6170b404f 793
nexpaq 1:55a6170b404f 794 OK (1 tests, 1 ran, 10 checks, 0 ignored, 0 filtered out, 10 ms)
nexpaq 1:55a6170b404f 795
nexpaq 1:55a6170b404f 796 {{success}}
nexpaq 1:55a6170b404f 797 {{end}}
nexpaq 1:55a6170b404f 798 Test::Output::Finish
nexpaq 1:55a6170b404f 799 TargetTest::LPC1768::ARM::UT_2::Semihost file system [OK] in 2.43 of 10 sec
nexpaq 1:55a6170b404f 800 Test summary:
nexpaq 1:55a6170b404f 801 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 802 | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
nexpaq 1:55a6170b404f 803 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 804 | OK | LPC1768 | ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 805 | OK | LPC1768 | ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 806 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 807 Result: 2 OK
nexpaq 1:55a6170b404f 808
nexpaq 1:55a6170b404f 809 Completed in 12.02 sec
nexpaq 1:55a6170b404f 810 ```
nexpaq 1:55a6170b404f 811
nexpaq 1:55a6170b404f 812 You can compile unit tests using various number of supported compilers, below just few examples with working solutions:
nexpaq 1:55a6170b404f 813 ```
nexpaq 1:55a6170b404f 814 Test summary:
nexpaq 1:55a6170b404f 815 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 816 | Result | Target | Toolchain | Test ID | Test Description | Elapsed Time (sec) | Timeout (sec) | Loops |
nexpaq 1:55a6170b404f 817 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 818 | OK | LPC1768 | ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 819 | OK | LPC1768 | ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 820 | OK | LPC1768 | uARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 821 | OK | LPC1768 | uARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 822 | OK | LPC1768 | GCC_ARM | UT_1 | Basic | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 823 | OK | LPC1768 | GCC_ARM | UT_2 | Semihost file system | 2.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 824 | OK | LPC1768 | GCC_CR | UT_1 | Basic | 3.44 | 10 | 1/1 |
nexpaq 1:55a6170b404f 825 | OK | LPC1768 | GCC_CR | UT_2 | Semihost file system | 3.43 | 10 | 1/1 |
nexpaq 1:55a6170b404f 826 +--------+---------+-----------+---------+----------------------+--------------------+---------------+-------+
nexpaq 1:55a6170b404f 827 Result: 8 OK
nexpaq 1:55a6170b404f 828
nexpaq 1:55a6170b404f 829 Completed in 55.85 sec
nexpaq 1:55a6170b404f 830 ```