EL4121 Embedded System / mbed-os

mbed-os

Dependents:   cobaLCDJoyMotor_Thread odometry_omni_3roda_v3 odometry_omni_3roda_v1 odometry_omni_3roda_v2 ... more

features/frameworks/greentea-client/README.md@0:b74591d5ab33, 2017-12-11 (annotated)

Committer:
be_bryan
Date:
Mon Dec 11 17:54:04 2017 +0000
Revision:
0:b74591d5ab33
motor ++

Who changed what in which revision?

UserRevisionLine numberNew contents of line
be_bryan 0:b74591d5ab33 1 # Table of Contents
be_bryan 0:b74591d5ab33 2
be_bryan 0:b74591d5ab33 3 * [greentea-client](#greentea-client)
be_bryan 0:b74591d5ab33 4 * [mbed-drivers dependencies](#mbed-drivers-dependencies)
be_bryan 0:b74591d5ab33 5 * [Greentea test tools](#greentea-test-tools)
be_bryan 0:b74591d5ab33 6 * [Compatibility](#compatibility)
be_bryan 0:b74591d5ab33 7 * [Greentea support](#greentea-support)
be_bryan 0:b74591d5ab33 8 * [utest support](#utest-support)
be_bryan 0:b74591d5ab33 9 * [greentea-client support in your module](#greentea-client-support-in-your-module)
be_bryan 0:b74591d5ab33 10 * [Terms](#terms)
be_bryan 0:b74591d5ab33 11 * [Test suite](#test-suite)
be_bryan 0:b74591d5ab33 12 * [Test case](#test-case)
be_bryan 0:b74591d5ab33 13 * [key-value protocol](#key-value-protocol)
be_bryan 0:b74591d5ab33 14 * [Where can I use it?](#where-can-i-use-it)
be_bryan 0:b74591d5ab33 15 * [Test suite model](#test-suite-model)
be_bryan 0:b74591d5ab33 16 * [utest support template](#utest-support-template)
be_bryan 0:b74591d5ab33 17 * [No utest support template](#no-utest-support-template)
be_bryan 0:b74591d5ab33 18
be_bryan 0:b74591d5ab33 19 # greentea-client
be_bryan 0:b74591d5ab33 20
be_bryan 0:b74591d5ab33 21 **Tip:** If you are unsure about the terms used in this document, please see the [Terms](#Terms) section.
be_bryan 0:b74591d5ab33 22
be_bryan 0:b74591d5ab33 23 ```greentea-client``` is a C++ client library for [the Greentea test tool](https://github.com/ARMmbed/greentea) when used in an [mbed](www.mbed.com) project.
be_bryan 0:b74591d5ab33 24 This package implements the slave side of the simple key-value protocol used for communication between the device under test (DUT) and the host. ```Greentea``` on the host side implements the protocol's master behavior. ```greentea-client``` is released through the [yotta registry](https://yotta.mbed.com/#/module/greentea-client/0.1.8).
be_bryan 0:b74591d5ab33 25
be_bryan 0:b74591d5ab33 26
be_bryan 0:b74591d5ab33 27 ```
be_bryan 0:b74591d5ab33 28 DUT <--- serial port connection ---> host
be_bryan 0:b74591d5ab33 29 (slave) . (master)
be_bryan 0:b74591d5ab33 30 .
be_bryan 0:b74591d5ab33 31 [greentea-client] . [conn_process] [htrun]
be_bryan 0:b74591d5ab33 32 ===== . ================ =========
be_bryan 0:b74591d5ab33 33 | . | |
be_bryan 0:b74591d5ab33 34 | . | |
be_bryan 0:b74591d5ab33 35 | {{ key ; value }} | |
be_bryan 0:b74591d5ab33 36 |------------------------->| (key, value, timestamp) |
be_bryan 0:b74591d5ab33 37 | . |------------------------>|
be_bryan 0:b74591d5ab33 38 | . | |
be_bryan 0:b74591d5ab33 39 | . | |
be_bryan 0:b74591d5ab33 40 | . | |
be_bryan 0:b74591d5ab33 41 | . | |
be_bryan 0:b74591d5ab33 42 | . | |
be_bryan 0:b74591d5ab33 43 | . | (key, value, timestamp) |
be_bryan 0:b74591d5ab33 44 | {{ key ; value }} |<------------------------|
be_bryan 0:b74591d5ab33 45 |<-------------------------| |
be_bryan 0:b74591d5ab33 46 | . | |
be_bryan 0:b74591d5ab33 47 .
be_bryan 0:b74591d5ab33 48 ```
be_bryan 0:b74591d5ab33 49
be_bryan 0:b74591d5ab33 50 ```greentea-client``` is a [yotta module](http://yottadocs.mbed.com/reference/module.html). You can easily include it in your yotta project as ```dependency```/```testDependency```.
be_bryan 0:b74591d5ab33 51
be_bryan 0:b74591d5ab33 52 ## mbed-drivers dependencies
be_bryan 0:b74591d5ab33 53
be_bryan 0:b74591d5ab33 54 This package was introduced as a future replacement for the [mbed-drivers/test_env.h](https://github.com/ARMmbed/mbed-drivers/blob/master/mbed-drivers/test_env.h) test framework. ```mbed-drivers/test_env.h``` is no longer considered the correct way to write tests for mbed modules.
be_bryan 0:b74591d5ab33 55
be_bryan 0:b74591d5ab33 56 Examples of test cases ported from the old ```mbed-drivers/test_env``` to the ```greentea-client``` model can be found here:
be_bryan 0:b74591d5ab33 57 * [core-util](https://github.com/ARMmbed/core-util/pull/95)
be_bryan 0:b74591d5ab33 58 * [ualloc](https://github.com/ARMmbed/ualloc/pull/24)
be_bryan 0:b74591d5ab33 59 * [minar](https://github.com/ARMmbed/minar/pull/36)
be_bryan 0:b74591d5ab33 60 * [mbed-drivers](https://github.com/ARMmbed/mbed-drivers/pull/165)
be_bryan 0:b74591d5ab33 61 * [sal](https://github.com/ARMmbed/sal/pull/46)
be_bryan 0:b74591d5ab33 62 * [sockets](https://github.com/ARMmbed/sockets/pull/64)
be_bryan 0:b74591d5ab33 63 * [utest](https://github.com/ARMmbed/utest/pull/6)
be_bryan 0:b74591d5ab33 64
be_bryan 0:b74591d5ab33 65 ## Greentea test tools
be_bryan 0:b74591d5ab33 66
be_bryan 0:b74591d5ab33 67 ```Greentea``` is a test automation tool written in Python and designed to automate test execution for [mbed](www.mbed.com) projects encapsulated as [yotta](http://yottadocs.mbed.com) modules. Its key features include: integration with yotta module, test automation for yotta module's tests and reporting.
be_bryan 0:b74591d5ab33 68
be_bryan 0:b74591d5ab33 69 ## Compatibility
be_bryan 0:b74591d5ab33 70
be_bryan 0:b74591d5ab33 71 ```greentea-client``` is compatible with:
be_bryan 0:b74591d5ab33 72 * [Greentea](https://github.com/ARMmbed/greentea) v0.2.x onwards
be_bryan 0:b74591d5ab33 73 * [htrun](https://github.com/ARMmbed/htrun) v0.2.x onwards
be_bryan 0:b74591d5ab33 74 * [utest](https://github.com/ARMmbed/utest) v1.10.0 onwards
be_bryan 0:b74591d5ab33 75 * [unity](https://github.com/ARMmbed/utest) v2.0.1 onwards
be_bryan 0:b74591d5ab33 76
be_bryan 0:b74591d5ab33 77 ### Greentea support
be_bryan 0:b74591d5ab33 78
be_bryan 0:b74591d5ab33 79 If you wish to use ```greentea-client``` please make sure you are using latest ```Greentea``` tools from [PyPI](https://pypi.python.org/pypi/mbed-greentea) (you will need Python 2.7):
be_bryan 0:b74591d5ab33 80 ```
be_bryan 0:b74591d5ab33 81 $ pip install mbed-greentea --upgrade
be_bryan 0:b74591d5ab33 82 ```
be_bryan 0:b74591d5ab33 83
be_bryan 0:b74591d5ab33 84 *Note*: If you previously used ```mbed-drivers/test_env.h``` features to write your test cases please downgrade ```Greentea``` to a version lower than ```v0.2.0``` to stay compatible:
be_bryan 0:b74591d5ab33 85 ```
be_bryan 0:b74591d5ab33 86 $ pip install "mbed-greentea<0.2.0" --upgrade
be_bryan 0:b74591d5ab33 87 ```
be_bryan 0:b74591d5ab33 88
be_bryan 0:b74591d5ab33 89 *Note*: [Greentea v0.1.x](https://github.com/ARMmbed/greentea/tree/master_v0_1_x) is still developed on a ```master_v0_1_x``` branch. We will only apply critical patches to version 0.1.x, no feature development is planned.
be_bryan 0:b74591d5ab33 90
be_bryan 0:b74591d5ab33 91 ### utest support
be_bryan 0:b74591d5ab33 92
be_bryan 0:b74591d5ab33 93 ```utest``` is our preferred test harness which allows you to execute a series of (asynchronous) C++ test cases. ```greentea-client``` includes as test dependency ```utest``` yotta module. See the ```greentea-client``` [module.json](https://github.com/ARMmbed/greentea-client/blob/master/module.json) ```testDependency``` section:
be_bryan 0:b74591d5ab33 94 ```json
be_bryan 0:b74591d5ab33 95 {
be_bryan 0:b74591d5ab33 96 "testDependencies": {
be_bryan 0:b74591d5ab33 97 "utest": "^1.10.0"
be_bryan 0:b74591d5ab33 98 }
be_bryan 0:b74591d5ab33 99 }
be_bryan 0:b74591d5ab33 100 ```
be_bryan 0:b74591d5ab33 101
be_bryan 0:b74591d5ab33 102 ### greentea-client support in your module
be_bryan 0:b74591d5ab33 103
be_bryan 0:b74591d5ab33 104 Currently ```greentea-client``` is in on version ```1.0.x```. Please use a ```^1.0.0``` dependency version in your ```module.json``` file.
be_bryan 0:b74591d5ab33 105
be_bryan 0:b74591d5ab33 106 Example of ```module.json``` file with ```greentea-client``` as a test dependency:
be_bryan 0:b74591d5ab33 107 ```json
be_bryan 0:b74591d5ab33 108 {
be_bryan 0:b74591d5ab33 109 "testDependencies": {
be_bryan 0:b74591d5ab33 110 "greentea-client": "^1.0.0",
be_bryan 0:b74591d5ab33 111 "utest" : "^1.10.0",
be_bryan 0:b74591d5ab33 112 "unity" : "^2.1.0"
be_bryan 0:b74591d5ab33 113 }
be_bryan 0:b74591d5ab33 114 }
be_bryan 0:b74591d5ab33 115 ```
be_bryan 0:b74591d5ab33 116
be_bryan 0:b74591d5ab33 117 ## Terms
be_bryan 0:b74591d5ab33 118
be_bryan 0:b74591d5ab33 119 ### Test suite
be_bryan 0:b74591d5ab33 120 A test suite is a binary containing test cases we execute on hardware. The test suite has a beginning and an end (like your ```main()``` function would. The test suite may pass, fail or be in an error state (for example if test suite times out or there was a serial port connection problem).
be_bryan 0:b74591d5ab33 121
be_bryan 0:b74591d5ab33 122 ### Test case
be_bryan 0:b74591d5ab33 123
be_bryan 0:b74591d5ab33 124 Preferably you will use ```utest``` to define test cases . test case has the beginning and the end. During test case execution you will use ``ùnity``` assert macros, schedule MINAR callbacks, check for timeouts in your code. Your test cases may pass, fail or be in an error state which means something went wrong and we were unable to determine exactly what that was (you may have to check the logs).
be_bryan 0:b74591d5ab33 125
be_bryan 0:b74591d5ab33 126 ### key-value protocol
be_bryan 0:b74591d5ab33 127
be_bryan 0:b74591d5ab33 128 The key-value protocol (also called ```KiVi```) is a simple protocol introduced to the ```Greentea``` test tools. It is used to send simple text messages (events) between the DUT and the host. Each message consists of a _key_ and corresponding _value_ pair.
be_bryan 0:b74591d5ab33 129 A ```KiVi``` message is defined as a string encapsulated between double curly braces. The key and value are separated by a semicolon (```;```).
be_bryan 0:b74591d5ab33 130 For example: the ```{{timeout;120}}}``` string is a simple key-value message where the key "_timeout_" is associated with the value "_120_". Both ```greentea-client``` and ```Greentea``` understand this format and can detect key-value messages in a data stream. Both _key_ and _value_ are ASCII strings.
be_bryan 0:b74591d5ab33 131 This protocol is a master-slave protocol. The host has the role of _master_ and the DUT is the _slave_.
be_bryan 0:b74591d5ab33 132
be_bryan 0:b74591d5ab33 133 ```greentea-client``` implements the key-value protocol tokenizer and parser.
be_bryan 0:b74591d5ab33 134
be_bryan 0:b74591d5ab33 135 ## Where can I use it?
be_bryan 0:b74591d5ab33 136 It is possible to write test cases that use ```greentea-client``` and at the same time support ```mbed``` features such as [MINAR scheduler](https://github.com/ARMmbed/minar). It is also possible to mix ```greentea-client``` with other test tools we use at ```mbed``` such as [utest](https://github.com/ARMmbed/utest) and [unity](https://github.com/ARMmbed/unity).
be_bryan 0:b74591d5ab33 137
be_bryan 0:b74591d5ab33 138 You can also find references to ```greentea-client``` in many ```mbed``` packages. For example:
be_bryan 0:b74591d5ab33 139 * [mbed-drivers](https://github.com/ARMmbed/mbed-drivers) - check its [module.json](https://github.com/ARMmbed/mbed-drivers/blob/master/module.json)
be_bryan 0:b74591d5ab33 140 * [core-util](https://github.com/ARMmbed/core-util) - check its [module.json](https://github.com/ARMmbed/core-util/blob/master/module.json)
be_bryan 0:b74591d5ab33 141
be_bryan 0:b74591d5ab33 142 # Test suite model
be_bryan 0:b74591d5ab33 143 ## utest support template
be_bryan 0:b74591d5ab33 144 ```c++
be_bryan 0:b74591d5ab33 145 #include "greentea-client/test_env.h"
be_bryan 0:b74591d5ab33 146 #include "utest/utest.h"
be_bryan 0:b74591d5ab33 147 #include "unity/unity.h"
be_bryan 0:b74591d5ab33 148
be_bryan 0:b74591d5ab33 149 void test_case_1_func() {
be_bryan 0:b74591d5ab33 150 // Test case #1 body
be_bryan 0:b74591d5ab33 151 // Here you can run your test cases and assertions
be_bryan 0:b74591d5ab33 152 TEST_ASSERT_TRUE(true);
be_bryan 0:b74591d5ab33 153 TEST_ASSERT_FALSE(false);
be_bryan 0:b74591d5ab33 154 }
be_bryan 0:b74591d5ab33 155
be_bryan 0:b74591d5ab33 156 void test_case_2_func() {
be_bryan 0:b74591d5ab33 157 // Test case #2 body
be_bryan 0:b74591d5ab33 158 // Here you can run your test cases and assertions
be_bryan 0:b74591d5ab33 159 TEST_ASSERT_TRUE(true);
be_bryan 0:b74591d5ab33 160 TEST_ASSERT_FALSE(false);
be_bryan 0:b74591d5ab33 161 }
be_bryan 0:b74591d5ab33 162
be_bryan 0:b74591d5ab33 163 const Case cases[] = {
be_bryan 0:b74591d5ab33 164 Case("Test case #1 name", test_case_1_func),
be_bryan 0:b74591d5ab33 165 Case("Test case #1 name", test_case_2_func)
be_bryan 0:b74591d5ab33 166 };
be_bryan 0:b74591d5ab33 167
be_bryan 0:b74591d5ab33 168 status_t greentea_setup(const size_t number_of_cases) {
be_bryan 0:b74591d5ab33 169 GREENTEA_SETUP(5, "default_auto");
be_bryan 0:b74591d5ab33 170 return greentea_test_setup_handler(number_of_cases);
be_bryan 0:b74591d5ab33 171 }
be_bryan 0:b74591d5ab33 172
be_bryan 0:b74591d5ab33 173 void app_start(int, char*[]) {
be_bryan 0:b74591d5ab33 174 Harness::run(specification);
be_bryan 0:b74591d5ab33 175 }
be_bryan 0:b74591d5ab33 176 ```
be_bryan 0:b74591d5ab33 177 ## No utest support template
be_bryan 0:b74591d5ab33 178 ```c++
be_bryan 0:b74591d5ab33 179 #include "greentea-client/test_env.h"
be_bryan 0:b74591d5ab33 180 #include "unity/unity.h"
be_bryan 0:b74591d5ab33 181
be_bryan 0:b74591d5ab33 182 void app_start(int, char*[]) {
be_bryan 0:b74591d5ab33 183 bool result = true;
be_bryan 0:b74591d5ab33 184 GREENTEA_SETUP(15, "default_auto");
be_bryan 0:b74591d5ab33 185
be_bryan 0:b74591d5ab33 186 // Test suite body
be_bryan 0:b74591d5ab33 187 // Here you can run your test cases and or assertions
be_bryan 0:b74591d5ab33 188 TEST_ASSERT_TRUE(true);
be_bryan 0:b74591d5ab33 189 TEST_ASSERT_FALSE(false);
be_bryan 0:b74591d5ab33 190
be_bryan 0:b74591d5ab33 191 GREENTEA_TESTSUITE_RESULT(result);
be_bryan 0:b74591d5ab33 192 }
be_bryan 0:b74591d5ab33 193 ```