Maxim nexpaq / nexpaq_dev

Important changes to repositories hosted on mbed.com

Mbed hosted mercurial repositories are deprecated and are due to be permanently deleted in July 2026.

To keep a copy of this software download the repository Zip archive or clone locally using Mercurial.

It is also possible to export all your personal repositories from the account settings page.

features/frameworks/greentea-client/README.md@0:6c56fb4bc5f0, 2016-11-04 (annotated)

Committer:
nexpaq
Date:
Fri Nov 04 20:27:58 2016 +0000
Revision:
0:6c56fb4bc5f0
Moving to library for sharing updates

Who changed what in which revision?

UserRevisionLine numberNew contents of line
nexpaq 0:6c56fb4bc5f0 1 # Table of Contents
nexpaq 0:6c56fb4bc5f0 2
nexpaq 0:6c56fb4bc5f0 3 * [greentea-client](#greentea-client)
nexpaq 0:6c56fb4bc5f0 4 * [mbed-drivers dependencies](#mbed-drivers-dependencies)
nexpaq 0:6c56fb4bc5f0 5 * [Greentea test tools](#greentea-test-tools)
nexpaq 0:6c56fb4bc5f0 6 * [Compatibility](#compatibility)
nexpaq 0:6c56fb4bc5f0 7 * [Greentea support](#greentea-support)
nexpaq 0:6c56fb4bc5f0 8 * [utest support](#utest-support)
nexpaq 0:6c56fb4bc5f0 9 * [greentea-client support in your module](#greentea-client-support-in-your-module)
nexpaq 0:6c56fb4bc5f0 10 * [Terms](#terms)
nexpaq 0:6c56fb4bc5f0 11 * [Test suite](#test-suite)
nexpaq 0:6c56fb4bc5f0 12 * [Test case](#test-case)
nexpaq 0:6c56fb4bc5f0 13 * [key-value protocol](#key-value-protocol)
nexpaq 0:6c56fb4bc5f0 14 * [Where can I use it?](#where-can-i-use-it)
nexpaq 0:6c56fb4bc5f0 15 * [Test suite model](#test-suite-model)
nexpaq 0:6c56fb4bc5f0 16 * [utest support template](#utest-support-template)
nexpaq 0:6c56fb4bc5f0 17 * [No utest support template](#no-utest-support-template)
nexpaq 0:6c56fb4bc5f0 18
nexpaq 0:6c56fb4bc5f0 19 # greentea-client
nexpaq 0:6c56fb4bc5f0 20
nexpaq 0:6c56fb4bc5f0 21 **Tip:** If you are unsure about the terms used in this document, please see the [Terms](#Terms) section.
nexpaq 0:6c56fb4bc5f0 22
nexpaq 0:6c56fb4bc5f0 23 ```greentea-client``` is a C++ client library for [the Greentea test tool](https://github.com/ARMmbed/greentea) when used in an [mbed](www.mbed.com) project.
nexpaq 0:6c56fb4bc5f0 24 This package implements the slave side of the simple key-value protocol used for communication between the device under test (DUT) and the host. ```Greentea``` on the host side implements the protocol's master behavior. ```greentea-client``` is released through the [yotta registry](https://yotta.mbed.com/#/module/greentea-client/0.1.8).
nexpaq 0:6c56fb4bc5f0 25
nexpaq 0:6c56fb4bc5f0 26
nexpaq 0:6c56fb4bc5f0 27 ```
nexpaq 0:6c56fb4bc5f0 28 DUT <--- serial port connection ---> host
nexpaq 0:6c56fb4bc5f0 29 (slave) . (master)
nexpaq 0:6c56fb4bc5f0 30 .
nexpaq 0:6c56fb4bc5f0 31 [greentea-client] . [conn_process] [htrun]
nexpaq 0:6c56fb4bc5f0 32 ===== . ================ =========
nexpaq 0:6c56fb4bc5f0 33 | . | |
nexpaq 0:6c56fb4bc5f0 34 | . | |
nexpaq 0:6c56fb4bc5f0 35 | {{ key ; value }} | |
nexpaq 0:6c56fb4bc5f0 36 |------------------------->| (key, value, timestamp) |
nexpaq 0:6c56fb4bc5f0 37 | . |------------------------>|
nexpaq 0:6c56fb4bc5f0 38 | . | |
nexpaq 0:6c56fb4bc5f0 39 | . | |
nexpaq 0:6c56fb4bc5f0 40 | . | |
nexpaq 0:6c56fb4bc5f0 41 | . | |
nexpaq 0:6c56fb4bc5f0 42 | . | |
nexpaq 0:6c56fb4bc5f0 43 | . | (key, value, timestamp) |
nexpaq 0:6c56fb4bc5f0 44 | {{ key ; value }} |<------------------------|
nexpaq 0:6c56fb4bc5f0 45 |<-------------------------| |
nexpaq 0:6c56fb4bc5f0 46 | . | |
nexpaq 0:6c56fb4bc5f0 47 .
nexpaq 0:6c56fb4bc5f0 48 ```
nexpaq 0:6c56fb4bc5f0 49
nexpaq 0:6c56fb4bc5f0 50 ```greentea-client``` is a [yotta module](http://yottadocs.mbed.com/reference/module.html). You can easily include it in your yotta project as ```dependency```/```testDependency```.
nexpaq 0:6c56fb4bc5f0 51
nexpaq 0:6c56fb4bc5f0 52 ## mbed-drivers dependencies
nexpaq 0:6c56fb4bc5f0 53
nexpaq 0:6c56fb4bc5f0 54 This package was introduced as a future replacement for the [mbed-drivers/test_env.h](https://github.com/ARMmbed/mbed-drivers/blob/master/mbed-drivers/test_env.h) test framework. ```mbed-drivers/test_env.h``` is no longer considered the correct way to write tests for mbed modules.
nexpaq 0:6c56fb4bc5f0 55
nexpaq 0:6c56fb4bc5f0 56 Examples of test cases ported from the old ```mbed-drivers/test_env``` to the ```greentea-client``` model can be found here:
nexpaq 0:6c56fb4bc5f0 57 * [core-util](https://github.com/ARMmbed/core-util/pull/95)
nexpaq 0:6c56fb4bc5f0 58 * [ualloc](https://github.com/ARMmbed/ualloc/pull/24)
nexpaq 0:6c56fb4bc5f0 59 * [minar](https://github.com/ARMmbed/minar/pull/36)
nexpaq 0:6c56fb4bc5f0 60 * [mbed-drivers](https://github.com/ARMmbed/mbed-drivers/pull/165)
nexpaq 0:6c56fb4bc5f0 61 * [sal](https://github.com/ARMmbed/sal/pull/46)
nexpaq 0:6c56fb4bc5f0 62 * [sockets](https://github.com/ARMmbed/sockets/pull/64)
nexpaq 0:6c56fb4bc5f0 63 * [utest](https://github.com/ARMmbed/utest/pull/6)
nexpaq 0:6c56fb4bc5f0 64
nexpaq 0:6c56fb4bc5f0 65 ## Greentea test tools
nexpaq 0:6c56fb4bc5f0 66
nexpaq 0:6c56fb4bc5f0 67 ```Greentea``` is a test automation tool written in Python and designed to automate test execution for [mbed](www.mbed.com) projects encapsulated as [yotta](http://yottadocs.mbed.com) modules. Its key features include: integration with yotta module, test automation for yotta module's tests and reporting.
nexpaq 0:6c56fb4bc5f0 68
nexpaq 0:6c56fb4bc5f0 69 ## Compatibility
nexpaq 0:6c56fb4bc5f0 70
nexpaq 0:6c56fb4bc5f0 71 ```greentea-client``` is compatible with:
nexpaq 0:6c56fb4bc5f0 72 * [Greentea](https://github.com/ARMmbed/greentea) v0.2.x onwards
nexpaq 0:6c56fb4bc5f0 73 * [htrun](https://github.com/ARMmbed/htrun) v0.2.x onwards
nexpaq 0:6c56fb4bc5f0 74 * [utest](https://github.com/ARMmbed/utest) v1.10.0 onwards
nexpaq 0:6c56fb4bc5f0 75 * [unity](https://github.com/ARMmbed/utest) v2.0.1 onwards
nexpaq 0:6c56fb4bc5f0 76
nexpaq 0:6c56fb4bc5f0 77 ### Greentea support
nexpaq 0:6c56fb4bc5f0 78
nexpaq 0:6c56fb4bc5f0 79 If you wish to use ```greentea-client``` please make sure you are using latest ```Greentea``` tools from [PyPI](https://pypi.python.org/pypi/mbed-greentea) (you will need Python 2.7):
nexpaq 0:6c56fb4bc5f0 80 ```
nexpaq 0:6c56fb4bc5f0 81 $ pip install mbed-greentea --upgrade
nexpaq 0:6c56fb4bc5f0 82 ```
nexpaq 0:6c56fb4bc5f0 83
nexpaq 0:6c56fb4bc5f0 84 *Note*: If you previously used ```mbed-drivers/test_env.h``` features to write your test cases please downgrade ```Greentea``` to a version lower than ```v0.2.0``` to stay compatible:
nexpaq 0:6c56fb4bc5f0 85 ```
nexpaq 0:6c56fb4bc5f0 86 $ pip install "mbed-greentea<0.2.0" --upgrade
nexpaq 0:6c56fb4bc5f0 87 ```
nexpaq 0:6c56fb4bc5f0 88
nexpaq 0:6c56fb4bc5f0 89 *Note*: [Greentea v0.1.x](https://github.com/ARMmbed/greentea/tree/master_v0_1_x) is still developed on a ```master_v0_1_x``` branch. We will only apply critical patches to version 0.1.x, no feature development is planned.
nexpaq 0:6c56fb4bc5f0 90
nexpaq 0:6c56fb4bc5f0 91 ### utest support
nexpaq 0:6c56fb4bc5f0 92
nexpaq 0:6c56fb4bc5f0 93 ```utest``` is our preferred test harness which allows you to execute a series of (asynchronous) C++ test cases. ```greentea-client``` includes as test dependency ```utest``` yotta module. See the ```greentea-client``` [module.json](https://github.com/ARMmbed/greentea-client/blob/master/module.json) ```testDependency``` section:
nexpaq 0:6c56fb4bc5f0 94 ```json
nexpaq 0:6c56fb4bc5f0 95 {
nexpaq 0:6c56fb4bc5f0 96 "testDependencies": {
nexpaq 0:6c56fb4bc5f0 97 "utest": "^1.10.0"
nexpaq 0:6c56fb4bc5f0 98 }
nexpaq 0:6c56fb4bc5f0 99 }
nexpaq 0:6c56fb4bc5f0 100 ```
nexpaq 0:6c56fb4bc5f0 101
nexpaq 0:6c56fb4bc5f0 102 ### greentea-client support in your module
nexpaq 0:6c56fb4bc5f0 103
nexpaq 0:6c56fb4bc5f0 104 Currently ```greentea-client``` is in on version ```1.0.x```. Please use a ```^1.0.0``` dependency version in your ```module.json``` file.
nexpaq 0:6c56fb4bc5f0 105
nexpaq 0:6c56fb4bc5f0 106 Example of ```module.json``` file with ```greentea-client``` as a test dependency:
nexpaq 0:6c56fb4bc5f0 107 ```json
nexpaq 0:6c56fb4bc5f0 108 {
nexpaq 0:6c56fb4bc5f0 109 "testDependencies": {
nexpaq 0:6c56fb4bc5f0 110 "greentea-client": "^1.0.0",
nexpaq 0:6c56fb4bc5f0 111 "utest" : "^1.10.0",
nexpaq 0:6c56fb4bc5f0 112 "unity" : "^2.1.0"
nexpaq 0:6c56fb4bc5f0 113 }
nexpaq 0:6c56fb4bc5f0 114 }
nexpaq 0:6c56fb4bc5f0 115 ```
nexpaq 0:6c56fb4bc5f0 116
nexpaq 0:6c56fb4bc5f0 117 ## Terms
nexpaq 0:6c56fb4bc5f0 118
nexpaq 0:6c56fb4bc5f0 119 ### Test suite
nexpaq 0:6c56fb4bc5f0 120 A test suite is a binary containing test cases we execute on hardware. The test suite has a beginning and an end (like your ```main()``` function would. The test suite may pass, fail or be in an error state (for example if test suite times out or there was a serial port connection problem).
nexpaq 0:6c56fb4bc5f0 121
nexpaq 0:6c56fb4bc5f0 122 ### Test case
nexpaq 0:6c56fb4bc5f0 123
nexpaq 0:6c56fb4bc5f0 124 Preferably you will use ```utest``` to define test cases . test case has the beginning and the end. During test case execution you will use ``ùnity``` assert macros, schedule MINAR callbacks, check for timeouts in your code. Your test cases may pass, fail or be in an error state which means something went wrong and we were unable to determine exactly what that was (you may have to check the logs).
nexpaq 0:6c56fb4bc5f0 125
nexpaq 0:6c56fb4bc5f0 126 ### key-value protocol
nexpaq 0:6c56fb4bc5f0 127
nexpaq 0:6c56fb4bc5f0 128 The key-value protocol (also called ```KiVi```) is a simple protocol introduced to the ```Greentea``` test tools. It is used to send simple text messages (events) between the DUT and the host. Each message consists of a _key_ and corresponding _value_ pair.
nexpaq 0:6c56fb4bc5f0 129 A ```KiVi``` message is defined as a string encapsulated between double curly braces. The key and value are separated by a semicolon (```;```).
nexpaq 0:6c56fb4bc5f0 130 For example: the ```{{timeout;120}}}``` string is a simple key-value message where the key "_timeout_" is associated with the value "_120_". Both ```greentea-client``` and ```Greentea``` understand this format and can detect key-value messages in a data stream. Both _key_ and _value_ are ASCII strings.
nexpaq 0:6c56fb4bc5f0 131 This protocol is a master-slave protocol. The host has the role of _master_ and the DUT is the _slave_.
nexpaq 0:6c56fb4bc5f0 132
nexpaq 0:6c56fb4bc5f0 133 ```greentea-client``` implements the key-value protocol tokenizer and parser.
nexpaq 0:6c56fb4bc5f0 134
nexpaq 0:6c56fb4bc5f0 135 ## Where can I use it?
nexpaq 0:6c56fb4bc5f0 136 It is possible to write test cases that use ```greentea-client``` and at the same time support ```mbed``` features such as [MINAR scheduler](https://github.com/ARMmbed/minar). It is also possible to mix ```greentea-client``` with other test tools we use at ```mbed``` such as [utest](https://github.com/ARMmbed/utest) and [unity](https://github.com/ARMmbed/unity).
nexpaq 0:6c56fb4bc5f0 137
nexpaq 0:6c56fb4bc5f0 138 You can also find references to ```greentea-client``` in many ```mbed``` packages. For example:
nexpaq 0:6c56fb4bc5f0 139 * [mbed-drivers](https://github.com/ARMmbed/mbed-drivers) - check its [module.json](https://github.com/ARMmbed/mbed-drivers/blob/master/module.json)
nexpaq 0:6c56fb4bc5f0 140 * [core-util](https://github.com/ARMmbed/core-util) - check its [module.json](https://github.com/ARMmbed/core-util/blob/master/module.json)
nexpaq 0:6c56fb4bc5f0 141
nexpaq 0:6c56fb4bc5f0 142 # Test suite model
nexpaq 0:6c56fb4bc5f0 143 ## utest support template
nexpaq 0:6c56fb4bc5f0 144 ```c++
nexpaq 0:6c56fb4bc5f0 145 #include "greentea-client/test_env.h"
nexpaq 0:6c56fb4bc5f0 146 #include "utest/utest.h"
nexpaq 0:6c56fb4bc5f0 147 #include "unity/unity.h"
nexpaq 0:6c56fb4bc5f0 148
nexpaq 0:6c56fb4bc5f0 149 void test_case_1_func() {
nexpaq 0:6c56fb4bc5f0 150 // Test case #1 body
nexpaq 0:6c56fb4bc5f0 151 // Here you can run your test cases and assertions
nexpaq 0:6c56fb4bc5f0 152 TEST_ASSERT_TRUE(true);
nexpaq 0:6c56fb4bc5f0 153 TEST_ASSERT_FALSE(false);
nexpaq 0:6c56fb4bc5f0 154 }
nexpaq 0:6c56fb4bc5f0 155
nexpaq 0:6c56fb4bc5f0 156 void test_case_2_func() {
nexpaq 0:6c56fb4bc5f0 157 // Test case #2 body
nexpaq 0:6c56fb4bc5f0 158 // Here you can run your test cases and assertions
nexpaq 0:6c56fb4bc5f0 159 TEST_ASSERT_TRUE(true);
nexpaq 0:6c56fb4bc5f0 160 TEST_ASSERT_FALSE(false);
nexpaq 0:6c56fb4bc5f0 161 }
nexpaq 0:6c56fb4bc5f0 162
nexpaq 0:6c56fb4bc5f0 163 const Case cases[] = {
nexpaq 0:6c56fb4bc5f0 164 Case("Test case #1 name", test_case_1_func),
nexpaq 0:6c56fb4bc5f0 165 Case("Test case #1 name", test_case_2_func)
nexpaq 0:6c56fb4bc5f0 166 };
nexpaq 0:6c56fb4bc5f0 167
nexpaq 0:6c56fb4bc5f0 168 status_t greentea_setup(const size_t number_of_cases) {
nexpaq 0:6c56fb4bc5f0 169 GREENTEA_SETUP(5, "default_auto");
nexpaq 0:6c56fb4bc5f0 170 return greentea_test_setup_handler(number_of_cases);
nexpaq 0:6c56fb4bc5f0 171 }
nexpaq 0:6c56fb4bc5f0 172
nexpaq 0:6c56fb4bc5f0 173 void app_start(int, char*[]) {
nexpaq 0:6c56fb4bc5f0 174 Harness::run(specification);
nexpaq 0:6c56fb4bc5f0 175 }
nexpaq 0:6c56fb4bc5f0 176 ```
nexpaq 0:6c56fb4bc5f0 177 ## No utest support template
nexpaq 0:6c56fb4bc5f0 178 ```c++
nexpaq 0:6c56fb4bc5f0 179 #include "greentea-client/test_env.h"
nexpaq 0:6c56fb4bc5f0 180 #include "unity/unity.h"
nexpaq 0:6c56fb4bc5f0 181
nexpaq 0:6c56fb4bc5f0 182 void app_start(int, char*[]) {
nexpaq 0:6c56fb4bc5f0 183 bool result = true;
nexpaq 0:6c56fb4bc5f0 184 GREENTEA_SETUP(15, "default_auto");
nexpaq 0:6c56fb4bc5f0 185
nexpaq 0:6c56fb4bc5f0 186 // Test suite body
nexpaq 0:6c56fb4bc5f0 187 // Here you can run your test cases and or assertions
nexpaq 0:6c56fb4bc5f0 188 TEST_ASSERT_TRUE(true);
nexpaq 0:6c56fb4bc5f0 189 TEST_ASSERT_FALSE(false);
nexpaq 0:6c56fb4bc5f0 190
nexpaq 0:6c56fb4bc5f0 191 GREENTEA_TESTSUITE_RESULT(result);
nexpaq 0:6c56fb4bc5f0 192 }
nexpaq 0:6c56fb4bc5f0 193 ```