PES4 / Mbed OS Queue_02

Important changes to repositories hosted on mbed.com

Mbed hosted mercurial repositories are deprecated and are due to be permanently deleted in July 2026.

To keep a copy of this software download the repository Zip archive or clone locally using Mercurial.

It is also possible to export all your personal repositories from the account settings page.

mbed-os/features/frameworks/utest/README.md@0:6bf0743ece18, 2020-03-28 (annotated)

Committer:
demayer
Date:
Sat Mar 28 15:28:19 2020 +0000
Revision:
0:6bf0743ece18
IMU Thread with an event-queue running parallel to handle tasks like a 5 times blinking LED. Button with interrupt detected.

Who changed what in which revision?

UserRevisionLine numberNew contents of line
demayer 0:6bf0743ece18 1 # utest: Asynchronous C++ Test Harness
demayer 0:6bf0743ece18 2
demayer 0:6bf0743ece18 3 This test harness allows you to execute a specified series of (asynchronous) C++ test cases with sensible default reporting and useful customization options.
demayer 0:6bf0743ece18 4
demayer 0:6bf0743ece18 5 Please note that this is a purposefully lean test harness, only dealing with test execution and providing default reporting handlers. It specifically does not support auto-discovery of test cases and does not provide you with test macros or other convenience functions.
demayer 0:6bf0743ece18 6 Instead, the macros in the [unity module](https://github.com/ARMmbed/mbed-os/tree/master/features/frameworks/unity) can be used for this purpose. However, you are not required to use these, and can use your own macros if you wish.
demayer 0:6bf0743ece18 7
demayer 0:6bf0743ece18 8 Furthermore, test failure recovery through the use of exceptions or `longjmp` is not supported; the test will either continue and ignore failures or die by busy-waiting.
demayer 0:6bf0743ece18 9
demayer 0:6bf0743ece18 10 ## Theory of Operation
demayer 0:6bf0743ece18 11
demayer 0:6bf0743ece18 12 A test specification contains a setup handler, several test cases and a teardown handler.
demayer 0:6bf0743ece18 13 Each test case contains a textual description, setup, teardown and failure handler as well as the actual test handler.
demayer 0:6bf0743ece18 14
demayer 0:6bf0743ece18 15 The order of handler execution is:
demayer 0:6bf0743ece18 16
demayer 0:6bf0743ece18 17 1. Test setup handler.
demayer 0:6bf0743ece18 18 1. For each test case:
demayer 0:6bf0743ece18 19 1. Test case setup handler.
demayer 0:6bf0743ece18 20 1. Test case execution handler.
demayer 0:6bf0743ece18 21 1. (wait for timeout or callback validation in case of an asynchronous test case.)
demayer 0:6bf0743ece18 22 1. (repeat test case execution handler if specified.)
demayer 0:6bf0743ece18 23 1. Test case teardown handler.
demayer 0:6bf0743ece18 24 1. Test teardown handler.
demayer 0:6bf0743ece18 25
demayer 0:6bf0743ece18 26 ## Example
demayer 0:6bf0743ece18 27
demayer 0:6bf0743ece18 28 The following example showcases a lot of functionality and proper integration with the [Greentea testing automation framework](https://github.com/ARMmbed/greentea), while making use of the [unity test macros](https://github.com/ARMmbed/mbed-os/tree/master/features/frameworks/unity):
demayer 0:6bf0743ece18 29
demayer 0:6bf0743ece18 30 ```cpp
demayer 0:6bf0743ece18 31 #include "mbed-drivers/test_env.h"
demayer 0:6bf0743ece18 32 #include "utest/utest.h"
demayer 0:6bf0743ece18 33 #include "unity/unity.h"
demayer 0:6bf0743ece18 34
demayer 0:6bf0743ece18 35 using namespace utest::v1;
demayer 0:6bf0743ece18 36
demayer 0:6bf0743ece18 37 void test_simple() {
demayer 0:6bf0743ece18 38 TEST_ASSERT_EQUAL(0, 0);
demayer 0:6bf0743ece18 39 printf("Simple test called\n");
demayer 0:6bf0743ece18 40 }
demayer 0:6bf0743ece18 41
demayer 0:6bf0743ece18 42 status_t test_repeats_setup(const Case *const source, const size_t index_of_case) {
demayer 0:6bf0743ece18 43 // Call the default handler for proper reporting
demayer 0:6bf0743ece18 44 status_t status = greentea_case_setup_handler(source, index_of_case);
demayer 0:6bf0743ece18 45 printf("Setting up for '%s'\n", source->get_description());
demayer 0:6bf0743ece18 46 return status;
demayer 0:6bf0743ece18 47 }
demayer 0:6bf0743ece18 48 control_t test_repeats(const size_t call_count) {
demayer 0:6bf0743ece18 49 printf("Called for the %u. time\n", call_count);
demayer 0:6bf0743ece18 50 TEST_ASSERT_NOT_EQUAL(3, call_count);
demayer 0:6bf0743ece18 51 // Specify how often this test is repeated ie. n total calls
demayer 0:6bf0743ece18 52 return (call_count < 2) ? CaseRepeatAll : CaseNext;
demayer 0:6bf0743ece18 53 }
demayer 0:6bf0743ece18 54
demayer 0:6bf0743ece18 55 void test_callback_validate() {
demayer 0:6bf0743ece18 56 // You may also use assertions here!
demayer 0:6bf0743ece18 57 TEST_ASSERT_EQUAL_PTR(0, 0);
demayer 0:6bf0743ece18 58 // Validate the callback
demayer 0:6bf0743ece18 59 Harness::validate_callback();
demayer 0:6bf0743ece18 60 }
demayer 0:6bf0743ece18 61 control_t test_asynchronous() {
demayer 0:6bf0743ece18 62 TEST_ASSERT_TRUE_MESSAGE(true, "(true == false) o_O");
demayer 0:6bf0743ece18 63 // Set up a callback in the future. This may also be an interrupt!
demayer 0:6bf0743ece18 64 minar::Scheduler::postCallback(test_callback_validate).delay(minar::milliseconds(100));
demayer 0:6bf0743ece18 65 // Set a 200ms timeout starting from now
demayer 0:6bf0743ece18 66 return CaseTimeout(200);
demayer 0:6bf0743ece18 67 }
demayer 0:6bf0743ece18 68
demayer 0:6bf0743ece18 69 control_t test_asynchronous_timeout(const size_t call_count) {
demayer 0:6bf0743ece18 70 TEST_ASSERT_TRUE_MESSAGE(true, "(true == false) o_O");
demayer 0:6bf0743ece18 71 // Set a 200ms timeout starting from now,
demayer 0:6bf0743ece18 72 // but automatically repeat only this handler on timeout.
demayer 0:6bf0743ece18 73 if (call_count >= 5) {
demayer 0:6bf0743ece18 74 // but after the 5th call, the callback finally gets validated
demayer 0:6bf0743ece18 75 minar::Scheduler::postCallback(test_callback_validate).delay(minar::milliseconds(100));
demayer 0:6bf0743ece18 76 }
demayer 0:6bf0743ece18 77 return CaseRepeatHandlerOnTimeout(200);
demayer 0:6bf0743ece18 78 }
demayer 0:6bf0743ece18 79
demayer 0:6bf0743ece18 80 // Custom setup handler required for proper Greentea support
demayer 0:6bf0743ece18 81 status_t greentea_setup(const size_t number_of_cases) {
demayer 0:6bf0743ece18 82 GREENTEA_SETUP(20, "default_auto");
demayer 0:6bf0743ece18 83 // Call the default reporting function
demayer 0:6bf0743ece18 84 return greentea_test_setup_handler(number_of_cases);
demayer 0:6bf0743ece18 85 }
demayer 0:6bf0743ece18 86
demayer 0:6bf0743ece18 87 // Specify all your test cases here
demayer 0:6bf0743ece18 88 Case cases[] = {
demayer 0:6bf0743ece18 89 Case("Simple Test", test_simple),
demayer 0:6bf0743ece18 90 Case("Repeating Test", test_repeats_setup, test_repeats),
demayer 0:6bf0743ece18 91 Case("Asynchronous Test (200ms timeout)", test_asynchronous),
demayer 0:6bf0743ece18 92 Case("Asynchronous Timeout Repeat", test_asynchronous_timeout)
demayer 0:6bf0743ece18 93 };
demayer 0:6bf0743ece18 94
demayer 0:6bf0743ece18 95 // Declare your test specification with a custom setup handler
demayer 0:6bf0743ece18 96 Specification specification(greentea_setup, cases);
demayer 0:6bf0743ece18 97
demayer 0:6bf0743ece18 98 void app_start(int, char**)
demayer 0:6bf0743ece18 99 { // Run the test specification
demayer 0:6bf0743ece18 100 Harness::run(specification);
demayer 0:6bf0743ece18 101 }
demayer 0:6bf0743ece18 102 ```
demayer 0:6bf0743ece18 103
demayer 0:6bf0743ece18 104 Running this test will output the following:
demayer 0:6bf0743ece18 105
demayer 0:6bf0743ece18 106 ```
demayer 0:6bf0743ece18 107 {{timeout;20}}
demayer 0:6bf0743ece18 108 {{host_test_name;default_auto}}
demayer 0:6bf0743ece18 109 {{description;utest greentea example}}
demayer 0:6bf0743ece18 110 {{test_id;MBED_OS}}
demayer 0:6bf0743ece18 111 {{start}}
demayer 0:6bf0743ece18 112 >>> Running 4 test cases...
demayer 0:6bf0743ece18 113
demayer 0:6bf0743ece18 114 >>> Running case #1: 'Simple Test'...
demayer 0:6bf0743ece18 115 Simple test called
demayer 0:6bf0743ece18 116 >>> 'Simple Test': 1 passed, 0 failed
demayer 0:6bf0743ece18 117
demayer 0:6bf0743ece18 118 >>> Running case #2: 'Repeating Test'...
demayer 0:6bf0743ece18 119 Setting up for 'Repeating Test'
demayer 0:6bf0743ece18 120 Called for the 1. time
demayer 0:6bf0743ece18 121 >>> 'Repeating Test': 1 passed, 0 failed
demayer 0:6bf0743ece18 122
demayer 0:6bf0743ece18 123 >>> Running case #2: 'Repeating Test'...
demayer 0:6bf0743ece18 124 Setting up for 'Repeating Test'
demayer 0:6bf0743ece18 125 Called for the 2. time
demayer 0:6bf0743ece18 126 >>> 'Repeating Test': 2 passed, 0 failed
demayer 0:6bf0743ece18 127
demayer 0:6bf0743ece18 128 >>> Running case #3: 'Asynchronous Test (200ms timeout)'...
demayer 0:6bf0743ece18 129 >>> 'Asynchronous Test (200ms timeout)': 1 passed, 0 failed
demayer 0:6bf0743ece18 130
demayer 0:6bf0743ece18 131 >>> Running case #4: 'Asynchronous Timeout Repeat'...
demayer 0:6bf0743ece18 132 >>> failure with reason 'Ignored: Timed Out'
demayer 0:6bf0743ece18 133 >>> failure with reason 'Ignored: Timed Out'
demayer 0:6bf0743ece18 134 >>> failure with reason 'Ignored: Timed Out'
demayer 0:6bf0743ece18 135 >>> failure with reason 'Ignored: Timed Out'
demayer 0:6bf0743ece18 136 >>> failure with reason 'Ignored: Timed Out'
demayer 0:6bf0743ece18 137 >>> 'Asynchronous Timeout Repeat': 1 passed, 0 failed
demayer 0:6bf0743ece18 138
demayer 0:6bf0743ece18 139 >>> Test cases: 4 passed, 0 failed
demayer 0:6bf0743ece18 140 {{success}}
demayer 0:6bf0743ece18 141 {{end}}
demayer 0:6bf0743ece18 142 ```
demayer 0:6bf0743ece18 143
demayer 0:6bf0743ece18 144 ## Detailed Description
demayer 0:6bf0743ece18 145
demayer 0:6bf0743ece18 146 ### Handlers
demayer 0:6bf0743ece18 147
demayer 0:6bf0743ece18 148 There are five handler types you can, but do not have to, override to customize operation.
demayer 0:6bf0743ece18 149 Please see the `utest/types.h` file for a detailed description.
demayer 0:6bf0743ece18 150
demayer 0:6bf0743ece18 151 1. `status_t test_setup_handler_t(const size_t number_of_cases)`: called before execution of any test case.
demayer 0:6bf0743ece18 152 1. `void test_teardown_handler_t(const size_t passed, const size_t failed, const failure_t failure)`: called after execution of all test cases, and if testing is aborted.
demayer 0:6bf0743ece18 153 1. `void test_failure_handler_t(const failure_t failure)`: called whenever a failure occurs anywhere in the specification.
demayer 0:6bf0743ece18 154 1. `status_t case_setup_handler_t(const Case *const source, const size_t index_of_case)`: called before execution of each test case.
demayer 0:6bf0743ece18 155 1. `status_t case_teardown_handler_t(const Case *const source, const size_t passed, const size_t failed, const failure_t reason)`: called after execution of each test case, and if testing is aborted.
demayer 0:6bf0743ece18 156 1. `status_t case_failure_handler_t(const Case *const source, const failure_t reason)`: called whenever a failure occurs during the execution of a test case.
demayer 0:6bf0743ece18 157
demayer 0:6bf0743ece18 158 All handlers are defaulted for integration with the [Greentea testing automation framework](https://github.com/ARMmbed/greentea).
demayer 0:6bf0743ece18 159
demayer 0:6bf0743ece18 160 ### Test Case Handlers
demayer 0:6bf0743ece18 161
demayer 0:6bf0743ece18 162 There are three test case handlers:
demayer 0:6bf0743ece18 163
demayer 0:6bf0743ece18 164 1. `void case_handler_t(void)`: executes once, if the case setup succeeded.
demayer 0:6bf0743ece18 165 1. `control_t case_control_handler_t(void)`: executes (asynchronously) as many times as you specify, if the case setup succeeded.
demayer 0:6bf0743ece18 166 1. `control_t case_call_count_handler_t(const size_t call_count)`: executes (asynchronously) as many times as you specify, if the case setup succeeded.
demayer 0:6bf0743ece18 167
demayer 0:6bf0743ece18 168 To specify a test case you must wrap it into a `Case` class: `Case("mandatory description", case_handler)`. You may override the setup, teardown and failure handlers in this wrapper class as well.
demayer 0:6bf0743ece18 169 The `Case` constructor is overloaded to allow you a comfortable declaration of all your callbacks and the order of arguments is:
demayer 0:6bf0743ece18 170
demayer 0:6bf0743ece18 171 1. Description (required).
demayer 0:6bf0743ece18 172 1. Setup handler (optional).
demayer 0:6bf0743ece18 173 1. Test case handler (required).
demayer 0:6bf0743ece18 174 1. Teardown handler (optional).
demayer 0:6bf0743ece18 175 1. Failure handler (optional).
demayer 0:6bf0743ece18 176
demayer 0:6bf0743ece18 177 #### Test Case Attributes
demayer 0:6bf0743ece18 178
demayer 0:6bf0743ece18 179 You can modify test case behavior by returning `control_t` modifiers:
demayer 0:6bf0743ece18 180
demayer 0:6bf0743ece18 181 - `CaseNext`: never repeats and immediately moves to next test case
demayer 0:6bf0743ece18 182 - `CaseNoRepeat`: never repeats.
demayer 0:6bf0743ece18 183 - `CaseRepeatAll`: repeats test case **with** setup and teardown handlers.
demayer 0:6bf0743ece18 184 - `CaseRepeatHandler`: repeats test case **without** set and teardown handlers.
demayer 0:6bf0743ece18 185 - `CaseNoTimeout`: immediately moves to next test case.
demayer 0:6bf0743ece18 186 - `CaseAwait`: waits indefinitely for callback validation (*use with caution*).
demayer 0:6bf0743ece18 187 - `CaseTimeout(uint32_t ms)`: waits for callback validation for `ms` milliseconds, times out after that (fails with `REASON_TIMEOUT`).
demayer 0:6bf0743ece18 188 - `CaseRepeatAllOnTimeout(uint32_t ms)`: waits for callback validation for `ms` milliseconds, repeats test case **with** setup and teardown handlers on time out.
demayer 0:6bf0743ece18 189 - `CaseRepeatHandlerOnTimeout(uint32_t ms)`: waits for callback validation for `ms` milliseconds, repeats test case **without** setup and teardown handlers on time out.
demayer 0:6bf0743ece18 190
demayer 0:6bf0743ece18 191 Returning `CaseRepeatAll` from your test case handler tells the test harness to repeat the test handler. You can use the `call_count` (starts counting at 1) to decide when to stop.
demayer 0:6bf0743ece18 192 By default the setup and teardown handlers are called on every repeated test cases, however, you may only repeat the case handler by returning `CaseRepeatHandler`. To stop the harness from repeating the test case, return `CaseNext`.
demayer 0:6bf0743ece18 193
demayer 0:6bf0743ece18 194 For asynchronous test cases, you must return a `CaseTimeout(uint32_t ms)`.
demayer 0:6bf0743ece18 195 If you want to automatically repeat the test case on a timeout, use `CaseRepeatAllOnTimeout(uint32_t ms)` and `CaseRepeatHandlerOnTimeout(uint32_t ms)`.
demayer 0:6bf0743ece18 196
demayer 0:6bf0743ece18 197 To validate your callback, you must call `Harness::validate_callback()` in your asynchronous callback before the timeout fires.
demayer 0:6bf0743ece18 198 This will schedule the execution of the next test case.
demayer 0:6bf0743ece18 199
demayer 0:6bf0743ece18 200 For repeating asynchronous cases, you can "add" both attributes together: `CaseTimeout(200) + CaseRepeatAll` will wait for 200ms for the callback validation and then repeat the test case. See the section on arbitration logic for more details.
demayer 0:6bf0743ece18 201
demayer 0:6bf0743ece18 202 Note that you can also add attributes during callback validation, however, only repeat attributes are considered. This allows you to return `CaseTimeout(500)` to wait up to 500ms for the callback validation and delegate the decision to repeat to the time the callback occurs: `Harness::validate_callback(CaseRepeatHandler)`.
demayer 0:6bf0743ece18 203
demayer 0:6bf0743ece18 204 Keep in mind that you can only validate a callback once. If you need to wait for several callbacks, you need to write your own helper function that validates the expected callback only when all your custom callbacks arrive.
demayer 0:6bf0743ece18 205 This custom functionality is purposefully not part of this test harness, you can achieve it externally with additional code.
demayer 0:6bf0743ece18 206
demayer 0:6bf0743ece18 207 ### Failure Handlers
demayer 0:6bf0743ece18 208
demayer 0:6bf0743ece18 209 A failure may occur during any phase of the test. The appropriate failure handler is then called with `failure_t`, which contains the failure reason and location.
demayer 0:6bf0743ece18 210
demayer 0:6bf0743ece18 211 The failure reasons are:
demayer 0:6bf0743ece18 212
demayer 0:6bf0743ece18 213 - `REASON_NONE`: No failure occurred
demayer 0:6bf0743ece18 214 - `REASON_UNKNOWN`: An unknown failure occurred
demayer 0:6bf0743ece18 215 - `REASON_CASES`: A failure occurred in at least one test case
demayer 0:6bf0743ece18 216 - `REASON_EMPTY_CASE`: The test case contains only empty handlers
demayer 0:6bf0743ece18 217 - `REASON_TIMEOUT`: An expected asynchronous call timed out
demayer 0:6bf0743ece18 218 - `REASON_ASSERTION`: An assertion failed
demayer 0:6bf0743ece18 219 - `REASON_TEST_SETUP`: Test setup failed
demayer 0:6bf0743ece18 220 - `REASON_TEST_TEARDOWN`: Test teardown failed
demayer 0:6bf0743ece18 221 - `REASON_CASE_SETUP`: Case setup failed
demayer 0:6bf0743ece18 222 - `REASON_CASE_HANDLER`: Case handler failed
demayer 0:6bf0743ece18 223 - `REASON_CASE_TEARDOWN`: Case teardown failed
demayer 0:6bf0743ece18 224 - `REASON_CASE_INDEX`: Case index returned from test setup or case teardown handler is invalid
demayer 0:6bf0743ece18 225 - `REASON_SCHEDULER`: Underlying scheduler is not asynchronous
demayer 0:6bf0743ece18 226
demayer 0:6bf0743ece18 227 The failure locations are:
demayer 0:6bf0743ece18 228
demayer 0:6bf0743ece18 229 - `LOCATION_NONE`: No location information
demayer 0:6bf0743ece18 230 - `LOCATION_UNKNOWN`: A failure occurred in an unknown location
demayer 0:6bf0743ece18 231 - `LOCATION_TEST_SETUP`: A failure occurred in the test setup
demayer 0:6bf0743ece18 232 - `LOCATION_TEST_TEARDOWN`: A failure occurred in the test teardown
demayer 0:6bf0743ece18 233 - `LOCATION_CASE_SETUP`: A failure occurred in the case setup
demayer 0:6bf0743ece18 234 - `LOCATION_CASE_HANDLER`: A failure occurred in the case handler
demayer 0:6bf0743ece18 235 - `LOCATION_CASE_TEARDOWN`: A failure occurred in the case teardown
demayer 0:6bf0743ece18 236
demayer 0:6bf0743ece18 237 If the setup or teardown handlers fail, they may return a `STATUS_ABORT` code, which will call the failure handler with the appropriate failure reason (`REASON_CASE_{SETUP|TEARDOWN}`) and failure location (`LOCATION_CASE_{SETUP|TEARDOWN}`).
demayer 0:6bf0743ece18 238 If the setup handler fails, the test case is never executed. Instead, the teardown handler is called in an attempt to salvage the situation.
demayer 0:6bf0743ece18 239 Please note that if a teardown handler fails, the system can be considered too unstable to continue testing.
demayer 0:6bf0743ece18 240
demayer 0:6bf0743ece18 241 You may also raise a failure manually by calling `Harness::raise_failure(failure_reason_t reason)`. In fact, this is how you can integrate assertion failures from custom test macros, as done with the unity macros, which raise a failure with the `REASON_ASSERTION` reason.
demayer 0:6bf0743ece18 242
demayer 0:6bf0743ece18 243 When waiting for an asynchronous callback, if the timeout fires, `REASON_TIMEOUT` is raised.
demayer 0:6bf0743ece18 244
demayer 0:6bf0743ece18 245 The failure handler decides whether to continue or abort testing by returning `STATUS_CONTINUE` or `STATUS_ABORT` respectively.
demayer 0:6bf0743ece18 246 You can also ignore any raised failure by returning `STATUS_IGNORE` and the harness will then not count this failure.
demayer 0:6bf0743ece18 247 In case of an abort, the test harness dies by busy waiting in a forever loop.
demayer 0:6bf0743ece18 248 This is needed because we cannot unwind the stack without exception support, and the asynchronous nature of the test harness breaks with using `longjmp`s.
demayer 0:6bf0743ece18 249
demayer 0:6bf0743ece18 250 Note that when `REASON_IGNORE` is `OR`ed into the failure reason, the failure handler is expected to return `STATUS_IGNORE`.
demayer 0:6bf0743ece18 251 This is done automatically for test cases repeating after a timeout, and the default failure handlers also report this failure, but tell the harness to ignore it.
demayer 0:6bf0743ece18 252 Furthermore, the unity macros may decide to ignore assertion failures as well, in which case the assertion is ignored intentionally.
demayer 0:6bf0743ece18 253
demayer 0:6bf0743ece18 254 ### Default Handlers
demayer 0:6bf0743ece18 255
demayer 0:6bf0743ece18 256 Three sets of default handlers with different behaviors are provided for your convenience:
demayer 0:6bf0743ece18 257
demayer 0:6bf0743ece18 258 1. `greentea_abort_handlers` (default): Greentea-style reporting, aborts on the first failure, but requires custom test setup handler.
demayer 0:6bf0743ece18 259 1. `greentea_continue_handlers`: Greentea-style reporting, always continues testing, but requires custom test setup handler.
demayer 0:6bf0743ece18 260 1. `verbose_continue_handlers`: always continues testing and reporting, except when a teardown failed.
demayer 0:6bf0743ece18 261 1. `selftest_handlers`: Greentea-style reporting, but aborts on the first assertion failure raised. This allows the use of unity macros for self testing without recursive failure handler calls.
demayer 0:6bf0743ece18 262
demayer 0:6bf0743ece18 263 These default handlers are called when you have not overridden a custom handler, and they only contain reporting functionality and do not modify global state.
demayer 0:6bf0743ece18 264
demayer 0:6bf0743ece18 265 You can specify which default handlers you want to use when wrapping your test cases in the `Specification` class:
demayer 0:6bf0743ece18 266
demayer 0:6bf0743ece18 267 ```cpp
demayer 0:6bf0743ece18 268 // Declare your test specification with a custom setup handler
demayer 0:6bf0743ece18 269 // and set the default handlers to the predefined “greentea continue” behavior
demayer 0:6bf0743ece18 270 Specification specification(greentea_setup, cases, greentea_continue_handlers);
demayer 0:6bf0743ece18 271 ```
demayer 0:6bf0743ece18 272
demayer 0:6bf0743ece18 273 ### Custom Handlers
demayer 0:6bf0743ece18 274
demayer 0:6bf0743ece18 275 You may override any of the default handlers with your own custom handler.
demayer 0:6bf0743ece18 276
demayer 0:6bf0743ece18 277 To ignore a handler completely and not call a custom or default handler, you may use the `ignore_handler` hint.
demayer 0:6bf0743ece18 278 To explicitly invoke the default handler, use the `default_handler` hint.
demayer 0:6bf0743ece18 279
demayer 0:6bf0743ece18 280 To use your own custom handler, provide a function with the correct signature for the handler that you want to customize and provide it in your test case wrapper or specification wrapper.
demayer 0:6bf0743ece18 281 To turn a `failure_t` into a meaningful string use the `stringify(failure_t)` method.
demayer 0:6bf0743ece18 282
demayer 0:6bf0743ece18 283 **We strongly recommend that you call the predefined `greentea_*` handlers inside your custom callback, as they report the current condition in a properly formatted fashion.**
demayer 0:6bf0743ece18 284 By calling these handlers inside your custom callback your unit test does not need to be modified if the test logging needs to be changed in the future.
demayer 0:6bf0743ece18 285
demayer 0:6bf0743ece18 286 For the `Specification` class the order of arguments is:
demayer 0:6bf0743ece18 287
demayer 0:6bf0743ece18 288 1. Test setup handler (optional).
demayer 0:6bf0743ece18 289 1. Array of test cases (required).
demayer 0:6bf0743ece18 290 1. Test teardown handler (optional).
demayer 0:6bf0743ece18 291 1. Test failure handler (optional).
demayer 0:6bf0743ece18 292 1. Default handlers (optional).
demayer 0:6bf0743ece18 293
demayer 0:6bf0743ece18 294 ### Test Case Attribute Arbitration
demayer 0:6bf0743ece18 295
demayer 0:6bf0743ece18 296 When adding conflicting modifiers together
demayer 0:6bf0743ece18 297
demayer 0:6bf0743ece18 298 - the more restrictive (=shorter) timeout is chosen, but `CaseNoTimeout` always wins arbitration: `CaseNoTimeout` > `CaseTimeout(100)` > `CaseTimeout(200)` > `CaseAwait`.
demayer 0:6bf0743ece18 299 - the more invasive repeat method is chosen, but `CaseNoRepeat` always wins arbitration: `CaseNoRepeat` > `CaseRepeatAll`/`CaseRepeatAllOnTimeout(ms)` > `CaseRepeatHandler`/`CaseRepeatHandlerOnTimeout(ms)`.
demayer 0:6bf0743ece18 300 - `CaseNext` always wins arbitration.
demayer 0:6bf0743ece18 301
demayer 0:6bf0743ece18 302 The following table shows this arbitration logic in detail:
demayer 0:6bf0743ece18 303
demayer 0:6bf0743ece18 304 | + | `CaseNext` | `CaseNoRepeat` | `CaseRepeatAll` | `CaseRepeatHandler` | `CaseNoTimeout` | `CaseAwait` | `CaseTimeout(aa)` | `CaseRepeatAllOnTimeout(aa)` | `CaseRepeatHandlerOnTimeout(aa)`
demayer 0:6bf0743ece18 305 |:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
demayer 0:6bf0743ece18 306 | `CaseNext` | no repeat &<br> no timeout
demayer 0:6bf0743ece18 307 | `CaseNoRepeat` | no repeat &<br> no timeout | no repeat
demayer 0:6bf0743ece18 308 | `CaseRepeatAll` | no repeat &<br> no timeout | no repeat | repeat all
demayer 0:6bf0743ece18 309 | `CaseRepeatHandler` | no repeat &<br> no timeout | no repeat | repeat all | repeat handler
demayer 0:6bf0743ece18 310 | `CaseNoTimeout` | no repeat &<br> no timeout | no repeat &<br> no timeout | repeat all &<br> no timeout | repeat handler &<br> no timeout | no timeout
demayer 0:6bf0743ece18 311 | `CaseAwait` | no repeat &<br> no timeout | no repeat &<br> infinite timeout | repeat all &<br> infinite timeout | repeat handler &<br> infinite timeout | no timeout | infinite timeout
demayer 0:6bf0743ece18 312 | `CaseTimeout(bb)` | no repeat &<br> no timeout | no repeat &<br> `bb`ms timeout | repeat all &<br> `bb`ms timeout | repeat handler &<br> `bb`ms timeout | no timeout | `bb`ms timeout | `min(aa,bb)`ms timeout
demayer 0:6bf0743ece18 313 | `CaseRepeatAllOnTimeout(bb)` | no repeat &<br> no timeout | no repeat &<br> `bb`ms timeout | repeat all on validate & repeat all on `bb`ms timeout | repeat all on validate & repeat all on `bb`ms timeout | repeat all & no timeout | repeat all on `bb`ms timeout | repeat all on `min(aa,bb)`ms timeout | repeat all on `min(aa,bb)`ms timeout |
demayer 0:6bf0743ece18 314 | `CaseRepeatHandlerOnTimeout(bb)` | no repeat &<br> no timeout | no repeat &<br> `bb`ms timeout | repeat all on validate & repeat all on `bb`ms timeout | repeat handler on validate & repeat handler on `bb`ms timeout | repeat handler & no timeout | repeat handler on `bb`ms timeout | repeat handler on `min(aa,bb)`ms timeout | repeat all on `min(aa,bb)`ms timeout | repeat handler on `min(aa,bb)`ms timeout
demayer 0:6bf0743ece18 315
demayer 0:6bf0743ece18 316 ### Atomicity
demayer 0:6bf0743ece18 317
demayer 0:6bf0743ece18 318 All handlers execute with interrupts enabled, **except the case failure handler!**.
demayer 0:6bf0743ece18 319 This means you can write test cases that poll for interrupts to be completed inside any handler, except the failure handler.
demayer 0:6bf0743ece18 320
demayer 0:6bf0743ece18 321 If you setup an interrupt that validates its callback using `Harness::validate_callback()` inside a test case and it fires before the test case completed, the validation will be buffered.
demayer 0:6bf0743ece18 322 If the test case then returns a timeout value, but the callback is already validated, the test harness just continues normally.
demayer 0:6bf0743ece18 323
demayer 0:6bf0743ece18 324 ### Custom Scheduler
demayer 0:6bf0743ece18 325
demayer 0:6bf0743ece18 326 By default, a Timeout object is used for scheduling the harness operations.
demayer 0:6bf0743ece18 327 In case this is not available you can provide your own custom scheduler implementation and make the harness use it with the `Harness::set_scheduler(your_custom_implementation)` function.
demayer 0:6bf0743ece18 328
demayer 0:6bf0743ece18 329 The scheduler requirements are very simple: Execute a `void(void)` function in you main loop (with a delay of *N* ms). Only one function is scheduled by the harness *at any given time*.
demayer 0:6bf0743ece18 330 Note that you do not need to implement the delay functionality, if your tests do not require timeouts. You will still be able to use repeating test cases, but an error is thrown if your tests attempt to use a timeout, when your underlying scheduler does not support it.
demayer 0:6bf0743ece18 331
demayer 0:6bf0743ece18 332 There are two functions you need to implement:
demayer 0:6bf0743ece18 333
demayer 0:6bf0743ece18 334 - `void* post_callback(const utest_v1_harness_callback_t callback, const uint32_t delay_ms)`: schedules a `void(void)` callback function in *N* ms.
demayer 0:6bf0743ece18 335 - `int32_t cancel_callback_t(void *handle)`: cancels an asynchronous callback.
demayer 0:6bf0743ece18 336
demayer 0:6bf0743ece18 337 Please see [their doxygen documentation for implementation details](utest/scheduler.h).
demayer 0:6bf0743ece18 338
demayer 0:6bf0743ece18 339 ### Example Synchronous Scheduler
demayer 0:6bf0743ece18 340
demayer 0:6bf0743ece18 341 Here is the most [basic scheduler implementation without any asynchronous support](test/minimal_scheduler/main.cpp). Note that this does not require any hardware support at all, but you cannot use timeouts in your test cases!
demayer 0:6bf0743ece18 342 ```cpp
demayer 0:6bf0743ece18 343 volatile utest_v1_harness_callback_t minimal_callback;
demayer 0:6bf0743ece18 344
demayer 0:6bf0743ece18 345 static void* utest_minimal_post(const utest_v1_harness_callback_t callback, const uint32_t delay_ms) {
demayer 0:6bf0743ece18 346 minimal_callback = callback;
demayer 0:6bf0743ece18 347 // this scheduler does not support asynchronous callbacks
demayer 0:6bf0743ece18 348 return (delay_ms ? NULL : (void*)1);
demayer 0:6bf0743ece18 349 }
demayer 0:6bf0743ece18 350 static int32_t utest_minimal_cancel(void*) {
demayer 0:6bf0743ece18 351 return -1; // canceling not supported either
demayer 0:6bf0743ece18 352 }
demayer 0:6bf0743ece18 353 static const utest_v1_scheduler_t utest_minimal_scheduler = {utest_minimal_post, utest_minimal_cancel};
demayer 0:6bf0743ece18 354
demayer 0:6bf0743ece18 355 // [...] Add your test cases and specification here.
demayer 0:6bf0743ece18 356
demayer 0:6bf0743ece18 357 void main() // or whatever your custom entry point is
demayer 0:6bf0743ece18 358 {
demayer 0:6bf0743ece18 359 // You MUST set the custom scheduler before running the specification.
demayer 0:6bf0743ece18 360 Harness::set_scheduler(utest_minimal_scheduler);
demayer 0:6bf0743ece18 361 Harness::run(specification);
demayer 0:6bf0743ece18 362
demayer 0:6bf0743ece18 363 while(1) {
demayer 0:6bf0743ece18 364 if (minimal_callback) {
demayer 0:6bf0743ece18 365 // copy the callback and reset the shared memory
demayer 0:6bf0743ece18 366 utest_v1_harness_callback_t callback = minimal_callback;
demayer 0:6bf0743ece18 367 minimal_callback = NULL;
demayer 0:6bf0743ece18 368 callback(); // execute the copied callback
demayer 0:6bf0743ece18 369 }
demayer 0:6bf0743ece18 370 }
demayer 0:6bf0743ece18 371 }
demayer 0:6bf0743ece18 372 ```
demayer 0:6bf0743ece18 373
demayer 0:6bf0743ece18 374 ### Example Asynchronous Scheduler
demayer 0:6bf0743ece18 375
demayer 0:6bf0743ece18 376 Here is the a [complete scheduler implementation with any asynchronous support](test/minimal_scheduler_async/main.cpp). Note that this does require at least a hardware timer, in this case we have used `mbed-hal/us_ticker`! Note that you must not execute the callback in the timer interrupt context, but in the main loop context!
demayer 0:6bf0743ece18 377 ```cpp
demayer 0:6bf0743ece18 378 volatile utest_v1_harness_callback_t minimal_callback;
demayer 0:6bf0743ece18 379 volatile utest_v1_harness_callback_t ticker_callback;
demayer 0:6bf0743ece18 380 const ticker_data_t *ticker_data;
demayer 0:6bf0743ece18 381 ticker_event_t ticker_event;
demayer 0:6bf0743ece18 382
demayer 0:6bf0743ece18 383 static void ticker_handler(uint32_t) {
demayer 0:6bf0743ece18 384 minimal_callback = ticker_callback; // interrupt context!
demayer 0:6bf0743ece18 385 }
demayer 0:6bf0743ece18 386 static void* utest_minimal_post(const utest_v1_harness_callback_t callback, const uint32_t delay_ms) {
demayer 0:6bf0743ece18 387 if (delay_ms) {
demayer 0:6bf0743ece18 388 ticker_callback = callback;
demayer 0:6bf0743ece18 389 ticker_insert_event(ticker_data, &ticker_event, ticker_read(ticker_data) + delay_ms * 1000, 0);
demayer 0:6bf0743ece18 390 }
demayer 0:6bf0743ece18 391 else minimal_callback = callback;
demayer 0:6bf0743ece18 392 return (void*)1;
demayer 0:6bf0743ece18 393 }
demayer 0:6bf0743ece18 394 static int32_t utest_minimal_cancel(void*) {
demayer 0:6bf0743ece18 395 ticker_remove_event(ticker_data, &ticker_event);
demayer 0:6bf0743ece18 396 return 0; // canceling is supported
demayer 0:6bf0743ece18 397 }
demayer 0:6bf0743ece18 398 static const utest_v1_scheduler_t utest_minimal_scheduler = {utest_minimal_post, utest_minimal_cancel};
demayer 0:6bf0743ece18 399
demayer 0:6bf0743ece18 400 // [...] Add your test cases and specification here.
demayer 0:6bf0743ece18 401
demayer 0:6bf0743ece18 402 void main() // or whatever your custom entry point is
demayer 0:6bf0743ece18 403 {
demayer 0:6bf0743ece18 404 ticker_data = get_us_ticker_data(); // initialize the ticker data.
demayer 0:6bf0743ece18 405 ticker_set_handler(ticker_data, ticker_handler);
demayer 0:6bf0743ece18 406 // You MUST set the custom scheduler before running the specification.
demayer 0:6bf0743ece18 407 Harness::set_scheduler(utest_minimal_scheduler);
demayer 0:6bf0743ece18 408 Harness::run(specification);
demayer 0:6bf0743ece18 409
demayer 0:6bf0743ece18 410 while(1) {
demayer 0:6bf0743ece18 411 if (minimal_callback) {
demayer 0:6bf0743ece18 412 // copy the callback and reset the shared memory
demayer 0:6bf0743ece18 413 utest_v1_harness_callback_t callback = minimal_callback;
demayer 0:6bf0743ece18 414 minimal_callback = NULL;
demayer 0:6bf0743ece18 415 callback(); // execute the copied callback
demayer 0:6bf0743ece18 416 }
demayer 0:6bf0743ece18 417 }
demayer 0:6bf0743ece18 418 }
demayer 0:6bf0743ece18 419 ```