HTTP and HTTPS library for Mbed OS 5

Dependents:   MQTTGateway2 MQTTGatewayK64 http-example-wnc GuardRoom ... more

For the example program, see: sandbox/http-example.

This library is used to make HTTP and HTTPS calls from Mbed OS 5 applications.

HTTP Request API

NetworkInterface* network = /* obtain a NetworkInterface object */

const char body[] = "{\"hello\":\"world\"}";

HttpRequest* request = new HttpRequest(network, HTTP_POST, "http://httpbin.org/post");
request->set_header("Content-Type", "application/json");
HttpResponse* response = request->send(body, strlen(body));
// if response is NULL, check response->get_error()

printf("status is %d - %s\n", response->get_status_code(), response->get_status_message());
printf("body is:\n%s\n", response->get_body_as_string().c_str());

delete request; // also clears out the response

HTTPS Request API

// pass in the root certificates that you trust, there is no central CA registry in Mbed OS
const char SSL_CA_PEM[] = "-----BEGIN CERTIFICATE-----\n"
    /* rest of the CA root certificates */;

NetworkInterface* network = /* obtain a NetworkInterface object */

const char body[] = "{\"hello\":\"world\"}";

HttpsRequest* request = new HttpsRequest(network, SSL_CA_PEM, HTTP_GET "https://httpbin.org/status/418");
HttpResponse* response = request->send();
// if response is NULL, check response->get_error()

printf("status is %d - %s\n", response->get_status_code(), response->get_status_message());
printf("body is:\n%s\n", response->get_body().c_str());

delete request;

Note: You can get the root CA for a domain easily from Firefox. Click on the green padlock, click More information > Security > View certificate > Details. Select the top entry in the 'Certificate Hierarchy' and click Export.... This gives you a PEM file. Add the content of the PEM file to your root CA list (here's an image).

Mbed TLS Entropy configuration

If your target does not have a built-in TRNG, or other entropy sources, add the following macros to your mbed_app.json file to disable entropy:

{
    "macros": [
        "MBEDTLS_TEST_NULL_ENTROPY",
        "MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES"
    ]
}

Note that this is not secure, and you should not deploy this device into production with this configuration.

Memory usage

Small requests where the body of the response is cached by the library (like the one found in main-http.cpp), require 4K of RAM. When the request is finished they require 1.5K of RAM, depending on the size of the response. This applies both to HTTP and HTTPS. If you need to handle requests that return a large response body, see 'Dealing with large body'.

HTTPS requires additional memory: on FRDM-K64F about 50K of heap space (at its peak). This means that you cannot use HTTPS on devices with less than 128K of memory, asyou also need to reserve memory for the stack and network interface.

Dealing with large response body

By default the library will store the full request body on the heap. This works well for small responses, but you'll run out of memory when receiving a large response body. To mitigate this you can pass in a callback as the last argument to the request constructor. This callback will be called whenever a chunk of the body is received. You can set the request chunk size in the HTTP_RECEIVE_BUFFER_SIZE macro (see mbed_lib.json for the definition) although it also depends on the buffer size ofthe underlying network connection.

void body_callback(const char* data, uint32_t data_len) {
    // do something with the data
}

HttpRequest* req = new HttpRequest(network, HTTP_GET, "http://pathtolargefile.com", &body_callback);
req->send(NULL, 0);

Dealing with a large request body

If you cannot load the full request into memory, you can pass a callback into the send function. Through this callback you can feed in chunks of the request body. This is very useful if you want to send files from a file system.

const void * get_chunk(uint32_t* out_size) {
    // set the value of out_size (via *out_size = 10) to the size of the buffer
    // return the buffer

    // if you don't have any more data, set *out_size to 0
}

HttpRequest* req = new HttpRequest(network, HTTP_POST, "http://my_api.com/upload");
req->send(callback(&get_chunk));

Socket re-use

By default the library opens a new socket per request. This is wasteful, especially when dealing with TLS requests. You can re-use sockets like this:

HTTP

TCPSocket* socket = new TCPSocket();

nsapi_error_t open_result = socket->open(network);
// check open_result

nsapi_error_t connect_result = socket->connect("httpbin.org", 80);
// check connect_result

// Pass in `socket`, instead of `network` as first argument
HttpRequest* req = new HttpRequest(socket, HTTP_GET, "http://httpbin.org/status/418");

HTTPS

TLSSocket* socket = new TLSSocket();

nsapi_error_t r;
// make sure to check the return values for the calls below (should return NSAPI_ERROR_OK)
r = socket->open(network);
r = socket->set_root_ca_cert(SSL_CA_PEM);
r = socket->connect("httpbin.org", 443);

// Pass in `socket`, instead of `network` as first argument, and omit the `SSL_CA_PEM` argument
HttpsRequest* get_req = new HttpsRequest(socket, HTTP_GET, "https://httpbin.org/status/418");

Request logging

To make debugging easier you can log the raw request body that goes over the line. This also works with chunked encoding.

uint8_t *request_buffer = (uint8_t*)calloc(2048, 1);
req->set_request_log_buffer(request_buffer, 2048);

// after the request is done:
printf("\n----- Request buffer -----\n");
for (size_t ix = 0; ix < req->get_request_log_buffer_length(); ix++) {
    printf("%02x ", request_buffer[ix]);
}
printf("\n");

Integration tests

Integration tests are located in the TESTS folder and are ran through Greentea. Instructions on how to run the tests are in http-example.

Mbed OS 5.10 or lower

If you want to use this library on Mbed OS 5.10 or lower, you need to add the TLSSocket library to your project. This library is included in Mbed OS 5.11 and up.

Tested on

  • K64F with Ethernet.
  • NUCLEO_F411RE with ESP8266.
  • ODIN-W2 with WiFi.
  • K64F with Atmel 6LoWPAN shield.
  • DISCO-L475VG-IOT01A with WiFi.
  • Mbed Simulator.
Committer:
Jan Jongboom
Date:
Mon Aug 12 11:45:31 2019 +0200
Revision:
39:a8d157986ad8
Parent:
0:910f5949759f
Fix parsed url leaking memory if path is empty

Who changed what in which revision?

UserRevisionLine numberNew contents of line
Jan Jongboom 0:910f5949759f 1 Apache License
Jan Jongboom 0:910f5949759f 2 Version 2.0, January 2004
Jan Jongboom 0:910f5949759f 3 http://www.apache.org/licenses/
Jan Jongboom 0:910f5949759f 4
Jan Jongboom 0:910f5949759f 5 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Jan Jongboom 0:910f5949759f 6
Jan Jongboom 0:910f5949759f 7 1. Definitions.
Jan Jongboom 0:910f5949759f 8
Jan Jongboom 0:910f5949759f 9 "License" shall mean the terms and conditions for use, reproduction, and
Jan Jongboom 0:910f5949759f 10 distribution as defined by Sections 1 through 9 of this document.
Jan Jongboom 0:910f5949759f 11
Jan Jongboom 0:910f5949759f 12 "Licensor" shall mean the copyright owner or entity authorized by the copyright
Jan Jongboom 0:910f5949759f 13 owner that is granting the License.
Jan Jongboom 0:910f5949759f 14
Jan Jongboom 0:910f5949759f 15 "Legal Entity" shall mean the union of the acting entity and all other entities
Jan Jongboom 0:910f5949759f 16 that control, are controlled by, or are under common control with that entity.
Jan Jongboom 0:910f5949759f 17 For the purposes of this definition, "control" means (i) the power, direct or
Jan Jongboom 0:910f5949759f 18 indirect, to cause the direction or management of such entity, whether by
Jan Jongboom 0:910f5949759f 19 contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the
Jan Jongboom 0:910f5949759f 20 outstanding shares, or (iii) beneficial ownership of such entity.
Jan Jongboom 0:910f5949759f 21
Jan Jongboom 0:910f5949759f 22 "You" (or "Your") shall mean an individual or Legal Entity exercising
Jan Jongboom 0:910f5949759f 23 permissions granted by this License.
Jan Jongboom 0:910f5949759f 24
Jan Jongboom 0:910f5949759f 25 "Source" form shall mean the preferred form for making modifications, including
Jan Jongboom 0:910f5949759f 26 but not limited to software source code, documentation source, and configuration
Jan Jongboom 0:910f5949759f 27 files.
Jan Jongboom 0:910f5949759f 28
Jan Jongboom 0:910f5949759f 29 "Object" form shall mean any form resulting from mechanical transformation or
Jan Jongboom 0:910f5949759f 30 translation of a Source form, including but not limited to compiled object code,
Jan Jongboom 0:910f5949759f 31 generated documentation, and conversions to other media types.
Jan Jongboom 0:910f5949759f 32
Jan Jongboom 0:910f5949759f 33 "Work" shall mean the work of authorship, whether in Source or Object form, made
Jan Jongboom 0:910f5949759f 34 available under the License, as indicated by a copyright notice that is included
Jan Jongboom 0:910f5949759f 35 in or attached to the work (an example is provided in the Appendix below).
Jan Jongboom 0:910f5949759f 36
Jan Jongboom 0:910f5949759f 37 "Derivative Works" shall mean any work, whether in Source or Object form, that
Jan Jongboom 0:910f5949759f 38 is based on (or derived from) the Work and for which the editorial revisions,
Jan Jongboom 0:910f5949759f 39 annotations, elaborations, or other modifications represent, as a whole, an
Jan Jongboom 0:910f5949759f 40 original work of authorship. For the purposes of this License, Derivative Works
Jan Jongboom 0:910f5949759f 41 shall not include works that remain separable from, or merely link (or bind by
Jan Jongboom 0:910f5949759f 42 name) to the interfaces of, the Work and Derivative Works thereof.
Jan Jongboom 0:910f5949759f 43
Jan Jongboom 0:910f5949759f 44 "Contribution" shall mean any work of authorship, including the original version
Jan Jongboom 0:910f5949759f 45 of the Work and any modifications or additions to that Work or Derivative Works
Jan Jongboom 0:910f5949759f 46 thereof, that is intentionally submitted to Licensor for inclusion in the Work
Jan Jongboom 0:910f5949759f 47 by the copyright owner or by an individual or Legal Entity authorized to submit
Jan Jongboom 0:910f5949759f 48 on behalf of the copyright owner. For the purposes of this definition,
Jan Jongboom 0:910f5949759f 49 "submitted" means any form of electronic, verbal, or written communication sent
Jan Jongboom 0:910f5949759f 50 to the Licensor or its representatives, including but not limited to
Jan Jongboom 0:910f5949759f 51 communication on electronic mailing lists, source code control systems, and
Jan Jongboom 0:910f5949759f 52 issue tracking systems that are managed by, or on behalf of, the Licensor for
Jan Jongboom 0:910f5949759f 53 the purpose of discussing and improving the Work, but excluding communication
Jan Jongboom 0:910f5949759f 54 that is conspicuously marked or otherwise designated in writing by the copyright
Jan Jongboom 0:910f5949759f 55 owner as "Not a Contribution."
Jan Jongboom 0:910f5949759f 56
Jan Jongboom 0:910f5949759f 57 "Contributor" shall mean Licensor and any individual or Legal Entity on behalf
Jan Jongboom 0:910f5949759f 58 of whom a Contribution has been received by Licensor and subsequently
Jan Jongboom 0:910f5949759f 59 incorporated within the Work.
Jan Jongboom 0:910f5949759f 60
Jan Jongboom 0:910f5949759f 61 2. Grant of Copyright License.
Jan Jongboom 0:910f5949759f 62
Jan Jongboom 0:910f5949759f 63 Subject to the terms and conditions of this License, each Contributor hereby
Jan Jongboom 0:910f5949759f 64 grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
Jan Jongboom 0:910f5949759f 65 irrevocable copyright license to reproduce, prepare Derivative Works of,
Jan Jongboom 0:910f5949759f 66 publicly display, publicly perform, sublicense, and distribute the Work and such
Jan Jongboom 0:910f5949759f 67 Derivative Works in Source or Object form.
Jan Jongboom 0:910f5949759f 68
Jan Jongboom 0:910f5949759f 69 3. Grant of Patent License.
Jan Jongboom 0:910f5949759f 70
Jan Jongboom 0:910f5949759f 71 Subject to the terms and conditions of this License, each Contributor hereby
Jan Jongboom 0:910f5949759f 72 grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free,
Jan Jongboom 0:910f5949759f 73 irrevocable (except as stated in this section) patent license to make, have
Jan Jongboom 0:910f5949759f 74 made, use, offer to sell, sell, import, and otherwise transfer the Work, where
Jan Jongboom 0:910f5949759f 75 such license applies only to those patent claims licensable by such Contributor
Jan Jongboom 0:910f5949759f 76 that are necessarily infringed by their Contribution(s) alone or by combination
Jan Jongboom 0:910f5949759f 77 of their Contribution(s) with the Work to which such Contribution(s) was
Jan Jongboom 0:910f5949759f 78 submitted. If You institute patent litigation against any entity (including a
Jan Jongboom 0:910f5949759f 79 cross-claim or counterclaim in a lawsuit) alleging that the Work or a
Jan Jongboom 0:910f5949759f 80 Contribution incorporated within the Work constitutes direct or contributory
Jan Jongboom 0:910f5949759f 81 patent infringement, then any patent licenses granted to You under this License
Jan Jongboom 0:910f5949759f 82 for that Work shall terminate as of the date such litigation is filed.
Jan Jongboom 0:910f5949759f 83
Jan Jongboom 0:910f5949759f 84 4. Redistribution.
Jan Jongboom 0:910f5949759f 85
Jan Jongboom 0:910f5949759f 86 You may reproduce and distribute copies of the Work or Derivative Works thereof
Jan Jongboom 0:910f5949759f 87 in any medium, with or without modifications, and in Source or Object form,
Jan Jongboom 0:910f5949759f 88 provided that You meet the following conditions:
Jan Jongboom 0:910f5949759f 89
Jan Jongboom 0:910f5949759f 90 You must give any other recipients of the Work or Derivative Works a copy of
Jan Jongboom 0:910f5949759f 91 this License; and
Jan Jongboom 0:910f5949759f 92 You must cause any modified files to carry prominent notices stating that You
Jan Jongboom 0:910f5949759f 93 changed the files; and
Jan Jongboom 0:910f5949759f 94 You must retain, in the Source form of any Derivative Works that You distribute,
Jan Jongboom 0:910f5949759f 95 all copyright, patent, trademark, and attribution notices from the Source form
Jan Jongboom 0:910f5949759f 96 of the Work, excluding those notices that do not pertain to any part of the
Jan Jongboom 0:910f5949759f 97 Derivative Works; and
Jan Jongboom 0:910f5949759f 98 If the Work includes a "NOTICE" text file as part of its distribution, then any
Jan Jongboom 0:910f5949759f 99 Derivative Works that You distribute must include a readable copy of the
Jan Jongboom 0:910f5949759f 100 attribution notices contained within such NOTICE file, excluding those notices
Jan Jongboom 0:910f5949759f 101 that do not pertain to any part of the Derivative Works, in at least one of the
Jan Jongboom 0:910f5949759f 102 following places: within a NOTICE text file distributed as part of the
Jan Jongboom 0:910f5949759f 103 Derivative Works; within the Source form or documentation, if provided along
Jan Jongboom 0:910f5949759f 104 with the Derivative Works; or, within a display generated by the Derivative
Jan Jongboom 0:910f5949759f 105 Works, if and wherever such third-party notices normally appear. The contents of
Jan Jongboom 0:910f5949759f 106 the NOTICE file are for informational purposes only and do not modify the
Jan Jongboom 0:910f5949759f 107 License. You may add Your own attribution notices within Derivative Works that
Jan Jongboom 0:910f5949759f 108 You distribute, alongside or as an addendum to the NOTICE text from the Work,
Jan Jongboom 0:910f5949759f 109 provided that such additional attribution notices cannot be construed as
Jan Jongboom 0:910f5949759f 110 modifying the License.
Jan Jongboom 0:910f5949759f 111 You may add Your own copyright statement to Your modifications and may provide
Jan Jongboom 0:910f5949759f 112 additional or different license terms and conditions for use, reproduction, or
Jan Jongboom 0:910f5949759f 113 distribution of Your modifications, or for any such Derivative Works as a whole,
Jan Jongboom 0:910f5949759f 114 provided Your use, reproduction, and distribution of the Work otherwise complies
Jan Jongboom 0:910f5949759f 115 with the conditions stated in this License.
Jan Jongboom 0:910f5949759f 116
Jan Jongboom 0:910f5949759f 117 5. Submission of Contributions.
Jan Jongboom 0:910f5949759f 118
Jan Jongboom 0:910f5949759f 119 Unless You explicitly state otherwise, any Contribution intentionally submitted
Jan Jongboom 0:910f5949759f 120 for inclusion in the Work by You to the Licensor shall be under the terms and
Jan Jongboom 0:910f5949759f 121 conditions of this License, without any additional terms or conditions.
Jan Jongboom 0:910f5949759f 122 Notwithstanding the above, nothing herein shall supersede or modify the terms of
Jan Jongboom 0:910f5949759f 123 any separate license agreement you may have executed with Licensor regarding
Jan Jongboom 0:910f5949759f 124 such Contributions.
Jan Jongboom 0:910f5949759f 125
Jan Jongboom 0:910f5949759f 126 6. Trademarks.
Jan Jongboom 0:910f5949759f 127
Jan Jongboom 0:910f5949759f 128 This License does not grant permission to use the trade names, trademarks,
Jan Jongboom 0:910f5949759f 129 service marks, or product names of the Licensor, except as required for
Jan Jongboom 0:910f5949759f 130 reasonable and customary use in describing the origin of the Work and
Jan Jongboom 0:910f5949759f 131 reproducing the content of the NOTICE file.
Jan Jongboom 0:910f5949759f 132
Jan Jongboom 0:910f5949759f 133 7. Disclaimer of Warranty.
Jan Jongboom 0:910f5949759f 134
Jan Jongboom 0:910f5949759f 135 Unless required by applicable law or agreed to in writing, Licensor provides the
Jan Jongboom 0:910f5949759f 136 Work (and each Contributor provides its Contributions) on an "AS IS" BASIS,
Jan Jongboom 0:910f5949759f 137 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied,
Jan Jongboom 0:910f5949759f 138 including, without limitation, any warranties or conditions of TITLE,
Jan Jongboom 0:910f5949759f 139 NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are
Jan Jongboom 0:910f5949759f 140 solely responsible for determining the appropriateness of using or
Jan Jongboom 0:910f5949759f 141 redistributing the Work and assume any risks associated with Your exercise of
Jan Jongboom 0:910f5949759f 142 permissions under this License.
Jan Jongboom 0:910f5949759f 143
Jan Jongboom 0:910f5949759f 144 8. Limitation of Liability.
Jan Jongboom 0:910f5949759f 145
Jan Jongboom 0:910f5949759f 146 In no event and under no legal theory, whether in tort (including negligence),
Jan Jongboom 0:910f5949759f 147 contract, or otherwise, unless required by applicable law (such as deliberate
Jan Jongboom 0:910f5949759f 148 and grossly negligent acts) or agreed to in writing, shall any Contributor be
Jan Jongboom 0:910f5949759f 149 liable to You for damages, including any direct, indirect, special, incidental,
Jan Jongboom 0:910f5949759f 150 or consequential damages of any character arising as a result of this License or
Jan Jongboom 0:910f5949759f 151 out of the use or inability to use the Work (including but not limited to
Jan Jongboom 0:910f5949759f 152 damages for loss of goodwill, work stoppage, computer failure or malfunction, or
Jan Jongboom 0:910f5949759f 153 any and all other commercial damages or losses), even if such Contributor has
Jan Jongboom 0:910f5949759f 154 been advised of the possibility of such damages.
Jan Jongboom 0:910f5949759f 155
Jan Jongboom 0:910f5949759f 156 9. Accepting Warranty or Additional Liability.
Jan Jongboom 0:910f5949759f 157
Jan Jongboom 0:910f5949759f 158 While redistributing the Work or Derivative Works thereof, You may choose to
Jan Jongboom 0:910f5949759f 159 offer, and charge a fee for, acceptance of support, warranty, indemnity, or
Jan Jongboom 0:910f5949759f 160 other liability obligations and/or rights consistent with this License. However,
Jan Jongboom 0:910f5949759f 161 in accepting such obligations, You may act only on Your own behalf and on Your
Jan Jongboom 0:910f5949759f 162 sole responsibility, not on behalf of any other Contributor, and only if You
Jan Jongboom 0:910f5949759f 163 agree to indemnify, defend, and hold each Contributor harmless for any liability
Jan Jongboom 0:910f5949759f 164 incurred by, or claims asserted against, such Contributor by reason of your
Jan Jongboom 0:910f5949759f 165 accepting any such warranty or additional liability.
Jan Jongboom 0:910f5949759f 166
Jan Jongboom 0:910f5949759f 167 ---
Jan Jongboom 0:910f5949759f 168
Jan Jongboom 0:910f5949759f 169 http_parser.c is based on src/http/ngx_http_parse.c from NGINX copyright
Jan Jongboom 0:910f5949759f 170 Igor Sysoev.
Jan Jongboom 0:910f5949759f 171
Jan Jongboom 0:910f5949759f 172 Additional changes are licensed under the same terms as NGINX and
Jan Jongboom 0:910f5949759f 173 copyright Joyent, Inc. and other Node contributors. All rights reserved.
Jan Jongboom 0:910f5949759f 174
Jan Jongboom 0:910f5949759f 175 Permission is hereby granted, free of charge, to any person obtaining a copy
Jan Jongboom 0:910f5949759f 176 of this software and associated documentation files (the "Software"), to
Jan Jongboom 0:910f5949759f 177 deal in the Software without restriction, including without limitation the
Jan Jongboom 0:910f5949759f 178 rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
Jan Jongboom 0:910f5949759f 179 sell copies of the Software, and to permit persons to whom the Software is
Jan Jongboom 0:910f5949759f 180 furnished to do so, subject to the following conditions:
Jan Jongboom 0:910f5949759f 181
Jan Jongboom 0:910f5949759f 182 The above copyright notice and this permission notice shall be included in
Jan Jongboom 0:910f5949759f 183 all copies or substantial portions of the Software.
Jan Jongboom 0:910f5949759f 184
Jan Jongboom 0:910f5949759f 185 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
Jan Jongboom 0:910f5949759f 186 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
Jan Jongboom 0:910f5949759f 187 FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
Jan Jongboom 0:910f5949759f 188 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
Jan Jongboom 0:910f5949759f 189 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
Jan Jongboom 0:910f5949759f 190 FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
Jan Jongboom 0:910f5949759f 191 IN THE SOFTWARE.