HTTP and HTTPS library for Mbed OS 5
Dependents: MQTTGateway2 MQTTGatewayK64 http-example-wnc GuardRoom ... more
For the example program, see: sandbox/http-example.
This library is used to make HTTP and HTTPS calls from Mbed OS 5 applications.
HTTP Request API
NetworkInterface* network = /* obtain a NetworkInterface object */ const char body[] = "{\"hello\":\"world\"}"; HttpRequest* request = new HttpRequest(network, HTTP_POST, "http://httpbin.org/post"); request->set_header("Content-Type", "application/json"); HttpResponse* response = request->send(body, strlen(body)); // if response is NULL, check response->get_error() printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); printf("body is:\n%s\n", response->get_body_as_string().c_str()); delete request; // also clears out the response
HTTPS Request API
// pass in the root certificates that you trust, there is no central CA registry in Mbed OS const char SSL_CA_PEM[] = "-----BEGIN CERTIFICATE-----\n" /* rest of the CA root certificates */; NetworkInterface* network = /* obtain a NetworkInterface object */ const char body[] = "{\"hello\":\"world\"}"; HttpsRequest* request = new HttpsRequest(network, SSL_CA_PEM, HTTP_GET "https://httpbin.org/status/418"); HttpResponse* response = request->send(); // if response is NULL, check response->get_error() printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); printf("body is:\n%s\n", response->get_body().c_str()); delete request;
Note: You can get the root CA for a domain easily from Firefox. Click on the green padlock, click More information > Security > View certificate > Details. Select the top entry in the 'Certificate Hierarchy' and click Export.... This gives you a PEM file. Add the content of the PEM file to your root CA list (here's an image).
Mbed TLS Entropy configuration
If your target does not have a built-in TRNG, or other entropy sources, add the following macros to your mbed_app.json
file to disable entropy:
{ "macros": [ "MBEDTLS_TEST_NULL_ENTROPY", "MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES" ] }
Note that this is not secure, and you should not deploy this device into production with this configuration.
Memory usage
Small requests where the body of the response is cached by the library (like the one found in main-http.cpp), require 4K of RAM. When the request is finished they require 1.5K of RAM, depending on the size of the response. This applies both to HTTP and HTTPS. If you need to handle requests that return a large response body, see 'Dealing with large body'.
HTTPS requires additional memory: on FRDM-K64F about 50K of heap space (at its peak). This means that you cannot use HTTPS on devices with less than 128K of memory, asyou also need to reserve memory for the stack and network interface.
Dealing with large response body
By default the library will store the full request body on the heap. This works well for small responses, but you'll run out of memory when receiving a large response body. To mitigate this you can pass in a callback as the last argument to the request constructor. This callback will be called whenever a chunk of the body is received. You can set the request chunk size in the HTTP_RECEIVE_BUFFER_SIZE
macro (see mbed_lib.json
for the definition) although it also depends on the buffer size ofthe underlying network connection.
void body_callback(const char* data, uint32_t data_len) { // do something with the data } HttpRequest* req = new HttpRequest(network, HTTP_GET, "http://pathtolargefile.com", &body_callback); req->send(NULL, 0);
Dealing with a large request body
If you cannot load the full request into memory, you can pass a callback into the send
function. Through this callback you can feed in chunks of the request body. This is very useful if you want to send files from a file system.
const void * get_chunk(uint32_t* out_size) { // set the value of out_size (via *out_size = 10) to the size of the buffer // return the buffer // if you don't have any more data, set *out_size to 0 } HttpRequest* req = new HttpRequest(network, HTTP_POST, "http://my_api.com/upload"); req->send(callback(&get_chunk));
Socket re-use
By default the library opens a new socket per request. This is wasteful, especially when dealing with TLS requests. You can re-use sockets like this:
HTTP
TCPSocket* socket = new TCPSocket(); nsapi_error_t open_result = socket->open(network); // check open_result nsapi_error_t connect_result = socket->connect("httpbin.org", 80); // check connect_result // Pass in `socket`, instead of `network` as first argument HttpRequest* req = new HttpRequest(socket, HTTP_GET, "http://httpbin.org/status/418");
HTTPS
TLSSocket* socket = new TLSSocket(); nsapi_error_t r; // make sure to check the return values for the calls below (should return NSAPI_ERROR_OK) r = socket->open(network); r = socket->set_root_ca_cert(SSL_CA_PEM); r = socket->connect("httpbin.org", 443); // Pass in `socket`, instead of `network` as first argument, and omit the `SSL_CA_PEM` argument HttpsRequest* get_req = new HttpsRequest(socket, HTTP_GET, "https://httpbin.org/status/418");
Request logging
To make debugging easier you can log the raw request body that goes over the line. This also works with chunked encoding.
uint8_t *request_buffer = (uint8_t*)calloc(2048, 1); req->set_request_log_buffer(request_buffer, 2048); // after the request is done: printf("\n----- Request buffer -----\n"); for (size_t ix = 0; ix < req->get_request_log_buffer_length(); ix++) { printf("%02x ", request_buffer[ix]); } printf("\n");
Integration tests
Integration tests are located in the TESTS
folder and are ran through Greentea. Instructions on how to run the tests are in http-example.
Mbed OS 5.10 or lower
If you want to use this library on Mbed OS 5.10 or lower, you need to add the TLSSocket library to your project. This library is included in Mbed OS 5.11 and up.
Tested on
- K64F with Ethernet.
- NUCLEO_F411RE with ESP8266.
- ODIN-W2 with WiFi.
- K64F with Atmel 6LoWPAN shield.
- DISCO-L475VG-IOT01A with WiFi.
- Mbed Simulator.
source/http_response.h@4:539df159e058, 2017-02-23 (annotated)
- Committer:
- Jan Jongboom
- Date:
- Thu Feb 23 13:50:43 2017 +0100
- Revision:
- 4:539df159e058
- Parent:
- 3:8a6b003e3874
- Child:
- 7:2e3eedb9ca5c
Allocate body on the heap, not on the stack
Who changed what in which revision?
User | Revision | Line number | New contents of line |
---|---|---|---|
Jan Jongboom |
0:910f5949759f | 1 | /* |
Jan Jongboom |
0:910f5949759f | 2 | * PackageLicenseDeclared: Apache-2.0 |
Jan Jongboom |
0:910f5949759f | 3 | * Copyright (c) 2017 ARM Limited |
Jan Jongboom |
0:910f5949759f | 4 | * |
Jan Jongboom |
0:910f5949759f | 5 | * Licensed under the Apache License, Version 2.0 (the "License"); |
Jan Jongboom |
0:910f5949759f | 6 | * you may not use this file except in compliance with the License. |
Jan Jongboom |
0:910f5949759f | 7 | * You may obtain a copy of the License at |
Jan Jongboom |
0:910f5949759f | 8 | * |
Jan Jongboom |
0:910f5949759f | 9 | * http://www.apache.org/licenses/LICENSE-2.0 |
Jan Jongboom |
0:910f5949759f | 10 | * |
Jan Jongboom |
0:910f5949759f | 11 | * Unless required by applicable law or agreed to in writing, software |
Jan Jongboom |
0:910f5949759f | 12 | * distributed under the License is distributed on an "AS IS" BASIS, |
Jan Jongboom |
0:910f5949759f | 13 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
Jan Jongboom |
0:910f5949759f | 14 | * See the License for the specific language governing permissions and |
Jan Jongboom |
0:910f5949759f | 15 | * limitations under the License. |
Jan Jongboom |
0:910f5949759f | 16 | */ |
Jan Jongboom |
0:910f5949759f | 17 | |
Jan Jongboom |
0:910f5949759f | 18 | #ifndef _MBED_HTTP_HTTP_RESPONSE |
Jan Jongboom |
0:910f5949759f | 19 | #define _MBED_HTTP_HTTP_RESPONSE |
Jan Jongboom |
0:910f5949759f | 20 | #include <string> |
Jan Jongboom |
0:910f5949759f | 21 | #include <vector> |
Jan Jongboom |
0:910f5949759f | 22 | #include "http_parser.h" |
Jan Jongboom |
0:910f5949759f | 23 | |
Jan Jongboom |
0:910f5949759f | 24 | using namespace std; |
Jan Jongboom |
0:910f5949759f | 25 | |
Jan Jongboom |
0:910f5949759f | 26 | class HttpResponse { |
Jan Jongboom |
0:910f5949759f | 27 | public: |
Jan Jongboom |
0:910f5949759f | 28 | HttpResponse() { |
Jan Jongboom |
0:910f5949759f | 29 | status_code = 0; |
Jan Jongboom |
0:910f5949759f | 30 | concat_header_field = false; |
Jan Jongboom |
0:910f5949759f | 31 | concat_header_value = false; |
Jan Jongboom |
3:8a6b003e3874 | 32 | expected_content_length = 0; |
Jan Jongboom |
3:8a6b003e3874 | 33 | body_length = 0; |
Jan Jongboom |
4:539df159e058 | 34 | body_offset = 0; |
Jan Jongboom |
4:539df159e058 | 35 | body = NULL; |
Jan Jongboom |
4:539df159e058 | 36 | } |
Jan Jongboom |
4:539df159e058 | 37 | |
Jan Jongboom |
4:539df159e058 | 38 | ~HttpResponse() { |
Jan Jongboom |
4:539df159e058 | 39 | if (body != NULL) { |
Jan Jongboom |
4:539df159e058 | 40 | free(body); |
Jan Jongboom |
4:539df159e058 | 41 | } |
Jan Jongboom |
0:910f5949759f | 42 | } |
Jan Jongboom |
0:910f5949759f | 43 | |
Jan Jongboom |
0:910f5949759f | 44 | void set_status(int a_status_code, string a_status_message) { |
Jan Jongboom |
0:910f5949759f | 45 | status_code = a_status_code; |
Jan Jongboom |
0:910f5949759f | 46 | status_message = a_status_message; |
Jan Jongboom |
0:910f5949759f | 47 | } |
Jan Jongboom |
0:910f5949759f | 48 | |
Jan Jongboom |
0:910f5949759f | 49 | int get_status_code() { |
Jan Jongboom |
0:910f5949759f | 50 | return status_code; |
Jan Jongboom |
0:910f5949759f | 51 | } |
Jan Jongboom |
0:910f5949759f | 52 | |
Jan Jongboom |
0:910f5949759f | 53 | string get_status_message() { |
Jan Jongboom |
0:910f5949759f | 54 | return status_message; |
Jan Jongboom |
0:910f5949759f | 55 | } |
Jan Jongboom |
0:910f5949759f | 56 | |
Jan Jongboom |
0:910f5949759f | 57 | void set_header_field(string field) { |
Jan Jongboom |
0:910f5949759f | 58 | concat_header_value = false; |
Jan Jongboom |
0:910f5949759f | 59 | |
Jan Jongboom |
0:910f5949759f | 60 | // headers can be chunked |
Jan Jongboom |
0:910f5949759f | 61 | if (concat_header_field) { |
Jan Jongboom |
0:910f5949759f | 62 | header_fields[header_fields.size() - 1] = header_fields[header_fields.size() - 1] + field; |
Jan Jongboom |
0:910f5949759f | 63 | } |
Jan Jongboom |
0:910f5949759f | 64 | else { |
Jan Jongboom |
0:910f5949759f | 65 | header_fields.push_back(field); |
Jan Jongboom |
0:910f5949759f | 66 | } |
Jan Jongboom |
0:910f5949759f | 67 | |
Jan Jongboom |
0:910f5949759f | 68 | concat_header_field = true; |
Jan Jongboom |
0:910f5949759f | 69 | } |
Jan Jongboom |
0:910f5949759f | 70 | |
Jan Jongboom |
0:910f5949759f | 71 | void set_header_value(string value) { |
Jan Jongboom |
0:910f5949759f | 72 | concat_header_field = false; |
Jan Jongboom |
0:910f5949759f | 73 | |
Jan Jongboom |
0:910f5949759f | 74 | // headers can be chunked |
Jan Jongboom |
0:910f5949759f | 75 | if (concat_header_value) { |
Jan Jongboom |
0:910f5949759f | 76 | header_values[header_values.size() - 1] = header_values[header_values.size() - 1] + value; |
Jan Jongboom |
0:910f5949759f | 77 | } |
Jan Jongboom |
0:910f5949759f | 78 | else { |
Jan Jongboom |
0:910f5949759f | 79 | header_values.push_back(value); |
Jan Jongboom |
0:910f5949759f | 80 | } |
Jan Jongboom |
0:910f5949759f | 81 | |
Jan Jongboom |
0:910f5949759f | 82 | concat_header_value = true; |
Jan Jongboom |
0:910f5949759f | 83 | } |
Jan Jongboom |
0:910f5949759f | 84 | |
Jan Jongboom |
3:8a6b003e3874 | 85 | void set_headers_complete() { |
Jan Jongboom |
3:8a6b003e3874 | 86 | // @todo: chunked encoding |
Jan Jongboom |
3:8a6b003e3874 | 87 | for (size_t ix = 0; ix < header_fields.size(); ix++) { |
Jan Jongboom |
3:8a6b003e3874 | 88 | if (strcicmp(header_fields[ix].c_str(), "content-length") == 0) { |
Jan Jongboom |
3:8a6b003e3874 | 89 | expected_content_length = (size_t)atoi(header_values[ix].c_str()); |
Jan Jongboom |
3:8a6b003e3874 | 90 | break; |
Jan Jongboom |
3:8a6b003e3874 | 91 | } |
Jan Jongboom |
3:8a6b003e3874 | 92 | } |
Jan Jongboom |
3:8a6b003e3874 | 93 | } |
Jan Jongboom |
3:8a6b003e3874 | 94 | |
Jan Jongboom |
0:910f5949759f | 95 | size_t get_headers_length() { |
Jan Jongboom |
0:910f5949759f | 96 | return header_fields.size(); |
Jan Jongboom |
0:910f5949759f | 97 | } |
Jan Jongboom |
0:910f5949759f | 98 | |
Jan Jongboom |
0:910f5949759f | 99 | vector<string> get_headers_fields() { |
Jan Jongboom |
0:910f5949759f | 100 | return header_fields; |
Jan Jongboom |
0:910f5949759f | 101 | } |
Jan Jongboom |
0:910f5949759f | 102 | |
Jan Jongboom |
0:910f5949759f | 103 | vector<string> get_headers_values() { |
Jan Jongboom |
0:910f5949759f | 104 | return header_values; |
Jan Jongboom |
0:910f5949759f | 105 | } |
Jan Jongboom |
0:910f5949759f | 106 | |
Jan Jongboom |
4:539df159e058 | 107 | void set_body(const char *at, size_t length) { |
Jan Jongboom |
4:539df159e058 | 108 | // only malloc when this fn is called, so we don't alloc when body callback's are enabled |
Jan Jongboom |
4:539df159e058 | 109 | if (body == NULL) { |
Jan Jongboom |
4:539df159e058 | 110 | body = (char*)malloc(expected_content_length); |
Jan Jongboom |
4:539df159e058 | 111 | } |
Jan Jongboom |
4:539df159e058 | 112 | |
Jan Jongboom |
4:539df159e058 | 113 | memcpy(body + body_offset, at, length); |
Jan Jongboom |
4:539df159e058 | 114 | |
Jan Jongboom |
4:539df159e058 | 115 | body_offset += length; |
Jan Jongboom |
0:910f5949759f | 116 | } |
Jan Jongboom |
0:910f5949759f | 117 | |
Jan Jongboom |
4:539df159e058 | 118 | void* get_body() { |
Jan Jongboom |
4:539df159e058 | 119 | return (void*)body; |
Jan Jongboom |
4:539df159e058 | 120 | } |
Jan Jongboom |
4:539df159e058 | 121 | |
Jan Jongboom |
4:539df159e058 | 122 | string get_body_as_string() { |
Jan Jongboom |
4:539df159e058 | 123 | string s(body, body_offset); |
Jan Jongboom |
4:539df159e058 | 124 | return s; |
Jan Jongboom |
0:910f5949759f | 125 | } |
Jan Jongboom |
0:910f5949759f | 126 | |
Jan Jongboom |
3:8a6b003e3874 | 127 | void increase_body_length(size_t length) { |
Jan Jongboom |
3:8a6b003e3874 | 128 | body_length += length; |
Jan Jongboom |
3:8a6b003e3874 | 129 | } |
Jan Jongboom |
3:8a6b003e3874 | 130 | |
Jan Jongboom |
4:539df159e058 | 131 | size_t get_body_length() { |
Jan Jongboom |
4:539df159e058 | 132 | return body_offset; |
Jan Jongboom |
4:539df159e058 | 133 | } |
Jan Jongboom |
4:539df159e058 | 134 | |
Jan Jongboom |
3:8a6b003e3874 | 135 | bool is_body_complete() { |
Jan Jongboom |
3:8a6b003e3874 | 136 | return body_length == expected_content_length; |
Jan Jongboom |
3:8a6b003e3874 | 137 | } |
Jan Jongboom |
3:8a6b003e3874 | 138 | |
Jan Jongboom |
0:910f5949759f | 139 | private: |
Jan Jongboom |
3:8a6b003e3874 | 140 | // from http://stackoverflow.com/questions/5820810/case-insensitive-string-comp-in-c |
Jan Jongboom |
3:8a6b003e3874 | 141 | int strcicmp(char const *a, char const *b) { |
Jan Jongboom |
3:8a6b003e3874 | 142 | for (;; a++, b++) { |
Jan Jongboom |
3:8a6b003e3874 | 143 | int d = tolower(*a) - tolower(*b); |
Jan Jongboom |
3:8a6b003e3874 | 144 | if (d != 0 || !*a) { |
Jan Jongboom |
3:8a6b003e3874 | 145 | return d; |
Jan Jongboom |
3:8a6b003e3874 | 146 | } |
Jan Jongboom |
3:8a6b003e3874 | 147 | } |
Jan Jongboom |
3:8a6b003e3874 | 148 | } |
Jan Jongboom |
3:8a6b003e3874 | 149 | |
Jan Jongboom |
0:910f5949759f | 150 | int status_code; |
Jan Jongboom |
0:910f5949759f | 151 | string status_message; |
Jan Jongboom |
0:910f5949759f | 152 | |
Jan Jongboom |
0:910f5949759f | 153 | vector<string> header_fields; |
Jan Jongboom |
0:910f5949759f | 154 | vector<string> header_values; |
Jan Jongboom |
0:910f5949759f | 155 | |
Jan Jongboom |
0:910f5949759f | 156 | bool concat_header_field; |
Jan Jongboom |
0:910f5949759f | 157 | bool concat_header_value; |
Jan Jongboom |
0:910f5949759f | 158 | |
Jan Jongboom |
3:8a6b003e3874 | 159 | size_t expected_content_length; |
Jan Jongboom |
3:8a6b003e3874 | 160 | |
Jan Jongboom |
4:539df159e058 | 161 | char * body; |
Jan Jongboom |
3:8a6b003e3874 | 162 | size_t body_length; |
Jan Jongboom |
4:539df159e058 | 163 | size_t body_offset; |
Jan Jongboom |
0:910f5949759f | 164 | }; |
Jan Jongboom |
0:910f5949759f | 165 | #endif |