Fork of SandBox's original mbed-http (https://os.mbed.com/teams/sandbox/code/mbed-http/) and update for MbedOS6+ Content of TESTS folder was replaced with basic examples form original SandBox's HelloWorld
README.md@33:5b2869cc8934, 2019-01-04 (annotated)
- Committer:
- Jan Jongboom
- Date:
- Fri Jan 04 13:27:23 2019 +0100
- Revision:
- 33:5b2869cc8934
- Parent:
- 32:fa4d71265625
Remove TLSSocket (now in Mbed OS 5.11)
Who changed what in which revision?
User | Revision | Line number | New contents of line |
---|---|---|---|
Jan Jongboom |
0:910f5949759f | 1 | # HTTP and HTTPS library for mbed OS 5 |
Jan Jongboom |
0:910f5949759f | 2 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 3 | This library is used to make HTTP and HTTPS calls from Mbed OS 5 applications. |
Jan Jongboom |
0:910f5949759f | 4 | |
Jan Jongboom |
0:910f5949759f | 5 | ## HTTP Request API |
Jan Jongboom |
0:910f5949759f | 6 | |
Jan Jongboom |
0:910f5949759f | 7 | ```cpp |
Jan Jongboom |
0:910f5949759f | 8 | NetworkInterface* network = /* obtain a NetworkInterface object */ |
Jan Jongboom |
0:910f5949759f | 9 | |
Jan Jongboom |
0:910f5949759f | 10 | const char body[] = "{\"hello\":\"world\"}"; |
Jan Jongboom |
0:910f5949759f | 11 | |
Jan Jongboom |
5:2456c90f02e9 | 12 | HttpRequest* request = new HttpRequest(network, HTTP_POST, "http://httpbin.org/post"); |
Jan Jongboom |
5:2456c90f02e9 | 13 | request->set_header("Content-Type", "application/json"); |
Jan Jongboom |
5:2456c90f02e9 | 14 | HttpResponse* response = request->send(body, strlen(body)); |
Jan Jongboom |
0:910f5949759f | 15 | // if response is NULL, check response->get_error() |
Jan Jongboom |
0:910f5949759f | 16 | |
Jan Jongboom |
0:910f5949759f | 17 | printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); |
Jan Jongboom |
5:2456c90f02e9 | 18 | printf("body is:\n%s\n", response->get_body_as_string().c_str()); |
Jan Jongboom |
5:2456c90f02e9 | 19 | |
Jan Jongboom |
5:2456c90f02e9 | 20 | delete request; // also clears out the response |
Jan Jongboom |
0:910f5949759f | 21 | ``` |
Jan Jongboom |
0:910f5949759f | 22 | |
Jan Jongboom |
0:910f5949759f | 23 | ## HTTPS Request API |
Jan Jongboom |
0:910f5949759f | 24 | |
Jan Jongboom |
0:910f5949759f | 25 | ```cpp |
Jennifer Plunkett |
30:3ad153a3fdfd | 26 | // pass in the root certificates that you trust, there is no central CA registry in Mbed OS |
Jan Jongboom |
0:910f5949759f | 27 | const char SSL_CA_PEM[] = "-----BEGIN CERTIFICATE-----\n" |
Jan Jongboom |
0:910f5949759f | 28 | /* rest of the CA root certificates */; |
Jan Jongboom |
0:910f5949759f | 29 | |
Jan Jongboom |
0:910f5949759f | 30 | NetworkInterface* network = /* obtain a NetworkInterface object */ |
Jan Jongboom |
0:910f5949759f | 31 | |
Jan Jongboom |
0:910f5949759f | 32 | const char body[] = "{\"hello\":\"world\"}"; |
Jan Jongboom |
0:910f5949759f | 33 | |
Jan Jongboom |
5:2456c90f02e9 | 34 | HttpsRequest* request = new HttpsRequest(network, SSL_CA_PEM, HTTP_GET "https://httpbin.org/status/418"); |
Jan Jongboom |
5:2456c90f02e9 | 35 | HttpResponse* response = request->send(); |
Jan Jongboom |
0:910f5949759f | 36 | // if response is NULL, check response->get_error() |
Jan Jongboom |
0:910f5949759f | 37 | |
Jan Jongboom |
0:910f5949759f | 38 | printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); |
Jan Jongboom |
0:910f5949759f | 39 | printf("body is:\n%s\n", response->get_body().c_str()); |
Jan Jongboom |
5:2456c90f02e9 | 40 | |
Jan Jongboom |
5:2456c90f02e9 | 41 | delete request; |
Jan Jongboom |
0:910f5949759f | 42 | ``` |
Jan Jongboom |
0:910f5949759f | 43 | |
Jan Jongboom |
25:47d5c90c9ceb | 44 | **Note:** You can get the root CA for a domain easily from Firefox. Click on the green padlock, click *More information > Security > View certificate > Details*. Select the top entry in the 'Certificate Hierarchy' and click *Export...*. This gives you a PEM file. Add the content of the PEM file to your root CA list ([here's an image](img/root-ca-selection.png)). |
Jan Jongboom |
25:47d5c90c9ceb | 45 | |
Jan Jongboom |
32:fa4d71265625 | 46 | ### Mbed TLS Entropy configuration |
Jan Jongboom |
32:fa4d71265625 | 47 | |
Jan Jongboom |
32:fa4d71265625 | 48 | If your target does not have a built-in TRNG, or other entropy sources, add the following macros to your `mbed_app.json` file to disable entropy: |
Jan Jongboom |
32:fa4d71265625 | 49 | |
Jan Jongboom |
32:fa4d71265625 | 50 | ```json |
Jan Jongboom |
32:fa4d71265625 | 51 | { |
Jan Jongboom |
32:fa4d71265625 | 52 | "macros": [ |
Jan Jongboom |
32:fa4d71265625 | 53 | "MBEDTLS_TEST_NULL_ENTROPY", |
Jan Jongboom |
32:fa4d71265625 | 54 | "MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES" |
Jan Jongboom |
32:fa4d71265625 | 55 | ] |
Jan Jongboom |
32:fa4d71265625 | 56 | } |
Jan Jongboom |
32:fa4d71265625 | 57 | ``` |
Jan Jongboom |
32:fa4d71265625 | 58 | |
Jan Jongboom |
32:fa4d71265625 | 59 | Note that this is **not** secure, and you should not deploy this device into production with this configuration. |
Jan Jongboom |
32:fa4d71265625 | 60 | |
Jan Jongboom |
19:a5371b71de6f | 61 | ## Memory usage |
Jan Jongboom |
19:a5371b71de6f | 62 | |
Jan Jongboom |
19:a5371b71de6f | 63 | Small requests where the body of the response is cached by the library (like the one found in main-http.cpp), require ~4K of RAM. When the request is finished they require ~1.5K of RAM, depending on the size of the response. This applies both to HTTP and HTTPS. If you need to handle requests that return a large response body, see 'Dealing with large body'. |
Jan Jongboom |
19:a5371b71de6f | 64 | |
Jan Jongboom |
32:fa4d71265625 | 65 | HTTPS requires additional memory: on FRDM-K64F about 50K of heap space (at its peak). This means that you cannot use HTTPS on devices with less than 128K of memory, as you also need to reserve memory for the stack and network interface. |
Jan Jongboom |
19:a5371b71de6f | 66 | |
Jan Jongboom |
24:6c1651fd26b9 | 67 | ### Dealing with large response body |
Jan Jongboom |
0:910f5949759f | 68 | |
Jan Jongboom |
5:2456c90f02e9 | 69 | By default the library will store the full request body on the heap. This works well for small responses, but you'll run out of memory when receiving a large response body. To mitigate this you can pass in a callback as the last argument to the request constructor. This callback will be called whenever a chunk of the body is received. You can set the request chunk size in the `HTTP_RECEIVE_BUFFER_SIZE` macro (see `mbed_lib.json` for the definition) although it also depends on the buffer size of the underlying network connection. |
Jan Jongboom |
0:910f5949759f | 70 | |
Jan Jongboom |
0:910f5949759f | 71 | ```cpp |
Jan Jongboom |
29:383e9bfbfbed | 72 | void body_callback(const char* data, uint32_t data_len) { |
Jan Jongboom |
0:910f5949759f | 73 | // do something with the data |
Jan Jongboom |
0:910f5949759f | 74 | } |
Jan Jongboom |
0:910f5949759f | 75 | |
Jan Jongboom |
22:71fc1b1894f8 | 76 | HttpRequest* req = new HttpRequest(network, HTTP_GET, "http://pathtolargefile.com", &body_callback); |
Jan Jongboom |
22:71fc1b1894f8 | 77 | req->send(NULL, 0); |
Jan Jongboom |
0:910f5949759f | 78 | ``` |
Jan Jongboom |
0:910f5949759f | 79 | |
Jan Jongboom |
24:6c1651fd26b9 | 80 | ### Dealing with a large request body |
Jan Jongboom |
24:6c1651fd26b9 | 81 | |
Jan Jongboom |
24:6c1651fd26b9 | 82 | If you cannot load the full request into memory, you can pass a callback into the `send` function. Through this callback you can feed in chunks of the request body. This is very useful if you want to send files from a file system. |
Jan Jongboom |
24:6c1651fd26b9 | 83 | |
Jan Jongboom |
24:6c1651fd26b9 | 84 | ```cpp |
Jan Jongboom |
29:383e9bfbfbed | 85 | const void * get_chunk(uint32_t* out_size) { |
Jan Jongboom |
24:6c1651fd26b9 | 86 | // set the value of out_size (via *out_size = 10) to the size of the buffer |
Jan Jongboom |
24:6c1651fd26b9 | 87 | // return the buffer |
Jan Jongboom |
24:6c1651fd26b9 | 88 | |
Jan Jongboom |
24:6c1651fd26b9 | 89 | // if you don't have any more data, set *out_size to 0 |
Jan Jongboom |
24:6c1651fd26b9 | 90 | } |
Jan Jongboom |
24:6c1651fd26b9 | 91 | |
Jan Jongboom |
24:6c1651fd26b9 | 92 | HttpRequest* req = new HttpRequest(network, HTTP_POST, "http://my_api.com/upload"); |
Jan Jongboom |
24:6c1651fd26b9 | 93 | req->send(callback(&get_chunk)); |
Jan Jongboom |
24:6c1651fd26b9 | 94 | ``` |
Jan Jongboom |
24:6c1651fd26b9 | 95 | |
Jan Jongboom |
12:530c2ebee349 | 96 | ## Socket re-use |
Jan Jongboom |
12:530c2ebee349 | 97 | |
Jan Jongboom |
12:530c2ebee349 | 98 | By default the library opens a new socket per request. This is wasteful, especially when dealing with TLS requests. You can re-use sockets like this: |
Jan Jongboom |
12:530c2ebee349 | 99 | |
Jan Jongboom |
12:530c2ebee349 | 100 | ### HTTP |
Jan Jongboom |
12:530c2ebee349 | 101 | |
Jan Jongboom |
12:530c2ebee349 | 102 | ```cpp |
Jan Jongboom |
12:530c2ebee349 | 103 | TCPSocket* socket = new TCPSocket(); |
Jan Jongboom |
12:530c2ebee349 | 104 | |
Jan Jongboom |
12:530c2ebee349 | 105 | nsapi_error_t open_result = socket->open(network); |
Jan Jongboom |
12:530c2ebee349 | 106 | // check open_result |
Jan Jongboom |
12:530c2ebee349 | 107 | |
Jan Jongboom |
12:530c2ebee349 | 108 | nsapi_error_t connect_result = socket->connect("httpbin.org", 80); |
Jan Jongboom |
12:530c2ebee349 | 109 | // check connect_result |
Jan Jongboom |
12:530c2ebee349 | 110 | |
Jan Jongboom |
12:530c2ebee349 | 111 | // Pass in `socket`, instead of `network` as first argument |
Jan Jongboom |
12:530c2ebee349 | 112 | HttpRequest* req = new HttpRequest(socket, HTTP_GET, "http://httpbin.org/status/418"); |
Jan Jongboom |
12:530c2ebee349 | 113 | ``` |
Jan Jongboom |
12:530c2ebee349 | 114 | |
Jan Jongboom |
12:530c2ebee349 | 115 | ### HTTPS |
Jan Jongboom |
12:530c2ebee349 | 116 | |
Jan Jongboom |
12:530c2ebee349 | 117 | ```cpp |
Jan Jongboom |
31:b3730a2c4f39 | 118 | TLSSocket* socket = new TLSSocket(); |
Jan Jongboom |
32:fa4d71265625 | 119 | |
Jan Jongboom |
32:fa4d71265625 | 120 | nsapi_error_t r; |
Jan Jongboom |
31:b3730a2c4f39 | 121 | // make sure to check the return values for the calls below (should return NSAPI_ERROR_OK) |
Jan Jongboom |
32:fa4d71265625 | 122 | r = socket->open(network); |
Jan Jongboom |
32:fa4d71265625 | 123 | r = socket->set_root_ca_cert(SSL_CA_PEM); |
Jan Jongboom |
32:fa4d71265625 | 124 | r = socket->connect("httpbin.org", 443); |
Jan Jongboom |
12:530c2ebee349 | 125 | |
Jan Jongboom |
12:530c2ebee349 | 126 | // Pass in `socket`, instead of `network` as first argument, and omit the `SSL_CA_PEM` argument |
Jan Jongboom |
12:530c2ebee349 | 127 | HttpsRequest* get_req = new HttpsRequest(socket, HTTP_GET, "https://httpbin.org/status/418"); |
Jan Jongboom |
12:530c2ebee349 | 128 | ``` |
Jan Jongboom |
12:530c2ebee349 | 129 | |
Jan Jongboom |
31:b3730a2c4f39 | 130 | ## Request logging |
Jan Jongboom |
31:b3730a2c4f39 | 131 | |
Jan Jongboom |
31:b3730a2c4f39 | 132 | To make debugging easier you can log the raw request body that goes over the line. This also works with chunked encoding. |
Jan Jongboom |
31:b3730a2c4f39 | 133 | |
Jan Jongboom |
31:b3730a2c4f39 | 134 | ```cpp |
Jan Jongboom |
31:b3730a2c4f39 | 135 | uint8_t *request_buffer = (uint8_t*)calloc(2048, 1); |
Jan Jongboom |
31:b3730a2c4f39 | 136 | req->set_request_log_buffer(request_buffer, 2048); |
Jan Jongboom |
31:b3730a2c4f39 | 137 | |
Jan Jongboom |
31:b3730a2c4f39 | 138 | // after the request is done: |
Jan Jongboom |
31:b3730a2c4f39 | 139 | printf("\n----- Request buffer -----\n"); |
Jan Jongboom |
31:b3730a2c4f39 | 140 | for (size_t ix = 0; ix < req->get_request_log_buffer_length(); ix++) { |
Jan Jongboom |
31:b3730a2c4f39 | 141 | printf("%02x ", request_buffer[ix]); |
Jan Jongboom |
31:b3730a2c4f39 | 142 | } |
Jan Jongboom |
31:b3730a2c4f39 | 143 | printf("\n"); |
Jan Jongboom |
31:b3730a2c4f39 | 144 | ``` |
Jan Jongboom |
31:b3730a2c4f39 | 145 | |
Jan Jongboom |
33:5b2869cc8934 | 146 | ## Integration tests |
Jan Jongboom |
33:5b2869cc8934 | 147 | |
Jan Jongboom |
33:5b2869cc8934 | 148 | Integration tests are located in the `TESTS` folder and are ran through [Greentea](https://github.com/ARMmbed/greentea). Instructions on how to run the tests are in [http-example](https://os.mbed.com/teams/sandbox/code/http-example/). |
Jan Jongboom |
33:5b2869cc8934 | 149 | |
Jan Jongboom |
33:5b2869cc8934 | 150 | ## Mbed OS 5.10 or lower |
Jan Jongboom |
33:5b2869cc8934 | 151 | |
Jan Jongboom |
33:5b2869cc8934 | 152 | If you want to use this library on Mbed OS 5.10 or lower, you need to add the [TLSSocket](https://github.com/ARMmbed/TLSSocket) library to your project. This library is included in Mbed OS 5.11 and up. |
Jan Jongboom |
33:5b2869cc8934 | 153 | |
Jan Jongboom |
0:910f5949759f | 154 | ## Tested on |
Jan Jongboom |
0:910f5949759f | 155 | |
Jan Jongboom |
0:910f5949759f | 156 | * K64F with Ethernet. |
Jan Jongboom |
0:910f5949759f | 157 | * NUCLEO_F411RE with ESP8266. |
Jan Jongboom |
19:a5371b71de6f | 158 | * ODIN-W2 with WiFi. |
Jan Jongboom |
24:6c1651fd26b9 | 159 | * K64F with Atmel 6LoWPAN shield. |
Jan Jongboom |
33:5b2869cc8934 | 160 | * DISCO-L475VG-IOT01A with WiFi. |
Jan Jongboom |
32:fa4d71265625 | 161 | * [Mbed Simulator](https://github.com/janjongboom/mbed-simulator). |
Jan Jongboom |
24:6c1651fd26b9 | 162 | |
Jan Jongboom |
24:6c1651fd26b9 | 163 | But this should work with any Mbed OS 5 device that implements the `NetworkInterface` API. |