Experimental HTTP and HTTPS library for mbed OS 5
README.md@29:383e9bfbfbed, 2018-08-10 (annotated)
- Committer:
- Jan Jongboom
- Date:
- Fri Aug 10 11:30:37 2018 +0100
- Revision:
- 29:383e9bfbfbed
- Parent:
- 25:47d5c90c9ceb
- Child:
- 30:3ad153a3fdfd
Force usage of uint32_t instead of size_t - required for compilation on 64-bit systems
Who changed what in which revision?
User | Revision | Line number | New contents of line |
---|---|---|---|
Jan Jongboom |
0:910f5949759f | 1 | # HTTP and HTTPS library for mbed OS 5 |
Jan Jongboom |
0:910f5949759f | 2 | |
Jan Jongboom |
0:910f5949759f | 3 | This library is used to make HTTP and HTTPS calls from mbed OS 5 applications. |
Jan Jongboom |
0:910f5949759f | 4 | |
Jan Jongboom |
0:910f5949759f | 5 | ## HTTP Request API |
Jan Jongboom |
0:910f5949759f | 6 | |
Jan Jongboom |
0:910f5949759f | 7 | ```cpp |
Jan Jongboom |
0:910f5949759f | 8 | NetworkInterface* network = /* obtain a NetworkInterface object */ |
Jan Jongboom |
0:910f5949759f | 9 | |
Jan Jongboom |
0:910f5949759f | 10 | const char body[] = "{\"hello\":\"world\"}"; |
Jan Jongboom |
0:910f5949759f | 11 | |
Jan Jongboom |
5:2456c90f02e9 | 12 | HttpRequest* request = new HttpRequest(network, HTTP_POST, "http://httpbin.org/post"); |
Jan Jongboom |
5:2456c90f02e9 | 13 | request->set_header("Content-Type", "application/json"); |
Jan Jongboom |
5:2456c90f02e9 | 14 | HttpResponse* response = request->send(body, strlen(body)); |
Jan Jongboom |
0:910f5949759f | 15 | // if response is NULL, check response->get_error() |
Jan Jongboom |
0:910f5949759f | 16 | |
Jan Jongboom |
0:910f5949759f | 17 | printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); |
Jan Jongboom |
5:2456c90f02e9 | 18 | printf("body is:\n%s\n", response->get_body_as_string().c_str()); |
Jan Jongboom |
5:2456c90f02e9 | 19 | |
Jan Jongboom |
5:2456c90f02e9 | 20 | delete request; // also clears out the response |
Jan Jongboom |
0:910f5949759f | 21 | ``` |
Jan Jongboom |
0:910f5949759f | 22 | |
Jan Jongboom |
0:910f5949759f | 23 | ## HTTPS Request API |
Jan Jongboom |
0:910f5949759f | 24 | |
Jan Jongboom |
0:910f5949759f | 25 | ```cpp |
Jan Jongboom |
0:910f5949759f | 26 | // pass in the root certificates that you trust, there is no central CA registry in mbed OS |
Jan Jongboom |
0:910f5949759f | 27 | const char SSL_CA_PEM[] = "-----BEGIN CERTIFICATE-----\n" |
Jan Jongboom |
0:910f5949759f | 28 | /* rest of the CA root certificates */; |
Jan Jongboom |
0:910f5949759f | 29 | |
Jan Jongboom |
0:910f5949759f | 30 | NetworkInterface* network = /* obtain a NetworkInterface object */ |
Jan Jongboom |
0:910f5949759f | 31 | |
Jan Jongboom |
0:910f5949759f | 32 | const char body[] = "{\"hello\":\"world\"}"; |
Jan Jongboom |
0:910f5949759f | 33 | |
Jan Jongboom |
5:2456c90f02e9 | 34 | HttpsRequest* request = new HttpsRequest(network, SSL_CA_PEM, HTTP_GET "https://httpbin.org/status/418"); |
Jan Jongboom |
5:2456c90f02e9 | 35 | HttpResponse* response = request->send(); |
Jan Jongboom |
0:910f5949759f | 36 | // if response is NULL, check response->get_error() |
Jan Jongboom |
0:910f5949759f | 37 | |
Jan Jongboom |
0:910f5949759f | 38 | printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); |
Jan Jongboom |
0:910f5949759f | 39 | printf("body is:\n%s\n", response->get_body().c_str()); |
Jan Jongboom |
5:2456c90f02e9 | 40 | |
Jan Jongboom |
5:2456c90f02e9 | 41 | delete request; |
Jan Jongboom |
0:910f5949759f | 42 | ``` |
Jan Jongboom |
0:910f5949759f | 43 | |
Jan Jongboom |
25:47d5c90c9ceb | 44 | **Note:** You can get the root CA for a domain easily from Firefox. Click on the green padlock, click *More information > Security > View certificate > Details*. Select the top entry in the 'Certificate Hierarchy' and click *Export...*. This gives you a PEM file. Add the content of the PEM file to your root CA list ([here's an image](img/root-ca-selection.png)). |
Jan Jongboom |
25:47d5c90c9ceb | 45 | |
Jan Jongboom |
19:a5371b71de6f | 46 | ## Memory usage |
Jan Jongboom |
19:a5371b71de6f | 47 | |
Jan Jongboom |
19:a5371b71de6f | 48 | Small requests where the body of the response is cached by the library (like the one found in main-http.cpp), require ~4K of RAM. When the request is finished they require ~1.5K of RAM, depending on the size of the response. This applies both to HTTP and HTTPS. If you need to handle requests that return a large response body, see 'Dealing with large body'. |
Jan Jongboom |
19:a5371b71de6f | 49 | |
Jan Jongboom |
19:a5371b71de6f | 50 | HTTPS requires additional memory. On FRDM-K64F: |
Jan Jongboom |
19:a5371b71de6f | 51 | |
Jan Jongboom |
19:a5371b71de6f | 52 | * TLS handshake requires 53K of heap space. |
Jan Jongboom |
19:a5371b71de6f | 53 | * Keeping TLS socket open requires 43K of heap space. |
Jan Jongboom |
19:a5371b71de6f | 54 | |
Jan Jongboom |
19:a5371b71de6f | 55 | This means that you cannot use HTTPS on devices with less than 128K of memory, as you also need to reserve memory for the stack and network interface. |
Jan Jongboom |
19:a5371b71de6f | 56 | |
Jan Jongboom |
24:6c1651fd26b9 | 57 | ### Dealing with large response body |
Jan Jongboom |
0:910f5949759f | 58 | |
Jan Jongboom |
5:2456c90f02e9 | 59 | By default the library will store the full request body on the heap. This works well for small responses, but you'll run out of memory when receiving a large response body. To mitigate this you can pass in a callback as the last argument to the request constructor. This callback will be called whenever a chunk of the body is received. You can set the request chunk size in the `HTTP_RECEIVE_BUFFER_SIZE` macro (see `mbed_lib.json` for the definition) although it also depends on the buffer size of the underlying network connection. |
Jan Jongboom |
0:910f5949759f | 60 | |
Jan Jongboom |
0:910f5949759f | 61 | ```cpp |
Jan Jongboom |
29:383e9bfbfbed | 62 | void body_callback(const char* data, uint32_t data_len) { |
Jan Jongboom |
0:910f5949759f | 63 | // do something with the data |
Jan Jongboom |
0:910f5949759f | 64 | } |
Jan Jongboom |
0:910f5949759f | 65 | |
Jan Jongboom |
22:71fc1b1894f8 | 66 | HttpRequest* req = new HttpRequest(network, HTTP_GET, "http://pathtolargefile.com", &body_callback); |
Jan Jongboom |
22:71fc1b1894f8 | 67 | req->send(NULL, 0); |
Jan Jongboom |
0:910f5949759f | 68 | ``` |
Jan Jongboom |
0:910f5949759f | 69 | |
Jan Jongboom |
24:6c1651fd26b9 | 70 | ### Dealing with a large request body |
Jan Jongboom |
24:6c1651fd26b9 | 71 | |
Jan Jongboom |
24:6c1651fd26b9 | 72 | If you cannot load the full request into memory, you can pass a callback into the `send` function. Through this callback you can feed in chunks of the request body. This is very useful if you want to send files from a file system. |
Jan Jongboom |
24:6c1651fd26b9 | 73 | |
Jan Jongboom |
24:6c1651fd26b9 | 74 | ```cpp |
Jan Jongboom |
29:383e9bfbfbed | 75 | const void * get_chunk(uint32_t* out_size) { |
Jan Jongboom |
24:6c1651fd26b9 | 76 | // set the value of out_size (via *out_size = 10) to the size of the buffer |
Jan Jongboom |
24:6c1651fd26b9 | 77 | // return the buffer |
Jan Jongboom |
24:6c1651fd26b9 | 78 | |
Jan Jongboom |
24:6c1651fd26b9 | 79 | // if you don't have any more data, set *out_size to 0 |
Jan Jongboom |
24:6c1651fd26b9 | 80 | } |
Jan Jongboom |
24:6c1651fd26b9 | 81 | |
Jan Jongboom |
24:6c1651fd26b9 | 82 | HttpRequest* req = new HttpRequest(network, HTTP_POST, "http://my_api.com/upload"); |
Jan Jongboom |
24:6c1651fd26b9 | 83 | req->send(callback(&get_chunk)); |
Jan Jongboom |
24:6c1651fd26b9 | 84 | ``` |
Jan Jongboom |
24:6c1651fd26b9 | 85 | |
Jan Jongboom |
12:530c2ebee349 | 86 | ## Socket re-use |
Jan Jongboom |
12:530c2ebee349 | 87 | |
Jan Jongboom |
12:530c2ebee349 | 88 | By default the library opens a new socket per request. This is wasteful, especially when dealing with TLS requests. You can re-use sockets like this: |
Jan Jongboom |
12:530c2ebee349 | 89 | |
Jan Jongboom |
12:530c2ebee349 | 90 | ### HTTP |
Jan Jongboom |
12:530c2ebee349 | 91 | |
Jan Jongboom |
12:530c2ebee349 | 92 | ```cpp |
Jan Jongboom |
12:530c2ebee349 | 93 | TCPSocket* socket = new TCPSocket(); |
Jan Jongboom |
12:530c2ebee349 | 94 | |
Jan Jongboom |
12:530c2ebee349 | 95 | nsapi_error_t open_result = socket->open(network); |
Jan Jongboom |
12:530c2ebee349 | 96 | // check open_result |
Jan Jongboom |
12:530c2ebee349 | 97 | |
Jan Jongboom |
12:530c2ebee349 | 98 | nsapi_error_t connect_result = socket->connect("httpbin.org", 80); |
Jan Jongboom |
12:530c2ebee349 | 99 | // check connect_result |
Jan Jongboom |
12:530c2ebee349 | 100 | |
Jan Jongboom |
12:530c2ebee349 | 101 | // Pass in `socket`, instead of `network` as first argument |
Jan Jongboom |
12:530c2ebee349 | 102 | HttpRequest* req = new HttpRequest(socket, HTTP_GET, "http://httpbin.org/status/418"); |
Jan Jongboom |
12:530c2ebee349 | 103 | ``` |
Jan Jongboom |
12:530c2ebee349 | 104 | |
Jan Jongboom |
12:530c2ebee349 | 105 | ### HTTPS |
Jan Jongboom |
12:530c2ebee349 | 106 | |
Jan Jongboom |
12:530c2ebee349 | 107 | ```cpp |
Jan Jongboom |
12:530c2ebee349 | 108 | TLSSocket* socket = new TLSSocket(network, "httpbin.org", 443, SSL_CA_PEM); |
Jan Jongboom |
12:530c2ebee349 | 109 | socket->set_debug(true); |
Jan Jongboom |
12:530c2ebee349 | 110 | if (socket->connect() != 0) { |
Jan Jongboom |
12:530c2ebee349 | 111 | printf("TLS Connect failed %d\n", socket->error()); |
Jan Jongboom |
12:530c2ebee349 | 112 | return 1; |
Jan Jongboom |
12:530c2ebee349 | 113 | } |
Jan Jongboom |
12:530c2ebee349 | 114 | |
Jan Jongboom |
12:530c2ebee349 | 115 | // Pass in `socket`, instead of `network` as first argument, and omit the `SSL_CA_PEM` argument |
Jan Jongboom |
12:530c2ebee349 | 116 | HttpsRequest* get_req = new HttpsRequest(socket, HTTP_GET, "https://httpbin.org/status/418"); |
Jan Jongboom |
12:530c2ebee349 | 117 | ``` |
Jan Jongboom |
12:530c2ebee349 | 118 | |
Jan Jongboom |
0:910f5949759f | 119 | ## Tested on |
Jan Jongboom |
0:910f5949759f | 120 | |
Jan Jongboom |
0:910f5949759f | 121 | * K64F with Ethernet. |
Jan Jongboom |
0:910f5949759f | 122 | * NUCLEO_F411RE with ESP8266. |
Jan Jongboom |
19:a5371b71de6f | 123 | * ODIN-W2 with WiFi. |
Jan Jongboom |
24:6c1651fd26b9 | 124 | * K64F with Atmel 6LoWPAN shield. |
Jan Jongboom |
24:6c1651fd26b9 | 125 | |
Jan Jongboom |
24:6c1651fd26b9 | 126 | But this should work with any Mbed OS 5 device that implements the `NetworkInterface` API. |