Experimental HTTP and HTTPS library for mbed OS 5
README.md@30:3ad153a3fdfd, 2018-09-06 (annotated)
- Committer:
- Jennifer Plunkett
- Date:
- Thu Sep 06 14:01:09 2018 -0500
- Revision:
- 30:3ad153a3fdfd
- Parent:
- 29:383e9bfbfbed
- Child:
- 32:fa4d71265625
Updated README with Mbed TLS config info
Who changed what in which revision?
User | Revision | Line number | New contents of line |
---|---|---|---|
Jan Jongboom |
0:910f5949759f | 1 | # HTTP and HTTPS library for mbed OS 5 |
Jan Jongboom |
0:910f5949759f | 2 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 3 | This library is used to make HTTP and HTTPS calls from Mbed OS 5 applications. |
Jan Jongboom |
0:910f5949759f | 4 | |
Jan Jongboom |
0:910f5949759f | 5 | ## HTTP Request API |
Jan Jongboom |
0:910f5949759f | 6 | |
Jan Jongboom |
0:910f5949759f | 7 | ```cpp |
Jan Jongboom |
0:910f5949759f | 8 | NetworkInterface* network = /* obtain a NetworkInterface object */ |
Jan Jongboom |
0:910f5949759f | 9 | |
Jan Jongboom |
0:910f5949759f | 10 | const char body[] = "{\"hello\":\"world\"}"; |
Jan Jongboom |
0:910f5949759f | 11 | |
Jan Jongboom |
5:2456c90f02e9 | 12 | HttpRequest* request = new HttpRequest(network, HTTP_POST, "http://httpbin.org/post"); |
Jan Jongboom |
5:2456c90f02e9 | 13 | request->set_header("Content-Type", "application/json"); |
Jan Jongboom |
5:2456c90f02e9 | 14 | HttpResponse* response = request->send(body, strlen(body)); |
Jan Jongboom |
0:910f5949759f | 15 | // if response is NULL, check response->get_error() |
Jan Jongboom |
0:910f5949759f | 16 | |
Jan Jongboom |
0:910f5949759f | 17 | printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); |
Jan Jongboom |
5:2456c90f02e9 | 18 | printf("body is:\n%s\n", response->get_body_as_string().c_str()); |
Jan Jongboom |
5:2456c90f02e9 | 19 | |
Jan Jongboom |
5:2456c90f02e9 | 20 | delete request; // also clears out the response |
Jan Jongboom |
0:910f5949759f | 21 | ``` |
Jan Jongboom |
0:910f5949759f | 22 | |
Jan Jongboom |
0:910f5949759f | 23 | ## HTTPS Request API |
Jan Jongboom |
0:910f5949759f | 24 | |
Jan Jongboom |
0:910f5949759f | 25 | ```cpp |
Jennifer Plunkett |
30:3ad153a3fdfd | 26 | // pass in the root certificates that you trust, there is no central CA registry in Mbed OS |
Jan Jongboom |
0:910f5949759f | 27 | const char SSL_CA_PEM[] = "-----BEGIN CERTIFICATE-----\n" |
Jan Jongboom |
0:910f5949759f | 28 | /* rest of the CA root certificates */; |
Jan Jongboom |
0:910f5949759f | 29 | |
Jan Jongboom |
0:910f5949759f | 30 | NetworkInterface* network = /* obtain a NetworkInterface object */ |
Jan Jongboom |
0:910f5949759f | 31 | |
Jan Jongboom |
0:910f5949759f | 32 | const char body[] = "{\"hello\":\"world\"}"; |
Jan Jongboom |
0:910f5949759f | 33 | |
Jan Jongboom |
5:2456c90f02e9 | 34 | HttpsRequest* request = new HttpsRequest(network, SSL_CA_PEM, HTTP_GET "https://httpbin.org/status/418"); |
Jan Jongboom |
5:2456c90f02e9 | 35 | HttpResponse* response = request->send(); |
Jan Jongboom |
0:910f5949759f | 36 | // if response is NULL, check response->get_error() |
Jan Jongboom |
0:910f5949759f | 37 | |
Jan Jongboom |
0:910f5949759f | 38 | printf("status is %d - %s\n", response->get_status_code(), response->get_status_message()); |
Jan Jongboom |
0:910f5949759f | 39 | printf("body is:\n%s\n", response->get_body().c_str()); |
Jan Jongboom |
5:2456c90f02e9 | 40 | |
Jan Jongboom |
5:2456c90f02e9 | 41 | delete request; |
Jan Jongboom |
0:910f5949759f | 42 | ``` |
Jan Jongboom |
0:910f5949759f | 43 | |
Jan Jongboom |
25:47d5c90c9ceb | 44 | **Note:** You can get the root CA for a domain easily from Firefox. Click on the green padlock, click *More information > Security > View certificate > Details*. Select the top entry in the 'Certificate Hierarchy' and click *Export...*. This gives you a PEM file. Add the content of the PEM file to your root CA list ([here's an image](img/root-ca-selection.png)). |
Jan Jongboom |
25:47d5c90c9ceb | 45 | |
Jan Jongboom |
19:a5371b71de6f | 46 | ## Memory usage |
Jan Jongboom |
19:a5371b71de6f | 47 | |
Jan Jongboom |
19:a5371b71de6f | 48 | Small requests where the body of the response is cached by the library (like the one found in main-http.cpp), require ~4K of RAM. When the request is finished they require ~1.5K of RAM, depending on the size of the response. This applies both to HTTP and HTTPS. If you need to handle requests that return a large response body, see 'Dealing with large body'. |
Jan Jongboom |
19:a5371b71de6f | 49 | |
Jan Jongboom |
19:a5371b71de6f | 50 | HTTPS requires additional memory. On FRDM-K64F: |
Jan Jongboom |
19:a5371b71de6f | 51 | |
Jan Jongboom |
19:a5371b71de6f | 52 | * TLS handshake requires 53K of heap space. |
Jan Jongboom |
19:a5371b71de6f | 53 | * Keeping TLS socket open requires 43K of heap space. |
Jan Jongboom |
19:a5371b71de6f | 54 | |
Jan Jongboom |
19:a5371b71de6f | 55 | This means that you cannot use HTTPS on devices with less than 128K of memory, as you also need to reserve memory for the stack and network interface. |
Jan Jongboom |
19:a5371b71de6f | 56 | |
Jan Jongboom |
24:6c1651fd26b9 | 57 | ### Dealing with large response body |
Jan Jongboom |
0:910f5949759f | 58 | |
Jan Jongboom |
5:2456c90f02e9 | 59 | By default the library will store the full request body on the heap. This works well for small responses, but you'll run out of memory when receiving a large response body. To mitigate this you can pass in a callback as the last argument to the request constructor. This callback will be called whenever a chunk of the body is received. You can set the request chunk size in the `HTTP_RECEIVE_BUFFER_SIZE` macro (see `mbed_lib.json` for the definition) although it also depends on the buffer size of the underlying network connection. |
Jan Jongboom |
0:910f5949759f | 60 | |
Jan Jongboom |
0:910f5949759f | 61 | ```cpp |
Jan Jongboom |
29:383e9bfbfbed | 62 | void body_callback(const char* data, uint32_t data_len) { |
Jan Jongboom |
0:910f5949759f | 63 | // do something with the data |
Jan Jongboom |
0:910f5949759f | 64 | } |
Jan Jongboom |
0:910f5949759f | 65 | |
Jan Jongboom |
22:71fc1b1894f8 | 66 | HttpRequest* req = new HttpRequest(network, HTTP_GET, "http://pathtolargefile.com", &body_callback); |
Jan Jongboom |
22:71fc1b1894f8 | 67 | req->send(NULL, 0); |
Jan Jongboom |
0:910f5949759f | 68 | ``` |
Jan Jongboom |
0:910f5949759f | 69 | |
Jan Jongboom |
24:6c1651fd26b9 | 70 | ### Dealing with a large request body |
Jan Jongboom |
24:6c1651fd26b9 | 71 | |
Jan Jongboom |
24:6c1651fd26b9 | 72 | If you cannot load the full request into memory, you can pass a callback into the `send` function. Through this callback you can feed in chunks of the request body. This is very useful if you want to send files from a file system. |
Jan Jongboom |
24:6c1651fd26b9 | 73 | |
Jan Jongboom |
24:6c1651fd26b9 | 74 | ```cpp |
Jan Jongboom |
29:383e9bfbfbed | 75 | const void * get_chunk(uint32_t* out_size) { |
Jan Jongboom |
24:6c1651fd26b9 | 76 | // set the value of out_size (via *out_size = 10) to the size of the buffer |
Jan Jongboom |
24:6c1651fd26b9 | 77 | // return the buffer |
Jan Jongboom |
24:6c1651fd26b9 | 78 | |
Jan Jongboom |
24:6c1651fd26b9 | 79 | // if you don't have any more data, set *out_size to 0 |
Jan Jongboom |
24:6c1651fd26b9 | 80 | } |
Jan Jongboom |
24:6c1651fd26b9 | 81 | |
Jan Jongboom |
24:6c1651fd26b9 | 82 | HttpRequest* req = new HttpRequest(network, HTTP_POST, "http://my_api.com/upload"); |
Jan Jongboom |
24:6c1651fd26b9 | 83 | req->send(callback(&get_chunk)); |
Jan Jongboom |
24:6c1651fd26b9 | 84 | ``` |
Jan Jongboom |
24:6c1651fd26b9 | 85 | |
Jan Jongboom |
12:530c2ebee349 | 86 | ## Socket re-use |
Jan Jongboom |
12:530c2ebee349 | 87 | |
Jan Jongboom |
12:530c2ebee349 | 88 | By default the library opens a new socket per request. This is wasteful, especially when dealing with TLS requests. You can re-use sockets like this: |
Jan Jongboom |
12:530c2ebee349 | 89 | |
Jan Jongboom |
12:530c2ebee349 | 90 | ### HTTP |
Jan Jongboom |
12:530c2ebee349 | 91 | |
Jan Jongboom |
12:530c2ebee349 | 92 | ```cpp |
Jan Jongboom |
12:530c2ebee349 | 93 | TCPSocket* socket = new TCPSocket(); |
Jan Jongboom |
12:530c2ebee349 | 94 | |
Jan Jongboom |
12:530c2ebee349 | 95 | nsapi_error_t open_result = socket->open(network); |
Jan Jongboom |
12:530c2ebee349 | 96 | // check open_result |
Jan Jongboom |
12:530c2ebee349 | 97 | |
Jan Jongboom |
12:530c2ebee349 | 98 | nsapi_error_t connect_result = socket->connect("httpbin.org", 80); |
Jan Jongboom |
12:530c2ebee349 | 99 | // check connect_result |
Jan Jongboom |
12:530c2ebee349 | 100 | |
Jan Jongboom |
12:530c2ebee349 | 101 | // Pass in `socket`, instead of `network` as first argument |
Jan Jongboom |
12:530c2ebee349 | 102 | HttpRequest* req = new HttpRequest(socket, HTTP_GET, "http://httpbin.org/status/418"); |
Jan Jongboom |
12:530c2ebee349 | 103 | ``` |
Jan Jongboom |
12:530c2ebee349 | 104 | |
Jan Jongboom |
12:530c2ebee349 | 105 | ### HTTPS |
Jan Jongboom |
12:530c2ebee349 | 106 | |
Jan Jongboom |
12:530c2ebee349 | 107 | ```cpp |
Jan Jongboom |
12:530c2ebee349 | 108 | TLSSocket* socket = new TLSSocket(network, "httpbin.org", 443, SSL_CA_PEM); |
Jan Jongboom |
12:530c2ebee349 | 109 | socket->set_debug(true); |
Jan Jongboom |
12:530c2ebee349 | 110 | if (socket->connect() != 0) { |
Jan Jongboom |
12:530c2ebee349 | 111 | printf("TLS Connect failed %d\n", socket->error()); |
Jan Jongboom |
12:530c2ebee349 | 112 | return 1; |
Jan Jongboom |
12:530c2ebee349 | 113 | } |
Jan Jongboom |
12:530c2ebee349 | 114 | |
Jan Jongboom |
12:530c2ebee349 | 115 | // Pass in `socket`, instead of `network` as first argument, and omit the `SSL_CA_PEM` argument |
Jan Jongboom |
12:530c2ebee349 | 116 | HttpsRequest* get_req = new HttpsRequest(socket, HTTP_GET, "https://httpbin.org/status/418"); |
Jan Jongboom |
12:530c2ebee349 | 117 | ``` |
Jan Jongboom |
12:530c2ebee349 | 118 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 119 | **Note:** For HTTPS, if you are using a **K64F**, **K22F**, or **ODIN-W2** target, you will need to include the following `mbedtls_entropy_config.h` file to enable Mbed TLS entropy (placed in the root directory of your application): |
Jennifer Plunkett |
30:3ad153a3fdfd | 120 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 121 | ```cpp |
Jennifer Plunkett |
30:3ad153a3fdfd | 122 | /* Enable entropy for K64F, K22F, ODIN-W2. This means entropy is disabled for all other targets. */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 123 | /* Do **NOT** deploy this code in production on other targets! */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 124 | /* See https://tls.mbed.org/kb/how-to/add-entropy-sources-to-entropy-pool */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 125 | #if defined(TARGET_K64F) || defined(TARGET_K22F) || defined(TARGET_UBLOX_EVK_ODIN_W2) |
Jennifer Plunkett |
30:3ad153a3fdfd | 126 | #undef MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES |
Jennifer Plunkett |
30:3ad153a3fdfd | 127 | #undef MBEDTLS_TEST_NULL_ENTROPY |
Jennifer Plunkett |
30:3ad153a3fdfd | 128 | #endif |
Jennifer Plunkett |
30:3ad153a3fdfd | 129 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 130 | #if !defined(MBEDTLS_ENTROPY_HARDWARE_ALT) && \ |
Jennifer Plunkett |
30:3ad153a3fdfd | 131 | !defined(MBEDTLS_ENTROPY_NV_SEED) && !defined(MBEDTLS_TEST_NULL_ENTROPY) |
Jennifer Plunkett |
30:3ad153a3fdfd | 132 | #error "This hardware does not have an entropy source." |
Jennifer Plunkett |
30:3ad153a3fdfd | 133 | #endif /* !MBEDTLS_ENTROPY_HARDWARE_ALT && !MBEDTLS_ENTROPY_NV_SEED && |
Jennifer Plunkett |
30:3ad153a3fdfd | 134 | * !MBEDTLS_TEST_NULL_ENTROPY */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 135 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 136 | #if !defined(MBEDTLS_SHA1_C) |
Jennifer Plunkett |
30:3ad153a3fdfd | 137 | #define MBEDTLS_SHA1_C |
Jennifer Plunkett |
30:3ad153a3fdfd | 138 | #endif /* !MBEDTLS_SHA1_C */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 139 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 140 | #if !defined(MBEDTLS_RSA_C) |
Jennifer Plunkett |
30:3ad153a3fdfd | 141 | #define MBEDTLS_RSA_C |
Jennifer Plunkett |
30:3ad153a3fdfd | 142 | #endif /* !MBEDTLS_RSA_C */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 143 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 144 | /* |
Jennifer Plunkett |
30:3ad153a3fdfd | 145 | * This value is sufficient for handling 2048 bit RSA keys. |
Jennifer Plunkett |
30:3ad153a3fdfd | 146 | * |
Jennifer Plunkett |
30:3ad153a3fdfd | 147 | * Set this value higher to enable handling larger keys, but be aware that this |
Jennifer Plunkett |
30:3ad153a3fdfd | 148 | * will increase the stack usage. |
Jennifer Plunkett |
30:3ad153a3fdfd | 149 | */ |
Jennifer Plunkett |
30:3ad153a3fdfd | 150 | #define MBEDTLS_MPI_MAX_SIZE 1024 |
Jennifer Plunkett |
30:3ad153a3fdfd | 151 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 152 | #define MBEDTLS_MPI_WINDOW_SIZE 1 |
Jennifer Plunkett |
30:3ad153a3fdfd | 153 | ``` |
Jennifer Plunkett |
30:3ad153a3fdfd | 154 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 155 | You will also need to enable a custom user config file macro in an `mbed_app.json` file (placed in the root of your application): |
Jennifer Plunkett |
30:3ad153a3fdfd | 156 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 157 | ```json |
Jennifer Plunkett |
30:3ad153a3fdfd | 158 | { |
Jennifer Plunkett |
30:3ad153a3fdfd | 159 | "macros": ["MBEDTLS_USER_CONFIG_FILE=\"mbedtls_entropy_config.h\"", |
Jennifer Plunkett |
30:3ad153a3fdfd | 160 | "MBEDTLS_TEST_NULL_ENTROPY", "MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES" ] |
Jennifer Plunkett |
30:3ad153a3fdfd | 161 | } |
Jennifer Plunkett |
30:3ad153a3fdfd | 162 | ``` |
Jennifer Plunkett |
30:3ad153a3fdfd | 163 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 164 | You will also need to include an `mbedtls_config.h` file in the root directory of your application: [`mbedtls_config.h`](https://os.mbed.com/teams/sandbox/code/http-example/file/5ad8f931e4ff/mbedtls_config.h) |
Jennifer Plunkett |
30:3ad153a3fdfd | 165 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 166 | For all other targets, you will need the following macro present in an `mbed_app.json` file (placed in the root of your application): |
Jennifer Plunkett |
30:3ad153a3fdfd | 167 | |
Jennifer Plunkett |
30:3ad153a3fdfd | 168 | ```json |
Jennifer Plunkett |
30:3ad153a3fdfd | 169 | { |
Jennifer Plunkett |
30:3ad153a3fdfd | 170 | "macros": ["MBEDTLS_TEST_NULL_ENTROPY", "MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES" ] |
Jennifer Plunkett |
30:3ad153a3fdfd | 171 | } |
Jennifer Plunkett |
30:3ad153a3fdfd | 172 | ``` |
Jennifer Plunkett |
30:3ad153a3fdfd | 173 | |
Jan Jongboom |
0:910f5949759f | 174 | ## Tested on |
Jan Jongboom |
0:910f5949759f | 175 | |
Jan Jongboom |
0:910f5949759f | 176 | * K64F with Ethernet. |
Jan Jongboom |
0:910f5949759f | 177 | * NUCLEO_F411RE with ESP8266. |
Jan Jongboom |
19:a5371b71de6f | 178 | * ODIN-W2 with WiFi. |
Jan Jongboom |
24:6c1651fd26b9 | 179 | * K64F with Atmel 6LoWPAN shield. |
Jan Jongboom |
24:6c1651fd26b9 | 180 | |
Jan Jongboom |
24:6c1651fd26b9 | 181 | But this should work with any Mbed OS 5 device that implements the `NetworkInterface` API. |