HTTPClient get with small response has to time out before it returns

08 Mar 2014

Hi,

My code uses HTTPClient (actually this fork: http://mbed.org/users/ollie8/code/HTTPClient/) to read a value from a very small file on a web server. The call:

HTTPResult r = http.get( "http://192.168.1.245/mbed/VERSION.TXT", ver, 10 );

doesn't return until the timeout occurs.

I believe it is caused by this piece of code in HTTPClient.cpp:

Receive response DBG("Receiving response"); ret = recv(buf, CHUNK_SIZE - 1, CHUNK_SIZE - 1, &trfLen); Read n bytes

which tells recv to return a minimum of 255 bytes, and a maximum of 255 bytes. If I change that last line to:

ret = recv(buf, 1, CHUNK_SIZE - 1, &trfLen); Read n bytes

then the file contents is retrieved much quicker, without waiting for the timeout.

Really, it should keep receiving bytes until the whole header has been received...

Rob.

09 Mar 2014

Hi Rob, I have not had a similar experience, but it might be that you misread the API. The timeout value is documented as milliseconds, not seconds. So, you should have almost constantly timed out.

Since I was first trying to replicate your failure mode (and couldn't), I didn't evaluate your proposed changes.

But I did run a few tests with different timeout parameters, and as you see below, this is a simple modification of the HTTPClient test application.

One more disclaimer - I'm using my own derivative of HTTPClient, which took a different path from Donatien Garnier.

#include "mbed.h"
#include "HTTPClient.h"
#include "EthernetInterface.h"

Serial pc(USBTX, USBRX);

EthernetInterface eth;
HTTPClient http;
Timer timer;
char message[2000];

DigitalOut myled(LED1);

int main() {
    pc.baud(460800);
    printf("HTTP Client - Build " __DATE__ " - " __TIME__ "\r\n");
    eth.init();
    eth.connect();

    while(1) {
        myled = 1;
        timer.start();
        int ret = http.get("http://mbed.org/media/uploads/mbed_official/hello.txt", message, sizeof(message), 200);
        int elapsed = timer.read_ms();
        if (!ret) {
            printf("Success - read %d characters in %d msec.\r\n", strlen(message), elapsed);
            printf("%s\r\n", message);
        } else {
            printf("%d msec timeout in awaiting response.\r\n", elapsed);
        }
        myled = 0;
        wait(5);
    }
    // eth.disconnect();
}

I changed the timeout value as you see in this table. Naturally, the response time across the internet is variable, and I only monitored for results for a 5 or 10 cycles on each test.

timeoutactualactualNotes
(ms)min (ms)max (ms)=================================================
15000185245"Hello world!"
200189295almost all "Hello world!" and very rarely "timeout awaiting response."
190225310mostly "Hello world!" and rarely "timeout awaiting response."
150225279sometimes "Hello world!" and more often "timeout awaiting response."
100175238"timeout awaiting response."
1086104"timeout awaiting response."
17686"timeout awaiting response."
075134"timeout awaiting response."
09 Mar 2014

Hi David,

Thanks for looking into it.

What you can't see from the one line of code I quoted is that "ver" is a char*, and therefore 10 is the count of maximum bytes to read, not the HTTP timeout value. Apologies - I should have posted a complete test case.

The entire response that comes from the webserver (a Readynas Duo, in this case) is less than 255 bytes, and the server keeps the connection open (Keepalive), the combination of which is what triggers the problem, I think. The HTTPClient::recv method does a blocking read for 255 bytes, and because the response is smaller than that, it does not return until the timeout (which is 15000ms=15 seconds by default).

The webserver for http://mbed.org/media/uploads/mbed_official/hello.txt returns more than 255 characters of HTTP headers, so it won't trigger the problem.

Rob.

10 Mar 2014

Ah, yes I misread for sure, and focused on the wrong aspect completely. My window for testing closed for now - hopefully opening again later in the week.