slow code

01 Feb 2011

Hi,

I have wrote this program:

main.cpp

#include "mbed.h"
#include "EthernetNetIf.h"
#include "HTTPClient.h"

EthernetNetIf eth;
HTTPClient http;

DigitalOut led1(LED1);
DigitalOut led2(LED2);
DigitalOut led3(LED3);
DigitalOut led4(LED4);

int main() {

    printf("Setting up...\r\n");
    led1 = true; //state 1
    
    EthernetErr ethErr = eth.setup();
    if (ethErr) {
        printf("Error %d in setup.\r\n", ethErr);
        return -1;
    }
    printf("Setup OK\r\n");
    led2 = true; //state 2

    HTTPText get_data;
    string data;
    int V1, V2, V3, V4;
    HTTPResult r;
    led3 = true; //state 3

    while (1) {
        r = http.get("http://10.0.0.151:8080/data.txt", &get_data);
        if (r==HTTP_OK) {
            data = get_data.get();
            V1 = atoi(data.substr(data.find("rr")+2,2).c_str());
            V2 = atoi(data.substr(data.find("rl")+2,2).c_str());
            V3 = atoi(data.substr(data.find("ch")+2,2).c_str());
            V4 = atoi(data.substr(data.find("cv")+2,2).c_str());
            printf("test: [%i], [%i] , [%i] , [%i] \r\n", V1, V2, V3, V4);
            led3 = !led3;
            led4 = !led4;
            //when led 3&4 change, the MBED has checked the values again.
        } else {
            printf("Error %d\r\n", r);
        }
    }
}

What this program does, is checking all the time 4 numbers in a file (data.txt) on my server (10.0.0.151:8080). This works excellent, this script does about two checks per second. (I've got two printf-results per second)

Is there any way to do is faster?? (It would be great if its something like 20 per second, but I'm happy if it's something like 10 per second)

01 Feb 2011

Which part of the code is slow? I would suspect that it is the http.get, since you're doing it wrong. When you look at the examples, you will see that it works asynchronously, so I think it depends on a luck of timing to work properly.

01 Feb 2011

Hendrik Lipka wrote:

When you look at the examples, you will see that it works asynchronously, so I think it depends on a luck of timing to work properly.

To be more precise: you need to wait for the http.get to execute, and can read the result only afterwards. You do this immediately, so this works only if the get was fast enough.

You can try to use my TCPLineStream class to execute the HTTP request manually, which might give you a round trip time of about 100ms (see the SimpleWebService class for an example). But in the end it depends on how fast your server can send the reply (I have tested with a google web service: DNS takes about 100-200ms, and the request about 300ms to get the first packet back).

01 Feb 2011

OK, thank you for your quick reply. I'll take a look at these codes.

01 Feb 2011

I think I don't understand all these codes enough...

Maybe you can help me with this: I want to visit "http://10.0.0.151:8080/data.txt" (this could be something different) and save the content of that file into a string, so I can do something whit that string. This must be repeat so fast as possible. (So I've got an update of that file every time) That httpclient-example from me is perfect, except it is to slow.

I hope you can help me, because this all is (very) difficult to me.

01 Feb 2011

I must correct myself. After reading the documentation again I saw that you used the correct (read: blocking) version of http.get, which means your code works properly (you should have also seen an error message otherwise).

This also means that your HTTP request just takes 500 milliseconds to complete - and the mbed cannot change this.

01 Feb 2011

Hendrik Lipka wrote:

and the mbed cannot change this

In other words: faster is impossible?

01 Feb 2011

Maybe with some tricks. First task would be to find out what really takes the time.

Maybe you can do some comparisions how fast a PC would take for the same task (you can use wget or curl to retrieve the file).

If it is much faster then the mbed, the server isn't the problem - then you need to find out which part of the HTTPClient take so long.

If the server is too slow, you would need to retrieve multiple files in parallel, which would need some experimentation to get it run.

02 Feb 2011

The server isn't the problem. The server (a program, UsbWebserver) is fast enough and the file wich I try to open is 16 bytes, so that is very small.

I'm afraid I don't know enough about C++/MBED, so I think I cannot find out wich part of the HTTPClient library is slow... I'll try, but maybe you have a simple alternative?

02 Feb 2011

if you set a pin high before you call GET and clear it afterwards you can measure the time spent fetching the packet. Your server may be pretty quick but you seem to be forgetting there's a whole load of network transport between you and it. Network programming is generally considered "slow" when compared against CPU speeds.

02 Feb 2011

How do you know the web server is fast enough? How did you measure this? I have made tests with google server, and you easily get round trip times of several 100ms or some more, so I would consider even 500ms from sending your request to receiving the reply as fast.

If you happen to have access to the machine running your webserver (which, I assume, is a windows box), you can try to install wireshark and have a look at the network traffic. With that you can see how long the time between establishing the connection (the first SYN packet) and sending the complete response (the last FIN&ACK packet) really is.

You also can add a Timer object to your main method, and read the time stamp at several point in the program, and print them. (But don't forget, printf takes a considerable amount of time because of the serial connection!).

I'm using HTTP extensively in my project, and I don't think that your code is slow.

02 Feb 2011

Hendrik Lipka wrote:

How do you know the web server is fast enough

If I wrote an JavaScript script (with jquery) I can do a HTTP-request every 40 ms using the JavaScript-function setInterval().

Is there a function in C++ that is doing the same as the JS-function setInterval()??

02 Feb 2011

Olle Kelderman wrote:

If I wrote an JavaScript script (with jquery) I can do a HTTP-request every 40 ms using the JavaScript-function setInterval().

This means you try to retrieve the web page every 40ms, but it doesn't necessarily mean that the retrieval is finished in that time. It might just mean that you have multiple requests in parallel, which in the end looks like you have a new result every 40ms, but it is just a little bit delayed.

Olle Kelderman wrote:

Is there a function in C++ that is doing the same as the JS-function setInterval()??

Yes, it is called a Ticker.

You still should try to time your execution, for example like this (not tested, and I omitted the header stuff):

    Timer tim;
    tim.reset();
    tim.start();
    HTTPText get_data;
    string data;
    int V1, V2, V3, V4;
    HTTPResult r;

    while (1) {
        int time1=tim.read_ms();
        r = http.get("http://10.0.0.151:8080/data.txt", &get_data);
        int time2=tim.read_ms();
        if (r==HTTP_OK) {
            data = get_data.get();
            int time3=tim.read_ms();
            V1 = atoi(data.substr(data.find("rr")+2,2).c_str());
            V2 = atoi(data.substr(data.find("rl")+2,2).c_str());
            V3 = atoi(data.substr(data.find("ch")+2,2).c_str());
            V4 = atoi(data.substr(data.find("cv")+2,2).c_str());
            int time4=tim.read_ms();
            printf("test: [%i], [%i] , [%i] , [%i] \r\n", V1, V2, V3, V4);
            int time5=tim.read_ms();
            printf("times: request=%i / read=%i / calc=%i / print=%i",(time2-time1),(time3-time2),(time4-time3),(time5-time4));
        } else {
            printf("Error %d\r\n", r);
        }
    }
02 Feb 2011

Quote:

If I wrote an JavaScript script (with jquery) I can do a HTTP-request every 40 ms using the JavaScript-function setInterval().

Why not just call the page in the browser in the normal way and using Firebug's "Net" tab option it shows you the retreival time for the page. Then you'll know how long it takes Firefox to get the page.

/media/uploads/AjK/slow-get.jpg

[Edit: I should add you need to do this on a computer that's not the web server assuming is on your PC is running the USBWebserver. Otherwise you are not testing the infrastructure. Your browser has an unfair advantage if it's only getting from an Apache on localhost]

02 Feb 2011

Hendrik Lipka wrote:

This means you try to retrieve the web page every 40ms, but it doesn't necessarily mean that the retrieval is finished in that time. It might just mean that you have multiple requests in parallel, which in the end looks like you have a new result every 40ms, but it is just a little bit delayed.

OK, I understand. Thanks for the lesson. That was one thing I didn't know.

Your code works perfect. This is de output:

test: [0], [0] , [0] , [0]
times: request=3261 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=417 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=417 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=465 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=377 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=419 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=419 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=424 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=415 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=425 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=414 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=418 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=421 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=421 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=419 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=421 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=417 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=1 / print=28
test: [0], [0] , [0] , [0]
times: request=418 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=3419 / read=0 / calc=1 / print=28
test: [0], [0] , [0] , [0]
times: request=419 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=420 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=419 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=422 / read=0 / calc=0 / print=29
test: [0], [0] , [0] , [0]
times: request=417 / read=0 / calc=0 / print=28
test: [0], [0] , [0] , [0]
times: request=421 / read=0 / calc=1 / print=28

etc.

You can see that de request time is something about 420 ms, but the first request is more than 3 seconds. Here is it the first request, but sometimes (not often) it is somewhere in de middle at an random time....

I'll take al look at that Ticker and try something. I hope I can do multiple requests in parallel, just like that JavaScript example.

02 Feb 2011

The ticker works perfect, but it hasn't the result I hoped for. My control-LED change every 500 ms, so it does the same as my first code. My code at this time:

main.cpp

#include "mbed.h"
#include "EthernetNetIf.h"
#include "HTTPClient.h"
#include <string>

EthernetNetIf eth;
HTTPClient http;
Ticker request_handler;
HTTPText get_data;
string data;
int V1, V2, V3, V4;
HTTPResult r;
Timer tim;
int count = 0;

DigitalOut led1(LED1);
DigitalOut led2(LED2);
DigitalOut led3(LED3);
DigitalOut led4(LED4);


void do_request() {
    int time1=tim.read_ms();
    r = http.get("http://10.0.0.151:8080/data.txt", &get_data);
    int time2=tim.read_ms();
    if (r==HTTP_OK) {
        data = get_data.get();
        int time3=tim.read_ms();
        V1 = atoi(data.substr(data.find("rr")+2,2).c_str());
        V2 = atoi(data.substr(data.find("rl")+2,2).c_str());
        V3 = atoi(data.substr(data.find("ch")+2,2).c_str());
        V4 = atoi(data.substr(data.find("cv")+2,2).c_str());
        int time4=tim.read_ms();
        printf("test%i: [%i], [%i] , [%i] , [%i] \r\n", count++, V1, V2, V3, V4);
        int time5=tim.read_ms();
        printf("times: request=%i / read=%i / calc=%i / print=%i \r\n",(time2-time1),(time3-time2),(time4-time3),(time5-time4));
        led3 = !led3;
        led4 = !led4;
        // led3 and led4 are the control-LEDs. When they change, a new request is handled.
    } else {
        printf("Error %d\r\n", r);
    }
}

int main() {
    printf("Setting up...\r\n");
    led1 = true; //state 1

    EthernetErr ethErr = eth.setup();
    if (ethErr) {
        printf("Error %d in setup.\r\n", ethErr);
        return -1;
    }
    printf("Setup OK\r\n");
    led2 = true; //state 2

    tim.reset();
    tim.start();
    led3 = true; //state 3

    request_handler.attach(&do_request, 0.04);
}
02 Feb 2011

Olle Kelderman wrote:

The ticker works perfect, but it hasn't the result I hoped for. My control-LED change every 500 ms, so it does the same as my first code.

I did not mention the most important thing - the Ticker is not multi-threaded. If the called function takes longer then the specified time period, the next call is delayed. This is what you are seeing, and is the difference to setInterval().

02 Feb 2011

So there is no way to do multiple requests in parallel?

02 Feb 2011

Like I said - not without some tricks. The networking stack as precompiled supports only 3 or 4 parallel TCP sockets. Also, since the mbed doesn't provide a RTOS or some direct form of multi-threading, you need to do some things by yourself:

  • modifying the NetServices library configuration to allow more parallel sockets
  • open multiple sockets and execute requests with them (e.g. 5 sockets)
  • do something like a round-robin between these sockets
    • in the first round, open all the sockets and send asynchronous requests
    • after that, read the response from each of the sockets
    • and then you start again

Another option might be doing all the HTTP stuff by yourself, and do HTTP pipelining. As a basis, you can look at my SimpleWebService library, which does the simplest HTTP request possible (the notebook page for TcpLineStream shows the HTTP part only. The trick would then be to use only a single socket, send a request every 20 milliseconds and parse the results as they come in.

Can you tell us what you want to achieve? There might be other ways to get what you need - when you want to have continuous stream of data a simple TCP stream might be simpler...

02 Feb 2011

The content of http://10.0.0.151:8080/data.txt looks always like "rr##,rl##,ch##,cv##". Each '##' represents a number. What I want is four integers with the value of each number. That file changes all the time (by a website that runs on that webserver) and I need a constant update of those four numbers.

That 4 numbers represents the speed 4 little motors (witch I still haven't connected to the MBED). That's my next problem: making the motors working... (I have no idea witch pins I have to use..., PwmOut???)

EDIT: Maybe I have to tell that the first 2 numbers (rr and rl) can have a value from -6 to 6 (6=full speed forward) and de last 2 numbers (ch and cv) from -1 to 1 (1=full speed forward).

02 Feb 2011

So you want to control these four motors from a web page? Why not doing this by having the mbed running as web server?

02 Feb 2011

Hendrik Lipka wrote:

Why not doing this by having the mbed running as web server?

I have thinking about this, but I don't know how to do that. I'm a beginner and I have to finish this very soon...

One question: how can I run PHP-scripts on the MBED. It's not a simple site! There are a lot of JavaScript, AJAX, PHP and more like that in my site, that makes that 4 numbers.

02 Feb 2011

There is an HTTPServer example, but this only serves static pages - so it would need some serious enhancements to handle at least post requests.

No, the mbed cannot run PHP, it is just not equipped for that.

My best guess for what you want to achieve is to do some additional changes to your server application. You can run the mbed as client to an additional TCP server on your HTTP server, where you can exchange just the data you need, without all the HTTP stuff going on. Or you can, even more simpler, send UDP packets to the mbed with the raw numbers in them - this sould be possible at an even higher rate.

02 Feb 2011

On a similar line to "reversing the flow" (previously mentioned as making the Mbed the web server) PHP can establish out going connections. Maybe that may be more use, esp if you can pump UDP sockets out (do you need TCP reliability?). That may be an alternative.

05 Feb 2011

I found this program:

Import programEthernetTester

See http://mbed.org/users/no2chem/notebook/ethernet-testing/ For usage information.

and I think that its possible to do what I want with this. Every time when you visit the "mbed test page" I see in my terminal-program a lot of request-info and also the query-string, so I can do something with that. I'll try something and you'll hear from me whether it works or not.

08 Feb 2011

It works!! Setting up a web-server (what that program is actually doing) takes a few seconds, but when it is complete, the mbed react immediately when I visit the webpage of the mbed. So I can do now requests directly to the mbed and it works much faster than my first code. Thanks a lot to everyone who helped me here.

It's now time for me to solve my second problem: how to control my motors with the mbed...