Sensor data (AnalogIn) to PC, via Ethernet. (fast, low latency)?

23 Mar 2012

Hello,

/* I started with mbed a few weeks ago, so there is still a lot to learn. Sadly, time is of the essence (thesis...), and there is not much tp spare for try and error.

  • /

For an experiment setup, I want to get sensor data (voltage on 1 Pin AnalogIn) to a server via Ethernet.

What I hope to accomplish: About 100-1000 sensor values transmitted per second, with latency as low as possible (under 5ms). The order the values is important, on the server side.

What general approach would you recommend?

My thoughts: TCPSocket on mbed and on server, mbed sends one packet per sensor value - the amount of packets could be too hight, right?

What about UDP? Or, no packets at all, and some kind of Stream? Is that easy to implement, in that case?

(Programming language on server side is Objective-C with Cocoa) Local Ethernet is Gigabit, fast, reliable. Ping to UDPSocket on mbed works with 0.1 ms response time.

24 Mar 2012

are the server and mbed on the same broadcast domain (i.e VLAN)? did you write your own server software, i.e do you have complete control to the server's Ethernet port?

you can get away with just layer 2 communication and scratch IP altogether.

Just build your Ethernet frame and pack your data inside the frame and have the server process it using layer 2 addressing only. This would allow you the whole bandwidth of the mbed's ethernet hardware which I believe is close to 90mbps

24 Mar 2012

try this and scope it on wireshark

#include "mbed.h"

Ethernet eth;
char eth_txs_buffer[1536];
uint16_t packet_length;

int main(void)
{
    uint16_t count;

    //set destination MAC, your server's MAC address or broadcast address, example shows broadcast
    eth_txs_buffer[0]=0xFF;
    eth_txs_buffer[1]=0xFF;
    eth_txs_buffer[2]=0xFF;
    eth_txs_buffer[3]=0xFF;
    eth_txs_buffer[4]=0xFF;
    eth_txs_buffer[5]=0xFF;

    //set source MAC, the mbed's MAC address (use your own)
    eth_txs_buffer[6]=0x35;
    eth_txs_buffer[7]=0x5F;
    eth_txs_buffer[8]=0x22;
    eth_txs_buffer[9]=0xAB;
    eth_txs_buffer[10]=0xA3;
    eth_txs_buffer[11]=0x55;

    //set ethertype, just use IPv4
    eth_txs_buffer[12]=0x08;
    eth_txs_buffer[13]=0x00;

    //fill your payload, example shows 100 bytes of the letter G, this is where you put your data to be sent to server
    for(count=0; count<100; count++)
       eth_txs_buffer[14+count]='G';

    //set frame length, 100 for message + 14 for ethernet header
    packet_length=100+14;

    //send it
    while(1){    
         //send packet
         eth.write(eth_txs_buffer, (int) packet_length);
         eth.send();
         
         //you can change the wait depending on how much frame you want to send, you don't want to flood your network
         wait(0.1);
         }

return 0;
}

I did not test it but I have a different program that uses the same concept

24 Mar 2012

Herson Bagay wrote:

are the server and mbed on the same broadcast domain (i.e VLAN)? did you write your own server software, i.e do you have complete control to the server's Ethernet port?

you can get away with just layer 2 communication and scratch IP altogether.

Just build your Ethernet frame and pack your data inside the frame and have the server process it using layer 2 addressing only. This would allow you the whole bandwidth of the mbed's ethernet hardware which I believe is close to 90mbps

Yes, same broadcast domain, just 1 switch, 2 ports used. And yes, I have complete control of the server side.

Herson Bagay wrote:

try this and scope it on wireshark

I did not test it but I have a different program that uses the same concept

Oh, looks nice. I will test that today.

Thanks for your fast replies! Wow :-)

Working without IP is something I did not do before... but heck, what can go wrong ;-).

24 Mar 2012

just try to use the actual server MAC though because there's a tendency that it can crash other devices in your network if you are going to broadcast layer 2 data at 90Mbps

24 Mar 2012

Ok, tried the code you suggested, changed the MAC adresses to the servers and the mbeds, but something is wrong: My code segment reads:

// for destination:
    eth_txs_buffer[0]=0x00;
    eth_txs_buffer[1]=0x25;
    eth_txs_buffer[2]=0x00;
    eth_txs_buffer[3]=0xF2;
    eth_txs_buffer[4]=0xA6;
    eth_txs_buffer[5]=0x8D;

For server MAC 00:25:00:F2:A6:8D. With that, wireshark sees incoming data every 5 seconds (I set the delay to 5 seconds).

But even if I change the destination MAC, wireshark sees incoming data from the mbed. That should not be, right? Wireshark was set to listen in promiscuous mode, but data with a non-broadcast MAC should be visibile only to the right destination network-adapter, in a switched network, or am I wrong here?

24 Mar 2012

well, if you changed the destination mac to an unknown MAC address (it doesn't exist in your network), your switch will keep flooding the broadcast domain until it learn which port that destination MAC is located. so if you use a non existent MAC address, your switch is never going to learn that MAC since no device will be transmitting with that MAC

24 Mar 2012

Yep. Just checked on another machine. With an existing "wrong" MAC, only the "wrong" machine sees the data, otherwise everything is fine.

Last question up to this point: Can sending out 1000 frames/second (non broadcasting!) still cause any trouble? The payload in the frames will be really small, 3 chars each time.

Is that safe to run at our office network?

24 Mar 2012

as long as it is not flooded, your server and your office switch should be able to handle that much traffic. by the way, the minimum Ethernet frame size is 64 bytes but I guess the mbed hardware will pad it so no worries on those.

24 Mar 2012

I am not able to create packets that wireshark does not mark as "bogus ip header length" or "malformed packet". I got it working that the packets are recognized as "Ethernet II", which should be correct, but can not recreate that. Spend 6 hours on that, very frustrating :-/

I will try using TCP, I guess... besides, the next problem would be, that it does not look promising getting Layer 2 data into my Objective C project... :-/

25 Mar 2012

You cannot because you did not include any IP header after the Ethertype bytes. You can re-create your own IP header, take a look at the frame format of an IP packet.

http://en.wikipedia.org/wiki/IPv4#Packet_structure

It's just a matter of populating the eth_txs_buffer with the proper bytes to construct a valid IP header. Then after that, you can build your own UDP/TCP packet inside the IP PDU.

void generateIPv4header(void)
{
    uint8_t counter=14;
    uint8_t i=0;
    uint16_t temp=0;
    uint32_t checksum=0;
    
    eth_txs_buffer[counter++]=xx;                //version+header length
    eth_txs_buffer[counter++]=xx;                //DSCP
    eth_txs_buffer[counter++]=xx;                //L3 length0
    eth_txs_buffer[counter++]=xx;                //L3 length1
    eth_txs_buffer[counter++]=xx;                //ID0
    eth_txs_buffer[counter++]=xx;                //ID1
    eth_txs_buffer[counter++]=xx;                //flags
    eth_txs_buffer[counter++]=xx;                //offset
    eth_txs_buffer[counter++]=xx;                //TTL
    eth_txs_buffer[counter++]=xx;                //protocol
    
    //header checksum
    eth_txs_buffer[counter++]=0x00;                                   //CHKSUM0
    eth_txs_buffer[counter++]=0x00;                                   //CHKSUM1
    
    //source IP
    eth_txs_buffer[counter++]=myip[0];
    eth_txs_buffer[counter++]=myip[1];
    eth_txs_buffer[counter++]=myip[2];
    eth_txs_buffer[counter++]=myip[3];
    
    //destination IP
    eth_txs_buffer[counter++]=destip[0];
    eth_txs_buffer[counter++]=destip[1];
    eth_txs_buffer[counter++]=destip[2];
    eth_txs_buffer[counter++]=destip[3];
    
    //compute header checksum
    counter=14;
    for(i=0; i<10; i++)
    {
        temp=0;
        temp=(uint16_t) eth_txs_buffer[counter];
        temp=(temp<<8) | (uint16_t) eth_txs_buffer[counter+1];
        checksum+=(uint32_t) temp;
        counter+=2;
    }
    temp=0xFFFF - ((checksum & 0x000000FF) + ((checksum & 0x00FF0000)>>16) + (checksum & 0x0000FF00));
    
    //update checksum field
    eth_txs_buffer[24]=temp>>8;                                   //CHKSUM0
    eth_txs_buffer[25]=(uint8_t) temp & 0x00FF;                   //CHKSUM1
}
25 Mar 2012

Herson Bagay, you are a huge help in very frustrating work hours. Thanks, I will look into that again, this afternoon.

Thank you very much.

25 Mar 2012

Ok, update for people in my situation, what I figured out by now: Sending "raw" frames constructed by hand is not necessary for the performance I need. But it was really worth the look into all that basic groundwork. Should have done that years ago. Thanks again, Herson!

I got a "clean" UDP solution (below) now:

- Performance (cpu and interface load) seems to be fine. 2000 packets/second work fine, wireshark sees almost perfect timestamps! Even with bigger payloads. (If someone is interested, i can do a table with some tests...)

Solution:

MBED Side: - EthernetNetIF and UDPSocket - set up UDPSocket called udp - loop in main(), which sends via udp.sendto(...) ...done!

Objective-C/Cocoa Side: - CocoaAsyncSocket (to be exact: GCDAsyncUdpSocket) - ...download CocoaAsyncSocket from GitHub, and see the UdpEchoServer example

I will clean up the mbed-code that works for me, and publish that later :)

14 Mar 2014

hello Mr Herson Baga what you done is exactelly what i need and i will be thankful if you give an example of sending an UDP paket without use the UDPSoket just i want to define all by me ( all the fields)