8 years, 5 months ago.

Fastest way to stream data to PC ?

Hi everybody,

I am using the mbed for fast (5kHz) measurements. I have an interrupt from an external sensor every 200us. After reading the 32bits data in the interrupt with SPI (2Mhz), I need to send it.

I have the reading and sending in the same callback but the speed limitation by the printf function is a problem since I want to log and plot the data in LabVIEW. With serial (baudrate 115200) printf (USBTX, USBRX), I see my incoming data with an inconsistent delta of 1-3ms.

Is there a better way to stream the data to my PC ?

Thanks!

Nice answer form Andy A. Just be careful not to confuse ASCII output by printf and binary data sent. For example if you send two binary bytes 0xf3 and 0xe0 that is only 2 bytes sent and printf would need 5 bytes to send it since if interpreted as integer it would be "62432". and those are 0x36 0x32 0x34 0x33 0x32 So if sending data as raw binary values you must use PC RS232 terminal such ad Docklight to show that data in raw format. Otherwise you will not be able to read it since in most cases it will not be ASCII readable characters. You can find docklight at https://docklight.de/ but you can also use some other advanced terminal program. Docklight is just my recommendation.

posted by Vladimir Milosavljevic 27 Jan 2017

1 Answer

8 years, 5 months ago.

A couple of tricks you can use -

First use a higher baud rate, you can increase the baud rate by at least a factor of 4. The UARTs in the mbed can handle it, the issue may be getting windows to let you run the port that fast, that comes down to a driver and application issue.

Secondly don't use printf. You know your data is 4 bytes, use 4 putc() commands to send the data byte by byte, that will cut your software overhead by a huge amount.

uint32 data;
char *bytePtr = (char *)&data;
putc(*bytePtr);      // reverse the order of these lines if you have endian issues
putc(*(bytePtr+1));
putc(*(bytePtr+2));
putc(*(bytePtr+3));

Accepted Answer

Thank you for your reply Andy. Regarding the tricks you mentioned, I sure can increase the baudrate but I saw some people having trouble at the maximum speed. That is the main reason I chose a lower rate.

As for chopping the transmission, don't you think this strategy is less efficient since you multiply the sending operation? Of course putc() doesn't have to guess the length and type of the data but anyway, being on an sub triggered by an interrupt, I fear that the total time in that interruption could be a problem.

Additionally, if I were to add a timestamp to the sampled data (with a simple timer for example), how could I modify the code to remain time efficient?

Thanks again

posted by Seight Seight 22 Nov 2015

Printf has to parse the format string, allocate an output buffer and then create the output string which then needs to be written to the serial port. A 32 bit number is potentially 9 digits long so your looking at writing at least that many bytes to the port. If you go over 16 bytes then you'll have to wait for some of the data to have been sent before you can put it all in the transmit buffer. On the other hand putc writes a single byte into the transmit buffer, assuming there is space this is very fast.

In other words the putc method isn't just slightly faster, it's a massive speed improvement.

You could easily add a timestamp, just keep the transmission size to a minimum. If the total message size is over the size of the hardware transmit buffer (16 bytes or less depending on the board) then use something like modserial to buffer the port of you'll end up blocking waiting for the transmit.

And do a sanity check on your baud rate, RS232 sends 10 bits per byte, make sure your message size multiplied by the measurement frequency is less than the baud rate.

posted by Andy A 22 Nov 2015

I really appreciate your insight on that. My main concern regarding chopping the 32 bits is the timing. If the total message needs to be split up then the acquisition loop becomes more complicated then potentially lasts longer.

I took the "bypass" option (LPC_TIM3->TC) in that post https://developer.mbed.org/forum/mbed/topic/2365/ to get the timer count. Is there any equivalent low level operation for the printf issue?

This post https://developer.mbed.org/forum/helloworld/topic/2095/ mentions other ways to stream data to PC, would you recommend a different approach?

Regards,

posted by Seight Seight 23 Nov 2015

The message isn't getting "split up" at all. RS232 sends data 1 byte at a time. The end result of what goes down the wire is identical for

putc('1');
putc('2');
putc('3');
<<//code>>
and
<<code>>
printf("123");

The only difference is how many CPU cycles are taken to send the data. If you split it up manually then at the very least you are saving a while loop which means it will be faster.

The low level version of printf would be to format the data yourself and then write the data into the UART tx buffer one byte at a time. putc() will have a more overhead than writing the buffer directly because it will also check that there is space and enable the transmit if needed but unless you need every last CPU cycle that overhead is worth it in exchange for the simpler code.

The downside to raw binary is there is no clear indication between values. Some sort of delimiter is probably a good idea (e.g. add a 0, 0xff of 0x55 byte at the end of the data) in order to give you a means of getting the two ends in sync. The value could show up in data but over time it would allow the two ends to sync up. A longer delimiter makes it easier to sync but means transmitting more bytes.

Text makes getting a sync between the two trivial but at the cost of a massive increase in message length.

As for other completely different methods, it depends on how much work you want to do and whether you need the data in real time. Generally they will add CPU overhead so it depends if you are bandwidth limited or CPU limited.

posted by Andy A 23 Nov 2015

Thanks again for this clear explanation. It seems that putc() is a good trade-off and I will manage to get it to work the right way.

Initially, I wanted to have the data in real time. The mbed is hooked up with the ADC AD7190 that can go up to 4.8kHz, so there is the bandwidth limit. The mbed then just format the data before sending it as I prefer to log raw data to work on it without losing something.

I got fooled by that real time constraint that can more or less never be respected since the serial link is not a real time priority, thus the timestamp.

posted by Seight Seight 24 Nov 2015