Best way to send/store data to a computer

04 Apr 2011

Hello Everyone,

I'm trying to send some data readed from mBed to a computer for further processing. I have 4 inputs (readed as float), a counter (int), and 4 char for separation format: counter;#;#;#;# . The main critiria I have is to read the data as fast as possible from the sensors for about 20 seconds, this time might increase (all data received at same frequency).

1) What would be the best way to receive the data and send it to the computer, storing the readed data into some array and after done with the reading beging to send the data to the computer (via serial)? Or send the data directly (via serial)? Can there be any better method?

2) As mBed has 64KB of RAM I do not know if I will be able to store in the board. What will be faster, store on an EEPROM (24LC256) or send the data directly to the computer (via serial)?

I will really appreciate your help. Thank you.

04 Apr 2011

via ethernet?

store data into the flash file system in the mbed?

04 Apr 2011

Hi,..

that serial port is slow, and an mbed is very fast. I think you need to quantify what you mean when you say "as fast as possible",

are we talking 1000 readings a second ? slower or faster ?

it depends what sensor you are reading, as to what makes sense.

If 64k is enough to hold your data, (what are we talking 8 bytes?) We are talking of the order of 300 readings/sec over 20 secs is possible. Hold that in an array, then you just have to send the 60k down the serial line.

you need to mention some numbers.

to some people 10hz is fast, to others 10khz is fast.

I've not really needed to do anything over a few hz for data acquisition yet, and I've been writing that down to an SDCard connected to an mbed.

I don't know how fast that could run, as I say, depends what you are calling fast. Maybe time for a benchtest!

04 Apr 2011

While I'm here... you could connect a USB stick to mbed and write it down to that ?

04 Apr 2011

David Styles wrote:

If 64k is enough to hold your data, (what are we talking 8 bytes?) We are talking of the order of 300 readings/sec over 20 secs is possible. Hold that in an array, then you just have to send the 60k down the serial line.

I will try what said Dave Styles, as it makes sense. But i think that one read will be more than 8 bytes, according to this http://mbed.org/handbook/C-Data-Types

05 Apr 2011

Updated - local file system test

I ran some simple benchmarks on a few memory devices I have. The test simply writes 500 K of text, one text record at a time. Nothing sophisticated (and I haven't baseline verified that there is not an error).

DeviceWrite SpeedRead Speed
USB Memory stick144 K bytes/sec164 K bytes/sec
microSD memory75 K bytes/sec92 K bytes/sec
Local File System33 K bytes/sec51 K bytes/sec

The code I used is part of my hardware test. The local file system test only works if defined at the module level, not function, and it seems to only work if I do not run the USB file system test before the local file system test. It appears that I can run the microSD test w/o disrupting the local file system test.

04 Apr 2011

With those sorts of speeds.. forget what I said...

just right it down to USB or microSD and attach that to your PC after.

(Presuming you don't need it connected to a PC permanently)

04 Apr 2011

Some hints: 1) you have 4 inputs...analog I assume. Each value you read is not a float, but 10 or 12 bits : a short integer is enough. This is 8 bytes for the 4 inputs.

2) fast readings of the ADC are noisy (there is a thread about the adc bad performance)

2) the available memory of the mbed is 32k, not 64 !!! This is good for 4000 samples (8 bytes each) or 200samples/sec during 20 sec

3) If you send a formatted line to the serial output , the length will be around 30 chars. At 115000 bauds you can send more than 300 samples /sec

4) if you prefer a local file, do not forget that the interrupts are masked....this can be a problem if you use a ticker

I would simply try with a simple program - led on - read 10000 samples - write the samples ( write to a file or write to a serial output ) - led of and measure the time

Robert

04 Apr 2011

If you have it connected to pc to use with serial, then use USB directly. It's a lot faster than serial. You can transfer at least at 1Mb/s. 10Mb/s can also be easily achieved with USB. Your acquisition is consider slow for USB. I recommend to use HID. It does not require a device driver on the host side (PC side). I don't particularly recommend CDC as it emulate a serial port. It slow down USB transfer to match with serial port. Although with CDC you acheive transfer at higher baud rate then regular serial port.

04 Apr 2011

It seems that USB or storing and then serial could be the way to go for this. But can I did not follow this "If you have it connected to pc to use with serial, then use USB directly. It's a lot faster than serial." or this "Your acquisition is consider slow for USB. I recommend to use HID." My apologies but I'm new to micro-controllers.

Does any one have any examples of this methods? specially the USB, I'm looking for the USB breakout but sparkfun does not have any.

04 Apr 2011

There are several tutorials/examples in the Cookbook under "Interfacing with other Languages" (or whatever lol). Try: http://mbed.org/cookbook/Interfacing-with-Java

I haven't done this yet as I have been putting it off until my prototype reaches a point where it feels necessary, but I will likely use Java and the option for the user to either use USB or 1-wire. My data communication requirements are very small however so speed is no problem for me. For you it will be very important that you buffer all of your data (up to a certain size) and then send that packet all at once. Depending on your sensors, 20 seconds of data is not likely to be all that much data. If it actually is 500K then a uSD for buffering is probably the best choice.

You don't need a USB breakout in my opinion. The mbed already has a USB port on the board (or if you must you could wire a new one up to the other pins available).

05 Apr 2011

What I meant was that, if you are doing acquisition while the mbed is connected to the PC. It is better to transfer data directly via USB rather than using RS-232. The data rate at which you are acquiring is much slow than the USB bandwidth (up to 400Mb/s for USB 2.0) but may be too fast for serial. USB has many ways to talk to host computer. HID (Human Interface Device) is one of the them. A few examples of HID devices are Mouse, Keyboard, Joystick, etc... The advantage using HID is supported by default on most modern operating systems such as Windows, Linux, OSX, etc... By making the mbed act has a HID device, it can be connected to those OSes and transfer data without requiring you to write low level device driver. You can write a high level application on the host computer to retrieve data at full USB speed. An other way to transfer data via USB is by emulating a serial (RS-232) device. It is called CDC. But this method often require you to install a CDC driver on the host computer and often limit the data rate so it could still be compatible with software that uses serial port. There are many sample code on this website to make the mbed emulation different devices (Serial, USB flash drive, etc...).

Here is a link to a document from NXP about HID. It is written for the LPC214x series processor but should give you a good idea about HID programming.

http://www.nxp.com/documents/application_note/AN10736.pdf

05 Apr 2011

I like the USB idea, then by using the mBed as HID would I be able to send the data to the PC without need of an actual USBstrick (basically, directly to the PC just using the wire that mBed has)? I hope that there are examples on how to do this around.

05 Apr 2011

Yes, exactly. The mbed board has 2 pins labeled as USB_D+ & USB_D-. You need to patch the USB cable to it. Do an import from the compiler and search for USB keyword or HID keyword or search mbed website. You'll find a bunch of published codes.

05 Apr 2011

Hi

Be aware that HID has its speed limitations too. Mice are sampled at max 500Hz by the CPU (USB1.1) on WinXP+. Good old Windows 2K had a higher sampling rate for HID (1kHz+). HID is made for Human Interface Devices aka relatively slow changing stuff. But a major plus is you do not have to write a driver.

Size of hid packets is also limited if my memory is correct (has been a while).

There are btw drivers that are transaprent (like http://www.thesycon.de/eng/usbio.shtml) but they are mostly commercially. These drivers just extend kernel mode to user level makeing USB development easier (what goes in, goes into the device unchanged). Not cheap but if you substract labor costs & development time... We used usbio to get a device working and after that implement HID side-by-side.

05 Apr 2011

Yes there is a speed limits on HID. 8 bytes/msec for low speed device, 64 bytes/msec for full speed and 1024B/ms for high speed device per transaction. Which means 64Kb/s, 512Kb/s & 8Mb/s. That is per transaction, i.e per endpoint. You can have multiple endpoints in a single HID device. HID use interrupt transfer mode see chapter 5.7 of the specification. See also this posting http://www.keil.com/forum/11135/

05 Apr 2011

Has anyone written an MBED program with HID Device, I have searched forums and web, but cannot find anything that compiles, found a few from code red - lpc-xpresso etc., but they have lots of pices missing :(

Additionaly many of the links are broken.

05 Apr 2011

Ceri, there are a few samples on this site. Look for USBJoystick or USBMouse. The samples from LPCXpresso need to be compiled with CodeRed compiler. You need to do a few modification to compile them with mbed compiler. Typically removing the CMSIS dependency

05 Apr 2011

Indeed I have looked at the joystick/mouse/keyboard examples, but there are too many differences for me to be able to get it to compile AND work.

I have compared the differences between the PIC18F4550 HID & Joystick example, and have tried to crossover the differences, but with no joy (no pun intended)

Ceri.

05 Apr 2011

To tell you the truth, I haven't work with HID device on mbed yet. Only on AVR so far. On mbed/lpc, I use it acting as Host. In my previous post, there is a link to a document written by NXP people on programming HID device on LCP214x. It should be compatible to the LPC17xx. You can also browse the LCPXpresso support forum. There people working with HID there as well.

http://www.nxp.com/documents/application_note/AN10736.pdf

http://ics.nxp.com/support/lpcxpresso/

http://knowledgebase.nxp.com/lpcxpresso.php

05 Apr 2011

After a lot of work,(I had to delete all of my LPC-XPresso folders)

I managed to compile usb hid program,

to my utter dismay, it is JOYSTICK :(

Ceri

05 Apr 2011

I'm confused...

Why not just install the mBed USB Serial driver and use printf to send the data to the PC? You can read and store it with HyperTerminal or any other terminal program, or write your own GUI to read the serial port. I'm pushing data at 921.6 KBaud through the main USB connector, and have never needed to even consider the second USB port on pins 31 and 32. It really doesn't get any simpler.

Regards,

- Gary

05 Apr 2011

Gary, can you please post a sample of your post?

05 Apr 2011

Reynaldo -

It's in the "handbook" at http://mbed.org/handbook/Windows-serial-configuration. Just one of many hidden gems at mbed.org.... Good luck!

Regards,

- Gary

05 Apr 2011

Reynaldo -

Let me polish that gem for you just a little more. The SerialPC page http://mbed.org/handbook/SerialPC does not actually mention setting the baud rate, so just add it at the beginning of your code:

#include <mbed.h>
Serial pc(USBTX, USBRX);
int cycle = 0;

int main() {
    pc.baud(921600);
    while (1) {
        printf ("Cycle %d\r\n", cycle);
        cycle++;
        wait (0.01);
    }
}

Regards,

- Gary

05 Apr 2011

Thanks Gary, what is the maximum value for the baud rate? and what it the longest time that the board can hold on transmitting at the rate before it flips on me? Wooww, I have enought juice in this forum to be entertained for couple of weeks. I think this is the best forum ever.

05 Apr 2011

Reynaldo -

I have not checked, but I'm thinking 921600 is it (at least for HyperTerminal, it is). It is simply a serial port (UART0) that connects directly to the "mBed chip" on the bottom of the mBed. It will transmit for as long as you send it data, but in practice you'll get busy doing something else and it will get to come up for air....

Regards,

- Gary

P.S. On thinking about it a little more, the theoretical maximum for the LPC1768 is 6.25 MBaud since the baud rate generator has to run at 16 times the baud rate, and the clock will do 100 MHz max. Good luck getting your PC to talk to it, though.

26 Apr 2011

My primary reason for using HID Device interface, is because it is fast and the PC - Micro is fixed, Very similar to CAN, in that there is blocks of data and if you write the two halved well, then No parser like code is required, which is ideal especially for data acquisition, where repeated Formatted packets are being sent/received.

So far, I not have managed to get my MBED to be a HID DEVICE. BUT I Am desperate to do so.

I have been through dozens of example & sample, none of which work.

Ceri.

25 May 2012

When using the FAT-file-system via the USB port to write to a USB Flash drive, I get typical 30-byte writes in 0.14 ms, but roughly every 10th write takes about 3 ms. That's not bad, but about every 15 seconds one write takes about 600-700 ms!

Trying to write CAN Messages that might arrive each ms, it looks like I need an interrupt and a large CAN-message buffer to store enough CAN input to get through that 600 (or so) ms.

Does anybody know why this long "write" occurs, or how to reduce it?

Thanks, GaryG

25 May 2012

I belive there is quite some potential in the SD interface. I worked with this storing music. That was 10 years ago and we had it running at 80 Mbit (not NXP though).

You probably need to find the best size to write and also go down to some assembly programming.