I'm collecting data from an AD345 accelerometer with a 32 scan buffer using an interrupt to get 16 scans at a time, which are then written to SD card using the SDFile System and FAT library.  AT 200 scans/sec I need to repond to an interrupt within  <120 mSec at worst, or I loose data.  I am logging the  SD write time of  (fprintf( sd,.... ) by displaying average and max  write times for every 10 seconds to the serial port.  I write in about 300 byte chunks for every 16 scans - a total of about 4K bytes/sec. I get some very short write times  because of software buffering in the SD software to support 512 byte blocks(?). Average values are consistently around 3-4mSec.  But I still get big variations, max  values alternate between about 7mSec and 107 mSec every 10 secs ( on the run I'm watching now) so presumably something in the SD card is blocking data at ?64Kbytes? and taking time to open a new block?

I guess that mostly the SD card just swallows data, but occasionally says  hang on a mo while I mess with my internal arrangements before I accept more data?

My questions;  Can anyone throw any light on this - and ways to speed things up?  In particular, the SD write code seems to mask my interrupt, so I can't get a response to my interrupt while the disk is writing. Can I make a way for the long writes to be interrupted?  Has anyone fiddled with SDFileSystem to do this?

Has anyone measured write times for different SD cards?  While watching this, the times dropped to max values of 6 and 30 per 10 secs for about 10 minutes and then went back to 7 /117.  Changing SD card got Max values alternating between 20 and 30 mSec. Both cards alternate between hi and lo max values at the same rate.  The average values are always very low - around 3 - 4mSec.  Too much variablility for comfort!

Tim Owen

Having put the last post up, I tried an HP 2GByte card L1877A and got nothing greater then 8mSec after the first write on opening new  files in half an hour of logging.  Even opening a new file measured only about 58 mSec.  So if you want to log data, check the SD Card speed ! - I'll try to sort out a test program from my bits of code and post it later.

Tim

Hi Tim,

Sounds like some good investigations...

tim owen wrote:
My questions;  Can anyone throw any light on this - and ways to speed things up?  In particular, the SD write code seems to mask my interrupt, so I can't get a response to my interrupt while the disk is writing. Can I make a way for the long writes to be interrupted?  Has anyone fiddled with SDFileSystem to do this?

Sorry if this sounds like a silly question, but are you perhaps writing to the SD card *in* the interrupt? That'd be one explanation for not seeing the subsequent ones during that time?

Simon

Hi Simon, Thanks,  I am  doing the writes in the foreground - I did them in the interrupt to begin with, but the results didn't seem much different.  If the SD card write is interruptable, then I  need a deeper buffer in the foreground to soak up a few more pending scans - that could be my problem.  I haven't looked at the interrupt on a scope recently, but I'll take a look.

As I read it, the pin interrupts are effectively edge triggered, and don't interrupt a second time when the first completes unless the external line is toggled?

I'm now writing a test utility to plot histograms of write times at different data rates on different cards - I'l post it when I've finished it.

I did a lot of work on 68332 systems ages ago, but haven't touched an embedded system for 20 years, so its a slow process...... although  the Mbed is a pretty neat package  - maybe I'll tackle a 4 channel 24 bit seismic aquisition system next if I can lick the write speed issues!.

Tim:

I just compelted a power datalogger that had to write 2 channels of 16-bit data at 1KHz in a text (CSV) format.  I found that things were highly dependent on the card!   I ended up buying every uSD from newegg and tested each model.   In some cases, a card was 20 times slower.

I also had alot of buffering:

1.) An input buffer for raw ADC samples from my external converter

2.) An intermeddiate buffer for converting to text/string lines

3.) An output buffer that would only write in even sector sizes.

Also,  I have a published program that has long file name and proper date/time support. CHAN_FATFS

Eli Hughes wrote:
write ... data at 1KHz

Hi Eli

Can I just confirm for my understanding, that you managed to achieve a block write to the SD card in less than 1 millisecond so you did not jitter your ADC sample interval? Or alternatively that writes to the SD card are interruptible by ADC acquisition?

Thanks
Daniel

For what it's worth the Sd filesystem clocked the SPI bus at 3 MHz last time I checked, which is pretty slow.  I had to turn it up some to get it to deliver data fast enough to play back wave files.  I did some read benchmarking on the cookbook page that may be useful.

--steve

Steve: How can I speed up the SPI interface used by the SD filesystem?

For what its worth, I am no longer using SD cards for data logging. THe USB host controller and these parts were a better solution for me:

http://www.lacie.com/us/products/product.htm?pid=11546

and this connector:

http://www.samtec.com/ProductInformation/TechnicalSpecifications/Overview.aspx?series=USBR-A

its a high retention force USB type A.

SD cards exhibited alot of variability in write speeds. I have to come upwith some intense buffering schemes to deal with write speeds for high speed logging. The USB keys are much nicer. The write alot faster.

Here is an example of the USB host lite with Chan Fat FS file system (Long file names with real time clock support!)

http://mbed.org/users/emh203/programs/CHAN_USB/llhh3p