Fast ADC Sampling and Storing in a SD Card

Questions » Fast ADC Sampling and Storing in a SD Card

Aswin Natesh

7 years, 7 months ago.

Hello

I have a Mbed LPC1768 board and its connected to two analog sensors and an SD Card Reader. I would like to read the sensor values at a very high sampling rate, say 16Khz, for about 20 Seconds and store these values in an SD Card. This is for one trial and values for subsequent trials should be stored in a new file each time. I would be recording data for at least 100 trials with an input trigger pulse to start each trial at pin number 15.

The problem that I face is, that the array size is limited to somewhere around 3600, and exceeding which the array overflows. I could easily store around 3600 samples of each sensor and later write them to an SD card, but I require a huge number of samples for each sensor that's nearly (16,000 x 20 Sec x 2 Sensors) 6, 40, 000 Samples each trial.

I tried writing these values directly to the on-board storage too, as a txt file, but even this memory gets filled up completely once it reaches 2, 42, 000 Samples. Writing directly to an SD Card loses few samples and because of the write latency, and therefore didn’t want this to happen.

So are there any alternate solutions for my requirement?

Thanks in advance!

The code that I have for now:

#include "mbed.h"
#include "adc.h"
#include "SDFileSystem.h"
#define SAMPLE_RATE 18000 //Sampling Rate in Hertz
#define LENGTH_RESULT 3600 //Number of Samples

FILE *fp;
SDFileSystem sd(p5, p6, p7, p8, "sd");
ADC adc(SAMPLE_RATE, 1); //Initialise ADC to maximum SAMPLE_RATE
DigitalIn switchStatus(p15); //Trigger Pin
Serial uart(USBTX, USBRX);

volatile int result[LENGTH_RESULT];
volatile int result2[LENGTH_RESULT];
int count;

FILE *nextLogFile(void)
 {
 static unsigned int fileNbr = 0;
 char fileName[32];
 FILE *filePtr = NULL;
 do{
 if (filePtr != NULL)
 fclose(filePtr);
 sprintf(fileName,"/sd/Log%04u.txt",fileNbr++);
 filePtr = fopen(fileName,"r");
 } 
 while (filePtr != NULL);
 return fopen(fileName,"w");
}
int main()
 {
 while(1)
 {
 if(switchStatus == 1)
 {
 uart.printf("Command Received! \n");
 adc.setup(p20,1);
 adc.setup(p19,1);
 for(count = 0; count < LENGTH_RESULT; count++)
 {
 adc.select(p20);
 adc.start();
 while(!adc.done(p20));
 result[count] = adc.read(p20);
 adc.select(p19);
 adc.start();
 while(!adc.done(p19));
 result2[count] = adc.read(p19);
 }
 printf("SD Card File Handling!\n");
 fp = nextLogFile();
 if (!fp)
 {
 error("Could not open file for write\n");
 }
 for(count = 0; count < LENGTH_RESULT; count++){
 fprintf(fp, "%04u \t", result[count]);
 fprintf(fp, "%04u \n", result2[count]);
 }
 fclose(fp);
 printf("Task Complete!\n");
 }
 }
 }

2 Answers

Andreas Moser

6 years, 4 months ago.

Hello I'm nearly at the same problem. Do you found a solution? Danks

Andy A

6 years, 4 months ago.

There are a few tricks you can use for this.

1) user uint16_t not int. That will halve your memory requirements.

2) use more ram On the LPC1768 there is an extra 32k of RAM you can use if you don't need ethernet or CAN. This is in two 16k bit blocks but they are next to each other in memory so you can treat them as one large block if you want. If you save the ADC values as uint16_t rather than ints that gives you space for 16,384 samples.

uint16_t result1[8192] __attribute__((section("AHBSRAM0")));
uint16_t result2[8192] __attribute__((section("AHBSRAM1")));

3) Sample on a ticker interrupt and in the background loop write the data to the file. The code to do this is along the lines of:

uint16_t *adcBuffer = result1; // starts pointing to the first buffer
volatile uint16_t fileBuffer = null; 
int bufferPtr = 0;

void onSample() {
  *(adcBuffer +bufferPtr) = adc.read_u16();  // read the ADC
  bufferPtr++; 
  if (bufferPtr == BUFFERSIZE) {  // the buffer is full
    if (fileBuffer) { 
      // Buffer overflow, data has been lost.
    }
    fileBuffer =adcBuffer ;     // indicate that the current buffer should be written to the card.
    if (adcBuffer == result1)   // switch to the next buffer for storing reads.
      adcBuffer = result2;
    else 
      adcBuffer = result1;
    bufferPtr=0;
  }
}

main () {  

  FILE *outputFile = openNewOutput();  // open the file
  ticker.attach(SAMPLERATE,&onSample)   // start sampling
  while(!finished) {  // need some method to decide when to stop
    if (fileBuffer) {     // if data waiting to be written
      //   write filebuffer to outputFile here  
      filebuffer = NULL;  // indicate that write is done
    }
  }
  fclose(outputFile);
}

This method uses 2 buffers and flips between them, you can use more than two and queue them up waiting for the disk. More complex to code but better able to cope with variability in the cards write speed.

4) As a further improvement to the method above write the files as binary rather than text. That way you avoid the expensive printf calls, you just copy the contents of the buffer array directly to the file. You can always write a second program that converts the binary log to text if needed.

You'll probably get peak throughput if you make the buffers 512 bytes to 1024 bytes so that they match the block size that the file system is using in the background.

Aswin Natesh

Important changes to forums and questions

7 years, 7 months ago.