8 years, 6 months ago.

Saving analog data in SD card

Hello,

I have a mbed LPC1768 board and its connected to a analog sensor. What I want to do is read the sensor value and store it in a SD Card and I want to make a new file in the SD card every 5 minutes. How can I implement this, I know how to create a file but how do I create a new file in every 5 minutes? Reason behind creating a file in this order is to make sure the data doesn't get lost even if some files are corrupted. Is it also possible to write on the SD card and read from the sensor at the same time, so that I don't loose any incoming data from sensor while writing it to SD Card. Any info on this one for me? or any link where I can read about it.

I also have one additional question regarding the memory of this board how much can this processor buffer? It has 64KB SRAM and 512 KB flash memory. So can it buffer 64 KB or the 512 KB? I was thinking of adding multiple sensors, which means bigger data and it will go above 100 KB, if this is the case can this board still handle the data.

Thanks in advance.

1 Answer

8 years, 6 months ago.

This is fairly simple to do.

The way I'd structure it would be to have a Ticker running at the required analog sample rate. That makes the ADC measurement and saves it in a buffer.

The main program loop then constantly checks the buffer and if there is data waiting writes it to the SD card. The main loop also checks a timer to see how long since the file was opened and after 5 minutes closes the file and opens the next.

Something like this:

#include "mbed.h"
SDFileSystem sdCard(p5,p6,p7,p8, "sd");
FILE *myLogFile;
Ticker sampleTicker;
AnalogIn sensor(p20);
Timer fileOpenTimer;

#define bufferSize 1024
float sensorReading[bufferSize];
unsigned int readPointer = 0;
volatile unsigned int writePointer = 0; // volatile so that the main loop knows to check for changes.

// opens the next unused file name in the format set.
// This can be a little slow the first time if there are already lots of log files
// since it tries each number in turn but the second call on will be fairly quick.
FILE *nextLogFile(void)
{
    static unsigned int fileNumber = 0;
    char fileName[32];
    FILE *filePtr = NULL;
    do {
        if (filePtr != NULL)
            fclose(filePtr);
        sprintf(fileName,"/sd/log%04u.csv",fileNbr++);
        filePtr = fopen(fileName,"r");
    } while (filePtr != NULL);
    return fopen( fileName,"w");
}

void onSampleTick(void)
{
    sensorReading[writePointer++] = sensor*3.2; // scale to give a voltage rather than value from 0 to 1.
    if (writePointer == bufferSize)
        writePointer = 0;
    if (writePointer == readPointer) {
        // BUFFER OVERFLOW. You may want to print an error message or turn an LED on
    }
}

main()
{
    myLogFile = nextLogFile();
    if (!myLogFile) {
        // ERROR failed to open the first log file for writing.
        // The SD card is missing, not working, read only or full?

        return 1; // probably want to exit the program in this situation
    }

    fileOpenTimer.start();
    sampleTicker.attach(&onSampleTick,0.1); // sets the sample period in seconds

    while (true) {

        while (writePointer != readPointer) { // write any waiting data to the SD card
            fprintf(myLogFile,"%.2f\r\n",sensorReading[readPointer++]);
            if (readPointer == bufferSize)
                readPointer = 0;
        }

        if (fileOpenTimer > (5*60)) { // file has been open 5 minutes
            fclose(myLogFile); // close the current file
            myLogFile = nextLogFile(); // open a new file
            if (!myLogFile) {
                // ERROR failed to open the log file for writing.
                // card full maybe?

                break; // exit the while(true) loop
            }
            fileOpenTimer.reset() // restart the timer
        }
    }
}

Note: this doesn't allow any means to close the current file and stop logging. If you pull the card or power out you will lose the contents of the current file.

I'm not sure why you would ever need 100kB for buffering. You only need to buffer data until you've written it to the SD card, after that you don't need to keep it in memory any more, that's the whole point of writing it to a file. It's a good idea to allow enough buffer for a second or two worth of data in case the card is slow closing and opening files but there is little point in going much beyond that. In terms of memory size you normally have the 32k of RAM available for your program to use however it will need some of that for other variables. The compiler will give you an estimate of memory usage but that's not including things like the stack so you don't want to fill it all up.

Realistically assuming a fairly simple program you should easily have 16k free for buffers and still have plenty of safety margin. At 4 bytes for a float that gives you space to buffer up to 4096 values. If you read the ADC as a uint16_t and only convert to a float when saving to a file (or on a computer when reading the file) then you'll double the number of points you can store.

The LPC1786 has another 32k of RAM (making 64k in total) that is normally used for ethernet and USB buffering. If you aren't using either of those (the USB programming/serial port doesn't count) then it is possible to use that memory for buffering but why add complexity if you don't need it. If you do need it that gives you a full 32k of buffer.

How practical it would be to use both that 32k and the main memory at the same time would depend on what you were doing. e.g. If you wanted 3 16kB buffers for 3 different sensors then that would be simple. If on the other hand you wanted a single 48kB buffer then that would be nasty.