FAT, SD, and data logging

03 Jul 2010 . Edited: 03 Jul 2010

Hello, I'm working on a data-logging system for an aerospace application.  Here is what I have so far:

Started with standard, low-capacity SDcard and built a program to power up, initialize, write some dummy data, then read that data and printed to my computer.  I've now moved up through some cards and accomplished this with a 4GB microSDHC.  The next challenge is to learn how to use the FATFileSystem.

I can't seem to find answers to my questions anywhere on the site.  I noticed the library isn't anywhere to be found except through searching the compiler's importer, and even then there isn't much information on it.  Would it be possible to see the header file?  A list or description of the members would be helpful.

I'm also trying to figure out how the FAT blocks mesh with the SD blocks so I can set the correct size during initialization, I don't want to waste any space.  How do I tell the FAT system what sized blocks to use?

03 Jul 2010

Hi Thomas,

Take a look at http://mbed.org/projects/libraries/svn/FATFileSystem/trunk. Hope this will help get you started.

Simon

08 Jul 2010 . Edited: 20 Jul 2010

I'm noticing something about the SPI commands.  The frequency method takes an int as an argument, not an unsigned one.  A frequency cannot be negative, so wouldn't it be better to have an unsigned int as the argument?

--Update--

@ Simon - I have just completed work on a crc7 module, would you be interested in having a copy for your SD card library?  I'm working on the 16 bit CCITT crc now.

--Update--

all CRC modules are complete

14 Jul 2010 . Edited: 14 Jul 2010

I used cmd6 to ask the card what modes it supports.  It told me that it consumes 100 mA in normal mode and 200 mA in high speed mode.  Both of these are higher than the mbed's ability to sink/supply (40 mA maximum).

The card is working fine, but I'm worried about long-term wear on the mbed.

Should I be concerned about this?  Does anyone have a suggestion on how to fix it?

15 Jul 2010

Hi Thomas,

These power numbers you discuss are the power consumption of the card, not the amount of power you need to drive on the SPI control lines. So it really shouldn't be a problem (unless you have it wired up wrong!); the power will come from the power rails you have supplied to the SD card.

Simon

15 Jul 2010 . Edited: 20 Jul 2010

I get it now,

Thanks again.

18 Jul 2010 . Edited: 24 Jul 2010

I double-checked to make sure my CRCs work with low-capacity cards only to find the Kingston 128MB card says cmd59 is illegal.  All specifications dictate that this command shall be valid during idle state and I can't figure out why this is not working.  Does anyone have suggestions?

-EDIT-

I gave the code sort of a patch, it works with both cards again.  Apparently the CRC on/off command is not valid during initialization for some version 1 cards (though there are zero documents that say so).  I sent 59 again after initialization was complete.

21 Jul 2010

Are there any more documents on the FATFileSytem?

I would like to know the operation of certain areas of the class.  For instance, what does it do if initialization fails; does it retry or terminate?  How does it determine block size; is it fixed at 512?

I really don't mean to bother everyone so often, but I have researched for days and this is my last option.

Thanks,

Thomas

21 Jul 2010

Hi Thomas,

I've just posted the original code I wrote here which might be interesting:

The FATFileSystem is basically one of the results of an experiment I did around making a generic virtual file system abstraction. This one uses the CHaN FAT filesystem with some munging to allow it to have multiple instances of itself (to allow full virtualisation e.g. to have multiple FAT devices connected).

Note that it really was only an experiment and is not part of the core mbed libraries, but it has turned out to be very useful so is probably worthy of some love/updates; feel free to have a look and see what improvements you think might be useful. I'm happy to put the code under MIT license, and let it take on a life of it's own to help improve it over the experiments I started.

Is this useful?

Simon

22 Jul 2010

Not sure if it helps but I recompiled CHaN FAT to support long file names and proper time/date stamps (current FAT doesn't have time/date stamps)

I am using this code in a datalogger project.    http://mbed.org/users/emh203/notebook/long-file-name-example-code/

22 Jul 2010 . Edited: 22 Jul 2010

Hey, thanks for the help guys

@ Eli: Thanks man!  I wish I saw your code before I spent so much time writing all of my stuff!  Will definitely give it a look.

@ Simon: I would love to play with it.

I understand all of yours and ChaN's code save for 1 part:  how are we interfacing with the functions like fopen and fprintf.  I've been trying to figure that out for a while but cannot see the bridge.  I know that the FATFileSystem::open must be called at some point when the user calls fopen; could you explain this or point me toward some helpful documents?

It appears that "FileSystemLike.h" has something to do with it.  Do you know where I can find this?

Thomas

22 Jul 2010

The CHaN code does not use C standard "stream" code but uses its own.     Its pretty similiar (f_open instead of fopen), etc.   I think this was done to streamline the code (C streams are pretty bloated).

I searched far and wide for a open source, free FAT file system that supports FAT32 and long file names......  CHaN is by far the most mature, stable and easiest one to integrate.

 

27 Jul 2010 . Edited: 06 Aug 2010

Working on utilizing cmd25 to increase writing speed.  I ran into a strange problem and was wondering if any of you guys have encountered it.

I wrapped cmd25 in a method that accepts buffer, block count, address.  It loops cs low, 0xFC, send 512 buffer bytes, get response, step through busy, cs high; then it sends cs low, 0xFD, 0xFF, step through busy, cs high.

I tested it against single block mode by writing ten bytes in each mode.  It writes the data correctly, but the strange thing is it takes 250ms to call the continuous method to write all ten blocks the first time, then only 9ms on subsequent calls; while calling the single block mode ten times to write each block individually takes 50ms in total.

On further investigation, the busy signals are taking way to long to finish (writing 0xFF on the order of 100000 times).

-edit-

problem solved by presetting erase block size; large process time before hand is expected

06 Aug 2010

I'm having trouble with making new directories, has anyone been able to do this correctly?

I'm giving the FATFileSystem some "love", but I'm running into issues linking the ChaN FAT "f_mkdir" with the standard c functions.

First, I'm not sure how one would make a directory with a working system.  I figured one could just use fopen and any directories that don't exist would be created.  Does anyone know which standard c function is supposed to call the mkdir function of FileSystemLike?

06 Aug 2010

Should be "mkdir". See here.

mkdir("/test", 0666);

compiles, but I haven't tested it.

07 Aug 2010 . Edited: 07 Aug 2010

Ok, now we're getting somewhere.  The Chan FAT system is now giving me an "invalid path name" result.

I need to know what format the mbed FileSystemLike puts the dir name in when the virtual mkdir function is called.  Then I can figure out how to reformat it to suit the Chan FAT.

I cannot find docs on this, so I'm going to trial and error it to find out.

Thanks for the help man.

--edit--

I figured out how to fix the formatting issue; FileSystemLike passes in a "/" in the beginning of the path when passing a directory name.  It's confusing because it does not do this for file names.

Just one last bug to fix before the system is complete, it was discovered while trying to fix the path formatting

Is the FileSystemLike virtual method "rename" broken?  No matter what I try, it will not execute my block of code and will only return -1.  Has anyone else had this problem?

The declaration in the header file is

virtual int rename(const char* oldname, const char* newname);
The definition is
int FATFileSystem::rename(const char* oldname, const char* newname)
{
    char OldName[64];
    
    sprintf(OldName, "%d:/%s", FileSystemID, oldname);
    if (f_rename((const TCHAR*)OldName, (const TCHAR*)newname))
    {
        return -1;
    }
    else
    {
        return 0;
    }
}

02 Nov 2010

is there any support for date time stamp for SD file system ?

all my generated files have no entries in windows explorer.

 

Cheers

Ceri

02 Nov 2010

Hi Ceri,

The FATFileSystem doesn't support timestamps. In fact, when I put it together, it was just a test to see if it was possible to create a pluggable virtual filesystem supporting FAT in to the core mbed FileSystem handler. It seems like it has actually been used by a number of people, so more useful than I thought over just a test!

I'll look to dust it off sometime and add the timestamp, and publish it so others can tweak it, but for now it is no timestamps and 8.3 filenames only.

Simon

03 Nov 2010

I wrote a class to try and do a formal integration of the FAT file system based off of Simon's code (not just a test).

Everything works except the rename function (a question which, unfortunately, never got answered), and the chan FAT filesystem module was used for the FAT abstraction.  Perhaps, if you want to add timestamps, that would be a good place to start?

SDCard

The FAT system itself is in SDCard > FATFilesystem > Core

chan inculded a neat config header file to change the code to match your system.  Maybe you can edit this and the core program to create timestamps?

In any case, good luck to you.

03 Nov 2010

Hi Thomas,

Just seen your question re: rename too. I'll look at that at the same time.

Great work btw.

Simon

03 Nov 2010

You are quite gracious.  Thanks for your help.

16 Dec 2012

Sorry for the bump but I, too, would love to see timestamps supported by the FAT filesystem. I built a blackbox datalogger app and would like to automatically delete the oldest file on the SDRAM when the SDRAM starts to get full. I have a work-around but it's a kludge.

16 Dec 2012

I have used the SDHCfilesystem (with ChaN's FatFS) and it creates and uses timestamps.

20 Mar 2013

I would be most grateful if someone could make a very simple "Hello World!" example (write "Hello World!" to a file on the USB or local fs) using the ChaN FatFS with timestamps working. This is still eluding me.