Neil MacMullen

Important changes to forums and questions

All forums and questions are now archived. To start a new conversation or read the latest updates go to forums.mbed.com.

6 years, 3 months ago.

Block size constraints

This looks like an excellent addition to mbed. Could you say if there are any particular constraints on the block/device sizes that the file system can use?

The old FATfilesystem had several undocumented constraints. In particular:-

  • It required a large minimum device size (which made it impractical to support RAM drives on smaller MCUs)
  • It appeared to try to buffer an entire erase-block in RAM when writing data. This made it difficult to support flash devices where the erase block size is much larger than the write-block size. (In our case we have a flash part with a 512b write block but a 64K erase block.)

Thanks in advance...

Question relating to:

Last updated: 03 Jan 2018
file system, littlefs

1 Answer

Christopher Haster
6 years, 3 months ago.

Hi Neil,

LittleFS is designed to be quite a bit more flexible than FAT.

Quote:

It required a large minimum device size (which made it impractical to support RAM drives on smaller MCUs)

This is no longer the case! LittleFS scales better than FAT for tiny directory structures. LittleFS uses 2 blocks for the superblock, 2 blocks for each directory (including the root directory), and at least 1 block for each file.

However, it is still difficult to store much on small devices like RAM. The scale of memory is very different for normal forms of storage and MCU RAM, so I can't gauruntee you won't still run into problems.

Quote:

It appeared to try to buffer an entire erase-block in RAM when writing data. This made it difficult to support flash devices where the erase block size is much larger than the write-block size. (In our case we have a flash part with a 512b write block but a 64K erase block.)

This has also changed! LittleFS does buffer a full write block, but an erase block is never stored in its entirety. This was a big RAM saving even for devices with smaller erase sizes.

-

However, LittleFS is a block-based filesystem like FAT, so it does have other constraints to be aware of:

  • Each file requires at minimum a full erase block.

    This is OK for NOR based flash (where erase blocks are usually 4096B or 512B). But this is a concern for your block device with 64KB erase blocks. This can create a lot of wasted space if the files on disk are small.

    It may be that the increased density of the storage device may make this a non-problem, but I don't know yet having been focusing on NOR based flash. I suspect it depends on the filesystem's use case. Prefering large files over many small files helps.
  • Each directory requires at minimum two blocks
  • The superblock needs two blocks

    This does not include the root directory, so a completely empty filesystem uses 4 blocks
  • Block addresses are stored as 32-bit addresses

    This is the same as FAT and limits the maximum filesystem size to `2^32 * block` size blocks.

    Note that you can increase the block size to take advantage of more storage, but at the cost of increasing file slack as mentioned above.

    • for 512 B blocks = 2 TB
    • for 4096 B blocks = 16 TB
    • for 64 KB blocks = 256 TB
  • File size is also stored as a 32-bit value, this limits you to 4 GB files

-

Out of curiousity, what block device are you using? Were you able to use it successfully with the FAT filesystem?

Accepted Answer

Thanks for the detailed answer Christopher.

I take your point about the RAM drive not being a terribly efficient way to store data. My main interest was in using this for testing/emulation. As well as bulk sensor data we store some configuration settings in the filesystem so when testing on platforms that don't have an actual flash, it's convenient to put the filesystem in RAM. You didn't mention the minimum block size - I assume this is 512B? (It's tempting to think that we could squeeze a bit more efficiency out of the RAM drive by reporting the erase-block size as one byte otherwise!)

I'm also curious about whether there is any optimisation for journalling/append operations? The vast majority of our file-accesses are simply additions to the data-log. In principle a much simpler file-system could be written for this scenario but I'm not sure it's worth it.

The device with the large erase block size is the Cypress S25FL17S which can operate with either a mixture of 4KB and 64KB erase sectors or a uniform set of 256KB (!) erase sectors. I did manage to use this as a FAT filsystem by customising the flash driver to advertise 1K erase block sizes then using a virtual addressing scheme so that we ended up with only the first 1K of each physical sector in use. Obviously not very efficient and it sounds like LittleFS should be a much better match for this part! :)

On our other product variant we use the ATMEL AT45 which is better behaved (but much smaller.)

posted by Neil MacMullen 05 Jan 2018