Hi, people!

Well, I'm having a very weird problem here. I'm loading a textfile using LocalFIleSystem, and as the data is loaded from the file a lexical analyser checks whether the content is well written (the file contains some rules for a knowledge based system, and is written in a particular knowledge representation language). So, the file is readed line by line and, as soon as the line is readed, the lexical analyser analyses it. The problem is: at some point of the lexical analyser the program freezes. Here is a little piece of code from the lexical analyser that will be of interest:

...

string strToken; // string that will store a token from the file with the rules.
char charFile; // a charactere extracted from file.

...

strToken += charFile; // appends, given certain conditions, the character readed to the current token.

...

I added some printf's along the code to help me isolate the problem, and I discovered that it is happening on the concatenation of the string with the char (last line on the code above). I mean, the last "printf" that I could see on the serial terminal screen was the one before that line. But here is the weirdest part: it does not occur on every concatenation. The code runs well until a particular concatenation, where it freezes.

Then I changed every concatenation to:

...

strToken = strToken + charFile;

...

After making that change, the concatenation that was failing started to work, and the code started to fail at some other concatenation, even after changing all the others to the above format.

I already tried use ANSI C strings instead of std::strings, std::stringstream instead of reading from a file, and the maximum that I achieved was changing the concatenation where the program freezes. I am out of ideas about how to sove this. Does anyone have any ideas? Any suggestions? Please, I'm really desperate, anything may be helpful...

Thanks in advance!

My guess is that you are running out of memory. It is a common error to repeatedly use the + operator to lengthen strings. It is both slow and memory-hungry as, potentially, every time the string is lengthened a new copy is made and the space used by the old copy is not reclaimed (unless you have a garbage collector running).

I'm not a C++ programmer. If I was programming in Oberon I would use an array of characters and assign each new character to the next element of the array. If I was programming in C# I would use a StringBuilder class. You should be able to find similar alternative solutions in C++ by Googling "C++ string concatenation performance" or something like that.

I'm with Chris on this - most likely your app is out of memory.

Below is a simple free memory check. This finds the largest single free block, so could be misleading if you actually have a lot of small blocks free. Then if there is some strategic place in your code where you can run this, and emit the results, you might get an idea of the available space. Recognize that if you printf("Free mem is %u\r\n", Free_Mem());, that printf also has dynamic allocation requirements.

uint32_t Free_Mem() {
    void * p;
    uint32_t max = 100000;
    uint32_t x = max / 2;
    uint32_t min = 0;

    while (min < max-1) {
        void * p = malloc(x);
        if (p) {
            free(p);
            min = x;
        } else {
            max = x;
        }
        x = (max + min)/2;
    }
    return x;
}

Chris and David, thank you guys really much!

I could not fix the problem yet, but since yesterday I'm eliminating all uses of the string class on the code and replacing it with custom char array manipulation methods. The code is big and uses this class a lot, so it will take a while.

David, I called your function right before the concatenation the was freezing the app and it returned something between 9000 and 10000. Is this some kind of lower memory limit?

Thanks again, guys!

I would think that if memory allocations are gradual, that you would see that number work on downward much closer to zero before it hung.

And maybe you don't have to replace everything that was there, as Chris mentioned, it could be forever allocating, so maybe all you need to do is selectively delete the c++ string objects before you recycle them. I'm not sure what happens under the c++ runtime, but

string mystring = "hello";
mystring += " world";

initial allocation is 7 bytes + the overhead to manage it, but the += causes an allocation of 7 + 7 to first copy the "hello" in and then append with " world". And then hopefully it free's the old item. But this could cause fragmentation - so the Free_mem( ) might not be telling the whole story.

So, could the following free the current allocation and then start fresh? I think so.

mystring = "starting fresh";

But honestly, I've not prowled into c++ in this direction before.

Hi, David!

I tested that, and the error still occurs... It is also occuring during some calls to "string.h" C functions when I replaced the "string" objects by char arrays.

Well, I was wondering: what if I use an offline toolchain? Would it change something?

Thanks!

Hi Diego,

I probably won't be much help from here. I worked briefly with an offline tool chain for the mbed a year or so ago, but have lately done everything online. Different compilers, and the ability to configure different optimizations might make a difference, but it "feels" like something else is going on.

I haven't given it a go yet, but there has been some great work on a debugging interface - MRI.

Wish I could help more.