data buffer example code

03 Nov 2009

Does anyone have example code to setup the internal databuffer and then write to it?  I have searched the site and not yet found anything.

My application polls a serial device that returns a string of 120 bytes.  These bytes need to be replced in a buffer.  Additional bytes will be added to the buffer before interpretation and datalogging.

Communication to the serial device is working properly. .

Thanks.

03 Nov 2009

Something like this?

void get_buffer(Serial &s, char* buf, int bufsize)
{
  for ( int i=0; i <  bufsize; i++ )
    buf[i] = s.getc();
}

...

Serial device(p28, p27);
char msgbuffer[120];
get_buffer(device, msgbuffer, sizeof(msgbuffer));
09 Nov 2009 . Edited: 09 Nov 2009

Thank you for the example.  I finally had a chance to try it.  I can not get the code to compile.  I get a flag when I try to print the variable to the display (or pc).  The flag is on line 70 ( I commented the line to note the flag).  The flag says

 

"Expression must have (pointer-to-) function type (E109)" in file "MSruntimevariables/main.cpp"

 

I would appreciate any help.  Here is my current test code.

 

//Code Designed to display run-time variables from a Megasquirt Fuel Injection controller.
//This is the base code to load variables and print to LCD.

#include "mbed.h"
#include "TextLCD.h"

Serial usb(USBTX, USBRX); //define USB connection pins
Serial megasquirt(p28, p27); //define serial commection pins

DigitalOut usb_activity(LED1); //define USB activity
DigitalOut megasquirt_activity(LED2); //define Megasquirt activity
unsigned char      tmp_data[16];
unsigned char       buf[120];
int    tmp;
int    AFR;

/*LCD 2x20 parallel code pin assignment*/
TextLCD lcd(p21, p22, p23, p24, p25, p26, p30, 20, 2); // (rs, rw, e, d0, d1, d2, d3, n_column, n_rows)


//enter main loop
int main() {

megasquirt.baud(115200); //define MEgasquirt serial speed   
usb.baud(115200); //define USB connection speed
//    unsigned char tmp;
//    unsigned char data_in;

unsigned char data_out;

while (1) {

//usb.putc(usb.getc());//echo back character


char c = usb.getc();
//temporary call function using keyboard - in future will trigger automatically.
if ((c == 'a'))
{


while (megasquirt.readable())
{
//poll serial device for data stream (a,0,6) to (request, return, confirm)
megasquirt.putc(97);//send 97 for run parameters, 83 for board revision, 81 code for revision number
wait(0.005); //wiat for response
megasquirt.putc(0);
wait(0.005);
megasquirt.putc(6);
wait(0.005);        

data_out = megasquirt.getc();

/////////////Insert buffer code here
//Data_out should be a buffer.  I need 120 bytes
//need to know address of buffer
//buf = data_out; //test code for data buffer
{
for ( int i=0; i <  120; i++ )
buf[i] = megasquirt.getc();
}

//Want to pull certain bytes out for manipulation

///////////Define run time variables//////////////////////////          
//example code for AFR

//get Afr
tmp = buf[29];
//set position on disply
lcd.locate(0,0);  //Set LCD location
lcd.printf("AFR="); //text label on LCD
lcd.printf("%.3f" ,AFR()); // send data  ** compiler flags this line **


megasquirt_activity = 1;

usb.putc(data_out); //megasquirt write to USB read
lcd.locate(0,1);  //Set LCD location
lcd.putc(data_out); //megasquirt write to LCD
}
}


}
}//exit main loop

10 Nov 2009

I have simplified the code and now can get it to run.  However, the output from the data buffer is really wrong.

when I should get a count timer I get a fixed value of 2.212E-314.  This value prints no matter what buffer position I sample.

Also, I can only run my buffer loop to buf[65] instead of [120].  The program will not run and locks up.  Any ideas?  I need to buffer a string of 113 bytes, write the string to an SD card  and then pull out selected ones to print to an LCD.

 

here is my current code.

//Code Designed to display run-time variables from a Megasquirt Fuel Injection controller.
//This is the base code to load variables and print to LCD.

#include "mbed.h"
#include "TextLCD.h"

Serial usb(USBTX, USBRX); //define USB connection pins
Serial megasquirt(p28, p27); //define serial commection pins

DigitalOut usb_activity(LED1); //define USB activity
DigitalOut megasquirt_activity(LED2); //define Megasquirt activity
unsigned char      tmp_data[16];
unsigned char       buf[120];
char buffer[120];
int    tmp;
int    AFR;
unsigned char data_out;
int i;

/*LCD 2x20 parallel code pin assignment*/
TextLCD lcd(p21, p22, p23, p24, p25, p26, p30, 20, 2); // (rs, rw, e, d0, d1, d2, d3, n_column, n_rows)


//enter main loop
int main() {

megasquirt.baud(115200); //define MEgasquirt serial speed
usb.baud(115200); //define USB connection speed
//    unsigned char tmp;
//    unsigned char data_in;
megasquirt_activity = 1;



while (1) {
lcd.locate(0,0);  //Set LCD location
lcd.printf("AFR="); //text label on LCD
usb.printf("AFR="); //text label on LCD

if (megasquirt_activity = 1) {
//poll serial device for data stream (a,0,6) to (request, return, confirm)
megasquirt.putc(97);//send 97 for run parameters, 83 for board revision, 81 code for revision number
wait(0.005); //wiat for response
megasquirt.putc(0);
wait(0.005);
megasquirt.putc(6);
wait(0.005);


/////////////Insert buffer code here
//Data_out should be a buffer.  I need 120 bytes
//need to know address of buffer
//buf = data_out; //test code for data buffer

for ( int i=0; i <  10; i++ ) {
buf[i] = megasquirt.getc();
//usb.putc(buf[29]);
//lcd.putc(buf[29]);
AFR = buffer[1]; //changed to count timer
lcd.locate(5,0);
lcd.printf("%E" ,AFR);
usb.printf("%E" ,AFR);
}
}
else {
lcd.printf("no connection");
usb.printf("no connection");

}       

}

}

//exit main loop


11 Nov 2009

Hi James,

James Novak wrote:
However, the output from the data buffer is really wrong. when I should get a count timer I get a fixed value of 2.212E-314.  This value prints no matter what buffer position I sample.

The value "2.212E-314" is a representation of a floating point number, which is the first suspicious thing for a counter value. If you look at where it is being printed out:

lcd.printf("%E" ,AFR);

the use of %E tells printf to print in a scientific floating-point format. As AFR is an integer, you'll just get garbage. Try using %d, which is the format specifier for an int. i.e.

lcd.printf("%d", AFR);
This should hopefully show you what you have actually stored in the array (assuming the byte is a raw binary number, rather than ascii/text number!)

For more details on printf formatting specifiers, see:

Hope that makes you results start to appear.

Simon

11 Nov 2009

Simon,  thank you for your response.

I purposely put the engineering notation in place.  I was getting nothing but a "string of zeros" for my output and wanted to make sure I was not missing some information to leading zeros in the string.

You bring up an interesting question.  I think the string is binary.  If it was text/ascii can you make suggestions for the conversion.  Do I still use printf?

I still think there is an issue with loading the buffer.  If I increase the buffer size to anything larger than 65 the loop will not complete. The following snippet code will not run.

 


/////////////Insert buffer code here
//Data_out should be a buffer.  I need 120 bytes
//need to know address of buffer
//buf = data_out; //test code for data buffer

for ( int i=0; i <  120; i++ ) {
buf[i] = megasquirt.getc();
//usb.putc(buf[29]);
//lcd.putc(buf[29]);
AFR = buffer[27]; //changed to count timer
lcd.locate(5,0);
lcd.printf("%d", AFR);
usb.printf("%d" ,AFR);
}
}
else {
lcd.printf("no connection");
usb.printf("no connection");

11 Nov 2009 . Edited: 11 Nov 2009

Hi James,

I've had a quick look at your code, and I think one problem may be related to your buffers. Here is your code simplified:

for(int i=0; i<120; i++) {
    buf[i] = megasquirt.getc();
    AFR = buffer[27]; 
    lcd.printf("%d", AFR);
}

You seem to be reading in to one buffer ("buff"), but displaying another ("buffer").

Also, note you are doing all of this 120 times, including the AFR assignment, and the printf. I suspect you meant something more like:

for(int i=0; i<120; i++) {
    buffer[i] = megasquirt.getc();
}
AFR = buffer[27]; 
lcd.printf("%d", AFR);

This code would read 120 bytes, then print out the integer value of byte 28. Note that if you don't get 120 characters, it'll appear to just hang, so you'll want to check that is what is expected.

Simon

13 Nov 2009 . Edited: 13 Nov 2009

Simon,

This is the code that will work.  It writes each byte one at a time for test purposes.

 

while (megasquirt.readable())
{
data_out = megasquirt.getc();
megasquirt_activity = 1;

usb.putc(data_out); //megasquirt write to USB read
lcd.putc(data_out); //megasquirt write to LCD

}

 

However, with my code and the code you posted I have problems.  No matter what the size of the buffer definition I only get 17 bytes (n=0 -16).  Can this be changed in the header or library file?  Using a timer function I can see that all of the expected bytes are coming over on the oscilloscope.  They are just not making it to the buffer.

Is there a limitation on the buffer?  The reason I ask is the information in the following post.

http://mbed.org/forum/topic/170/

"The UART hardware has a buffer of 16-bytes, so you will get some buffering, but any more than this and they'll be getting dropped.

We are just experimenting with some interrupt driven UART abstractions which would allow you to build things like a software buffered UART very easily if that is of interest."

 

If this is the case then can I not define the buffer any larger.  There was a link to some trial code related to setting a timer to check buffer status.  I will see if I can get this code to work.

13 Nov 2009

James - I think you are having some of the same problems that I was.

 

Data is being sent to your serial port at some fixed rate, and you absolutely have to keep up, otherwise you will miss some of it.

So, your loop above is getting one byte, then going off to do something timeconsuming with the USB and LCD libs, then coming back to get the next byte. But it has taken too long to come back, and so missed a bunch of bytes.

A loop like this one:

char buffer[120];
for(int i=0; i<120; i++) {
    buffer[i] = megasquirt.getc();
}

will, I think, just about keep up with a 115200 baud serial stream, as long as nothing much else is going on in an interrupt handler.

 

The other mistake I made was to do too much work between asking for the data and starting to read it in, which meant I missed the beginning of the data. I see some wait(0.005) lines in your code, which might be responsible for something like that.

 

Other options might be:

- Running at a lower baud rate, if the megasquirt supports it. That's an easy experiment.

- Usign hardware handshake lines, if the megasquirt supports it.

 

Richard

16 Nov 2009

Richard,  Thank you for the reply.

The megasquirt system works by polling for data, followed by the return string of 113 bytes (0-112).  Unfortunately, the baud rate can not be changed as it is fixed by the ECU.

After modifying the code slightly based upon your suggestions, I can get no more than the first 30 bytes of the response. If I try to write to a data local data file I get only the first 16 bytes.  I know the program is executing up to a count of i = 120.  I can put test code lines to print the integer and everything works.  However, buffer placement does not occur past 30 bytes.

I can only assume there is a timing issue and I will have to continue and struggle through it.  the mbed should be easily capable of this task.  I am not sure what background processes are going on that prevent the buffer from loading.

16 Nov 2009

Hi Richard,

If possible, could you make a simple version of your program and publish it from the compiler, then just post the link in this thread. It'd be good to compare the code to the results you're seeing, and when you get stuck it is always a good way to help capture the problem (and often find the solution!).

I'll try and look at this later tomorrow if you can put something up, but hopefully someone else can have a look in the meantime.

Simon

18 Nov 2009

James - I've built a buffered serial reader to solve exactly this problem in my app. If you want to try it:

Import this project;

http://mbed.org/users/jarkman/published/80cd523739507c8bade52dfce149d900/SerialBufferedDemo.zip

Copy SerialBuffered.cpp and .h into your project.

Replace

Serial megasquirt(p28, p27);

with

SerialBuffered megasquirt(128, p28, p27);  // that 128 is the size of the read buffer it will make for you

Then, after

megasquirt.putc(6);

add this:

int actuallyRead = megasquirt.readBytes( buf, 120); // where 120 is the number of bytes you are expecting

After that, actuallyRead will contain the number of bytes that showed up before there was a timeout, or 120 if it got that far. And the bytes themselves will be in buf.

Good luck!

 

Richard

 

 

 

 

19 Nov 2009

Richard,  Thank you for the example code.  I just tried to load it.  I get a flag that line 15 in the SerialBuffered.cpp file "attach" is undefined.  Here is the line of code.

attach( this, &SerialBuffered::handleInterrupt );

I suspect that a definition might have been left out when you cleaned the code from your particular application.  Thanks again for the help solving this problem.

 

19 Nov 2009

James - you need to update your mbed library. Click on the 'mbed' line in your project, and it should offer you an update button. The attach method was only added a few days ago.

After that, all will be sweetness and light. Or, at least, you will have a fresh problem to chew on.. :-)

 

Richard

19 Nov 2009

Well, I should have known to update the header file.

On to find my new problem to chew on...  :)

 

19 Nov 2009

Found it.  I get a compile error I have not seen before.

 

" Undefined symbol loggerSerial (referred from main.o). (EL6218E)" in file "/"

 

I searched and did not find loggerSerial in any of the files.  Any suggestions?

19 Nov 2009 . Edited: 19 Nov 2009

Add this line to your main.cpp (should be global, i.e. before main() function) :

Serial loggerSerial(USBTX, USBRX);

Alternatively, you can remove all logging from SerialBuffered.cpp.

19 Nov 2009

thanks,  I just noticed that I had labled this connection as my usb connection.

I think I have everything working.  I can print selected pointers in the buffer.  Now I have to figure out how to print the entire buffer.

 

for example.

loggerSerial.printf("%d ", buf[27]);

works to print position 27 from the buffer.

loggerSerial.printf("%d ", buf[]); or loggerSerial.printf("%d ", buf);

do not work to print the entire contents of the buffer.  I want to dump the buffer to a file for storage.

 

Also, I want to add more information to the end of the buffer.  Is there a way to stack two buffers let's say (0 -n) and then (n+1 to m) and then print (0-m) onto a card?

Everyone has been most helpful. Thank you.

19 Nov 2009

When I try to rewrite the buffer contents to the local file or the hyperterminal I get the same character string.

"1073743072"

I have seen this before but do not know what it means.

19 Nov 2009 . Edited: 19 Nov 2009

Hi James,

James Novak wrote:
When I try to rewrite the buffer contents to the local file or the hyperterminal I get the same character string. "1073743072" I have seen this before but do not know what it means.

Here would be my hunch/deduction:

  • 1073743072 == 0x400004E0
  • The RAM on an LPC2368 starts at 0x40000000
  • You probably therefore have an mbed LPC2368
  • The value you are printing is a pointer to a memory location

For example:

char foo[64];            // an array
printf("%d", foo[0]);    // print the first value in the array 
printf("%d", foo);       // print the memory address of the array

Simon

20 Nov 2009

James - glad it's working.

 

You probably want to read a bit of an introduction to C programming, to make more sense of your buffers. You will not get far by experimenting with printf.

I'd suggest you want something like this:

for( int  i = 0; i < length_one; i ++ )

loggerSerial.printf("%d ", buffer_one[i] );

 

for( int  j = 0; j < length_two; j ++ )

loggerSerial.printf("%d ", buffer_two[ j ] );

 

20 Nov 2009

Simon - I like the simplicity of the existing Serial() implementation, but I do wonder if it might not be a little bit too hardcore for first-time users.

 

Its two vices - it hangs forever when you don't get the input you expect, and it drops bytes when you aren't reading quickly enough - are both pretty awkward to figure out from first principles.

 

Do you plan to build timeouts and buffers into the base implementaiton at some point ? I'm not trying to rush you , you understand. I'm sorted for now with the new interrupt. I'm just ruminating on the learning curve.

 

Richard

20 Nov 2009

Simon and Richard,

Thank you for your suggestions.  I am just beginning to learn C programming and embedded devices.  Fun and frustrating at the same time.

I have the device printing to the local file.  It took a bit of work to come up with a file format that could be imported into Excel.  It was the next line/CR that was a problem.  The next step is to move this over to the SD card.  I can get about 8Hz logging.  I would prefer at least 20Hz and will play with a few things to speed up timing.

 

I will try your suggestions for a dual buffer setup.

22 Nov 2009

James - I'm glad to hear it's working. Good luck with the speedup!

 

I'm curious, though. What are you building ?

 

R

23 Nov 2009

I now have the device logging to a micro-SD card.  Just by the switch in file location I went from 8Hz to 13 Hz.  Not too bad and probably acceptable.

 

Richard,  I am building an in-car datalogging system.  the system will log the engine's operating conditions from my EFI system (Megasquirt) and also vehicle dynamics using a 10Hz GPS and 3-axis accelerometer.  I have a track car (road course) and can not take a laptop to do all the functions.  I previously built an in-dash display for the EFI and have been trying to log the data simultaneously.  The problem I faced with expansion was integration of all the peripheral devices (acclerometer and GPS) that usually operate on a serial connection.  Most embedded MCUs do not have enough serial ports, nor operate a CAN bus for direct addressing.  A co-worker found this mbed device and sent the information my way.  So far, most everything has been very straightforward for my limited programming skills.

 

btw, I ended up using append to the SD file to put the extra data in place rather than a dual buffer.  Even with the dual buffer I would need to call two SD write functions.  .

 

23 Nov 2009

James - thanks - that's quite a project!

 

If you need more speedup, it may be worth trying to accumulate more data into buffers, so that you need to do fewer (but larger) write operations. That ought to be more efficient.

 

R