Overview

The lwIPSample1 demonstrates how to utilize the raw lwIP API set to create a simple TCP/IP based server application for the mbed device. The resulting server functionality is able to host a single piece of HTML which is hardcoded into the program as a constant C string. To verify the functionality of the sample, the user should be able to point their web browser at the device and see the following page rendered.

Attaching a terminal program to the mbed device via the USB based virtual serial port will show output summarizing the operations taking place on the device. This output includes the IP address that has been leased to the device via DHCP. Pointing your web browser at this IP address should show the webpage shown in the above image. If you need help getting a terminal connected to the mbed, you should check out this link and on Windows you will want to look at this link to learn more about the driver that will need to be installed.

lwIPSample1
Waiting for DHCP address...
IP     : 192.168.0.6
Gateway: 192.168.0.1
Mask   : 255.255.255.0
HTTP: Accepting client connection from: 192.168.0.4
HTTP: GET '/'
HTTP: Response send completed.
HTTP: Closing connection.
HTTP: Accepting client connection from: 192.168.0.4
HTTP: GET '/favicon.ico'
HTTP: File not found.
HTTP: Response send completed.
HTTP: Closing connection.

This sample program has minimal comments and error checking as it is just meant to help people see how to make use of the raw lwIP API set without getting bogged down in too much extraneous code. Therefore the code is not really appropriate for the use as a starting point for the development of other TCP/IP based applications but I hope that along with these notes, people will at least be able to read through the code, learn more about the lwIP API set and find it easier to start their own projects using this powerful lwIP stack directly. I should also note that while this example doesn't use the NetServices classes themselves, it does pull in Segundo Equipo's latest version of NetServices so that the sample can have access to the polled Ethernet driver and the lwIP network stack.

While implementing the HTTP protocol in the sample makes it a bit more complicated than necessary, I think it makes for a more interesting sample since people already know how to look at the content served up by such an example through the use of their favourite web browser.

Network Initialization

One of the first things to be done by the main() routine when this sample starts, is make a call to the NetworkInit() routine to initialize the network stack.

int main() 
{
    DigitalOut      ProgressLED(LED1);
    Timer           BlinkTimer;
    SNetwork        Network;
    
    printf("\r\nlwIPSample1\r\n");
    
    // Initialize the ethernet driver and lwIP network stack.
    NetworkInit(&Network);

    ...
}

The state of the network stack to be initialized is maintained in the Network variable which is local to the main routine. This variable doesn't need to be global as it only needs to exist for as long as the main() routine is running and a pointer to it will be passed into the NetworkInit(), HTTPInit() and NetworkPoll() routines. This variable is of type struct SNetwork:

struct SNetwork
{
    // Listening port for HTTP Server.
    tcp_pcb*        pHTTPListenPCB;
    // Timers used to determine when various network maintenance tasks should
    // be executed.
    Timer           ARP;
    Timer           DHCPCoarse;
    Timer           DHCPFine;
    Timer           TCP;
    Timer           DNS;
    // Object used to track the DHCP state.
    dhcp            DHCP;
    // The structure used to represent the LPC17xx polled Ethernet interface.
    netif           EthernetInterface;
};

NetworkInit()

I based the code in NetworkInit() on outdated documentation I found on the lwIP wiki and the code that already exists in the NetServices to bring up the network stack for the mbed device. I haven't fully analyzed what all of this network initialization code does yet so my description will be minimal. Its main responsibilities include:

• Zeroing out the SNetwork structure to ensure that everything has a known initial value before we start the process of bringing up the network stack.
• Call lwip_init() to let the lwIP network stack to prepare for accepting its first network interface.
• Initialize the EthernetInterface field from the SNetwork structure to represent the polled ethernet driver for the LPC17xx that exists in the eth_drv.h & eth_drv.cpp sources of the NetServices library. The eth_init() routine pointer passed into the netif_add() call will cause this interface to be initialized and the netif_add() call itself will hook this ethernet driver into the lwIP network stack's list of known network interfaces.
• Initialize a number of timers. These timers are going to be checked in the NetworkPoll() routine and when they timeout, they will call into appropriate parts of the lwIP network stack which need to be called at defined intervals.
• Setup the dhcp structure and start the process of attempting to get a lease on an IP address from the DHCP server by calling dhcp_start().

static void NetworkInit(SNetwork* pNetwork)
{
    ip_addr_t   IpAddress;
    ip_addr_t   NetMask;
    ip_addr_t   GatewayAddress;

    // Clear out the network object and fill it in during the initialization
    // process.
    memset(pNetwork, 0, sizeof(*pNetwork));
    
    // Initialize the lwIP network stack.
    lwip_init();
    
    // Clear the IP addresses and use DHCP instead
    ip_addr_set_zero(&IpAddress);
    ip_addr_set_zero(&NetMask);
    ip_addr_set_zero(&GatewayAddress);
    
    // Obtain the MAC address for the Ethernet device.
    // NOTE: This should really be done in eth_init()
    pNetwork->EthernetInterface.hwaddr_len = ETHARP_HWADDR_LEN;
    eth_address((char *)pNetwork->EthernetInterface.hwaddr);    
    
    // Connect the LPC17xx ethernet driver into the lwIP stack.
    netif_add(&pNetwork->EthernetInterface, 
              &IpAddress, 
              &NetMask, 
              &GatewayAddress, 
              NULL, 
              eth_init, 
              ethernet_input);
    netif_set_default(&pNetwork->EthernetInterface);
    
    // Start the required timers.
    pNetwork->ARP.start();
    pNetwork->DHCPCoarse.start();
    pNetwork->DHCPFine.start();
    pNetwork->TCP.start();
    pNetwork->DNS.start();
    
    // Start the DHCP request.
    pNetwork->EthernetInterface.hostname = "MBED";
    dhcp_set_struct(&pNetwork->EthernetInterface, &pNetwork->DHCP);
    dhcp_start(&pNetwork->EthernetInterface);

    // Wait for DHCP request to complete.
    Timer timeout;
    timeout.start();
    printf("Waiting for DHCP address...\r\n");

    // Wait until interface is up
    while (!netif_is_up(&pNetwork->EthernetInterface)) 
    {
        NetworkPoll(pNetwork);

        // Stop program if we get a timeout on DHCP attempt.
        if (timeout.read_ms() > 10000) 
        {
            error("DHCP Timeout.\r\n");
        }
    }

    // Print out the DHCP provided IP addresses.
    printf("IP     : "); 
    NetworkPrintAddress(&pNetwork->EthernetInterface.ip_addr);
    printf("\r\n");

    printf("Gateway: "); 
    NetworkPrintAddress(&pNetwork->EthernetInterface.gw);
    printf("\r\n");

    printf("Mask   : "); 
    NetworkPrintAddress(&pNetwork->EthernetInterface.netmask);
    printf("\r\n");
}

NetworkPoll()

Once the DHCP process has been started, the lwIP network stack has to actually be allowed to run so that packets can be pumped in and out of the ethernet port. To allow this to happen, the code starts calling the NetworkPoll() routine in a loop (which will timeout in 10 seconds due to the use of the timeout object.)

static void NetworkPoll(SNetwork* pNetwork)
{
    if (pNetwork->DHCPFine.read_ms() >= DHCP_FINE_TIMER_MSECS)
    {
        pNetwork->DHCPFine.reset();
        dhcp_fine_tmr();
    }
    if (pNetwork->DHCPCoarse.read() >= DHCP_COARSE_TIMER_SECS)
    {
        pNetwork->DHCPCoarse.reset();
        dhcp_coarse_tmr();
    }
    if (pNetwork->ARP.read_ms() >= ARP_TMR_INTERVAL)
    {
        pNetwork->ARP.reset();
        etharp_tmr();
    }
    if (pNetwork->TCP.read_ms() >= TCP_TMR_INTERVAL)
    {
        pNetwork->TCP.reset();
        tcp_tmr();
    }
    if (pNetwork->DNS.read_ms() >= DNS_TMR_INTERVAL)
    {
        pNetwork->DNS.reset();
        dns_tmr();
    }

    // Poll the ethernet driver to let it pull packets in and push new packets
    // into the lwIP network stack.
    eth_poll();
}

The NetworkPoll() routine first takes care of checking the mbed Timer objects that were setup in NetworkInit(). I am going to skip a discussion of most of these timers but one interesting one will actually show up later in our discussion of the HTTP server application code. The TCP timer object in SNetwork is used to determine when tcp_tmr() should be called. This timer will call the tcp_tmr() routine in lwIP every TCP_TMR_INTERVAL milliseconds. TCP_TMR_INTERVAL is set to 250 milliseconds in the lwIP tcp_impl.h header file. tcp_tmr() will call tcp_fasttmr() every 250 milliseconds and then it will call tcp_slowtmr() every second iteration or every 500 milliseconds (0.5 seconds.) One of the actions of the tcp_slowtmr() routine is to call the poll callbacks that have been configured by the application for any of its TCP/IP connections. We will later see that the HTTP server sets its HTTPPoll() routine to be called on every 4 invocations of the tcp_slowtmr() routine which would result in a polling interval of 2 seconds. This will be used for error recovery incase we run out of memory in an attempt to send data back to the client or an attempt to cleanly close down the connection.

The last thing done by NetworkPoll() also happens to be most important. It calls the eth_poll() routine which is implemented in eth_drv.cpp. This routine actually checks the ethernet peripheral on the mbed device to see if there are any inbound packets that need to be processed. If there are packets to be processed, then they will be sent into the lwIP network stack by a call to ethernet_input(). This means that the call graph for how lwIP actually gets called to process DHCP related traffic during the initialization process will look like this:

main() -> NetworkInit() -> NetworkPoll() -> eth_poll() -> ethernet_input()

Visual Hang Detector

In addition to the initialization of the Network variable, the main() routine also initializes the ProgressLED and BlinkTimer objects to be used for purpose of blinking LED1 in the main loop. This allows the user to determine whether the program has hung by simply looking at the mbed board to make sure that this LED is still blinking. If the LED stops blinking then something bad has probably happened. Within the main loop, it switches the state of LED1 every 500 milliseconds (0.5 seconds).

int main() 
{
    ...
    HTTPInit(&Network);
                
    BlinkTimer.start();
    while(1) 
    {
        NetworkPoll(&Network);

        if (BlinkTimer.read_ms() > 500)
        {
            BlinkTimer.reset();
            ProgressLED = !ProgressLED;
        }
    }
}

At this point, we have discussed how all of the fields except for pHTTPListenPCB in the Network variable have been initialized. This field is the only one specific to the HTTP functionality that we are implementing in this sample. Let's move on and start looking at how the call to HTTPInit() from main() gets the HTTP server initialized to use the lwIP network stack.

HTTP Listening on port 80

When you type an address like http://mbed.org into your browser, it is actually trying to connect to port 80 on the server at mbed.org. So to get our little web server up and running, we need to get it listening on port 80 and prepare it for accepting connections from web browsers out on the network. This is the code used to do this in the sample:

static void HTTPInit(SNetwork* pNetwork)
{
    tcp_pcb*    pcb = NULL;
    
    pcb = tcp_new();
    tcp_bind(pcb, IP_ADDR_ANY, 80);
    pNetwork->pHTTPListenPCB = tcp_listen(pcb);
    
    tcp_arg(pNetwork->pHTTPListenPCB, pNetwork);
    tcp_accept(pNetwork->pHTTPListenPCB, HTTPAccept);
}

tcp_new()

The first part of the process is to create a tcp_pcb structure through a call to tcp_new(). If you are accustomed to Berkeley socket programming then you can think of these TCP protocol control block structures as sockets. When you are using the socket interface on other systems, the socket is represented as an integer but most likely the operating system is using a very similar structure to these tcp_pcb structures under the covers to represent the actual socket connection. Since lwIP is meant to be light weight, it discards this extra level of indirection and has the application directly use pointers to these tcp_pcb structures. The tcp_new() routine will return a NULL pointer if it failed to allocate a tcp_pcb structure for your application to use. This could happen if there are no more tcp_pcb items left in its memory pool. The current NetServices library allocates space for 3 tcp_pcb items in its pool.

tcp_bind()

Every connection in TCP/IP is represented by a unique combination of 4 items: local IP address, local TCP port number, remote IP address, and remote TCP port number. The tcp_pcb that was just allocated is pretty much a clean slate and it doesn't even know what IP address and port number it should be assigned to on the local mbed device. The next step is to call tcp_bind(pcb, IP_ADDR_ANY, 80) which will associate this tcp_pcb structure with the specified IP address and TCP port 80 on the local device. The IP_ADDR_ANY parameter indicates that this tcp_pcb should listen to port 80 connections on all network interfaces known by lwIP. Since this sample has only called netif_add() once in NetworkInit(), it will only be listening on the single mbed polled ethernet device.

tcp_listen()

Now that the tcp_pcb is bound to port 80 on the local device, we need to have it start listening for new connections on this port. This is done in lwIP with a call to tcp_listen(pcb). This places the bound tcp_pcb into the listening state. The internals of the lwIP stack make it such that a smaller PCB structure can be used for connections that are in the listening state. Since lwIP is meant to be used on small devices, every byte of memory counts. The tcp_listen() routine actually allocates a smaller tcp_pcb_listen structure, copies the required information from the original tcp_pcb structure into the smaller structure, and then frees the original tcp_pcb structure. The smaller tcp_pcb_listen structures are allocated from a different memory pool than the tcp_pcb items. The current NetServices library allocates space for 2 tcp_pcb_listen items in this pool. Now that we have returned the original tcp_pcb back to its pool and allocated a tcp_pcb_listen in its place, we now have 3 items available in the tcp_pcb pool for accepting connections from HTTP clients and 1 item available in the tcp_pcb_listen pool for another server to use.

pcb = tcp_listen(pcb);

listen_pcb = tcp_listen(pcb);
if (NULL == list_pcb)
{
    tcp_close(pcb);
    
    // Have application handle failure.
}
else
{
    pcb = listen_pcb;
}

tcp_arg() and tcp_accept()

Now that the tcp_pcb is listening on port 80 for client connections, how will the HTTP server application know when such connections are made? The final step in the HTTPInit() routine is to setup the callback for accepting connection attempts on the listening tcp_pcb. The tcp_accept() routine is called to associate the HTTPAccept() callback routine with the listening tcp_pcb. Now when a connection comes in on the listening tcp_pcb, it will know to call this HTTPAccept() routine. This HTTPAccept() routine has the following prototype:

err_t HTTPAccept(void* pvArg, tcp_pcb* pNewPCB, err_t err);

The first parameter is an argument pointer that has meaning to the HTTP application. The HTTPInit() routine calls tcp_arg() to set what value should be passed in as the pvArg to HTTPAccept() when connections are made. The lwIP network stack just passes this pvArg through to the callback routine, while completely ignoring it itself. The pNetwork pointer gets used in the tcp_arg() call for this sample since it will provide the HTTPAccept() routine access to various pieces of application state, including the pHTTPListenPCB for the listening 'socket'. The next section of the document will discuss how this HTTPAccept() callback gets called and what it does in response to connection attempts.

Accepting Client Connections

Now that the main() routine has finished calling NetworkInit() and HTTPInit(), it enters its infinite loop, calling NetworkPoll() and blinking LED1. What is going to happen when a web browser attempts to connect to port 80 on this mbed device? We have setup an accept callback on the listening tcp_pcb but how does it get called? It all starts with the calls to NetworkPoll() in the main() infinite loop. As discussed before, this is going to make calls into eth_poll() which will actually poll the mbed ethernet peripheral to see if there are any inbound packets for it to process? If so, it reads out the contents of the packets and then passes it into the lwIP network stack by calling ethernet_input(). This packet will get passed up through the layers of the lwIP TCP/IP network stack until it realizes that the packet indicates a new connection attempt to TCP port 80. lwIP will search its list of listening tcp_pcb_listen connections and find the one which we created in the HTTPInit() routine. Once this tcp_pcb_listen is found, it will call the HTTPAccept() routine that was set as a callback for such accept events. This means that the call graph to HTTPAccept() will look something like this:

main()->NetworkPoll()->eth_poll()->ethernet_input()->ip_input()->tcp_input()->tcp_process()->HTTPAccept()

The ethernet packet enters lwIP at the call to ethernet_input() and leaves lwIP at the call to our HTTPAccept() routine. In between, we can see where each layer of the lwIP network stack (ethernet, ip, tcp) looks at information in its headers and passes it along to the next layer up based on what it finds in the header.

Now that we see how the application's HTTPAccept() callback routine gets called, lets start looking at the actual implementation of this routine to see how it manages to accept the new HTTP client connection and prepares for receiving of the HTTP request.

static err_t HTTPAccept(void* pvArg, tcp_pcb* pNewPCB, err_t err)
{
    SNetwork*       pNetwork = (SNetwork*)pvArg;
    SHTTPContext*   pHTTPContext = NULL;
    
    printf("HTTP: Accepting client connection from: ");
    NetworkPrintAddress(&pNewPCB->remote_ip);
    printf("\r\n");
    
    tcp_accepted(pNetwork->pHTTPListenPCB);

    pHTTPContext = (SHTTPContext*)malloc(sizeof(*pHTTPContext));
    memset(pHTTPContext, 0, sizeof(*pHTTPContext));
    tcp_arg(pNewPCB, pHTTPContext);
    
    tcp_recv(pNewPCB, HTTPRecv);
    tcp_sent(pNewPCB, HTTPSent);
    tcp_err(pNewPCB, HTTPError);
    tcp_poll(pNewPCB, HTTPPoll, 4);
    
    return ERR_OK;
}

HTTPAccept() Parameters

To start with, we will look at the parameters that are passed into the HTTPAccept() callback.

pvArg

This is the value that was set by the call to tcp_arg() in the HTTPInit() routine. In this sample it will be set to the SNetwork object from main().

pNewPCB

This tcp_pcb structure is allocated by the lwIP network stack to represent the new incoming client connection. This tcp_pcb would have been allocated from a pool in which NetServices reserves space for 3 such items. This means that if more than 3 clients try to connect to the HTTP server at the same time, this tcp_pcb allocation will fail and no ACK is sent back to the remote client. To the client, this makes it look like its connection attempt was lost by the network so it should try to establish the connection again, at which time we might have freed up a tcp_pcb to the pool.

Since this tcp_pcb structure represents an actual connection from port 80 on the mbed to a port on some remote machine, the remote machine's information is available to the HTTP server application in this structure. The NetworkPrintAddress(&pNewPCB->remote_ip) call takes advantage of this fact to display the IP address of the remote machine to which the connection is being made.

err

This sample ignores the err parameter to this callback routine. When looking through the lwIP sources, it does appear that when this callback is made from the tcp_process() routine, the err parameter is hardcoded to ERR_OK anyway.

Hello lwIP, We Accept the Charges

One of the first things done by the HTTPAccept() callback is to make the following call:

tcp_accepted(pNetwork->pHTTPListenPCB);

It should be noted that it is making the tcp_accepted() call on the pointer to the tcp_pcb which is listening on port 80 and not the pNewPCB which was passed in to represent the newest connection to this port. This call to tcp_accepted lets the lwIP stack know that the application has accepted this connection.

How to Track State for each Client

Server applications will typically want to keep track of a certain amount of information for each client that connects to it. In the case of HTTP, the server needs to track the state for the decoding of the web browser request and the state of the data stream being sent back to the browser. This state is tracked in a SHTTPContext structure.

struct SHTTPContext
{
    // Pointer to response text being sent back to client.
    const char*     pResponseCurr;
    // Number of bytes in response left to be sent back to client.
    size_t          ResponseBytesLeft;
    // Have we starting receiving the request yet?
    int             RequestStarted;
    // Have we finished receiving the request yet?
    int             RequestCompleted;
    // Number of outbound bytes outstanding, awaiting acknowledgement.
    int             UnacknowledgedBytes;
    // Number of tcp_write() retries attempted so far.
    int             RetryCount;
    // Buffer used to track the last four characters read when searching for
    // request terminator.
    char            LastFourChars[4];
};

The HTTPAccept() routine allocates one of these SHTTPContext structures, and assigns the pointer to the pHTTPContext local variable. The pointer to this context is then zero filled and associated with the new client tcp_pcb through the call to tcp_arg(), similar to what we saw in HTTPInit() earlier.

More callbacks

The only thing left for HTTPAccept() to do for this connection is to associate callbacks to the new tcp_pcb structure. These callbacks will let lwIP know what it should do when events occur on this connection in the future. The following table summarizes the callback associations that are made for this tcp_pcb and summarizes when each will be called.

Once the callbacks have been set on the tcp_pcb, the HTTP_Accept() routine returns with ERR_OK to let the lwIP stack know that everything was successful.

HTTPRecv()

This callback routine really takes care of a lot of the work that needs to be done by a HTTP server application. It is called as HTTP request data is received from the remote client (web browser), parses this request, attempts to open the 'file' being requested by the client, and starts sending the requested file contents (or an error response if the request or filename wasn't recognized) back to the remote client. The lwIP network stack will invoke this callback routine as part of the polling process whenever an inbound packet arrives for this connection. It can actually be called multiple times for a single request since the request from the client might be too long to fit in one TCP message segment.

static err_t HTTPRecv(void* pvArg, tcp_pcb* pPCB, pbuf* pBuf, err_t err)
{
    SHTTPContext* pHTTPContext = (SHTTPContext*)pvArg;
    
    if (!pBuf)
    {
        return ERR_OK;
    }
    
    tcp_recved(pPCB, pBuf->tot_len);
    
    if (pHTTPContext->RequestStarted)
    {
        // Check for the blank line terminator if we haven't seen it yet.
        if (!pHTTPContext->RequestCompleted)
        {
            pHTTPContext->RequestCompleted = 
                            HTTPContainsRequestTerminator(pBuf, pHTTPContext);
        }
    }
    else
    {
        // Start parsing the request.
        const char* pURI = NULL;
        int         ParseResult = 0;
        
        ParseResult = HTTPParseRequest(pBuf, &pURI);
        if (ParseResult & HTTP_PARSE_GET)
        {
            printf("HTTP: GET '%s'\r\n", pURI);
            pHTTPContext->pResponseCurr = 
                            HTTPOpenFile(pURI, 
                                         &pHTTPContext->ResponseBytesLeft);
            pHTTPContext->RequestCompleted = 
                            HTTPContainsRequestTerminator(pBuf, pHTTPContext);
        }
        else if (ParseResult & HTTP_PARSE_NOT_IMPLEMENTED)
        {
            static const char UnknownResponse[] = 
                "HTTP/1.0 501 Not Implemented\r\n"
                 HTTP_SERVER_NAME
                 "Content-type: text/html\r\n"
                 "\r\n"
                 "<html><body><h2>501: Not Implemented.</h2></body></html>"
                 "\r\n";
            
            printf("HTTP: Command not implemented.\r\n");
            pHTTPContext->pResponseCurr = UnknownResponse;
            pHTTPContext->ResponseBytesLeft = sizeof(UnknownResponse) - 1;
            pHTTPContext->RequestCompleted = 
                            HTTPContainsRequestTerminator(pBuf, pHTTPContext);
        }
        else if (ParseResult & HTTP_PARSE_BAD_REQUEST)
        {
            printf("HTTP: Bad Request.\r\n");
            HTTPCloseConnection(pPCB, pHTTPContext);
        }

        pHTTPContext->RequestStarted = 1;        
    }

    HTTPSendResponse(pPCB, pHTTPContext);

    pbuf_free(pBuf);
    
    return ERR_OK;
}

HTTPRecv() Parameters

Let's look at the parameters that are passed into the HTTPRecv() callback.

pvArg

This will point to the SHTTPContext structure that was allocated and associated with the client connection in the HTTPAccept() routine.

pPCB

This will point to the tcp_pcb structure for the TCP/IP connection which is receiving the HTTP client request.

pBuf

The pbuf structure represents a linked list of data that was received from the HTTP client. I will discuss this data structure in more detail below.

err

The sample ignores this err parameter.

What is the pbuf structure?

What is this pbuf structure that is passed into the HTTPRecv() callback? This data structure represents the incoming data from the HTTP client and has the following format:

struct pbuf {
  /** next pbuf in singly linked pbuf chain */
  struct pbuf *next;

  /** pointer to the actual data in the buffer */
  void *payload;

  /**
   * total length of this buffer and all next buffers in chain
   * belonging to the same packet.
   *
   * For non-queue packet chains this is the invariant:
   * p->tot_len == p->len + (p->next? p->next->tot_len: 0)
   */
  u16_t tot_len;

  /** length of this buffer */
  u16_t len;

  /** pbuf_type as u8_t instead of enum to save space */
  u8_t /*pbuf_type*/ type;

  /** misc flags */
  u8_t flags;

  /**
   * the reference count always equals the number of pointers
   * that refer to this pbuf. This can be pointers from an application,
   * the stack itself, or pbuf->next pointers from a chain.
   */
  u16_t ref;
};

Why such a complicated structure to represent what should be a simple array of bytes? This pbuf structure offers a lot of versatility to the lwIP network stack and application so that they have the possibility to maximize the use of memory and reduce the number of times that data has to be copied on small devices. The first thing you will notice when looking at this data structure is that it is actually implemented as a singly linked list. The HTTPRecv() routine is passed the head node of this linked list. If this head node is the only node in the linked list, then pBuf->next will be NULL. However, it is possible that the data is too large to fit into one pbuf node and in which case the pBuf->next pointer will point to the next chunk of data. The example below shows a HTTP request which is split across two pbuf nodes.

Typically the application will call pbuf_free() to decrement the reference count. In the example figure above, the reference count of the inbound pbuf list nodes is set to 1. This means that when the application calls pbuf_free(), the count will drop to 0 which will cause the memory for the nodes in this list to actually be returned to the memory pool and re-used for future allocations. If the application wanted to keep the data around after it returns from this HTTPRecv() routine, it could actually skip this pbuf_free() call, continue to use the data again later in other callbacks or the main loop, and then call pbuf_free later when it was truly done with it. This has the advantage that the application doesn't need to take a copy of the data that it wants to keep around for awhile longer but it does mean that this memory won't be returned to the pool for use by subsequent network packet handling.

The payload field of the pcb structure actually provides the pointer to the data for a node in the pbuf list. The amount of valid data in the payload buffer which can be used by the application is indicated by the byte count in the len field for the corresponding pbuf node. The tot_len indicates that sum of bytes in this node and all of those in the subsequent nodes of the linked list. This means that the tot_len in the head node will indicate the size of all the node payloads in the pbuf linked list.

The payload pointer in the pbuf node could point to any valid memory address on the mbed device but in reality, the pbuf type field will give you an indication of where this data will actually be located. For the most part, an application can ignore the location of this data and just use the payload and len fields of the pbuf structure to read in the data sent to it by a remote machine. However, it is very handy to know a bit about the different types of pbuf nodes supported by lwIP and get a better appreciation for how it tries to make optimal use of memory.

PBUF_POOL

These nodes are actually allocated from a pool that has been reserved within the lwIP network stack. The current NetServices library reserves 8 elements in this pool where each node can contain 592 bytes of payload data. Each element in the pool contains enough memory for this 592 bytes of payload data and the fields of the pbuf structure itself. This means that the payload field in the pbuf node will actually point to an address which directly follows the pbuf structure itself. The figure above shows two PBUF_POOL nodes and their payload pointers are actually pointing to memory which immediately follows the pbuf structure.

PBUF_RAM

These nodes are similar to the PBUF_POOL nodes, in that they too have their pbuf structure fields allocated alongside their actual payload data. However, pbuf nodes of this type aren't allocated from the pool but from a lwIP heap and typically the payload area will just be large enough for the data to be used for this node whereas the pool items reserve a fixed amount of space for each node (592 bytes). Later when we discuss the calls to tcp_write() for sending data with the TCP_WRITE_FLAG_COPY, it will be pbuf nodes of this type that lwIP allocates for the provided data to be copied into and sent down to the ethernet driver.

PBUF_ROM

These nodes are also allocated from a pool that has been reserved within the lwIP network stack. The current NetServices library reserves 16 elements in this pool where each node only contains enough memory to store the fields of the pbuf structure itself. The payload field will point off to some other memory location. This type of pbuf node would typically be used for tcp_write() calls when the TCP_WRITE_FLAG_COPY flag isn't specified. This helps to both conserve memory since the lwIP network stack can use the same memory as the application and it removes the CPU time it takes to copy the data into a PBUF_RAM node (however the CPU does still need to read the data to calculate the checksum required by TCP). Even in this scenario, the lwIP network stack will probably have to allocate some PBUF_RAM nodes to include in the pbuf list sent down to the ethernet driver since it needs to include ethernet, IP, and TCP headers in the packet. The cool thing about the linked list nature of the pbuf structure is that it is easy for the network stack to mix nodes of multiple types like this in a single packet of data to be sent and make the most of the limited memory resources of the mbed.

PBUF_REF

Nodes of this type are treated almost exactly the same as those of type PBUF_ROM except that the payload pointer is expected to be pointing off to RAM addresses instead of ROM addresses.

Check for NULL pbuf

The inbound pbuf data structure pointer can be passed in as NULL to the HTTPRecv() routine. I have seen this happen when the connection is forcibly closed by the remote client machine while the response data is being sent back for larger responses. There is nothing for this callback to do if no actual pbuf request data was received so return with ERR_OK.

Call to tcp_recved()

One of the first things that HTTPRecv() does is to call the tcp_recved() routine to let lwIP know that the application has received this packet. This lets the network stack know that it can advance the receive TCP window for this connection.

Free pbuf

Towards the end of the HTTPRecv() routine, there is another important call to the lwIP pbuf_free() routine. The most recently received pbuf list isn't required outside of this routine so it should be freed before returning from the callback.

Parsing Request

Now the HTTP server can start processing the HTTP request data that it has just received in the pbuf. If it is the first packet to be received for this connection then the pHTTPContext->RequestStarted will be 0. If it was set to 0, then HTTPRecv() will call the HTTPParseRequest() routine to parse the initial segment string to determine what the HTTP server is being requested to do. If it was set to 1, then the initial packet was already received and parsed to determine what command was being requested so the only thing left to do is to call HTTPContainsRequestTerminator() if the request terminator (a blank line in the request) wasn't encountered yet. The server code needs to know that it has finished receiving the complete request so that it doesn't send a short response back after parsing the initial segment and end up closing the connection before the server even finishes sending the request. This can actually happen when the client sends a large number of headers and we are sending short HTML files back (like this sample does.) The remote client will think that something has gone wrong if this pre-mature disconnect happens and it will indicate that it has had problems connecting to the specified server. We will later see how the pHTTPContext->RequestCompleted field is checked in HTTPSendResponse() so that it knows to delay the HTTPCloseConection() call if the full request hasn't been received yet.

HTTPParseRequest()

I won't go into much detail about how the HTTPParseRequest routine works as it is not directly related to lwIP but I will note a few things about the implementation:

static int HTTPParseRequest(pbuf* pBuf, const char** ppData)
{
    int         Return = HTTP_PARSE_BAD_REQUEST;
    u16_t       CharsLeft = pBuf->len;
    const char* pURI = NULL;
    
    if (CharsLeft >= 4)
    {
        // pbuf is large enough to contain command so attempt to parse it.
        if (0 == memcmp(pBuf->payload, "GET ", 4))
        {
            // This is GET request so search for next whitespace character as it
            // will terminate the URI.
            char* pCurr = (char*)pBuf->payload + 4;
            CharsLeft -= 4;
            pURI = pCurr;
                   
            while(CharsLeft > 0 &&
                  ' '  != *pCurr &&
                  '\r' != *pCurr &&
                  '\n' != *pCurr)
            {
                pCurr++;
                CharsLeft--;
            }
            
            // Have a valid URI if whitespace was found
            if (CharsLeft > 0)
            {
                if (' ' == *pCurr)
                {
                    // Protocol string follows URI in request so must be
                    // HTTP/1.0 or HTTP/1.1
                    Return = HTTP_PARSE_GET;
                }
                else
                {
                    // Protocol string didn't follow URI so request must be
                    // from older v0.9 client which this sample doesn't
                    // support.
                    Return = HTTP_PARSE_BAD_REQUEST;
                }
                
                // Null terminate the URI at the whitespace separator
                *pCurr = '\0';
                
                // Return a pointer to where the URI begins
                *ppData = pURI;
            }
        }
        else
        {
            Return = HTTP_PARSE_NOT_IMPLEMENTED;
        }
    }
        
    return Return;
}

• It is only used to parse the first pbuf node of the first packet received by the network stack. This means that the whole GET request, including the name of the file being requested, must fit in 536 bytes. If it doesn't then it will return HTTP_PARSE_BAD_REQUEST.
• It only recognizes GET requests. It will return HTTP_PARSE_NOT_IMPLEMENTED if the pbuf contains 4 or more character but it doesn't start with "GET ".
• It will also return a bad request error if a HTTP/0.9 request is made where the filename,URI, isn't followed by a space and then the HTTP protocol version string.
• It returns a bit mask instead of an enumeration as I was originally planning to have it support HTTP/0.9 requests but it made the code more complicated than it needed to be for a lwIP sample.
• If it finds a valid GET request, then it will set *ppData to the beginning of the URI string and the terminating space character will be changed to a NULL terminator. This means that the caller can use the pointer to issue file open calls.

HTTPContainsRequestTerminator()

This routine is also specific to the HTTP protocol so I will just make a few notes about its implementation as well and not go into detail.

static int HTTPContainsRequestTerminator(pbuf* pBuf, SHTTPContext* pHTTPContext)
{
    static const char   RequestTerminator[4] = { '\r', '\n', '\r', '\n' };
    
    // Loop through all chunks in the pbuf list.
    while (pBuf)
    {
        char* pCurr = (char*)pBuf->payload;
        u16_t len = pBuf->len;
        
        // Loop through the characters in this chunk.
        while (len)
        {
            // Shift the current byte into the buffer.
            memcpy(pHTTPContext->LastFourChars, 
                   pHTTPContext->LastFourChars+1, 
                   3);
            pHTTPContext->LastFourChars[3] = *pCurr;
            
            // Have we encountered the blank line request terminator.
            if (0 == memcmp(pHTTPContext->LastFourChars, 
                            RequestTerminator, 
                            sizeof(RequestTerminator)))
            {
                return 1;
            }
            
            // Advance to the next character in this chunk.
            pCurr++;
            len--;
        }
        
        // Advance to the next chunk in the pbuf list.
        pBuf = pBuf->next;
    }
    
    // If we get here then we didn't find the blank line terminator.
    return 0;
}

• The two levels of nested loops show an example of how to walk all of the characters in a pbuf list. The outer loop walks the nodes in the linked list and the inner loop walks through the bytes in the payload of each node.
• This code actually keeps the last four characters seen in pHTTPContext->LastFourChars field of the HTTP client context rather than a local variable. This allows for the code to find a blank line even if it spans pbuf nodes or calls to the HTTPRecv() routine.
• The callers of this routine use it to set the value of the pHTTPContext->RequestCompleted field in the HTTP client context.
• It will be called for the first invocation of HTTPRecv() and every subsequent call until the terminating blank line is found in the pbuf list.
• The only scenario that it is never called is when a bad request is received. In that scenario, no response is sent by the sample and it instead closes the connection.

Handling of GET Request

If HTTPParseRequest() returns HTTP_PARSE_GET then pURI will have been set to the name of the file being requested. This filename can be passed into the HTTPOpenFile() routine in an attempt to open the desired file. The pHTTPContext->pResponseCurr and pHTTPContext->ResponseBytesLeft fields of the HTTP client context will be initialized for the HTML text which should be returned to the client. This sample is written such that the responses will always be short constant strings that are stored in the FLASH of the device. As data is successfully written to the connection, the pResponseCurr pointer field can be advanced to indicate the data left to be sent and the ResponseBytesLeft can be decremented by a corresponding amount. This will continue until the ResponseBytesLeftCount decreases to 0 and there is no more data to be sent.

HTTPOpenFile()

This simple routine returns one of two constant string responses: one if the requested filename is "/" or "/index.html" and another if any other filename was requested. These responses includes the HTTP headers, the blank line header termination indicator, and then the HTML content itself. The pFileSize parameter will be set to the sizeof() the returned string minus 1 since the NULL terminator isn't sent to the HTTP client.

static const char* HTTPOpenFile(const char* pURI, size_t* pFileSize)
{
    static const char HelloWorldPage[] = 
        "HTTP/1.0 200 OK\r\n"
        HTTP_SERVER_NAME
        "Content-Type: text/html\r\n"
        "\r\n"
        "<html>"
        "<head><title>Hello World</title></head>"
        "<body>"
        "<h1>mbed + lwIP = Internet of Things</h1>"
        "</body>"
        "</html>\r\n";
    static const char FileNotFoundPage[] = 
        "HTTP/1.0 404 File not found\r\n"
        HTTP_SERVER_NAME
        "Content-Type: text/html\r\n"
        "\r\n"
        "<html>"
        "<head><title>404 - File not found</title></head>"
        "<body><h2>404: File not found.</h2></body>"
        "</html>\r\n";
                                          
    // Either return the main page if clients asks for "/" or "/index.html"
    // Otherwise return the 404 File Not Found page.
    if (0 == strcmp(pURI, "/") ||
        0 == strcmp(pURI, "/index.html"))
    {
        *pFileSize = sizeof(HelloWorldPage) - 1;
        return HelloWorldPage;
    }
    else
    {
        printf("HTTP: File not found.\r\n");
        *pFileSize = sizeof(FileNotFoundPage) - 1;
        return FileNotFoundPage;
    }
}

HTTPSendResponse()

The HTTPRecv() routine always calls HTTPSendResponse() just before it frees the pbuf and returns back to lwIP. This routine is responsible for sending as much of the response back to the remote machine as possible and then attempting to close the connection if all of the data has been successfully written to the tcp_pcb and the full request has been received from the remote client. There are cases where this routine will be called with 0 bytes to send and in which case the code will just attempt to close the connection if the request has been completely received as well.

static void HTTPSendResponse(tcp_pcb* pPCB, SHTTPContext* pHTTPContext)
{
    size_t BytesToSend = pHTTPContext->ResponseBytesLeft;
    err_t  WriteResult = ERR_MEM;
    
    if (BytesToSend > tcp_sndbuf(pPCB))
    {
        BytesToSend = (size_t)tcp_sndbuf(pPCB);
    }
    
    while (BytesToSend > 0)
    {
        WriteResult = tcp_write(pPCB, 
                                pHTTPContext->pResponseCurr, BytesToSend, 
                                TCP_WRITE_FLAG_COPY);
        if (ERR_OK == WriteResult)
        {
            pHTTPContext->pResponseCurr += BytesToSend;
            pHTTPContext->ResponseBytesLeft -= BytesToSend;
            pHTTPContext->UnacknowledgedBytes += BytesToSend;
            break;
        }
        else if (ERR_MEM == WriteResult)
        {
            if (0 == tcp_sndqueuelen(pPCB))
            {
                break;
            }
            BytesToSend >>= 1;
        }
        else
        {
            // Received unexpected error.
            break;
        }
    }
    
    if (0 == pHTTPContext->ResponseBytesLeft && pHTTPContext->RequestCompleted)
    {
        printf("HTTP: Response send completed.\r\n");
        HTTPCloseConnection(pPCB, pHTTPContext);
    }
}

The application uses the tcp_write() routine to queue data for sending back to the remote machine. It's not possible for an application to call the tcp_write() routine with any amount of data that it wants. There are limitations imposed by lwIP which could result in some tcp_write() calls to fail with errors like ERR_MEM. Lets take a look at some things that an application developer needs to be aware of before they start writing code which calls this routine. First of all, there is a limit to the number of bytes that a single invocation of the HTTPRecv() callback routine can queue up for the lwIP network stack. In the current NetServices library, this queue length is set to 3 times the maximum length of a TCP segment (3 * 536 or 1,608 bytes.) The application can call tcp_write() more than once during an invocation of a callback routine like HTTPRecv() but the total data queued up can't exceed this amount. It should also be noted that the data to be written along with the required headers (for ethernet, IP, and TCP) will be queued up in pbuf lists to be sent out the ethernet port. As we have already discussed, there is a limit to the number of these pcb structures that can be allocated as well. lwIP can queue up the data passed into the tcp_write() routine in two different types of pbuf structures:

• PBUF_RAM if the apiflags parameter has the TCP_WRITE_FLAG_COPY flag set. The pbuf structure and payload buffer for the data will be allocated from the lwIP heap and the application data passed into the tcp_write() routine will be copied into this newly allocated memory alongside the headers. This is useful if the application data buffer may be modified after the HTTPRecv() routine returns (ie. the data exists in a local stack based buffer) or the ethernet hardware can't access the memory region used for the application's data buffers (ie. in FLASH memory on the LPC17xx.) If the application calls tcp_write() multiple times during a single invocation of a callback like HTTPRecv() then lwIP has functionality which will try to append data to PBUF_RAM structures that were allocated on previous calls to tcp_write().
• PBUF_ROM if the apiflags parameter doesn't have the TCP_WRITE_FLAG_COPY flag set. The pbuf structures will point their payload pointers directly to the buffers passed into the tcp_write() routines. This reduces the amount of memory required for the write and removes the data copy but a few smaller PBUF_RAM structures will still need to be allocated for storing the ethernet, IP, and TCP headers for each packet. The data passed into the tcp_write() routine will still need to be read for the purpose of calculating the checksum required for the TCP header. It should also be noted that if your application calls tcp_write() multiple times without the TCP_WRITE_FLAG_COPY set, then the lwIP network stack will need to allocate a separate PBUF_ROM structure to track each block of the application data.

For each TCP segment to be sent to the remote machine, there will be 536 bytes or less of application data and 40+ bytes of headers. When not using the TCP_WRITE_FLAG_COPY flag for the tcp_write() calls, each segment will require at least 2 pcb structures, one PBUF_RAM for the header and another PBUF_ROM for the data (but there could be more than 1 PBUF_ROM if multiple calls to tcp_write() were made to queue up such data.) The send queue is used to track the pcb structures queued up for sending and is another resource that can be overflown causing ERR_MEM to be returned from tcp_write().

This routine will limit the write byte count to the smallest of: the number of bytes left in the response to be sent back, and the size of the send buffer for this tcp_pcb. In this sample, our responses are so small that the total size of the response string will be smaller than the send buffer. The send buffer size is extracted for the connection by calling tcp_sendbuff().

The routine then enters a loop where it tries to call tcp_write() to queue up this chunk of response data to be sent back to the remote machine. This call to tcp_write() passes in the TCP_WRITE_FLAG_COPY flag. This flag tells lwIP to allocate a PBUF_RAM buffer large enough to contain this chunk of data and any required headers (ethernet, IP, and TCP) and copy the response data into it.

If the tcp_write() routine was successful, then the pHTTPContext->pResponseCurr and pHTTPContext->ResponseBytesLeft fields of the HTTP client context can be updated. The pHTTPContext->UnacknowledgedBytes field will also be increased to indicate the number of bytes that should be sent to the remote machine. As these bytes are successfully received by the remote machine, a HTTPSent() callback will be invoked by lwIP and this routine will decrease this UnacknowledgedBytes field so that the application always knows how many bytes it has outstanding on the network back to the remote machine. We will later see how the HTTPPoll() routine can take advantage of this count to determine if it needs to retry a write or not.

If the tcp_write() call failed due to out of memory then the write size will be halved and retried. The only time that this doesn't happen is if the send queue can't hold any more pbuf nodes since it couldn't add the needed pbuf anyway, no matter how small of a payload.

After the write loop, there is a check to see if all of the request data has been received and all of the response data has been queued up for delivery to the remote client. If those conditions have been met, then call HTTPCloseConnection().

HTTPCloseConnection()

This routine is called from HTTPRecv() to close the connection on the receipt of a bad request, from HTTPSendResponse() when request has been fully received and the response sent to the remote machine, and HTTPPoll() when it has failed to send data after 8 retries.

static void HTTPCloseConnection(tcp_pcb* pPCB, SHTTPContext* pHTTPContext)
{
    err_t CloseResult = ERR_MEM;
    
    printf("HTTP: Closing connection.\r\n");
    
    if (pHTTPContext)
    {
        free(pHTTPContext);
    }
    
    tcp_arg(pPCB, NULL);
    tcp_accept(pPCB, NULL);
    tcp_recv(pPCB, NULL);
    tcp_sent(pPCB, NULL);
    tcp_poll(pPCB, NULL, 0);
    tcp_err(pPCB, NULL);
    
    CloseResult = tcp_close(pPCB);
    if (ERR_OK != CloseResult)
    {
        tcp_poll(pPCB, HTTPPoll, 4);
    }
}

If there is still a SHTTPContext state object associated with the tcp_pcb, then it will be passed into the routine as a non-NULL value. In this case, free up the memory for this state by calling free(pHTTPContext) and then call tcp_arg(pPCB, NULL) to disassociate the state object from the tcp_pcb object. The code then clears all of the event callbacks that were previously registered. At this point, if anything further were to occur on this tcp_pcb object, no callbacks would be invoked. The routine ends by calling tcp_close(pPCB) in an attempt to close the connection cleanly. If this fails then it reactivates the HTTPPoll() callback which will retry the close. When HTTPPoll() gets called after a failed close attempt, it will be able to determine that this is the case because the passed in pvArg parameter will now be NULL.

HTTPSent()

When the remote client machine receives the data sent to it from the mbed device, it will send an acknowledgment back to the mbed device. The lwIP network stack will issue a callback when it receives this acknowledgment. The callback to be called in this sample will be the HTTPSent() routine which was associated with the tcp_pcb via the tcp_sent() call in HTTPAccept().

static err_t HTTPSent(void* pvArg, tcp_pcb* pPCB, u16_t AcknowledgeCount)
{
    SHTTPContext* pHTTPContext = (SHTTPContext*)pvArg;
    
    pHTTPContext->UnacknowledgedBytes -= AcknowledgeCount;
    
    HTTPSendResponse(pPCB, pHTTPContext);
    
    return ERR_OK;
}

This callback routine will be passed the AcknowledgeCount parameter, containing the number of bytes that were just acknowledged by the remote client machine for this connection. The UnacknowledgedBytes field of the SHTTPContext object will be decreased by this amount so that it can maintain a current count of the number of bytes still waiting for acknowledgment. Since some bytes have been acknowledged by the remote machine, the send buffer for this connection should now have some free space, allowing HTTPSendResponse() the opportunity to queue up more response data to be sent to the client.

HTTPPoll()

This callback routine was setup in HTTPAccept() to be called every 2 seconds while the client connection is active. This repeating invocation gives the sample an opportunity to retry queueing up more data to be sent to the remote client or a clean shutdown of the connection if a previous attempt has failed.

static err_t HTTPPoll(void* pvArg, tcp_pcb* pPCB)
{
    SHTTPContext* pHTTPContext = (SHTTPContext*)pvArg;
    
    if (!pHTTPContext)
    {
        err_t CloseResult;
        
        CloseResult = tcp_close(pPCB);
        if (ERR_OK != CloseResult)
        {
            tcp_abort(pPCB);
            return ERR_ABRT;
        }
        else
        {
            return ERR_OK;
        }
    }
    
    if (0 == pHTTPContext->UnacknowledgedBytes && pHTTPContext->ResponseBytesLeft > 0)
    {
        if (pHTTPContext->RetryCount++ > 8)
        {
            HTTPCloseConnection(pPCB, pHTTPContext);
        }
        else
        {
            HTTPSendResponse(pPCB, pHTTPContext);
        }
    }
    
    return ERR_OK;
}

If HTTPCloseConnection() was previously called but failed in its call to tcp_close(), then the HTTPPoll() callback will still be enabled. This gives the HTTPPoll() routine an opportunity to retry these close attempts. The HTTPPoll() routine knows that it should retry a call to tcp_close() because its pvArg parameter will have been set to NULL by the previous call to HTTPCloseConnection(). It will try calling tcp_close() again and if that fails, it will resort to calling tcp_abort(). When calling tcp_abort() from any of the application callbacks, you need to return ERR_ABRT so that the calling lwIP routine knows that its current tcp_pcb is no longer valid.

If data was queued up successfully with a call to tcp_write() then the HTTPSent() callback will be called once the remote client acknowledges the receipt of the data. As previously discussed, the HTTPSent() routine will then attempt to queue up more data. However, if a tcp_write() should fail, then it is possible that the HTTPSent() routine won't get invoked (no data to be acknowledged.) At that point, the only way the HTTP server can recover is to use the periodic calls to HTTPPoll() as an opportunity to retry calls to HTTPSendResponse(). If the HTTP server still has response data to be sent for this connection and there are no unacknowledged bytes outstanding then retry the call to HTTPSendResponse() from this routine. A maximum of 8 retries will be attempted for the sending of the HTTP response data before resorting to closing of the connection.

HTTPError()

The HTTPError() callback will be called by lwIP when the remote client machine resets or otherwise closes the connection so that the application can have an opportunity to free any state that it had allocated for this connection. HTTPError() will free the SHTTPContext object it allocated for this connection.

static void HTTPError(void* pvArg, err_t Error)
{
    SHTTPContext* pHTTPContext = (SHTTPContext*)pvArg;
    
    if (pHTTPContext)
    {
        free(pHTTPContext);
    }
}