Bluetooth Low Energy (a.k.a Bluetooth LE, BTLE, Bluetooth Smart)

Firmware Over The Air

24 Feb 2015

Warning

There is a newer version of this information on The BLE Team wiki FOTA page

FOTA

We've got over-the-air firmware updates (FOTA) working for the platforms based on Nordic's nRF51822<<@pratul: LINK NEEDED>>. This means that it is possible to update the firmware of a nRF51822 based device from a BLE-enabled smartphone (and hopefully soon from a development PC over a BLE dongle).

Presently, there is very little security or safety in this firmware-update process, but this is being worked upon.

We're also working on standardization of the FOTA protocol; and for defining a security model for validating/authenticating firmwares.

FOTA requires the use of a (Nordic-specific) DFU-bootloader to provide the functionality for firmware download and reprogramming. Once the DFU-bootloader is resident, any BLE application may be updated as long as the operation of the BLE stack isn't impaired.

Components Involved

The following image shows the layout of the main software components which go into a FOTA capable system. These include:

  • Softdevice: this is Nordic's encapsulation of the bluetooth stack.
  • Application: the user's BLE application.
  • Bootloader: which is responsible for firmware updates.

/media/uploads/rgrover1/bootloader.png

Upon power-up, the softdevice initializes the system and forwards control to the bootloader. Unless it has been triggered to do firmware-updates, the bootloader forwards control to the application right-away.

The bootloader image needs to be installed initially; and thereafter the FOTA mechanism can update any DFU-service enabled application as long as it doesn't crash. The softdevice and the bootloader components don't change during FOTA updates; and therefore don't need to be combined with the firmware sent over the air.

Mbed platforms typically come with an interface chip, called CMSIS- DAP<<@pratul LINK>>, which offers drag-n-drop based programming over USB. CMSIS-DAP does a complete chip-erase before programming new firmware, so binaries meant for USB drag-n-drop need to combine the softdevice with the application. FOTA offers a complementary programming facility where once the initial bootloader image is installed (possibly using CMSIS-DAP), further programming involves only updating the application while keeping the softdevice and the bootloader unchanged.

We'd like to re-emphasize that FOTA applications don't need to be combined with the softdevice (as is needed for stand-alone applications used with CMSIS-DAP). For this reason, FOTA applications need a separate (virtual) build target for each Nordic platform. be generated by building for 'shadow' platforms meant for FOTA.

Here are some links to these alternate/virtual platforms:

for mKit

for nRF-DK

Please be sure to add these platforms to your online compiler; and build for them when generating FOTA binaries.

Default Bootloader

Here's an initial image for the Nordic mKit to install the bootloader.

Here's the corresponding initial image for the nRF-DK platform.

Please unpack the appropriate .zip archive into a .hex file for programmed onto the target.

Together with the bootloader, these images come with a default application offering Device-Information and Console Output services.

You can find more information about the internals of the DFU bootloader at: http://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy- community/topic/5201/.

DFU Service

FOTA enabled applications work by offering a DFUService. This service contains a 'control' characteristic which when written to forwards program control to the resident bootloader in preparation for FOTA. This division of responsibility implies that FOTA applications need bear only a small overhead while keeping most of the FOTA heavy-lifting in the bootloader.

When building for a FOTA-enabled platform (see discussion on virtual/shadow platforms above), an instance of the DFUService gets added to an application implicitly during the call to ble.init(); the assumption being that for such a platform an application will always want to enable FOTA.

If an application built for a non-FOTA platform wishes to introduce the DFUService explicitly, it takes a single line of code to do so:

code changes needed to add DFU-Service

/* Enable over-the-air firmware updates. Instantiating DFUSservice introduces a
 * control characteristic which can be used to trigger the application to
 * handover control to a resident bootloader. */
DFUService dfu(ble);

In that case you will also need to include DFUService.h. An example of such use would be the creation of the initial application which goes with the bootloader.

Driving FOTA

Updating a FOTA-capable application is a two stage process. It involves triggering the application to hand over control to the bootloader, and then letting the bootloader receive and apply the udpate safely.

Sequence Diagram for Firmware Over The Air

/media/uploads/rgrover1/fota-sequence.png

This process can currently be driven from an external BLE agent such as an Android phone. We'll soon be releasing our own reference apps for iOS and Android; together with SDKs to build custom apps for FOTA. For the moment, we recommend using Nordic's nRF Master Control Panel as a generic tool to drive FOTA. https://play.google.com/store/apps/details?id=no.nordicsemi.android.mcp

Here's a video showing FOTA (starting at around 2:00).

The following sections provide snapshots of the process as driven by the Master Control Panel.

Lising the device with DFU Service

The following images (taken using Nordic's master-control-panel Android-app) expand on the capabilities of the defaultApp bundled into the initial bootloader image (mentioned above). Among other services, this application offers a DFU-Service which allows FOTA. All DFU-capable applications should offer this service.

Triggering FOTA

The DFUService contains a single 'control-point' characteristic, which when written forces the application to enter DFU mode; internally it causes the application to forward control to the resident bootloader which takes over the firmware transfer.

The following set of images shows how Nordic's Master Control Panel can be used to cause a DFU capable application to forward control to the bootloader:

Driving FOTA

And these images show the main FOTA sequence using the bootloader:

At the end of this process, the new application is handed control automatically, and you can see it appear on the master-control-panel.

Attribute/Service Caching

You should be aware that to save energy a GATT client often caches the results from a service discovery. Changing/updating an application (especially when replacing it with a different application) may require the client to re- attempt service discovery. After a FOTA update, you may want to restart your client application, if this doesn't happen automatically; or restart the Bluetooth service on the client's system.

UART access over BLE

If you want to receive console output from an updated app, it is possible to do so over the BLE UART Service. For instance, the default app which comes bundled with the bootloader generates regular pings on the rx-characteristic of the UARTService. These pings can be received using one of several UART apps, such as Nordic's nRF UART.

Please be aware that presently it is possible to have only a single active connection to a BLE device; so let's say you're working with a heart-rate application and if you've connected to it using nRF UART for console output, then you cannot simultaneously connect to it from another heart-rate phone app.

Please also note that console messages are sent across in notification packets of up to 20 bytes in size; this limit is imposed by Bluetooth standard. So longer messages need to be cropped into a sequence of 20-byte packets. Output buffers internal to the UARTService are flushed upon encountering a newline character. The receiving UART application should be able to stitch together cropped portions of longer messages.

Using the UART service

The following program illustrates the use of UARTService to redirect something like printf() to use the BLE transport.

Import programBLE_UARTConsole

A console service.

code changes needed to redirect console output to UART Service

#if NEED_CONSOLE_OUTPUT
#define DEBUG(STR) { if (uart) uart->write(STR, strlen(STR)); }
#else
#define DEBUG(...) /* nothing */
#endif /* #if NEED_CONSOLE_OUTPUT */


    uart = new UARTService(ble);
    DEBUG("ping\r\n");

You will also need to include UARTService.h.

Limitations of our current implementation

  • There is no security or safety built into the process yet. Anyone with the right tools can update a FOTA capable target. Resolving this is very high on our priorities.
  • Building FOTA binaries currently requires using a 'shadow' build platform for every target. We shouldn't be cloning build platforms for FOTA; ideally this should be a selectable option.
  • FOTA requires installing an initial image containing the bootloader. For non-official mbed platforms, that would mean that a user would need to understand the internals well enough to be able to build an initial image, and also find a programming interface to transfer it to the target. We're working on releasing a USB->SWD adaptor which can target nRF51822 boards which don't have an mbed CMSIS-DAP interface.
  • The two stage FOTA process is cumbersome and error-prone. We're working on creating a simple reference app to drive FOTA. Eventually we hope to release an SDK to allow users to create their own FOTA driver applications.
30 Sep 2014

That's great! Perhaps we should create a new target or an option to output application hex file without merging the soft device.

30 Sep 2014

Yes! I'm excited about completely wireless development on nRF51

Is the bootloader something that we can rebuild? I've got a platform I'd like to use this on that has no LFCLK xtal and a different LED & button map than the mKit. It would be great to be able to trigger emergency DFU by holding down a button at boot (like the Beacon kit).

30 Sep 2014

Hi Rohit,

How does the bootloader find out that it is forwarded by the soft device or by an application?

I am trying to build a bootloader for the Arch BLE based on https://github.com/NordicSemiconductor/DFU-Bootloader-for-gcc-compile. But the FOTA is triggered by a button when power on.

- Yihui

01 Oct 2014

Hello Everyone,

Thanks for your responses. We've been a bit busy with TechCon this week; so answers may take some time in coming.

One of my colleagues who handles the IDE support and platform definitions will soon be cloning the Nordic-mkit into a 'shadow' platform for OTA DFU to be able to build applications without combining them with the soft-device. It will take him 30mins of his time, and hopefully it will happen in between the conference duties for the week, but please be patient for those 30 minutes to come together.

I'd be very happy to share the sources for the bootloader and how I build it. But that will happen around next week.

A DFU enabled application executes the following code when DFU is triggered by writing into the control characteristic of the DFU service:

sd_power_gpregret_set(BOOTLOADER_DFU_START);

The above sets a register in the processor's power domain, which can then be read back by the bootloader.

BOOTLOADER_DFU_START happens to be some constant which is understood by the bootloader as a special indication that control flowed into it from an application (instead of the softdevice).

03 Oct 2014

Hi Rohit,

Thanks for your explanation. I get the idea and have a look at the implementation. I think we can simplify the implementation:

sd_power_gpregret_set(BOOTLOADER_DFU_START);
NVIC_SystemReset();

Do a software reset instead of manually disabling all interrupts and softdevice and using assemble language to do a jump.

09 Oct 2014

I've created a document describing the internals of the Bootloader: https://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy-community/topic/5201/

enjoy.

17 Oct 2014

Rohit Grover wrote:

I've created a document describing the internals of the Bootloader: https://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy-community/topic/5201/

Thanks Rohit. While I couldn't build from your sources since I don't have a license for the ARM toolchain, your doc was enough to get me going with the Nordic DFU for GCC. I found that the .hex for the default app built for my platform could be trimmed to work with the Android Master Control Panel app:

"srec_cat BLE_Default_APP_HRM1017_.hex -intel -crop 0x16000 0x3c000 -line-length 44 -o ble_default_dfu.hex -intel"

The "-line-length 44" works around a bug in the Android MCP app (maybe others?) where parsing long Intel hex lines causes the app to crash.

20 Nov 2014

The mbed FOTA platform for Nordic now produces .hex files. Just start with the default app (above) [or find a way to install the bootloader and the softdevice], and then you're set to go with FOTA. Hex files allow easy operation through Nordic's apps [for andorid and iOS].

We'll soon be releasing an iOS app to drive FOTA; with an Android version following soon.

rohit.

25 Nov 2014

Hi Rohit,

When I run BLE_Default_APP on mkit and connect using MCP, the device name remains as "DefaultApp" after I tap the trigger icon in DFU Control Point. More specifically, after trigger icon is pressed the mkit shown disconnected then I press the connect buttom, mkit is reconnected but shown a local name of "DefaultApp" still (expect to see "DfuTarg"). For me I seems like the mkit is fail to enter DFU mode. Is anything I miss? Any comments are appreciated.

Cheers, Tsungta

25 Nov 2014

Hi Tsungta,

What mobile platform are you using to drive FOTA? I know that iOS is very sticky with its caching of device information to avoid having to fetch them repeatedly. Android is much less sticky in this regard; I know that the Android-MasterControlPanel is able to update from DefaultApp to DfuTarg during the FOTA process. One quick indicator for this switch is the LED. I believe the DefaultApp blinks the led, but this stops when switched to the booloader in DfuTarg mode.

We're working on a reference iOS app to drive the entire FOTA process seamlessly. This is expected to come out very shortly; within weeks if not days. Hopefully you'll have a much easier time with FOTA with our reference app.

26 Nov 2014

Hi Rohit,

Many thanks for your reply. I used Android with MasterControlPanel for the aforementioned experiment. According to your reply, the LED should blinks when mkit is powered on with local name of DefaultApp. Somehow, neither LED1 nor LED2 blink in my case. Please help me out of this.

Cheers, Tsungta

27 Nov 2014

Hi,

I have the nRF51-DK board; I have installed the nRF51-DK and nRF51-DK FOTA compilers; I have installed the http://developer.mbed.org/media/uploads/rgrover1/ble_default_bootloader_app.hex bootloader file (as per the instructions at http://developer.mbed.org/platforms/Nordic-nRF51-DK-FOTA/). I then imported the 'BLE_HeartRate' example (search for 'nRF51822' in the Import dialog), and compiled using the FOTA compiler.

The board shows up as 'DefaultApp' in nRF Master Control Panel on my Android phone. When I try to update the firmware using the 'DFU' button, the upload fails, but the board now shows up as 'DfuTarg' instead. When I retry updating with the DFU button, it will then upload the hex file (usually requires a few tries though). After this point though, the board does not show up in Master control panel anymore; I have tried this may times, always with the same result.

Note that if I compile the 'BLE_HeartRate' example with the nRF51-DK compiler, and load the hex file by dragging the file via usb, it works as expected.

Can someone please steer me in the right direction? Thanks!

Mordechai

27 Nov 2014

I apologize for the poor documentation/tutorial around FOTA. We could be doing it better, but we haven't managed due of scarce resources. We'd be very glad to receive suggestions for improvements to the existing documentation.

Here's a vedio containing a section about driving FOTA using Nordic's android app(s): https://www.youtube.com/watch?v=54V2_4TR9Wo

We're going to release our own iOS reference app for FOTA.

@TT: The default-app was compiled for the mKit; and therefore is unable to drive the LED on the DK platform. I'll recompile the default app for the DK and post shortly.

@Mordechai Brodt: Please note that FOTA is a two step process. We acknowledge that this is not intuitive; and we're working on creating seamless FOTA driver apps for iOS and Android. In the absence of these, you have the option of using Nordic's MCP, which offers a two stage solution. It is necessary to first trigger handover to the bootloader before downloading firmware. Please refer to the sequences (above) titled 'Triggering FOTA' and 'Driving FOTA'. You said: "When I try to update the firmware using the 'DFU' button, the upload fails, but the board now shows up as 'DfuTarg' instead." Please only trigger FOTA in the first step.

27 Nov 2014

Here's the initial bootloader app for the DK platform. I've also updated the original documentation with this image.

In the second stage of the FOTA process, when the device is running the bootloader and shows as DfuTarg, connecting to it requires a little longer in 'service-discovery'; and may even require pressing the 'connect' button. Otherwise, I am able to drive FOTA without any problem.

27 Nov 2014

Hi Rohit,

It works great now, thank you so much!

Mordechai

03 Dec 2014

This no longer works with the latest BLE library...

I get "Error: Class "UARTService" has no member "retargetStdout" in "main.cpp", Line: 69, Col: 17"

03 Dec 2014

@rohit - can you explain this step for someone who is working on the gcc platform (and not necessarily with the DK or mkit). We have the gcc build system working that builds the combines the app and softdevice using srec_cat. Where does the DFU fit in with that - as in, if I want to do subsequent updates using FOTA, how does the cmake build change ? It would be great if the cmake build creates all these binaries in one shot (manual firmware, DFU hex, FOTA hex, etc.)

03 Dec 2014

@Mark Baseggio: Yes, you are correct about that. Please read from the following post to get some history. http://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy-community/topic/5210/?page=1#comment-26027

I'll update the documentation shortly. retarget isn't available as it used to be.

03 Dec 2014

@sandeep ss:

Components and entities involved: softdevice (i.e. bluetooth stack), application, DFU-Bootloader, Bootloader settings page, UICR (register defined by nRF MCU which tells the softdevice where to forward control: bootloader or application).

The bootloader settings page tell the bootloader about forwarding control to the application.

When working with the mkit or any mbed platform, drag-n-drop of the firmware over USB results in a mass-erase followed by programming. In that case, the following options exist:

  • Softdevice + application [default builds using the online IDE; non FOTA]
  • Softdevice + application + Bootloader + UICR pointing to the bootloader [initial app; available for download]

When updating over FOTA, the following is needed:

  • application only [meant for FOTA; build platforms available from links mentioned in the individual platforms page; requires the presence of a bootloader and also DFUService in the current application]

if you need FOTA, you must somehow get the softdevice and bootloader and initialized-UICR on the device. srec_cat can be used to combine things; and also initialize UICR and bootloader settings.

03 Dec 2014

@rohit - thanks for the reply. Just want to understand this clearly

1. how does the FOTA work - the interplay of UICR and CLENR0 seems to be a moving target and has changed to S110 V7 (https://devzone.nordicsemi.com/question/7788/issue-with-programming-s110-v700-3alpha-with-nrfgo-studio/). Again, if

2. The GCC build in cmake , I am able to generate two targets - Softdevice and application. This is then srec_cat to make the combined hex. The same application hex can be used (without the softdevice) to do a FOTA correct ? or will the application build also need to change for a FOTA based system? (based on what I read here - https://devzone.nordicsemi.com/documentation/nrf51/4.4.1/html/group__uicr__config__example.html) Based on Q1, I think the application build will also need to be modified to accomodate for writing of UICR,etc.

3. Softdevice + application + Bootloader + UICR - can this be generated with the cmake build system ? How do we add support?

thanks!

03 Dec 2014

Rohit Grover wrote:

@Mark Baseggio: Yes, you are correct about that. Please read from the following post to get some history. http://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy-community/topic/5210/?page=1#comment-26027

I'll update the documentation shortly. retarget isn't available as it used to be.

Thanks @rohit. Is there documentation other than this post that I should be looking at?

I'd really appreciate examples as well.

Mark

03 Dec 2014

help out, guys. Please contribute documentation or examples where you feel it would help.

03 Dec 2014

Quote:

1. how does the FOTA work - the interplay of UICR and CLENR0 seems to be a moving target and has changed to S110 V7 (https://devzone.nordicsemi.com/question/7788/issue-with-programming-s110-v700-3alpha-with-nrfgo-studio/). Again, if

There will be some enhancements to introduce safety and security into the FOTA process. Safety requires that it should not be possible to update firmware meant for a different MCU; and security means the it should be possible to authenticate and trust the new firmware.

Quote:

2. The GCC build in cmake , I am able to generate two targets - Softdevice and application. This is then srec_cat to make the combined hex. The same application hex can be used (without the softdevice) to do a FOTA correct ?

correct. The application doesn't update the UICR. It gets set by the initial image. I have tried to capture some of these details in http://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy-community/topic/5201/

3: please study http://developer.mbed.org/forum/team-63-Bluetooth-Low-Energy-community/topic/5201/

03 Dec 2014

Rohit Grover wrote:

help out, guys. Please contribute documentation or examples where you feel it would help.

If I could figure out how it worked I would. I'm not on the boards asking for examples because I know what I'm doing :)

Also, which docs? Things kinda seem to be everywhere.

Thanks again for your help.

03 Dec 2014

@rohit - thanks for the link. I just went through it. Just one question. The srec_cat below:

srec_cat ${MBED_SRC_PATH}/targets/hal/TARGET_NORDIC/TARGET_MCU_NRF51822/Lib/s110_nrf51822_7_0_0/s110_nrf51822_7.0.0_softdevice.hex -intel BLE_Default_APP.hex -intel ../../BLE_BootLoader/Build/BLE_BOOTLOADER.hex -intel -exclude 0x3FC00 0x3FC20 -generate 0x3FC00 0x3FC04 -l-e-constant 0x01 4 -generate 0x3FC04 0x3FC08 -l-e-constant 0x00 4 -generate 0x3FC08 0x3FC0C -l-e-constant 0xFE 4 -generate 0x3FC0C 0x3FC20 -constant 0x00 -o combined.hex -intel

From your doc this generate a full hex - Softdevice + application + Bootloader + UICR . Can I use this as the default output hex for all my applications ? Till now, I have been using the Softdevice+application hex only - and am fairly new to the DFU concept. So I'm wondering if this kind of hex imposes certain limitations on application size, application complexity, etc. (for some non-trivial applications)

04 Dec 2014

@Sandeep ss: Yes, you should be able to use this as a default to build your apps. Adding the bootloader costs only 16K of flash. You won't be significantly constrained in any way.

28 Jan 2015

Hi,

I've create a minimal code example: http://developer.mbed.org/users/waynek/code/HelloBlue/

Hope it helps All the best Wayne

07 Apr 2015

Hi everyone, Great posts.

FYI: The company I work for is looking for Firmware Developer/Engineer. Similar position to the Hardware Engineer but we need a FW person. Great products, great company, especially if you love sport/fitness industry. http://www.wahoofitness.com/jobs

03 Aug 2015

@Rohit Grover

Hi Rohit,

I have a nrf DK board, and I tried a lot to do FOTA but I failed.

I downloaded the initial image(https://developer.mbed.org/teams/Bluetooth-Low-Energy/wiki/Firmware-Over-the-Air-FOTA-Updates),

and drag and drop to board.

/media/uploads/Smarty/img_3992.mov

I upload video, please check what is wrong with me.