Osamu Koizumi / Mbed OS Pelion-GR-LYCHEE-camera-firmware

Taking photo using GR-LYCHEE through Pelion Device Management.

This firmware project works for GR-LYCHEE and GR-PEACH.

Steps to build this firmware with Mbed CLI

Import project

$ mbed import http://os.mbed.com/users/coisme/code/Pelion-GR-LYCHEE-camera-firmware/

Setting

  • Ethernet (GR-PEACH only)
    • Change the file name mbed_app.RZ_A1H_Ethernet.json to mbed_app.json
  • Wi-Fi
    • Change the Wi-Fi setting in mbed_app.json, i.e. nsapi.default-wifi-ssid and nsapi.default-wifi-password

If you haven't set your Pelion API key, run the following command.

$ mbed config -G CLOUD_SDK_API_KEY <API_KEY>

Setting for Pelion Device Management

$ mbed dm init -d "example.com" --model-name "PELION_DEMO" -q --force

Compile

For GR-LYCHEE:

$ mbed compile -t GCC_ARM -m GR_LYCHEE

For GR-PEACH:

$ mbed compile -t GCC_ARM -m RZ_A1H

Write the created .bin file to your GR-LYCHEE/PEACH.

That's it! :-)

LED Status Indicator

LEDs next to UB0 show the status of your GR-LYCHEE/PEACH.

LED#GR-LYCHEEGR-PEACHStatusDescription
LED1greenredNormalTurned on after the device is registered to Pelion Device Management successfully.
LED2yellowgreenErrorTurned on when the network initialization failed. Check your Wi-Fi setting.
LED3orangeblueErrorTurned on when the Pelion Device Management Client initialization failed. Check your SD card.
LED4redredErrorTurned on when the network is disconnected. Check your Wi-Fi network status.

Clear device identity

If you want to clear the device's identity, connect to the device via serial terminal. Then input r command. The device flushes the identity storage, then reboot.

Known Issues

  • client_error(6) -> Client in reconnection mode NetworkError appears when connecting to network, but eventually connection will be established.
  • Warning message "MBEDTLS_TEST_NULL_ENTROPY has been enabled. This configuration is not secure and is not suitable for production use" appears when compiling the project for GR-PEACH. This can be ignored in development stage.

main.cpp@10:29d334f90c99, 2019-02-21 (annotated)

Committer:
Osamu Koizumi
Date:
Thu Feb 21 16:15:28 2019 +0900
Revision:
10:29d334f90c99
Parent:
9:eac9e0c9effd
Child:
14:0480150782c8
Changed camera trigger and cleaned up code.

Who changed what in which revision?

UserRevisionLine numberNew contents of line
MACRUM 0:6d2053b84a92 1 // ----------------------------------------------------------------------------
MACRUM 0:6d2053b84a92 2 // Copyright 2016-2018 ARM Ltd.
MACRUM 0:6d2053b84a92 3 //
MACRUM 0:6d2053b84a92 4 // SPDX-License-Identifier: Apache-2.0
MACRUM 0:6d2053b84a92 5 //
MACRUM 0:6d2053b84a92 6 // Licensed under the Apache License, Version 2.0 (the "License");
MACRUM 0:6d2053b84a92 7 // you may not use this file except in compliance with the License.
MACRUM 0:6d2053b84a92 8 // You may obtain a copy of the License at
MACRUM 0:6d2053b84a92 9 //
MACRUM 0:6d2053b84a92 10 // http://www.apache.org/licenses/LICENSE-2.0
MACRUM 0:6d2053b84a92 11 //
MACRUM 0:6d2053b84a92 12 // Unless required by applicable law or agreed to in writing, software
MACRUM 0:6d2053b84a92 13 // distributed under the License is distributed on an "AS IS" BASIS,
MACRUM 0:6d2053b84a92 14 // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
MACRUM 0:6d2053b84a92 15 // See the License for the specific language governing permissions and
MACRUM 0:6d2053b84a92 16 // limitations under the License.
MACRUM 0:6d2053b84a92 17 // ----------------------------------------------------------------------------
MACRUM 0:6d2053b84a92 18 #ifndef MBED_TEST_MODE
MACRUM 0:6d2053b84a92 19
MACRUM 0:6d2053b84a92 20 #include "mbed.h"
MACRUM 0:6d2053b84a92 21 #include "simple-mbed-cloud-client.h"
MACRUM 0:6d2053b84a92 22 #include "FATFileSystem.h"
dkato 9:eac9e0c9effd 23 #include "EasyAttach_CameraAndLCD.h"
dkato 9:eac9e0c9effd 24 #include "dcache-control.h"
dkato 9:eac9e0c9effd 25 #include "JPEG_Converter.h"
MACRUM 0:6d2053b84a92 26
Osamu Koizumi 10:29d334f90c99 27 // Initial photo data
Osamu Koizumi 10:29d334f90c99 28 #include "default_photo.h"
Osamu Koizumi 10:29d334f90c99 29
dkato 9:eac9e0c9effd 30 /**** User Selection *********/
dkato 9:eac9e0c9effd 31 #define VIDEO_PIXEL_HW (320u) /* QVGA */
dkato 9:eac9e0c9effd 32 #define VIDEO_PIXEL_VW (240u) /* QVGA */
dkato 9:eac9e0c9effd 33 #define JPEG_ENCODE_QUALITY (75) /* JPEG encode quality (min:1, max:75 (Considering the size of JpegBuffer, about 75 is the upper limit.)) */
dkato 9:eac9e0c9effd 34 /*****************************/
dkato 9:eac9e0c9effd 35
dkato 9:eac9e0c9effd 36 #define DATA_SIZE_PER_PIC (2u)
dkato 9:eac9e0c9effd 37 #define FRAME_BUFFER_STRIDE (((VIDEO_PIXEL_HW * DATA_SIZE_PER_PIC) + 31u) & ~31u)
dkato 9:eac9e0c9effd 38 #define FRAME_BUFFER_HEIGHT (VIDEO_PIXEL_VW)
dkato 9:eac9e0c9effd 39
dkato 9:eac9e0c9effd 40 uint8_t user_frame_buffer0[FRAME_BUFFER_STRIDE * FRAME_BUFFER_HEIGHT]__attribute((aligned(32)));
dkato 9:eac9e0c9effd 41 uint8_t JpegBuffer[1024 * 32]__attribute((aligned(32)));
dkato 9:eac9e0c9effd 42 DisplayBase Display;
dkato 9:eac9e0c9effd 43 JPEG_Converter Jcu;
Osamu Koizumi 4:7c58c47eca55 44
MACRUM 0:6d2053b84a92 45 // An event queue is a very useful structure to debounce information between contexts (e.g. ISR and normal threads)
MACRUM 0:6d2053b84a92 46 // This is great because things such as network operations are illegal in ISR, so updating a resource in a button's fall() function is not allowed
MACRUM 0:6d2053b84a92 47 EventQueue eventQueue;
MACRUM 0:6d2053b84a92 48
MACRUM 0:6d2053b84a92 49 // Default block device
MACRUM 0:6d2053b84a92 50 BlockDevice *bd = BlockDevice::get_default_instance();
MACRUM 0:6d2053b84a92 51 FATFileSystem fs("fs");
MACRUM 0:6d2053b84a92 52
MACRUM 0:6d2053b84a92 53 // Default network interface object
MACRUM 0:6d2053b84a92 54 NetworkInterface *net = NetworkInterface::get_default_instance();
MACRUM 0:6d2053b84a92 55
MACRUM 0:6d2053b84a92 56 InterruptIn btn(USER_BUTTON0);
MACRUM 0:6d2053b84a92 57 // Declaring pointers for access to Pelion Device Management Client resources outside of main()
MACRUM 0:6d2053b84a92 58 MbedCloudClientResource *button_res;
Osamu Koizumi 10:29d334f90c99 59 MbedCloudClientResource *camera_trigger_res;
Osamu Koizumi 10:29d334f90c99 60 MbedCloudClientResource *camera_capture_res;
MACRUM 0:6d2053b84a92 61
dkato 9:eac9e0c9effd 62 bool take_photo(uint8_t *photo_data, uint32_t *photo_data_len) {
dkato 9:eac9e0c9effd 63 JPEG_Converter::bitmap_buff_info_t buff_info;
dkato 9:eac9e0c9effd 64 JPEG_Converter::encode_options_t encode_opt;
dkato 9:eac9e0c9effd 65
dkato 9:eac9e0c9effd 66 if ((photo_data == NULL) || (photo_data_len == NULL)) {
dkato 9:eac9e0c9effd 67 return false;
dkato 9:eac9e0c9effd 68 }
dkato 9:eac9e0c9effd 69
dkato 9:eac9e0c9effd 70 // Jpeg setting
dkato 9:eac9e0c9effd 71 buff_info.width = VIDEO_PIXEL_HW;
dkato 9:eac9e0c9effd 72 buff_info.height = VIDEO_PIXEL_VW;
dkato 9:eac9e0c9effd 73 buff_info.format = JPEG_Converter::WR_RD_YCbCr422;
dkato 9:eac9e0c9effd 74 buff_info.buffer_address = (void *)user_frame_buffer0;
dkato 9:eac9e0c9effd 75 encode_opt.encode_buff_size = *photo_data_len;
dkato 9:eac9e0c9effd 76 encode_opt.input_swapsetting = JPEG_Converter::WR_RD_WRSWA_32_16_8BIT;
dkato 9:eac9e0c9effd 77
dkato 9:eac9e0c9effd 78 dcache_invalid(photo_data, *photo_data_len);
Osamu Koizumi 10:29d334f90c99 79 if (Jcu.encode(&buff_info, photo_data, (size_t *)photo_data_len, &encode_opt) != JPEG_Converter::JPEG_CONV_OK) {
dkato 9:eac9e0c9effd 80 return false;
dkato 9:eac9e0c9effd 81 }
dkato 9:eac9e0c9effd 82
dkato 9:eac9e0c9effd 83 return true;
dkato 9:eac9e0c9effd 84 }
dkato 9:eac9e0c9effd 85
Osamu Koizumi 10:29d334f90c99 86 bool take_and_send_photo() {
Osamu Koizumi 10:29d334f90c99 87 // Send photo data on button click
Osamu Koizumi 10:29d334f90c99 88 uint32_t photo_data_len = sizeof(JpegBuffer);
Osamu Koizumi 10:29d334f90c99 89 if (take_photo(JpegBuffer, &photo_data_len)) {
Osamu Koizumi 10:29d334f90c99 90 M2MResource* m2m_res = camera_capture_res->get_m2m_resource();
Osamu Koizumi 10:29d334f90c99 91 m2m_res->set_value((const uint8_t *)JpegBuffer, photo_data_len);
Osamu Koizumi 10:29d334f90c99 92 // success
Osamu Koizumi 10:29d334f90c99 93 return true;
Osamu Koizumi 10:29d334f90c99 94 }
Osamu Koizumi 10:29d334f90c99 95 // fail
Osamu Koizumi 10:29d334f90c99 96 return false;
Osamu Koizumi 10:29d334f90c99 97 }
Osamu Koizumi 10:29d334f90c99 98
MACRUM 0:6d2053b84a92 99 void button_press() {
MACRUM 0:6d2053b84a92 100 int v = button_res->get_value_int() + 1;
MACRUM 0:6d2053b84a92 101 button_res->set_value(v);
Osamu Koizumi 7:826cdcc3c8ec 102 printf("User button clicked %d times\n", v);
Osamu Koizumi 7:826cdcc3c8ec 103
Osamu Koizumi 10:29d334f90c99 104 if(!take_and_send_photo()) {
Osamu Koizumi 10:29d334f90c99 105 printf("Failed to send photo.");
dkato 9:eac9e0c9effd 106 }
Osamu Koizumi 4:7c58c47eca55 107 }
Osamu Koizumi 4:7c58c47eca55 108
Osamu Koizumi 10:29d334f90c99 109 void camera_capture_callback(MbedCloudClientResource *resource, const NoticationDeliveryStatus status)
Osamu Koizumi 4:7c58c47eca55 110 {
Osamu Koizumi 4:7c58c47eca55 111 printf("camera notification, status: %s (%d)\n",
Osamu Koizumi 4:7c58c47eca55 112 MbedCloudClientResource::delivery_status_to_string(status), status);
MACRUM 0:6d2053b84a92 113 }
MACRUM 0:6d2053b84a92 114
MACRUM 0:6d2053b84a92 115 /**
MACRUM 0:6d2053b84a92 116 * PUT handler
MACRUM 0:6d2053b84a92 117 * @param resource The resource that triggered the callback
MACRUM 0:6d2053b84a92 118 * @param newValue Updated value for the resource
MACRUM 0:6d2053b84a92 119 */
Osamu Koizumi 10:29d334f90c99 120 void camera_trigger_callback(MbedCloudClientResource *resource, m2m::String newValue) {
Osamu Koizumi 10:29d334f90c99 121 printf("Camera trigger received, new value: %s\n", newValue.c_str());
MACRUM 0:6d2053b84a92 122
Osamu Koizumi 10:29d334f90c99 123 if(!take_and_send_photo()) {
Osamu Koizumi 10:29d334f90c99 124 printf("Failed to send photo.");
dkato 9:eac9e0c9effd 125 }
MACRUM 0:6d2053b84a92 126 }
MACRUM 0:6d2053b84a92 127
MACRUM 0:6d2053b84a92 128 /**
MACRUM 0:6d2053b84a92 129 * Notification callback handler
MACRUM 0:6d2053b84a92 130 * @param resource The resource that triggered the callback
MACRUM 0:6d2053b84a92 131 * @param status The delivery status of the notification
MACRUM 0:6d2053b84a92 132 */
MACRUM 0:6d2053b84a92 133 void button_callback(MbedCloudClientResource *resource, const NoticationDeliveryStatus status) {
MACRUM 0:6d2053b84a92 134 printf("Button notification, status %s (%d)\n", MbedCloudClientResource::delivery_status_to_string(status), status);
MACRUM 0:6d2053b84a92 135 }
MACRUM 0:6d2053b84a92 136
MACRUM 0:6d2053b84a92 137 /**
MACRUM 0:6d2053b84a92 138 * Registration callback handler
MACRUM 0:6d2053b84a92 139 * @param endpoint Information about the registered endpoint such as the name (so you can find it back in portal)
MACRUM 0:6d2053b84a92 140 */
MACRUM 0:6d2053b84a92 141 void registered(const ConnectorClientEndpointInfo *endpoint) {
MACRUM 0:6d2053b84a92 142 printf("Connected to Pelion Device Management. Endpoint Name: %s\n", endpoint->internal_endpoint_name.c_str());
Osamu Koizumi 10:29d334f90c99 143 // Set default photo
Osamu Koizumi 10:29d334f90c99 144 M2MResource* m2m_res = camera_capture_res->get_m2m_resource();
Osamu Koizumi 10:29d334f90c99 145 m2m_res->set_value(DEFAULT_PHOTO, DEFAULT_PHOTO_LEN);
MACRUM 0:6d2053b84a92 146 }
Osamu Koizumi 10:29d334f90c99 147
MACRUM 0:6d2053b84a92 148
MACRUM 0:6d2053b84a92 149 int main(void) {
MACRUM 0:6d2053b84a92 150 printf("Starting Simple Pelion Device Management Client example\n");
MACRUM 0:6d2053b84a92 151 printf("Connecting to the network...\n");
MACRUM 0:6d2053b84a92 152
MACRUM 0:6d2053b84a92 153 // Connect to the internet (DHCP is expected to be on)
MACRUM 0:6d2053b84a92 154 nsapi_error_t status = net->connect();
MACRUM 0:6d2053b84a92 155
MACRUM 0:6d2053b84a92 156 if (status != NSAPI_ERROR_OK) {
MACRUM 0:6d2053b84a92 157 printf("Connecting to the network failed %d!\n", status);
MACRUM 0:6d2053b84a92 158 return -1;
MACRUM 0:6d2053b84a92 159 }
MACRUM 0:6d2053b84a92 160
MACRUM 0:6d2053b84a92 161 printf("Connected to the network successfully. IP address: %s\n", net->get_ip_address());
MACRUM 0:6d2053b84a92 162
MACRUM 0:6d2053b84a92 163 // SimpleMbedCloudClient handles registering over LwM2M to Pelion Device Management
MACRUM 0:6d2053b84a92 164 SimpleMbedCloudClient client(net, bd, &fs);
MACRUM 0:6d2053b84a92 165 int client_status = client.init();
MACRUM 0:6d2053b84a92 166 if (client_status != 0) {
MACRUM 0:6d2053b84a92 167 printf("Pelion Client initialization failed (%d)\n", client_status);
MACRUM 0:6d2053b84a92 168 return -1;
MACRUM 0:6d2053b84a92 169 }
MACRUM 0:6d2053b84a92 170
MACRUM 0:6d2053b84a92 171 // Creating resources, which can be written or read from the cloud
Osamu Koizumi 10:29d334f90c99 172 button_res = client.create_resource("3200/0/5501", "button_shutter");
MACRUM 0:6d2053b84a92 173 button_res->set_value(0);
MACRUM 0:6d2053b84a92 174 button_res->methods(M2MMethod::GET);
MACRUM 0:6d2053b84a92 175 button_res->observable(true);
MACRUM 0:6d2053b84a92 176 button_res->attach_notification_callback(button_callback);
MACRUM 0:6d2053b84a92 177
Osamu Koizumi 10:29d334f90c99 178 camera_trigger_res = client.create_resource("3201/0/5853", "camera_trigger");
Osamu Koizumi 10:29d334f90c99 179 camera_trigger_res->set_value(0);
Osamu Koizumi 10:29d334f90c99 180 camera_trigger_res->methods(M2MMethod::GET | M2MMethod::PUT);
Osamu Koizumi 10:29d334f90c99 181 camera_trigger_res->attach_put_callback(camera_trigger_callback);
MACRUM 0:6d2053b84a92 182
Osamu Koizumi 10:29d334f90c99 183 camera_capture_res = client.create_resource("3200/0/4014", "CameraCapture");
Osamu Koizumi 10:29d334f90c99 184 camera_capture_res->set_value(0);
Osamu Koizumi 10:29d334f90c99 185 camera_capture_res->methods(M2MMethod::GET);
Osamu Koizumi 10:29d334f90c99 186 camera_capture_res->observable(true);
Osamu Koizumi 10:29d334f90c99 187 camera_capture_res->attach_notification_callback(camera_capture_callback);
MACRUM 0:6d2053b84a92 188
MACRUM 0:6d2053b84a92 189 printf("Initialized Pelion Client. Registering...\n");
MACRUM 0:6d2053b84a92 190
dkato 9:eac9e0c9effd 191 // Camera start
dkato 9:eac9e0c9effd 192 EasyAttach_Init(Display);
dkato 9:eac9e0c9effd 193 Display.Video_Write_Setting(
dkato 9:eac9e0c9effd 194 DisplayBase::VIDEO_INPUT_CHANNEL_0,
dkato 9:eac9e0c9effd 195 DisplayBase::COL_SYS_NTSC_358,
dkato 9:eac9e0c9effd 196 (void *)user_frame_buffer0,
dkato 9:eac9e0c9effd 197 FRAME_BUFFER_STRIDE,
dkato 9:eac9e0c9effd 198 DisplayBase::VIDEO_FORMAT_YCBCR422,
dkato 9:eac9e0c9effd 199 DisplayBase::WR_RD_WRSWA_32_16BIT,
dkato 9:eac9e0c9effd 200 VIDEO_PIXEL_VW,
dkato 9:eac9e0c9effd 201 VIDEO_PIXEL_HW
dkato 9:eac9e0c9effd 202 );
dkato 9:eac9e0c9effd 203 EasyAttach_CameraStart(Display, DisplayBase::VIDEO_INPUT_CHANNEL_0);
dkato 9:eac9e0c9effd 204
dkato 9:eac9e0c9effd 205 // Jpeg setting
dkato 9:eac9e0c9effd 206 Jcu.SetQuality(JPEG_ENCODE_QUALITY);
dkato 9:eac9e0c9effd 207
MACRUM 0:6d2053b84a92 208 // Callback that fires when registering is complete
MACRUM 0:6d2053b84a92 209 client.on_registered(&registered);
MACRUM 0:6d2053b84a92 210
MACRUM 0:6d2053b84a92 211 // Register with Pelion Device Management
MACRUM 0:6d2053b84a92 212 client.register_and_connect();
MACRUM 0:6d2053b84a92 213
MACRUM 0:6d2053b84a92 214 // Setup the button
MACRUM 0:6d2053b84a92 215 btn.mode(PullUp);
MACRUM 0:6d2053b84a92 216
MACRUM 0:6d2053b84a92 217 // The button fall handler is placed in the event queue so it will run in
MACRUM 0:6d2053b84a92 218 // thread context instead of ISR context, which allows safely updating the cloud resource
MACRUM 0:6d2053b84a92 219 btn.fall(eventQueue.event(&button_press));
MACRUM 0:6d2053b84a92 220
MACRUM 0:6d2053b84a92 221 // You can easily run the eventQueue in a separate thread if required
MACRUM 0:6d2053b84a92 222 eventQueue.dispatch_forever();
MACRUM 0:6d2053b84a92 223 }
MACRUM 0:6d2053b84a92 224 #endif