![](/media/cache/group/renesas_logomark_l.jpg.50x50_q85.jpg)
Sample program that can send the recognition data from HVC-P2 to Fujitsu IoT Platform using REST (HTTP)
Dependencies: AsciiFont GR-PEACH_video GraphicsFramework LCD_shield_config R_BSP USBHost_custom easy-connect-gr-peach mbed-http picojson
Information
Here are both English and Japanese description. First, English description is shown followed by Japanese one. 本ページには英語版と日本語版の説明があります。まず英語版、続いて日本語版の説明となります。
Overview
This sample program shows how to send the cognitive data gathered by Omron HVC-P2 (Human Vision Components B5T-007001) to IoT Platform managed by FUJITSU ( http://jp.fujitsu.com/solutions/cloud/k5/function/paas/iot-platform/ ).
Hardware Configuration
- GR-PEACH 1 set ( https://developer.mbed.org/platforms/Renesas-GR-PEACH/ )
- LCD Shield 1 set ( https://developer.mbed.org/teams/Renesas/Wiki/LCD-shield )
- HVC-P2 1 set ( Human Vision Components B5T-007001 ) ( https://plus-sensin.omron.com/product/B5T-007001/ )
- USBA - Micro USB Cable 2 sets
- USBA (Female) - Micro USB (Male) Adapter 1 set
When executing this sample program, please configure the above H/W as shown below:
Also, please close JP3 of GR-PEACH as follows:
Application Preconfiguration
- Configure Ethernet settings. For details, please refer to the following link:
https://developer.mbed.org/teams/Renesas/code/GR-PEACH_IoT_Platform_HTTP_sample/wiki/Ethernet-settings - On IoT Platform, set up the resource and its access code where the gathered data would be stored. For details on resource and access code, please refer to the following links:
https://iot-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/userguide_en.pdf
https://iot-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/apireference_en.pdf
https://iot-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/portalmanual_en.pdf
Build Procedure
- Import this sample program onto mbed Compiler
- In GR-PEACH_HVC-P2_IoTPlatform_http/IoT_Platform/iot_platform.cpp, please replace <ACCESS CODE> with the access code you set up on IoT Platform. For details on how to set up Access Code, please refer to the above Application Setup. Then, please delete the line beginning with #error macro.
Access Code configuration
#define ACCESS_CODE <Access CODE> #error "You need to replace <Access CODE for your resource> with yours"
- In GR-PEACH_HVC-P2_IoTPlatform_http/IoT_Platform/iot_platform.cpp, please replace <Base URI>, <Tenant ID> and <Path-to-Resource> below with yours and delete the line beginning with #error macro. For details on <Base URI> and <Tenant ID>, please contact FUJITSU LIMITED. Also, for details on <Path-to-Resource>, please refer to the above Application Setup.
URI configuration
std::string put_uri_base("<Base URI>/v1/<Tenant ID>/<Path-to-Resource>.json"); #error "You need to replace <Base URI>, <Tenant ID> and <Path-to-Resource> with yours" **snip** std::string get_uri("<Base URI>/v1/<Tenant ID>/<Path-to-Resource>/_past.json"); #error "You need to replace <Base URI>, <Tenant ID> and <Path-to-Resource> with yours"
- Compile the program. If compilation is successfully terminated, the binary should be downloaded on your PC.
- Plug Ethernet cable into GR-PEACH
- Plug micro-USB cable into the port which lies on the next to RESET button. If GR-PEACH is successfully recognized, it appears as the USB flash disk named mbed as show below:
- Copy the downloaded binary to mbed drive
- Press RESET button on GR-PEACH in order to run the program. If it's successfully run, you can see the following message on terminal:
Format of Data to be sent to IoT Platform
In this program, the cognitive data sent from HVC-P2 is serialized in the following JSON format and send it to IoT Platform:
- Face detection data
{ "RecodeType": "HVC-P2(face)" "id": <GR-PEACH ID>-<Sensor ID>" "FaceRectangle": { "Height": xxxx, "Left": xxxx, "Top": xxxx, "Width": xxxx, }, "Gender": "male" or "female", "Scores": { "Anger": zzz, "Hapiness": zzz, "Neutral": zzz, "Sadness": zzz, "Surprise": zzz } } xxxx: Top-left coordinate, width and height of the rectangle circumscribing the detected face in LCD display coordinate system zzz: the value indicating the expression estimated from the detected face
//
- Body detection data
{ "RecodeType": "HVC-P2(body)" "id": <GR-PEACH ID>-<Sensor ID>" "BodyRectangle": { "Height": xxxx, "Left": xxxx, "Top": xxxx, "Width": xxxx, } } xxxx: Top-left coordinate, width and height of the rectangle circumscribing the detected body in LCD display coordinate system
概要
本プログラムは、オムロン社製HVC-P2 (Human Vision Components B5T-007001)で収集した各種認識データを、富士通社のIoT Platform ( http://jp.fujitsu.com/solutions/cloud/k5/function/paas/iot-platform/ ) に送信するサンプルプログラムです。
ハードウェア構成
- GR-PEACH 1式 ( https://developer.mbed.org/platforms/Renesas-GR-PEACH/ )
- LCD Shield 1式 ( https://developer.mbed.org/teams/Renesas/Wiki/LCD-shield )
- HVC-P2 1式 ( Human Vision Components B5T-007001 ) ( https://plus-sensin.omron.com/product/B5T-007001/ )
- USBA - Micro USBケーブル 2式
- USBA (メス) - Micro USB (オス)変換アダプタ 1式
本プログラムを動作させる場合、上記H/Wを下図のように接続してください。
また下図に示すGR-PEACHのJP3をショートしてください。
アプリケーションの事前設定
- Ethernetの設定を行います。詳細は下記リンクを参照ください。
https://developer.mbed.org/teams/Renesas/code/GR-PEACH_IoT_Platform_HTTP_sample/wiki/Ethernet-settings - HVC-P2で収集したデータを格納するリソース、およびそのアクセスコードをIoT Platform上で設定します。詳細は下記リンクを参照ください。
https://iot-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/userguide_en.pdf https://iot-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/apireference_en.pdf https://iot-docs.jp-east-1.paas.cloud.global.fujitsu.com/en/manual/portalmanual_en.pdf
ビルド手順
- 本サンプルプログラムをmbed Compilerにインポートします
- 下記に示すGR-PEACH_HVC-P2_IoTPlatform_http/IoT_Platform/iot_platform.cpp中の<ACCESS CODE> を、IoT Platform上で設定したアクセスコードで上書きしてください。<Access Code>の設定方法につきましては、上述のApplication Setupを参照願います。また #errorマクロで始まる行を削除してください。
Access Code configuration
#define ACCESS_CODE <Access CODE> #error "You need to replace <Access CODE for your resource> with yours"
- 下記に示すGR-PEACH_HVC-P2_IoTPlatform_http/IoT_Platform/iot_platform.cpp中の<Base URI>と <Tenant ID>、および<Path-to-Resource>>を適当な値に置換えるとともに、#errorマクロで始まる行を削除してください。ここで、<Base URI>、 <Tenant ID>の詳細につきましては富士通社へご確認願います。また<Path-to-Resource>>につきましては、Application Setupの項を参照ください。
URI configuration
std::string put_uri_base("<Base URI>/v1/<Tenant ID>/<Path-to-Resource>.json"); #error "You need to replace <Base URI>, <Tenant ID> and <Path-to-Resource> with yours" (中略) std::string get_uri("<Base URI>/v1/<Tenant ID>/<Path-to-Resource>/_past.json"); #error "You need to replace <Base URI>, <Tenant ID> and <Path-to-Resource> with yours"
- プログラムをコンパイルします。コンパイルが正常終了すると、バイナリがお使いのPCにダウンロードされます。
- GR-PEACHのRJ-45コネクタにEthernetケーブルを差し込んでください。
- USBA - Micro USBケーブルを、GR-PEACHのRESETボタンの隣に配置されたMicro USBポートに差し込んでください。GR-PEACHが正常に認識されると、下図に示すようにGR-PEACHがmbedという名称のUSBドライブとして認識されます。
- ダウンロードしたバイナリをmbedドライブにコピーします。
- RESETボタンを押下してプログラムを実行します。正常に実行された場合、下記に示すメッセージがターミナル上に表示されます。
送信データフォーマット
本プログラムでは、HVC-P2が収集した認識データを下記のJSONフォーマットにシリアライズし、IoT Platformへ送信します。
- Face detection data
{ "RecodeType": "HVC-P2(face)" "id": <GR-PEACH ID>-<Sensor ID>" "FaceRectangle": { "Height": xxxx, "Left": xxxx, "Top": xxxx, "Width": xxxx, }, "Gender": "male" or "female", "Scores": { "Anger": zzz, "Hapiness": zzz, "Neutral": zzz, "Sadness": zzz, "Surprise": zzz } } xxxx: LCD表示座標系における検出した顔に外接する矩形の左上座標・幅・高さ zzz: 検出した顔から推定した各種感情を示す数値
//
- Body detection data
{ "RecodeType": "HVC-P2(body)" "id": <GR-PEACH ID>-<Sensor ID>" "BodyRectangle": { "Height": xxxx, "Left": xxxx, "Top": xxxx, "Width": xxxx, } } xxxx: LCD表示座標系における検出した人体に外接する矩形の左上座標・幅・高さ
Diff: HVCApi/HVCDef.h
- Revision:
- 0:813a237f1c50
diff -r 000000000000 -r 813a237f1c50 HVCApi/HVCDef.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/HVCApi/HVCDef.h Thu Sep 07 18:15:54 2017 +0900 @@ -0,0 +1,235 @@ +/*---------------------------------------------------------------------------*/ +/* Copyright(C) 2017 OMRON Corporation */ +/* */ +/* Licensed under the Apache License, Version 2.0 (the "License"); */ +/* you may not use this file except in compliance with the License. */ +/* You may obtain a copy of the License at */ +/* */ +/* http://www.apache.org/licenses/LICENSE-2.0 */ +/* */ +/* Unless required by applicable law or agreed to in writing, software */ +/* distributed under the License is distributed on an "AS IS" BASIS, */ +/* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. */ +/* See the License for the specific language governing permissions and */ +/* limitations under the License. */ +/*---------------------------------------------------------------------------*/ + +#ifndef HVCDef_H__ +#define HVCDef_H__ + +/*----------------------------------------------------------------------------*/ +/* Execution flag */ +#define HVC_ACTIV_BODY_DETECTION 0x00000001 +#define HVC_ACTIV_HAND_DETECTION 0x00000002 +#define HVC_ACTIV_FACE_DETECTION 0x00000004 +#define HVC_ACTIV_FACE_DIRECTION 0x00000008 +#define HVC_ACTIV_AGE_ESTIMATION 0x00000010 +#define HVC_ACTIV_GENDER_ESTIMATION 0x00000020 +#define HVC_ACTIV_GAZE_ESTIMATION 0x00000040 +#define HVC_ACTIV_BLINK_ESTIMATION 0x00000080 +#define HVC_ACTIV_EXPRESSION_ESTIMATION 0x00000100 +#define HVC_ACTIV_FACE_RECOGNITION 0x00000200 + +/* Image info of Execute command */ +#define HVC_EXECUTE_IMAGE_NONE 0x00000000 +#define HVC_EXECUTE_IMAGE_QVGA 0x00000001 +#define HVC_EXECUTE_IMAGE_QVGA_HALF 0x00000002 + +/*----------------------------------------------------------------------------*/ +/* Error code */ + +/* Parameter error */ +#define HVC_ERROR_PARAMETER -1 + +/* Send signal timeout error */ +#define HVC_ERROR_SEND_DATA -10 + +/* Receive header signal timeout error */ +#define HVC_ERROR_HEADER_TIMEOUT -20 +/* Invalid header error */ +#define HVC_ERROR_HEADER_INVALID -21 +/* Receive data signal timeout error */ +#define HVC_ERROR_DATA_TIMEOUT -22 + + +/*----------------------------------------------------------------------------*/ +/* Album data size */ +#define HVC_ALBUM_SIZE_MIN 32 +#define HVC_ALBUM_SIZE_MAX 816032 + +/*----------------------------------------------------------------------------*/ +/* Expression */ +typedef enum { + EX_NEUTRAL = 1, + EX_HAPPINESS, + EX_SURPRISE, + EX_ANGER, + EX_SADNESS +}EXPRESSION; + +/*----------------------------------------------------------------------------*/ +/* Struct */ +/*----------------------------------------------------------------------------*/ +/*----------------------------------------------------------------------------*/ +/* Devicefs model and version info */ +/*----------------------------------------------------------------------------*/ +typedef struct { + UINT8 string[12]; + UINT8 major; + UINT8 minor; + UINT8 relese; + UINT8 revision[4]; +}HVC_VERSION; + +/*----------------------------------------------------------------------------*/ +/* Detection result */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 posX; /* Center x-coordinate */ + INT32 posY; /* Center y-coordinate */ + INT32 size; /* Size */ + INT32 confidence; /* Degree of confidence */ +}DETECT_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Face direction */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 yaw; /* Yaw angle */ + INT32 pitch; /* Pitch angle */ + INT32 roll; /* Roll angle */ + INT32 confidence; /* Degree of confidence */ +}DIR_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Age */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 age; /* Age */ + INT32 confidence; /* Degree of confidence */ +}AGE_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Gender */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 gender; /* Gender */ + INT32 confidence; /* Degree of confidence */ +}GENDER_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Gaze */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 gazeLR; /* Yaw angle */ + INT32 gazeUD; /* Pitch angle */ +}GAZE_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Blink */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 ratioL; /* Left eye blink result */ + INT32 ratioR; /* Right eye blink result */ +}BLINK_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Expression */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 topExpression; /* Top expression */ + INT32 topScore; /* Top score */ + INT32 score[5]; /* Score of 5 expression */ + INT32 degree; /* Negative-positive degree */ +}EXPRESSION_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Face Recognition */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 uid; /* User ID */ + INT32 confidence; /* Degree of confidence */ +}RECOGNITION_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Face Detection & Estimations result */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + DETECT_RESULT dtResult; /* Face detection result */ + DIR_RESULT dirResult; /* Face direction estimation result */ + AGE_RESULT ageResult; /* Age Estimation result */ + GENDER_RESULT genderResult; /* Gender Estimation result */ + GAZE_RESULT gazeResult; /* Gaze Estimation result */ + BLINK_RESULT blinkResult; /* Blink Estimation result */ + EXPRESSION_RESULT expressionResult; /* Expression Estimation result */ + RECOGNITION_RESULT recognitionResult; /* Face Recognition result */ +}FACE_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Human Body Detection results */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + UINT8 num; /* Number of Detection */ + DETECT_RESULT bdResult[35]; /* Detection result */ +}BD_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Hand Detection results */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + UINT8 num; /* Number of Detection */ + DETECT_RESULT hdResult[35]; /* Detection result */ +}HD_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Face Detection & Estimations results */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + UINT8 num; /* Number of Detection */ + FACE_RESULT fcResult[35]; /* Detection & Estimations result */ +}FD_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Image data */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 width; + INT32 height; + UINT8 image[320*240]; +}HVC_IMAGE; + +/*----------------------------------------------------------------------------*/ +/* Eesult data of Execute command */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 executedFunc; /* Execution flag */ + BD_RESULT bdResult; /* Human Body Detection results */ + HD_RESULT hdResult; /* Hand Detection results */ + FD_RESULT fdResult; /* Face Detection & Estimations results */ + HVC_IMAGE image; /* Image data */ +}HVC_RESULT; + +/*----------------------------------------------------------------------------*/ +/* Threshold of confidence */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 bdThreshold; /* Threshold of confidence of Human Body Detection */ + INT32 hdThreshold; /* Threshold of confidence of Hand Detection */ + INT32 dtThreshold; /* Threshold of confidence of Face Detection */ + INT32 rsThreshold; /* Threshold of confidence of Face Recognition */ +}HVC_THRESHOLD; + +/*----------------------------------------------------------------------------*/ +/* Detection size */ +/*----------------------------------------------------------------------------*/ +typedef struct{ + INT32 bdMinSize; /* Minimum detection size of Human Body Detection */ + INT32 bdMaxSize; /* Maximum detection size of Human Body Detection */ + INT32 hdMinSize; /* Minimum detection size of Hand Detection */ + INT32 hdMaxSize; /* Maximum detection size of Hand Detection */ + INT32 dtMinSize; /* Minimum detection size of Face Detection */ + INT32 dtMaxSize; /* Maximum detection size of Face Detection */ +}HVC_SIZERANGE; + +#endif /* HVCDef_H__ */ +