RT-Thread-在 ART-Pi Smart 上运行 TensorFlow LiteRT-Thread问答社区

在 ART-Pi Smart 上运行 TensorFlow Lite

发布于 2022-04-30 19:06:47 浏览：862 订阅该版

[tocm]

#### 在 ART-Pi Smart 上运行 TensorFlow Lite

* [TensorFlow Lite 官方仓库](https://github.com/iysheng/tflite-micro)
* [我的ART-Pi-smart仓库](https://gitee.com/iysheng/ART-Pi-smart)

经过了多天的摸索学习，今天将 TensorFlow Lite 的框架移植到了 **ART-Pi Smart** 上，但是还是存在一点问题，主要是 C++ 和 C 语言混合编译的问题，根据目前我的总结，具体体现在：
* TensorFlow Lite 在函数中定义的静态对象无法正常构造，必须放在全局变量的位置
* 全局变量定义的对象只能使用默认的构造函数，无发使用带有参数的构造函数初始对象

针对上述两个现象，我在运行测试 Hello world 例程时，对  TensorFlow Lite 做出来的主要改动是：
```diff
diff --git a/kernel/bsp/imx6ull-artpi-smart/applications/tflm_tree/examples/hello_world/main_functions.cc b/kernel/bsp/imx6ull-artpi-smart/applications/tflm_tree/examples/hello_world/main_functions.cc
index a585ba7..a318c32 100644
--- a/kernel/bsp/imx6ull-artpi-smart/applications/tflm_tree/examples/hello_world/main_functions.cc
+++ b/kernel/bsp/imx6ull-artpi-smart/applications/tflm_tree/examples/hello_world/main_functions.cc
@@ -37,18 +37,30 @@ constexpr int kTensorArenaSize = 2000;
 uint8_t tensor_arena[kTensorArenaSize];
 }  // namespace
 
+static tflite::MicroErrorReporter micro_error_reporter;
+  // This pulls in all the operation implementations we need.
+  // NOLINTNEXTLINE(runtime-global-variables)
+  static tflite::AllOpsResolver resolver;
+  static tflite::MicroInterpreter *static_interpreter;
+
 // The name of this function is important for Arduino compatibility.
 void setup() {
+
+  DebugLog("setup begin.\n");
   tflite::InitializeTarget();
 
+  DebugLog("initial ok.\n");
+
   // Set up logging. Google style is to avoid globals or statics because of
   // lifetime uncertainty, but since this has a trivial destructor it's okay.
   // NOLINTNEXTLINE(runtime-global-variables)
-  static tflite::MicroErrorReporter micro_error_reporter;
+  /* 这个地方有问题 */
   error_reporter = µ_error_reporter;
 
   // Map the model into a usable data structure. This doesn't involve any
   // copying or parsing, it's a very lightweight operation.
+  // printf("value:%x %u\n", g_hello_world_model_data[0], g_hello_world_model_data_size);
+
   model = tflite::GetModel(g_hello_world_model_data);
   if (model->version() != TFLITE_SCHEMA_VERSION) {
     TF_LITE_REPORT_ERROR(error_reporter,
@@ -57,15 +69,10 @@ void setup() {
                          model->version(), TFLITE_SCHEMA_VERSION);
     return;
   }
-
-  // This pulls in all the operation implementations we need.
-  // NOLINTNEXTLINE(runtime-global-variables)
-  static tflite::AllOpsResolver resolver;
-
   // Build an interpreter to run the model with.
-  static tflite::MicroInterpreter static_interpreter(
+  static_interpreter = new tflite::MicroInterpreter(
       model, resolver, tensor_arena, kTensorArenaSize, error_reporter);
-  interpreter = &static_interpreter;
+  interpreter = static_interpreter;
 
   // Allocate memory from the tensor_arena for the model's tensors.
   TfLiteStatus allocate_status = interpreter->AllocateTensors();
@@ -102,6 +109,7 @@ int loop() {
   if (invoke_status != kTfLiteOk) {
     TF_LITE_REPORT_ERROR(error_reporter, "Invoke failed on x: %f\n",
                          static_cast<double>(x));
+    printf("invoke status:%x\n", invoke_status);
     return -1;
   }
```
参考的资料主要有：
1. tinyML基TensorFlow Lite在Arduino
2. https://github.com/iysheng/tflite-micro/blob/main/tensorflow/lite/micro/docs/new_platform_support.md

特别关键的一点是提取基础框架的文件：
``python3 tensorflow/lite/micro/tools/project_generation/create_tflm_tree.py -e hello_world /tmp/tflm-tree``
根据官方的文档，上述命令就可以将 hello_world 工程相关的文件提取到指定的 /tmp/tflm-tree 目录，下一步做的就是将这部分内容添加到 RT-Thread Smart 工程中，编译出来。相关的代码，我都上传到了我的 **ART-Pi Smart** 的仓库。如果有问题欢迎和我讨论。

展示下，运行起来的效果，为了展示需要，删除了部分空行打印：
```
U-Boot 2017.03-g002b758ac9-dirty (Jul 08 2021 - 10:45:12 +0000)
CPU:   Freescale i.MX6ULL rev1.1 696 MHz (running at 396 MHz)
CPU:   Industrial temperature grade (-40C to 105C) at 44C
Reset cause: POR
Model: Freescale i.MX6 ULL 14x14 EVK Board
Board: MX6ULL 14x14 EVK
DRAM:  512 MiB
MMC:   FSL_SDHC: 0, FSL_SDHC: 1
Display: TFT7016 (1024x600)
Video: 1024x600x24
In:    serial
Out:   serial
Err:   serial
switch to partitions #0, OK
mmc1(part 0) is current device
Net:   eth0: ethernet@02188000 [PRIME]
Normal Boot
Hit any key to stop autoboot:  3  2  1  0 
ethernet@02188000 Waiting for PHY auto negotiation to complete.... done
Using ethernet@02188000 device
TFTP from server 10.20.52.39; our IP address is 10.20.52.89
Filename 'rtthread.bin'.
Load address: 0x80001000
Loading: *#################################################################
	 ###############################################################
	 159.2 KiB/s
done
Bytes transferred = 1866656 (1c7ba0 hex)
## Starting application at 0x80001000 ...
 \ | /
- RT -     Thread Smart Operating System
 / | \     5.0.0 build Apr 30 2022
 2006 - 2020 Copyright by rt-thread team
do components initialization.
initialize rti_board_end:0 done
initialize dfs_init:0 done
initialize rt_mmcsd_core_init:0 done
initialize rt_usbd_winusb_class_register:0 done
initialize futex_system_init:0 done
initialize pmutex_system_init:0 done
initialize lwip_system_initlwIP-2.1.2 initialized!
:0 done
initialize ulog_console_backend_init:0 done
initialize ulog_init:0 done
initialize rt_hw_adc_init:0 done
initialize rt_hw_csi_init:0 done
initialize rt_hw_dbg_mm_init:0 done
initialize imx6ul_eth_initred first va=f00here 0000
here 1111 mmu_l2=c107c000
[31m[63] E/drv.enet: emac device init success
[0m:0 done
initialize rt_hw_i2c_init[32m[70] I/I2C: I2C bus [i2c1] registered
[0m[32m[75] I/I2C: I2C bus [i2c3] registered
[0m[32m[79] I/I2C: I2C bus [i2c4] registered
[0m:0 done
initialize rt_hw_elcd_init[86] D/drv.lcd: fb address => 0x811c0000
:0 done
initialize rt_hw_pwm_init:0 done
initialize rt_hw_rtc_init:0 done
initialize imxrt_mci_init:0 done
initialize rt_hw_spi_init:0 done
initialize gt911_init[31m[158] E/drv.enet: PHY Link down, please check the cable connection and link partner setting.
[0m[32m[192] I/SDIO: emmc card capacity 3817472 KB.
[0mfound part[0], begin: 2098176, size: 500.0MB
found part[1], begin: 526386176, size: 1.0GB
found part[2], begin: 1600128000, size: 10.0MB
found part[3], begin: 1610613760, size: 200.0MB
[32m[328] I/touch: rt_touch init success
[0m[32m[332] I/gt911: touch device gt911 init success
[0m:0 done
initialize rt_usbd_init:0 done
initialize rt_hw_wdt_init:0 done
initialize null_device_init:0 done
initialize random_device_init:0 done
initialize urandom_device_init:0 done
initialize zero_device_init:0 done
initialize rt_work_sys_workqueue_init:0 done
initialize ptmx_register:0 done
initialize critical_init:0 done
initialize rt_hw_ov5640_init:0 done
initialize cplusplus_system_init:0 done
initialize elm_init:0 done
initialize dfs_romfs_init:0 done
initialize rt_i2c_core_init:0 done
initialize libc_system_init:0 done
initialize clock_time_system_init:0 done
initialize sal_init[32m[390] I/sal.skt: Socket Abstraction Layer initialize success.
[0m:0 done
initialize mnt_initemmc file system initialization done!
:0 done
initialize TF_lite_init[5309] D/drv.enet: enet1 link up
:0 done
initialize em_initopen em failed!
:-1 done
initialize set_adc_init:0 done
initialize set_pwm_default:0 done
initialize rt_sysinfo_init:0 done
initialize finsh_system_init:0 done
Red say: hello rt-smart. 
open lcd okwatchdog start ing.....
test tf.
setup begin.
go here
initial ok.
get mode begin.
get here ?
get here 000?
construct the class.
init ok.
x_value: 1.0*2^-127, y_value: 1.0297613*2^-7
x_value: 1.0053092*2^-5, y_value: 1.8020826*2^-5
x_value: 1.0053092*2^-4, y_value: 1.1584818*2^-4
x_value: 1.507964*2^-4, y_value: 1.9308027*2^-4
x_value: 1.0053092*2^-3, y_value: 1.0941217*2^-3
x_value: 1.2566366*2^-3, y_value: 1.4802819*2^-3
x_value: 1.507964*2^-3, y_value: 1.6733624*2^-3
x_value: 1.7592913*2^-3, y_value: 1.8664425*2^-3
x_value: 1.0053092*2^-2, y_value: 1.0619412*2^-2
x_value: 1.130973*2^-2, y_value: 1.1584818*2^-2
```
特别地：
```
x_value: 1.0*2^-127, y_value: 1.0297613*2^-7
x_value: 1.0053092*2^-5, y_value: 1.8020826*2^-5
x_value: 1.0053092*2^-4, y_value: 1.1584818*2^-4
x_value: 1.507964*2^-4, y_value: 1.9308027*2^-4
x_value: 1.0053092*2^-3, y_value: 1.0941217*2^-3
x_value: 1.2566366*2^-3, y_value: 1.4802819*2^-3
x_value: 1.507964*2^-3, y_value: 1.6733624*2^-3
x_value: 1.7592913*2^-3, y_value: 1.8664425*2^-3
x_value: 1.0053092*2^-2, y_value: 1.0619412*2^-2
x_value: 1.130973*2^-2, y_value: 1.1584818*2^-2
...
```
上述部分是模型估算的正弦函数的数据，我绘制成图表的效果如下：
![](https://oss-club.rt-thread.org/uploads/20220714/40166fd09f07e73f5ca6d0a78a7756e19d082182.png)

效果还是不错的，这使用的是官方例程 16 \* 16 \* 1 的三层模型。