RT-Thread-Classify images of clothing@ART-Pi SmartRT-Thread问答社区

Classify images of clothing@ART-Pi Smart

发布于 2022-05-08 23:28:14 浏览：624 订阅该版

[tocm]

#### 参考资料
* [Basic classification: Classify images of clothing](https://www.tensorflow.org/tutorials/keras/classification)
* [person_detection](https://github.com/tensorflow/tflite-micro/tree/main/tensorflow/lite/micro/examples/person_detection)

#### 部署的过程
![misc-Page-1.drawio.png](https://oss-club.rt-thread.org/uploads/20220508/01722ea9420c65a21eaa881de230ad15.png.webp)

通过上述过程，我了解到如下几点：
1. 量化模型的概念
2. 如何在 PC 端对量化后的模型进行测试，因为这个是部署到控制器上的模型，所以这样对比才有意义
3. 同样的数据，在模型训练的时候，数据类型不同训练出来的模型也会有一定程度的区别，特别地，以我实际的测试发现，针对 Basic classification: Classify images of clothing 例子，使用 int8_t 训练出来的模型准确率大概在 50%，使用 float32 训练出来的模型准确率大概在 85%。
4. 有大佬告诉我可以对输入的 int8_t 进行处理，可以改善模型准确度，这个暂未尝试。

在 PC 端对 tf 模型进行测试的脚本：
``` python
#!/usr/bin/env python

import numpy as np
import tensorflow as tf
import sys
#import cv2

# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="models/model.tflite")
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

print(input_details)
print(output_details)

# 指定识别的图片名字
if len(sys.argv) == 1:
    IMG_SRC = '../web_pics/boots.jpg'
else:
    IMG_SRC = '../web_pics/' + sys.argv[1]

# 将图片转换为 28*28 像素的灰度图
from PIL import Image
raw_image = Image.open(IMG_SRC)
image = raw_image.resize((28, 28))
image_gray = image.convert('L')
image_gray_array = np.array(image_gray, dtype=np.float32)
input_data = image_gray_array.reshape(1,28,28)
# 进行灰度反向
input_data = 255.0 - input_data
input_data = np.array(input_data, dtype=np.float32)

index = input_details[0]['index']
interpreter.set_tensor(index, input_data)

interpreter.invoke()
out_data = interpreter.get_tensor(output_details[0]['index'])
# 获取打印输出，查看 10 种类型的概率分布
print("outdata:", out_data)
```
在 PC 断对量化后的模型测试准确度通过之后，再次部署到 ART-Pi Smart 上。这时候发现输出正常了。
```
 \ | /
- RT -     Thread Smart Operating System
 / | \     5.0.0 build May  7 2022
 2006 - 2020 Copyright by rt-thread team
do components initialization.
initialize rti_board_end:0 done
initialize dfs_init:0 done
initialize rt_mmcsd_core_init:0 done
initialize rt_usbd_winusb_class_register:0 done
initialize futex_system_init:0 done
initialize pmutex_system_init:0 done
initialize lwip_system_initlwIP-2.1.2 initialized!
:0 done
initialize ulog_console_backend_init:0 done
initialize ulog_init:0 done
initialize rt_hw_adc_init:0 done
initialize rt_hw_csi_init:0 done
initialize rt_hw_dbg_mm_init:0 done
initialize imx6ul_eth_initred first va=f00here 0000
here 1111 mmu_l2=c107c000
[63] E/drv.enet: emac device init success
:0 done
initialize rt_hw_i2c_init[70] I/drv.i2c: src_clock=66000000.
[74] I/I2C: I2C bus [i2c1] registered
[79] I/I2C: I2C bus [i2c3] registered
[83] I/I2C: I2C bus [i2c4] registered
:0 done
initialize rt_hw_elcd_init[91] D/drv.lcd: fb address => 0x811c0000

:0 done
initialize rt_hw_pwm_init:0 done
initialize rt_hw_rtc_init:0 done
initialize imxrt_mci_init:0 done
initialize rt_hw_spi_init:0 done
initialize gt911_init[162] E/drv.enet: PHY Link down, please check the cable connection and link partner setting.
[196] I/SDIO: emmc card capacity 3817472 KB.
found part[0], begin: 2098176, size: 500.0MB
found part[1], begin: 526386176, size: 1.0GB
found part[2], begin: 1600128000, size: 10.0MB
found part[3], begin: 1610613760, size: 200.0MB
[233] I/touch: rt_touch init success
[237] I/gt911: touch device gt911 init success
:0 done
initialize rt_usbd_init:0 done
initialize rt_hw_wdt_init:0 done
initialize null_device_init:0 done
initialize random_device_init:0 done
initialize urandom_device_init:0 done
initialize zero_device_init:0 done
initialize rt_work_sys_workqueue_init:0 done
initialize ptmx_register:0 done
initialize critical_init:0 done
initialize rt_hw_ov5640_init:0 done
initialize cplusplus_system_init:0 done
initialize elm_init:0 done
initialize dfs_romfs_init:0 done
initialize rt_i2c_core_init:0 done
initialize libc_system_init:0 done
initialize clock_time_system_init:0 done
initialize sal_init[295] I/sal.skt: Socket Abstraction Layer initialize success.
:0 done
initialize mnt_initemmc file system initialization done!
:0 done
initialize TF_lite_init[5213] D/drv.enet: enet1 link up
:0 done
initialize em_initopen em failed!
:-1 done
initialize set_adc_init:0 done
initialize set_pwm_default:0 done
initialize rt_sysinfo_init:0 done
initialize finsh_system_init:0 done
Red say: hello rt-smart.
open lcd okwatchdog start ing.....
go here
get mode begin.
get here ?
get here 000?
construct the class.
init ok.
interpreter: 1:Setup done
get image car
show image car
get invoke done
---------BEGIN------------
T-shirt/top:1.8725515*2^2
Trouser:1.0142987*2^5
Pullover:1.2483676*2^1
Dress:1.1443371*2^4
Coat:-1.2483676*2^2
Sandal:-1.8725515*2^2
Shirt:1.2483676*2^2
Sneaker:-1.8985596*2^5
Bag:-1.0403066*2^3
Ankle boot:-1.8205365*2^4
---------END------------
```
上述打印就是我识别的一条裤子的打印，这条裤子大概这个样子：
![pants_gry.bmp](https://oss-club.rt-thread.org/uploads/20220508/8a822443233b2aad0bd482337331fc7c.bmp)

#### 补充完善

为了更加方便地对图像进行识别，我计划实现这样的效果：
1. 支持命令行根据传入的图片参数对图像进行识别
2. 同步通过将图片显示在屏幕
3. 将识别的概率分布输出到串口

根据上述要求，我这样做：
![misc-Page-3.drawio.png](https://oss-club.rt-thread.org/uploads/20220508/ae1a2a6adec04d36d526ba33d334d131.png.webp)

在这个过程中，我发现了一个 float 输出打印异常的奇怪现象，相关讨论在[浮点数打印，从内核态直接打印和从用户态访问设备再触发打印，效果不一致](https://club.rt-thread.org/ask/question/0092203e5a1e27b4.html)。此处就不再过多赘述。

#### 测试过程
测试我以从网上下载的一个包的图片为例进行展示，过程大概分为3个步骤：
![misc-Page-4.drawio.png](https://oss-club.rt-thread.org/uploads/20220508/ab43486e453f6d1dca6d6a2708f8dd5b.png.webp)

我从网上下载了图片是这样的：
![bag.png](https://oss-club.rt-thread.org/uploads/20220508/502bf7aba4e084dbcfb317bfc8a8e9f4.png.webp)

转换脚本大概是这样的：
``` python
#!/usr/bin/env python

from PIL import Image
import os
import numpy as np
import sys
import matplotlib.pyplot as plt

# 从指定的目录查找原图
WEB_PICS = 'web_pics'
if len(sys.argv) == 1:
    IMG_SRC = 'pants.bmp'
else:
    IMG_SRC = sys.argv[1]

# 设置转换后的灰度图的名字
IMG_TARGET = IMG_SRC.split('.')[0] + '_gry' + '.bmp'
CC_TARGET = IMG_SRC.split('.')[0] + '.cc'
print(IMG_TARGET)
FULL_IMG_SRC = os.path.join(WEB_PICS, IMG_SRC)
print(FULL_IMG_SRC)

# 打开原图，修改大小，灰度转换
raw_image = Image.open(FULL_IMG_SRC)
image = raw_image.resize((28, 28))
image_gray = image.convert('L')
print(image_gray)
image_gray_array = np.array(image_gray, dtype=np.uint8)
# 灰度反向处理
image_gray_array = 255 - image_gray_array
print(type(image_gray_array))
np.set_printoptions(threshold=sys.maxsize)
person = image_gray_array.reshape(1,28 * 28)
person = np.array(person, dtype=int)
np.savetxt(CC_TARGET, person, fmt='%d', delimiter=',', header="const unsigned char g_person[]={", footer="};", comments='')
#print(person)
#np.save("person.c", person)
print(image_gray_array.shape)

# 显示转换后的灰度图
plt.figure()
plt.imshow(image_gray_array, cmap=plt.cm.binary)
im = Image.fromarray(image_gray_array, mode='L')
# 将图片保存
im.save(IMG_TARGET)
plt.show()
exit()
```
将下载的图片放到 web_pics 目录，执行的该脚本：
``` bash
▸ ./convert2gray.py fbags.png
fbags_gry.bmp
web_pics/fbags.png
<PIL.Image.Image image mode=L size=28x28 at 0x7F054393AC70>
<class 'numpy.ndarray'>
(28, 28)
```
此时，当前目录下会生成一个 **fbags_gry.bmp**
![](../Projects/learn_tf/fbags_gry.bmp)

```bash
▸ file fbags_gry.bmp
fbags_gry.bmp: PC bitmap, Windows 3.x format, 28 x 28 x 8, image size 784, resolution 3780 x 3780 px/m, 256 important colors, cbSize 1862, bits offset 1078
```
将该文件通过 [tftpx](https://github.com/ideawu/tftpx/blob/master/client.c) 下载到 ART-Pi Smart 的 **root/bin/** 目录。
执行测试程序：
```
msh /root/bin>tftpx.elf 192.168.91.103 69
msh /root/bin>Usage: cmd  arg0[,arg1,arg2...]
  -Directory listing:
    list path
  -Download a file from the server:
    get remote_file[ local_file]
  -Upload a file to the server:
    put filename
  -Set blocksize:
    blocksize size
  -Quit this programm:
    quit
get fbags_gry.bmp
Connect to server at 192.168.91.103:69>> DATA: block=1, data_size=512
DATA: block=2, data_size=512
DATA: block=3, data_size=512
DATA: block=4, data_size=326
quit
>>
msh /root/bin>sho
show_bmp.elf
shoow_bmp.elf
msh /root/bin>show
msh /root/bin>show_bmp.elf fba
msh /root/bin>show_bmp.elf fbags_gry.bmp
msh /root/bin>screen: lcd - 0x00195000, size 261120
screen: bpp 16, width - 480, height - 272
init lcd ok
use default display width height and addr
[dbg_header](14)42 4d 46 7 0 0 0 0 0 0 36 4 0 0
size=1862 offset=1078
size header:14 infoheader:40
width=28 height=28
---------BEGIN------------
T-shirt/top:1.6644905*2^-1
Trouser:-1.6644905*2^0
Pullover:1.6644905*2^-1
Dress:1.6644905*2^0
Coat:1.2483676*2^2
Sandal:1.6644905*2^0
Shirt:1.6644905*2^0
Sneaker:-1.2483676*2^1
Bag:1.5604596*2^3        --------------------- 可以看出将这个图片识别为包的概率最大
Ankle boot:-1.2483676*2^2
---------END------------
show bmp ok
```
屏幕上的同步显示：
![fbag.jpg](https://oss-club.rt-thread.org/uploads/20220508/ba6a8dff237b883fb49a123503ff12ca.jpg.webp)

#### 相关源码
* [image_class](https://gitee.com/iysheng/ART-Pi-smart/tree/image_class/)
* [learn_tf](https://gitee.com/iysheng/learn_tf.git)