RT-Thread-【24嵌入式设计大赛】基于RT_Thread Vision Board的光伏板清理机器人RT-Thread问答社区

【24嵌入式设计大赛】基于RT_Thread Vision Board的光伏板清理机器人

发布于 2024-09-14 23:56:27 浏览：289 订阅该版

[tocm]

##1.前言
随着全球对可再生能源需求的日益增长，光伏发电作为一种清洁、可持续的能源形式，在能源结构中的地位逐渐提高。然而，光伏板的发电效率极易受到外部环境的影响，尤其是灰尘、污垢、鸟粪等遮挡物的积累，会显著降低光伏系统的能量输出。研究表明，光伏板表面附着的灰尘和污垢会导致发电效率下降5%至30%，甚至更高。因此，光伏板的定期清洁对于维持其高效运行至关重要。同时避免了人力维护成本高，投入时间长，部分难操作等情况。结合当前人工智能技术，有个能自动进行光伏板灰尘清理的机器人是个值得研究的方向。
## 2.总体框架
总体的设计思想，通过RA8D1控制器搭载的RT-Thread操作系统，与TensorFlow Lite ,OpenMV实现光伏板上灰尘，障碍识别，然后对障碍物，灰尘等进行清理。实现发电效率最大化，以下是总体的设计结构。
![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/fc37be2ac5e58082ff25c218f8b3d41b.png)
清洁光伏板的前端，通过两个圆形的洗刷工具，自带电机，只需要给控制命令，不需要精确的控制。同时洗刷工具也接了导水管，给控制命令后即可开始清洗工作。

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/07ed2e8e0ef70d0874837e4fadbef953.png.webp)
## 3.开发配置
下载Vision Board相关资料，包括SDK，Openmv软件等，进入Vision Board\SDK\sdk-bsp-ra8d1-vision-board\projects\vision_board_openmv下，将library文件夹和rt-thread 文件夹复制到此路径。

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/654a0c2a4df7dc1e043e6856b32beee0.png.webp)
右击工程空白的地方，使用ENV打开

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/450fb106905986d93cb493610f24250b.png.webp)
输入menuconfig打开配置界面，Enable OpenMV for RT-Thread—>目录下，打开Enable tflite support功能，然后保存。

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/19485cc4d66726ccef1578328534759d.png.webp)
输入scons —target=mdk5重新生成mdk工程

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/4478d67e1a76e7ad5733008b08790bad.png)
点击新生产的工程文件，编辑，烧录

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/fa2367008cc4ea109f514629a5cf4ddd.png.webp)
## 4.模型训练
本次使用的是Edge impluse，需要先注册账号，然后新建project，依次点击Dashboard->Add existing data->Upload data。这里的素材需要提前准备好，以及图片的大小需要先处理好，选择相关文件后选择上传即可。

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/2df92c80cd2c4bcb3b21c63507cdad21.png)
保存后点击左侧Impulse design->Create Impulse，依次点击Add a processing block、Add a learning block进行输入数据对象、训练模型的选择，并点击保存，为了生成特征

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/fcfb4e6b40f539e474cd974be88472d9.png.webp)
接下来点击左侧Image，Color depth选择RGB，点击保存。接下来会自动跳转到生成特征界面，点击生成特征按钮，等待特征生成。

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/b0fd32fd477244fbab4b49044b799851.png.webp)
本次生成的特征如下：

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/b1b77763597cc811225718ebabd3ec05.png)
接下来开始训练，这里主要是提供的训练数据原因，导致的训练结果异常，这里可以通过调整参数来进行训练，直到达到自己想要的效果，下面是第一次训练

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/0e2fffcef822d714d0e6a1bd4c531f64.png.webp)
调整参数后：

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/8e9dd9d97e0cf90a32823a639f88582c.png.webp)
## 5.模型部署
如下图所示，点击部署，选择openmv库，生成压缩包

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/9d196ff867d9235878e675ede4c6aac7.png)
以下是解压后的估计，标签，训练模型以及代码

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/d6d76748c1fa025dbd41967167f70e82.png)
将代码重命名，且将这几个拷贝在内存卡里面即可

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/2def84eacfb2c4ea4ee652c4cf485b0c.png)
打开Openmv软件，连接开发板，打开main.py，同时可以打开终端看识别效果，当然代码自己做了调整，已经根据自己的需求进行调整。

```c
# Edge Impulse - OpenMV Object Detection Example

import sensor, image, time, os, tf, math, uos, gc

sensor.reset()                         # Reset and initialize the sensor.
sensor.set_pixformat(sensor.RGB565)    # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)      # Set frame size to QVGA (320x240)
sensor.set_windowing((240, 240))       # Set 240x240 window.
sensor.skip_frames(time=2000)          # Let the camera adjust.

net = None
labels = None
min_confidence = 0.5

try:
    # load the model, alloc the model file on the heap if we have at least 64K free after loading
    net = tf.load("trained.tflite", load_to_fb=uos.stat('trained.tflite')[6] > (gc.mem_free() - (64*1024)))
except Exception as e:
    raise Exception('Failed to load "trained.tflite", did you copy the .tflite and labels.txt file onto the mass-storage device? (' + str(e) + ')')

try:
    labels = [line.rstrip('\n') for line in open("labels.txt")]
except Exception as e:
    raise Exception('Failed to load "labels.txt", did you copy the .tflite and labels.txt file onto the mass-storage device? (' + str(e) + ')')

colors = [ # Add more colors if you are detecting more than 7 types of classes at once.
    (255,   0,   0),
    (  0, 255,   0),
    (255, 255,   0),
    (  0,   0, 255),
    (255,   0, 255),
    (  0, 255, 255),
    (255, 255, 255),
]

clock = time.clock()
while(True):
    clock.tick()

img = sensor.snapshot()

# detect() returns all objects found in the image (splitted out per class already)
    # we skip class index 0, as that is the background, and then draw circles of the center
    # of our objects

for i, detection_list in enumerate(net.detect(img, thresholds=[(math.ceil(min_confidence * 255), 255)])):
        if (i == 0): continue # background class
        if (len(detection_list) == 0): continue # no detections for this class?

print("********** %s **********" % labels[i])
        for d in detection_list:
            [x, y, w, h] = d.rect()
            center_x = math.floor(x + (w / 2))
            center_y = math.floor(y + (h / 2))
            print('x %d\ty %d' % (center_x, center_y))
            img.draw_circle((center_x, center_y, 12), color=colors[i], thickness=2)

print(clock.fps(), "fps", end="\n\n")

```
## 6.效果
光伏板上的障碍物检测效果：

![screenshot_image.png](https://oss-club.rt-thread.org/uploads/20240914/2f3948ec0db8c210d4252df574ff3b83.png.webp)

[演示.zip](https://club.rt-thread.org/file_download/3b1071982b2c8001)