【Arduino】试验带识别模块的吃鸡助手

在前面的试验中,我们试验了声音触发点击,方面多指操作的辅助功能,

【Arduino】自制声控点击器(吼叫吃鸡助手)-CSDN博客

如果声控模块换成图像识别模块,就是一个自动识别并shot的功能了,

通过这个试验,可以了解arduino串口通信的使用,蓝牙模块和arduino交互就是通过串口交互的方式。

看看arduino自带的串口例子

cpp 复制代码
/*
  Serial Event example

  When new serial data arrives, this sketch adds it to a String.
  When a newline is received, the loop prints the string and clears it.

  A good test for this is to try it with a GPS receiver that sends out
  NMEA 0183 sentences.

  NOTE: The serialEvent() feature is not available on the Leonardo, Micro, or
  other ATmega32U4 based boards.

  created 9 May 2011
  by Tom Igoe

  This example code is in the public domain.

  https://www.arduino.cc/en/Tutorial/BuiltInExamples/SerialEvent
*/

String inputString = "";      // a String to hold incoming data
bool stringComplete = false;  // whether the string is complete

void setup() {
  // initialize serial:
  Serial.begin(9600);
  // reserve 200 bytes for the inputString:
  inputString.reserve(200);
}

void loop() {
  // print the string when a newline arrives:
  if (stringComplete) {
    Serial.println(inputString);
    // clear the string:
    inputString = "";
    stringComplete = false;
  }
}

/*
  SerialEvent occurs whenever a new data comes in the hardware serial RX. This
  routine is run between each time loop() runs, so using delay inside loop can
  delay response. Multiple bytes of data may be available.
*/
void serialEvent() {
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag so the main loop can
    // do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
  }
}

这里我们使用一个yb k210来作为识别设备,将其tx,rx和arduino的tx,rx交叉连接,

在其人脸识别模块的代码里添加串口通信处理serial.send,

python 复制代码
import sensor, image, time, lcd
from maix import KPU
import gc
from modules import ybserial

lcd.init()
sensor.reset()                      # Reset and initialize the sensor. It will
                                    # run automatically, call sensor.run(0) to stop
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 1000)     # Wait for settings take effect.
clock = time.clock()                # Create a clock object to track the FPS.

serial = ybserial()

anchor = (0.1075, 0.126875, 0.126875, 0.175, 0.1465625, 0.2246875, 0.1953125, 0.25375, 0.2440625, 0.351875, 0.341875, 0.4721875, 0.5078125, 0.6696875, 0.8984375, 1.099687, 2.129062, 2.425937)
kpu = KPU()
kpu.load_kmodel("/sd/KPU/yolo_face_detect/face_detect_320x240.kmodel")
kpu.init_yolo2(anchor, anchor_num=9, img_w=320, img_h=240, net_w=320 , net_h=240 ,layer_w=10 ,layer_h=8, threshold=0.5, nms_value=0.2, classes=1)

lm68_kpu = KPU()
print("ready load model")
lm68_kpu.load_kmodel("/sd/KPU/face_detect_with_68landmark/landmark68.kmodel")

RATIO = 0.08
def extend_box(x, y, w, h, scale):
    x1_t = x - scale*w
    x2_t = x + w + scale*w
    y1_t = y - scale*h
    y2_t = y + h + scale*h
    x1 = int(x1_t) if x1_t>1 else 1
    x2 = int(x2_t) if x2_t<320 else 319
    y1 = int(y1_t) if y1_t>1 else 1
    y2 = int(y2_t) if y2_t<240 else 239
    cut_img_w = x2-x1+1
    cut_img_h = y2-y1+1
    return x1, y1, cut_img_w, cut_img_h

while 1:
    gc.collect()
    #print("mem free:",gc.mem_free())
    clock.tick()                    # Update the FPS clock.
    img = sensor.snapshot()
    kpu.run_with_output(img)
    dect = kpu.regionlayer_yolo2()
    fps = clock.fps()
    if len(dect) > 0:
        print("dect:",dect)
        text = 'Hello11\n'
        num = serial.send(text)
        print("num:", num)
        for l in dect :
            x1, y1, cut_img_w, cut_img_h = extend_box(l[0], l[1], l[2], l[3], scale=RATIO) # 扩大人脸框
            face_cut = img.cut(x1, y1, cut_img_w, cut_img_h)
            a = img.draw_rectangle(l[0],l[1],l[2],l[3], color=(0, 255, 0))
            face_cut_128 = face_cut.resize(128, 128)
            face_cut_128.pix_to_ai()
            out = lm68_kpu.run_with_output(face_cut_128, getlist=True)
            #print("out:",len(out))
            for j in range(68):
                x = int(KPU.sigmoid(out[2 * j])*cut_img_w + x1)
                y = int(KPU.sigmoid(out[2 * j + 1])*cut_img_h + y1)
                #a = img.draw_cross(x, y, size=1, color=(0, 0, 255))
                a = img.draw_circle(x, y, 2, color=(0, 0, 255), fill=True)
            del (face_cut_128)
            del (face_cut)

    a = img.draw_string(0, 0, "%2.1ffps" %(fps), color=(0, 60, 255), scale=2.0)
    lcd.display(img)
    gc.collect()

kpu.deinit()
lm68_kpu.deinit()

arduino收到串口信号时,调用继电器来实现通电,达到点击屏幕的效果,

在串口接收上添加了一点修改

cpp 复制代码
/*
  Serial Event example

  When new serial data arrives, this sketch adds it to a String.
  When a newline is received, the loop prints the string and clears it.

  A good test for this is to try it with a GPS receiver that sends out
  NMEA 0183 sentences.

  NOTE: The serialEvent() feature is not available on the Leonardo, Micro, or
  other ATmega32U4 based boards.

  created 9 May 2011
  by Tom Igoe

  This example code is in the public domain.

  https://www.arduino.cc/en/Tutorial/BuiltInExamples/SerialEvent
*/
int relayPin = 4;

String inputString = "";      // a String to hold incoming data
bool stringComplete = false;  // whether the string is complete

void setup() {
  // initialize serial:
  Serial.begin(115200);
  // reserve 200 bytes for the inputString:
  inputString.reserve(200);
      pinMode(relayPin, OUTPUT);
}

void loop() {
  // print the string when a newline arrives:
  if (stringComplete) {
    Serial.println(inputString);
    // clear the string:
    inputString = "";
    
    digitalWrite(relayPin, HIGH);
    delay(3000);
    digitalWrite(relayPin, LOW);
    stringComplete = false;
  }
  delay(10);
}

/*
  SerialEvent occurs whenever a new data comes in the hardware serial RX. This
  routine is run between each time loop() runs, so using delay inside loop can
  delay response. Multiple bytes of data may be available.
*/
void serialEvent() {
  if (stringComplete) {
    return;
  }
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag so the main loop can
    // do something about it:
    if (inChar == '\n') {
      stringComplete = true;
      break;
    }
  }
}

这里使用的是人脸检测,在吃鸡实战中效果不好,

人体检测会好一些,

去训练的话可以参考

http://t.csdnimg.cn/ZfoSE

【手把手在K210上部署自己在线训练的YOLO模型 - CSDN App】http://t.csdnimg.cn/uwcb9

http://t.csdnimg.cn/uwcb9

相关推荐
XianxinMao8 分钟前
2024大模型双向突破:MoE架构创新与小模型崛起
人工智能·架构
费曼的黑板11 分钟前
国产低功耗带LCD驱动和触摸按键功能的MCU
单片机·嵌入式硬件
Francek Chen19 分钟前
【深度学习基础】多层感知机 | 模型选择、欠拟合和过拟合
人工智能·pytorch·深度学习·神经网络·多层感知机·过拟合
pchmi1 小时前
C# OpenCV机器视觉:红外体温检测
人工智能·数码相机·opencv·计算机视觉·c#·机器视觉·opencvsharp
认知作战壳吉桔1 小时前
中国认知作战研究中心:从认知战角度分析2007年iPhone发布
大数据·人工智能·新质生产力·认知战·认知战研究中心
小猪写代码1 小时前
STM32 FreeRTOS内存管理简介
stm32·单片机
软件公司.乐学2 小时前
安全生产算法一体机定制
人工智能·安全
kcarly2 小时前
知识图谱都有哪些常见算法
人工智能·算法·知识图谱
dddcyy2 小时前
利用现有模型处理面部视频获取特征向量(3)
人工智能·深度学习
Fxrain2 小时前
[Computer Vision]实验三:图像拼接
人工智能·计算机视觉