【Arduino】试验带识别模块的吃鸡助手

在前面的试验中,我们试验了声音触发点击,方面多指操作的辅助功能,

【Arduino】自制声控点击器(吼叫吃鸡助手)-CSDN博客

如果声控模块换成图像识别模块,就是一个自动识别并shot的功能了,

通过这个试验,可以了解arduino串口通信的使用,蓝牙模块和arduino交互就是通过串口交互的方式。

看看arduino自带的串口例子

cpp 复制代码
/*
  Serial Event example

  When new serial data arrives, this sketch adds it to a String.
  When a newline is received, the loop prints the string and clears it.

  A good test for this is to try it with a GPS receiver that sends out
  NMEA 0183 sentences.

  NOTE: The serialEvent() feature is not available on the Leonardo, Micro, or
  other ATmega32U4 based boards.

  created 9 May 2011
  by Tom Igoe

  This example code is in the public domain.

  https://www.arduino.cc/en/Tutorial/BuiltInExamples/SerialEvent
*/

String inputString = "";      // a String to hold incoming data
bool stringComplete = false;  // whether the string is complete

void setup() {
  // initialize serial:
  Serial.begin(9600);
  // reserve 200 bytes for the inputString:
  inputString.reserve(200);
}

void loop() {
  // print the string when a newline arrives:
  if (stringComplete) {
    Serial.println(inputString);
    // clear the string:
    inputString = "";
    stringComplete = false;
  }
}

/*
  SerialEvent occurs whenever a new data comes in the hardware serial RX. This
  routine is run between each time loop() runs, so using delay inside loop can
  delay response. Multiple bytes of data may be available.
*/
void serialEvent() {
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag so the main loop can
    // do something about it:
    if (inChar == '\n') {
      stringComplete = true;
    }
  }
}

这里我们使用一个yb k210来作为识别设备,将其tx,rx和arduino的tx,rx交叉连接,

在其人脸识别模块的代码里添加串口通信处理serial.send,

python 复制代码
import sensor, image, time, lcd
from maix import KPU
import gc
from modules import ybserial

lcd.init()
sensor.reset()                      # Reset and initialize the sensor. It will
                                    # run automatically, call sensor.run(0) to stop
sensor.set_pixformat(sensor.RGB565) # Set pixel format to RGB565 (or GRAYSCALE)
sensor.set_framesize(sensor.QVGA)   # Set frame size to QVGA (320x240)
sensor.skip_frames(time = 1000)     # Wait for settings take effect.
clock = time.clock()                # Create a clock object to track the FPS.

serial = ybserial()

anchor = (0.1075, 0.126875, 0.126875, 0.175, 0.1465625, 0.2246875, 0.1953125, 0.25375, 0.2440625, 0.351875, 0.341875, 0.4721875, 0.5078125, 0.6696875, 0.8984375, 1.099687, 2.129062, 2.425937)
kpu = KPU()
kpu.load_kmodel("/sd/KPU/yolo_face_detect/face_detect_320x240.kmodel")
kpu.init_yolo2(anchor, anchor_num=9, img_w=320, img_h=240, net_w=320 , net_h=240 ,layer_w=10 ,layer_h=8, threshold=0.5, nms_value=0.2, classes=1)

lm68_kpu = KPU()
print("ready load model")
lm68_kpu.load_kmodel("/sd/KPU/face_detect_with_68landmark/landmark68.kmodel")

RATIO = 0.08
def extend_box(x, y, w, h, scale):
    x1_t = x - scale*w
    x2_t = x + w + scale*w
    y1_t = y - scale*h
    y2_t = y + h + scale*h
    x1 = int(x1_t) if x1_t>1 else 1
    x2 = int(x2_t) if x2_t<320 else 319
    y1 = int(y1_t) if y1_t>1 else 1
    y2 = int(y2_t) if y2_t<240 else 239
    cut_img_w = x2-x1+1
    cut_img_h = y2-y1+1
    return x1, y1, cut_img_w, cut_img_h

while 1:
    gc.collect()
    #print("mem free:",gc.mem_free())
    clock.tick()                    # Update the FPS clock.
    img = sensor.snapshot()
    kpu.run_with_output(img)
    dect = kpu.regionlayer_yolo2()
    fps = clock.fps()
    if len(dect) > 0:
        print("dect:",dect)
        text = 'Hello11\n'
        num = serial.send(text)
        print("num:", num)
        for l in dect :
            x1, y1, cut_img_w, cut_img_h = extend_box(l[0], l[1], l[2], l[3], scale=RATIO) # 扩大人脸框
            face_cut = img.cut(x1, y1, cut_img_w, cut_img_h)
            a = img.draw_rectangle(l[0],l[1],l[2],l[3], color=(0, 255, 0))
            face_cut_128 = face_cut.resize(128, 128)
            face_cut_128.pix_to_ai()
            out = lm68_kpu.run_with_output(face_cut_128, getlist=True)
            #print("out:",len(out))
            for j in range(68):
                x = int(KPU.sigmoid(out[2 * j])*cut_img_w + x1)
                y = int(KPU.sigmoid(out[2 * j + 1])*cut_img_h + y1)
                #a = img.draw_cross(x, y, size=1, color=(0, 0, 255))
                a = img.draw_circle(x, y, 2, color=(0, 0, 255), fill=True)
            del (face_cut_128)
            del (face_cut)

    a = img.draw_string(0, 0, "%2.1ffps" %(fps), color=(0, 60, 255), scale=2.0)
    lcd.display(img)
    gc.collect()

kpu.deinit()
lm68_kpu.deinit()

arduino收到串口信号时,调用继电器来实现通电,达到点击屏幕的效果,

在串口接收上添加了一点修改

cpp 复制代码
/*
  Serial Event example

  When new serial data arrives, this sketch adds it to a String.
  When a newline is received, the loop prints the string and clears it.

  A good test for this is to try it with a GPS receiver that sends out
  NMEA 0183 sentences.

  NOTE: The serialEvent() feature is not available on the Leonardo, Micro, or
  other ATmega32U4 based boards.

  created 9 May 2011
  by Tom Igoe

  This example code is in the public domain.

  https://www.arduino.cc/en/Tutorial/BuiltInExamples/SerialEvent
*/
int relayPin = 4;

String inputString = "";      // a String to hold incoming data
bool stringComplete = false;  // whether the string is complete

void setup() {
  // initialize serial:
  Serial.begin(115200);
  // reserve 200 bytes for the inputString:
  inputString.reserve(200);
      pinMode(relayPin, OUTPUT);
}

void loop() {
  // print the string when a newline arrives:
  if (stringComplete) {
    Serial.println(inputString);
    // clear the string:
    inputString = "";
    
    digitalWrite(relayPin, HIGH);
    delay(3000);
    digitalWrite(relayPin, LOW);
    stringComplete = false;
  }
  delay(10);
}

/*
  SerialEvent occurs whenever a new data comes in the hardware serial RX. This
  routine is run between each time loop() runs, so using delay inside loop can
  delay response. Multiple bytes of data may be available.
*/
void serialEvent() {
  if (stringComplete) {
    return;
  }
  while (Serial.available()) {
    // get the new byte:
    char inChar = (char)Serial.read();
    // add it to the inputString:
    inputString += inChar;
    // if the incoming character is a newline, set a flag so the main loop can
    // do something about it:
    if (inChar == '\n') {
      stringComplete = true;
      break;
    }
  }
}

这里使用的是人脸检测,在吃鸡实战中效果不好,

人体检测会好一些,

去训练的话可以参考

http://t.csdnimg.cn/ZfoSE

【手把手在K210上部署自己在线训练的YOLO模型 - CSDN App】http://t.csdnimg.cn/uwcb9

http://t.csdnimg.cn/uwcb9

相关推荐
zzu123zsw22 分钟前
第五章:自动化脚本开发
人工智能·自动化
IT_陈寒24 分钟前
Python 3.12 新特性实战:10个性能优化技巧让你的代码快如闪电⚡
前端·人工智能·后端
siliconstorm.ai1 小时前
OpenAI与微软“再造合作”:重组背后的资本与生态博弈
人工智能·microsoft
张较瘦_1 小时前
[论文阅读] 告别“数量为王”:双轨道会议模型+LS,破解AI时代学术交流困局
论文阅读·人工智能
nju_spy1 小时前
GPT 系列论文1-2 两阶段半监督 + zero-shot prompt
人工智能·gpt·nlp·大语言模型·zero-shot·transformer架构·半监督训练
芝麻开门-新起点1 小时前
第30章 零售与电商AI应用
人工智能·零售
shuidaoyuxing1 小时前
机器人检验报告包含内容
人工智能·机器人
南山二毛1 小时前
机器人控制器开发(训练到Jetson本地部署)
人工智能·机器人
文火冰糖的硅基工坊1 小时前
[硬件电路-192]:基级与发射极两端的电压超过1.5v可能是什么原因
单片机·嵌入式硬件·系统架构·电路·跨学科融合
工藤学编程1 小时前
零基础学AI大模型之AI大模型常见概念
人工智能