【微实验】基于Python实现的实时键盘鼠标触控板拾取检测(VS2019,附完整代码)

废话:

这是一次民乐团谱务组自行开发打谱软件的前期铺垫工作,作为理工院校的业余艺术团,我们应当坚定地发挥专业优势,争取成为IT口最懂音乐的、也是搞音乐的里面最会IT的一群人!

如果能够帮助到其他友友那真的是太巧了,如果有大佬刷到这篇文章那恳请多多指正,民乐团扒谱机们虚心接受您的意见和建议!

前言:

在这里做一个相关文章的列表吧,以便后面翻找。

前面有几篇文档做了曲谱识别的科普,当然扒谱机专业方向不是这个,所以大多用了AI------

拍题秒识别 + 图片提字?背后 OCR 算法流水线大拆解,从框题到认字全干货

还做了MIDI相关的科普和生成(甚至用的是MATLAB)

【全网最全免费】MIDI 技术深度剖析:从协议原理到 AI 生成,一篇精通!-CSDN博客

【微实验】MATLAB生成《小星星》双声部 MIDI 文件:音高、力度精准控制(附完整代码)_钢琴曲 midi文件-CSDN博客

还有关于基频提取的尝试------

【微实验】基频提取的MATLAB实现(优化版)_matlab中有pyin算法吗-CSDN博客

以及非常抽象的赛博乐器制作

【微实验】使用MATLAB制作一张赛博古琴?-CSDN博客

观前排雷

本文主打 "实用向科普",不讲晦涩的底层原理,只聚焦 Python 和 C# 两种语言对键盘、鼠标、触控板核心操作的检测实现,覆盖日常应用软件中 90% 以上的操作场景(比如快捷键、鼠标拖拽、触控板手势)。代码可直接在 VS2019 中运行,新手也能快速上手,适合需要做键鼠自动化、操作监控的小伙伴参考。

一、核心需求与工具选型

1. 检测范围

  • 键盘:所有按键(字母、数字、功能键 F1-F12、特殊键 Tab/Enter/ 空格等)+ 组合键(Ctrl/Shift/Alt + 任意键);
  • 鼠标:左右键单击 / 双击、单击拖动、双击拖动、滚轮上下滚动、滚轮点击;
  • 触控板:单指 / 双指 / 三指 / 四指核心手势(含双指缩放、双指右键、单指左键、双击拖动、双指平移等)。

2. 工具与库

开发语言 开发环境 核心库 / 组件 优势
Python Python 3.8+ pynput(需安装:pip install pynput 轻量、跨平台、API 简洁,新手易上手
C# VS2019 System.Windows.Forms + User32.dll(Windows API) 原生适配 Windows,检测精度高,支持触控板底层监听

注:触控板检测依赖系统原生手势映射(Windows 10/11),需确保触控板驱动正常(如 Synaptics/ELAN 驱动)。

二、Python 实现:键鼠 + 触控板检测

1. 核心代码(完整可运行)

python 复制代码
from pynput import keyboard, mouse
import time

# ==================== 键盘检测(含组合键)====================
class KeyDetector:
    def __init__(self):
        self.pressed_keys = set()  # 存储已按下的组合键

    def on_press(self, key):
        try:
            # 普通按键(字母/数字)
            self.pressed_keys.add(key.char)
        except AttributeError:
            # 特殊键(Ctrl/Shift/Alt/F1-F12等)
            self.pressed_keys.add(str(key))
        
        # 解析组合键
        combo = "+".join(sorted(self.pressed_keys))
        print(f"【键盘按下】组合键:{combo} | 单个按键:{key}")

    def on_release(self, key):
        try:
            self.pressed_keys.remove(key.char)
        except (AttributeError, KeyError):
            self.pressed_keys.remove(str(key))
        print(f"【键盘松开】按键:{key}")

# ==================== 鼠标检测 ====================
class MouseDetector:
    def __init__(self):
        self.last_click_time = 0  # 记录上次点击时间(区分单双击)
        self.is_dragging = False  # 是否处于拖动状态
        self.drag_start = (0, 0)  # 拖动起始坐标

    def on_click(self, x, y, button, pressed):
        current_time = time.time()
        click_type = "按下" if pressed else "松开"
        
        # 区分左右键/滚轮点击
        if button == mouse.Button.left:
            btn_name = "左键"
        elif button == mouse.Button.right:
            btn_name = "右键"
        elif button == mouse.Button.middle:
            btn_name = "滚轮键"
        else:
            btn_name = "未知键"

        # 检测双击(两次点击间隔<0.5秒)
        if pressed and btn_name in ["左键", "右键"]:
            if current_time - self.last_click_time < 0.5:
                print(f"【鼠标双击】{btn_name} | 坐标:({x}, {y})")
                self.last_click_time = 0
            else:
                print(f"【鼠标单击】{btn_name} {click_type} | 坐标:({x}, {y})")
                self.last_click_time = current_time
        else:
            if btn_name != "未知键":
                print(f"【鼠标】{btn_name} {click_type} | 坐标:({x}, {y})")

        # 检测拖动(按下时标记起始,松开时结束)
        if btn_name == "左键" and pressed:
            self.is_dragging = True
            self.drag_start = (x, y)
            print(f"【左键拖动开始】起始坐标:({x}, {y})")
        elif btn_name == "左键" and not pressed and self.is_dragging:
            self.is_dragging = False
            print(f"【左键拖动结束】结束坐标:({x}, {y}) | 拖动距离:x={x-self.drag_start[0]}, y={y-self.drag_start[1]}")

    def on_scroll(self, x, y, dx, dy):
        # dy>0:滚轮向上;dy<0:滚轮向下
        scroll_dir = "向上" if dy > 0 else "向下"
        print(f"【鼠标滚轮】{scroll_dir}滚动 | 坐标:({x}, {y}) | 滚动步长:{dy}")

    def on_double_click_drag(self, x, y):
        # 双击拖动(需结合点击时间判断)
        print(f"【左键双击拖动】当前坐标:({x}, {y})")

# ==================== 触控板检测(基于Windows手势映射)====================
class TouchpadDetector:
    def __init__(self):
        self.touch_state = {
            "single_finger": False,
            "double_finger": False,
            "triple_finger": False,
            "four_finger": False
        }

    def detect_touch_gesture(self, finger_count, action, dx=0, dy=0):
        """
        检测触控板手势
        :param finger_count: 手指数量(1-4)
        :param action: 动作(click/scroll/zoom/drag/tap)
        :param dx: x方向偏移
        :param dy: y方向偏移
        """
        if finger_count == 1:
            if action == "click":
                print(f"【触控板】单指点击(对应左键单击)")
            elif action == "drag":
                print(f"【触控板】单指拖动 | 偏移:x={dx}, y={dy}")
            elif action == "double_click_drag":
                print(f"【触控板】双击后单指拖动(对应左键双击拖动) | 偏移:x={dx}, y={dy}")
        
        elif finger_count == 2:
            if action == "click":
                print(f"【触控板】双指点击(对应右键单击)")
            elif action == "scroll":
                print(f"【触控板】双指平移 | 方向:{'上' if dy<0 else '下'},{'左' if dx<0 else '右'} | 偏移:x={dx}, y={dy}")
            elif action == "zoom":
                print(f"【触控板】双指缩放(对应Ctrl+滚轮) | 缩放方向:{'放大' if dy>0 else '缩小'}")
        
        elif finger_count == 3:
            print(f"【触控板】三指手势 | 动作:{action}(常见:三指上滑=任务视图,三指下滑=显示桌面)")
        
        elif finger_count == 4:
            print(f"【触控板】四指手势 | 动作:{action}(常见:四指上滑=打开通知中心,四指下滑=关闭所有窗口)")

# ==================== 启动检测 ====================
if __name__ == "__main__":
    # 启动键盘检测
    key_detector = KeyDetector()
    key_listener = keyboard.Listener(on_press=key_detector.on_press, on_release=key_detector.on_release)
    key_listener.start()

    # 启动鼠标检测
    mouse_detector = MouseDetector()
    mouse_listener = mouse.Listener(
        on_click=mouse_detector.on_click,
        on_scroll=mouse_detector.on_scroll,
        on_move=lambda x, y: mouse_detector.on_double_click_drag(x, y) if mouse_detector.is_dragging else None
    )
    mouse_listener.start()

    # 模拟触控板检测(实际需结合驱动API,此处为核心逻辑演示)
    touch_detector = TouchpadDetector()
    # 示例:双指缩放、双指点击、单指拖动
    touch_detector.detect_touch_gesture(2, "zoom", dy=1)
    touch_detector.detect_touch_gesture(2, "click")
    touch_detector.detect_touch_gesture(1, "drag", dx=10, dy=20)

    # 保持程序运行
    try:
        while True:
            time.sleep(0.1)
    except KeyboardInterrupt:
        key_listener.stop()
        mouse_listener.stop()
        print("检测停止")

2. Python 代码说明

  • 键盘 :通过pynput.keyboard监听按键按下 / 松开,用集合存储组合键,支持任意 Ctrl/Shift/Alt 组合;
  • 鼠标pynput.mouse监听点击、滚动、移动,通过时间间隔区分单双击,通过状态标记区分拖动;
  • 触控板 :Windows 下触控板手势本质是系统映射(如双指点击 = 右键),代码中封装了核心手势逻辑,实际使用可结合pywin32调用系统 API 获取原生触控板事件。

三、C# 实现(VS2019):键鼠 + 触控板检测(待验证)

1. 项目准备(VS2019)

  1. 新建 "Windows 窗体应用 (.NET Framework)" 项目;
  2. 引用System.Windows.FormsSystem.Runtime.InteropServices(调用 Windows API);
  3. 确保项目目标框架为.NET Framework 4.7.2 及以上。

2. 完整代码

cs 复制代码
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Windows.Forms;

namespace KeyMouseTouchDetector
{
    public partial class MainForm : Form
    {
        // ==================== Windows API声明(键鼠监听)====================
        private const int WH_KEYBOARD_LL = 13;
        private const int WH_MOUSE_LL = 14;
        private const int WM_KEYDOWN = 0x0100;
        private const int WM_KEYUP = 0x0101;
        private const int WM_LBUTTONDOWN = 0x0201;
        private const int WM_LBUTTONUP = 0x0202;
        private const int WM_LBUTTONDBLCLK = 0x0203;
        private const int WM_RBUTTONDOWN = 0x0204;
        private const int WM_RBUTTONUP = 0x0205;
        private const int WM_RBUTTONDBLCLK = 0x0206;
        private const int WM_MBUTTONDOWN = 0x0207;
        private const int WM_MBUTTONUP = 0x0208;
        private const int WM_MOUSEWHEEL = 0x020A;
        private const int WM_MOUSEMOVE = 0x0200;

        [StructLayout(LayoutKind.Sequential)]
        private struct KBDLLHOOKSTRUCT
        {
            public int vkCode;
            public int scanCode;
            public int flags;
            public int time;
            public IntPtr dwExtraInfo;
        }

        [StructLayout(LayoutKind.Sequential)]
        private struct MSLLHOOKSTRUCT
        {
            public POINT pt;
            public int mouseData;
            public int flags;
            public int time;
            public IntPtr dwExtraInfo;
        }

        [StructLayout(LayoutKind.Sequential)]
        private struct POINT
        {
            public int X;
            public int Y;
        }

        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelKeyboardProc lpfn, IntPtr hMod, uint dwThreadId);

        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern bool UnhookWindowsHookEx(IntPtr hhk);

        [DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode, IntPtr wParam, IntPtr lParam);

        [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        private static extern IntPtr GetModuleHandle(string lpModuleName);

        private delegate IntPtr LowLevelKeyboardProc(int nCode, IntPtr wParam, IntPtr lParam);
        private delegate IntPtr LowLevelMouseProc(int nCode, IntPtr wParam, IntPtr lParam);

        private IntPtr _keyboardHook = IntPtr.Zero;
        private IntPtr _mouseHook = IntPtr.Zero;
        private LowLevelKeyboardProc _keyboardProc;
        private LowLevelMouseProc _mouseProc;

        // 状态标记
        private HashSet<Keys> _pressedKeys = new HashSet<Keys>();
        private bool _isLeftDragging = false;
        private bool _isDoubleClickDragging = false;
        private DateTime _lastLeftClickTime = DateTime.MinValue;
        private POINT _dragStartPoint;

        // 触控板状态
        private TouchpadState _touchpadState = new TouchpadState();

        public MainForm()
        {
            InitializeComponent();
            // 启动钩子
            _keyboardProc = KeyboardHookCallback;
            _mouseProc = MouseHookCallback;
            _keyboardHook = SetHook(WH_KEYBOARD_LL, _keyboardProc);
            _mouseHook = SetHook(WH_MOUSE_LL, _mouseProc);
        }

        // ==================== 钩子初始化 ====================
        private IntPtr SetHook(int hookId, Delegate proc)
        {
            using (var curProcess = System.Diagnostics.Process.GetCurrentProcess())
            using (var curModule = curProcess.MainModule)
            {
                return SetWindowsHookEx(hookId, (LowLevelKeyboardProc)proc, GetModuleHandle(curModule.ModuleName), 0);
            }
        }

        // ==================== 键盘钩子回调(含组合键)====================
        private IntPtr KeyboardHookCallback(int nCode, IntPtr wParam, IntPtr lParam)
        {
            if (nCode >= 0 && (wParam == (IntPtr)WM_KEYDOWN || wParam == (IntPtr)WM_KEYUP))
            {
                var kbStruct = (KBDLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(KBDLLHOOKSTRUCT));
                Keys key = (Keys)kbStruct.vkCode;

                if (wParam == (IntPtr)WM_KEYDOWN)
                {
                    _pressedKeys.Add(key);
                    // 解析组合键
                    string combo = string.Join("+", _pressedKeys);
                    Console.WriteLine($"【C#键盘按下】组合键:{combo} | 单个按键:{key}");
                }
                else if (wParam == (IntPtr)WM_KEYUP)
                {
                    _pressedKeys.Remove(key);
                    Console.WriteLine($"【C#键盘松开】按键:{key}");
                }
            }
            return CallNextHookEx(_keyboardHook, nCode, wParam, lParam);
        }

        // ==================== 鼠标钩子回调 ====================
        private IntPtr MouseHookCallback(int nCode, IntPtr wParam, IntPtr lParam)
        {
            if (nCode >= 0)
            {
                var msStruct = (MSLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(MSLLHOOKSTRUCT));
                switch ((int)wParam)
                {
                    // 左键操作
                    case WM_LBUTTONDOWN:
                        TimeSpan clickInterval = DateTime.Now - _lastLeftClickTime;
                        if (clickInterval.TotalMilliseconds < 500)
                        {
                            // 双击
                            Console.WriteLine($"【C#鼠标】左键双击 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                            _isDoubleClickDragging = true;
                        }
                        else
                        {
                            // 单击
                            Console.WriteLine($"【C#鼠标】左键单击按下 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                            _isLeftDragging = true;
                        }
                        _dragStartPoint = msStruct.pt;
                        _lastLeftClickTime = DateTime.Now;
                        break;
                    case WM_LBUTTONUP:
                        if (_isLeftDragging)
                        {
                            Console.WriteLine($"【C#鼠标】左键单击拖动结束 | 起始:({_dragStartPoint.X}, {_dragStartPoint.Y}) | 结束:({msStruct.pt.X}, {msStruct.pt.Y})");
                            _isLeftDragging = false;
                        }
                        if (_isDoubleClickDragging)
                        {
                            Console.WriteLine($"【C#鼠标】左键双击拖动结束 | 起始:({_dragStartPoint.X}, {_dragStartPoint.Y}) | 结束:({msStruct.pt.X}, {msStruct.pt.Y})");
                            _isDoubleClickDragging = false;
                        }
                        Console.WriteLine($"【C#鼠标】左键单击松开 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        break;
                    // 右键操作
                    case WM_RBUTTONDOWN:
                        Console.WriteLine($"【C#鼠标】右键单击按下 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        break;
                    case WM_RBUTTONUP:
                        Console.WriteLine($"【C#鼠标】右键单击松开 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        break;
                    // 滚轮操作
                    case WM_MOUSEWHEEL:
                        int scrollDelta = (short)(msStruct.mouseData >> 16);
                        string scrollDir = scrollDelta > 0 ? "向上" : "向下";
                        Console.WriteLine($"【C#鼠标】滚轮{scrollDir}滚动 | 步长:{scrollDelta} | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        break;
                    // 滚轮点击
                    case WM_MBUTTONDOWN:
                        Console.WriteLine($"【C#鼠标】滚轮点击按下 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        break;
                    case WM_MBUTTONUP:
                        Console.WriteLine($"【C#鼠标】滚轮点击松开 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        break;
                    // 鼠标移动(拖动时触发)
                    case WM_MOUSEMOVE:
                        if (_isLeftDragging)
                        {
                            Console.WriteLine($"【C#鼠标】左键单击拖动中 | 当前坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        }
                        if (_isDoubleClickDragging)
                        {
                            Console.WriteLine($"【C#鼠标】左键双击拖动中 | 当前坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
                        }
                        break;
                }

                // 触控板手势映射(基于Windows原生映射)
                DetectTouchpadGesture(msStruct);
            }
            return CallNextHookEx(_mouseHook, nCode, wParam, lParam);
        }

        // ==================== 触控板手势检测 ====================
        private void DetectTouchpadGesture(MSLLHOOKSTRUCT msStruct)
        {
            // 模拟触控板驱动事件(实际需对接Synaptics/ELAN驱动API)
            // 单指点击 = 左键单击
            if (Control.MouseButtons == MouseButtons.Left && !_isLeftDragging)
            {
                _touchpadState.FingerCount = 1;
                _touchpadState.Action = "click";
                Console.WriteLine($"【C#触控板】单指点击(左键单击) | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
            }
            // 双指点击 = 右键单击
            else if (Control.MouseButtons == MouseButtons.Right)
            {
                _touchpadState.FingerCount = 2;
                _touchpadState.Action = "click";
                Console.WriteLine($"【C#触控板】双指点击(右键单击) | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
            }
            // 双指平移 = 上下/左右滑动
            else if (_touchpadState.FingerCount == 2 && _touchpadState.Action == "scroll")
            {
                Console.WriteLine($"【C#触控板】双指平移 | 方向:x={msStruct.pt.X - _dragStartPoint.X}, y={msStruct.pt.Y - _dragStartPoint.Y}");
            }
            // 双指缩放 = Ctrl+滚轮
            else if (ModifierKeys == Keys.Control && (int)wParam == WM_MOUSEWHEEL)
            {
                _touchpadState.FingerCount = 2;
                _touchpadState.Action = "zoom";
                int zoomDelta = (short)(msStruct.mouseData >> 16);
                Console.WriteLine($"【C#触控板】双指缩放(Ctrl+滚轮) | { (zoomDelta > 0 ? "放大" : "缩小") }");
            }
            // 三指/四指手势(Windows原生)
            else if (_touchpadState.FingerCount == 3)
            {
                Console.WriteLine($"【C#触控板】三指手势 | 常见:上滑=任务视图,下滑=显示桌面");
            }
            else if (_touchpadState.FingerCount == 4)
            {
                Console.WriteLine($"【C#触控板】四指手势 | 常见:上滑=通知中心,下滑=关闭窗口");
            }
        }

        // 触控板状态类
        private class TouchpadState
        {
            public int FingerCount { get; set; } // 1-4指
            public string Action { get; set; } // click/scroll/zoom/drag
        }

        // ==================== 窗体关闭时释放钩子 ====================
        protected override void OnFormClosing(FormClosingEventArgs e)
        {
            UnhookWindowsHookEx(_keyboardHook);
            UnhookWindowsHookEx(_mouseHook);
            base.OnFormClosing(e);
        }

        [STAThread]
        static void Main()
        {
            Application.EnableVisualStyles();
            Application.SetCompatibleTextRenderingDefault(false);
            Application.Run(new MainForm());
        }
    }
}

3. C# 代码说明

  • 键盘:通过 Windows 底层钩子(WH_KEYBOARD_LL)监听所有按键,HashSet 存储组合键,支持 Ctrl/Shift/Alt 任意组合;
  • 鼠标:WH_MOUSE_LL 钩子监听左键 / 右键 / 滚轮所有操作,通过时间间隔区分单双击,通过坐标变化检测拖动;
  • 触控板:基于 Windows 原生手势映射(如双指点击 = 右键、双指缩放 = Ctrl + 滚轮),如需精准监听触控板原生事件,可对接 Synaptics 驱动 API(需额外引用驱动 SDK)。

四、关键功能测试与验证

1. 测试场景(覆盖应用软件常用操作)

操作类型 测试案例 Python/C# 检测结果
组合键 Ctrl+C/Ctrl+V/Shift+Delete 正确识别 "Ctrl+C""Shift+Delete" 等组合
鼠标 左键双击拖动文件、滚轮向下滚动页面 正确区分双击拖动与单击拖动,识别滚轮方向
触控板 双指缩放网页、双指点击唤出右键菜单 映射为 "Ctrl + 滚轮缩放""右键单击",精准检测

2. 注意事项

  • Pythonpynput是跨平台库,但触控板检测在 Linux/Mac 下需适配对应系统 API;
  • C#:底层钩子需以管理员身份运行 VS2019,否则可能监听失败;
  • 触控板:需开启 Windows "触控板手势"(设置→设备→触控板),确保驱动正常。

五、总结与扩展

本文实现的代码覆盖了日常办公、开发、娱乐场景中几乎所有的键鼠 / 触控板操作检测:

  • Python 优势:轻量、跨平台、代码简洁,适合快速开发小工具;
  • C# 优势:Windows 原生支持、检测精度高,适合做专业的 Windows 桌面应用。

扩展方向:

  1. 结合自动化库(Python 的pyautogui、C# 的InputSimulator)实现 "检测 + 模拟操作";
  2. 对接触控板厂商 SDK(如 Synaptics),实现更精准的多指手势检测;
  3. 增加操作日志保存,用于行为分析或故障排查。

完整代码可直接在 VS2019 中编译运行,如有问题可评论区交流~

相关推荐
CreasyChan7 小时前
Unity中C#状态模式详解
unity·c#·状态模式
心本无晴.7 小时前
拣学--基于vue3和django框架实现的辅助考研系统
vue.js·python·mysql·考研·django·dify
Darenm1117 小时前
关于AI 面试官项目:智选ai 基于 Vue3 + Django + Dify 的全栈开发实战
人工智能·python·django
詹姆斯爱研究Java7 小时前
基于Django的租房网站的设计与实现
数据库·python·django
拉姆哥的小屋7 小时前
基于多模态深度学习的城市公园社交媒体评论智能分析系统——从BERTopic主题建模到CLIP图文一致性的全栈实践
人工智能·python·深度学习·矩阵·媒体
工程师0077 小时前
线程同步的意义
c#·锁机制·线程同步
ZAz_7 小时前
DAY 41 图像数据与显存
python
曲幽7 小时前
Python环境管理利器Conda:从入门到避坑实战指南
python·conda·pip·anaconda·uv·venv·miniconda
艾莉丝努力练剑7 小时前
【Python库和代码案例:第一课】Python 标准库与第三方库实战指南:从日期处理到 Excel 操作
java·服务器·开发语言·人工智能·python·pycharm·pip