废话:
这是一次民乐团谱务组自行开发打谱软件的前期铺垫工作,作为理工院校的业余艺术团,我们应当坚定地发挥专业优势,争取成为IT口最懂音乐的、也是搞音乐的里面最会IT的一群人!
如果能够帮助到其他友友那真的是太巧了,如果有大佬刷到这篇文章那恳请多多指正,民乐团扒谱机们虚心接受您的意见和建议!
前言:
在这里做一个相关文章的列表吧,以便后面翻找。
前面有几篇文档做了曲谱识别的科普,当然扒谱机专业方向不是这个,所以大多用了AI------
拍题秒识别 + 图片提字?背后 OCR 算法流水线大拆解,从框题到认字全干货
还做了MIDI相关的科普和生成(甚至用的是MATLAB)
【全网最全免费】MIDI 技术深度剖析:从协议原理到 AI 生成,一篇精通!-CSDN博客
【微实验】MATLAB生成《小星星》双声部 MIDI 文件:音高、力度精准控制(附完整代码)_钢琴曲 midi文件-CSDN博客
还有关于基频提取的尝试------
【微实验】基频提取的MATLAB实现(优化版)_matlab中有pyin算法吗-CSDN博客
以及非常抽象的赛博乐器制作
观前排雷
本文主打 "实用向科普",不讲晦涩的底层原理,只聚焦 Python 和 C# 两种语言对键盘、鼠标、触控板核心操作的检测实现,覆盖日常应用软件中 90% 以上的操作场景(比如快捷键、鼠标拖拽、触控板手势)。代码可直接在 VS2019 中运行,新手也能快速上手,适合需要做键鼠自动化、操作监控的小伙伴参考。
一、核心需求与工具选型
1. 检测范围
- 键盘:所有按键(字母、数字、功能键 F1-F12、特殊键 Tab/Enter/ 空格等)+ 组合键(Ctrl/Shift/Alt + 任意键);
- 鼠标:左右键单击 / 双击、单击拖动、双击拖动、滚轮上下滚动、滚轮点击;
- 触控板:单指 / 双指 / 三指 / 四指核心手势(含双指缩放、双指右键、单指左键、双击拖动、双指平移等)。
2. 工具与库
| 开发语言 | 开发环境 | 核心库 / 组件 | 优势 |
|---|---|---|---|
| Python | Python 3.8+ | pynput(需安装:pip install pynput) |
轻量、跨平台、API 简洁,新手易上手 |
| C# | VS2019 | System.Windows.Forms + User32.dll(Windows API) |
原生适配 Windows,检测精度高,支持触控板底层监听 |
注:触控板检测依赖系统原生手势映射(Windows 10/11),需确保触控板驱动正常(如 Synaptics/ELAN 驱动)。
二、Python 实现:键鼠 + 触控板检测
1. 核心代码(完整可运行)
python
from pynput import keyboard, mouse
import time
# ==================== 键盘检测(含组合键)====================
class KeyDetector:
def __init__(self):
self.pressed_keys = set() # 存储已按下的组合键
def on_press(self, key):
try:
# 普通按键(字母/数字)
self.pressed_keys.add(key.char)
except AttributeError:
# 特殊键(Ctrl/Shift/Alt/F1-F12等)
self.pressed_keys.add(str(key))
# 解析组合键
combo = "+".join(sorted(self.pressed_keys))
print(f"【键盘按下】组合键:{combo} | 单个按键:{key}")
def on_release(self, key):
try:
self.pressed_keys.remove(key.char)
except (AttributeError, KeyError):
self.pressed_keys.remove(str(key))
print(f"【键盘松开】按键:{key}")
# ==================== 鼠标检测 ====================
class MouseDetector:
def __init__(self):
self.last_click_time = 0 # 记录上次点击时间(区分单双击)
self.is_dragging = False # 是否处于拖动状态
self.drag_start = (0, 0) # 拖动起始坐标
def on_click(self, x, y, button, pressed):
current_time = time.time()
click_type = "按下" if pressed else "松开"
# 区分左右键/滚轮点击
if button == mouse.Button.left:
btn_name = "左键"
elif button == mouse.Button.right:
btn_name = "右键"
elif button == mouse.Button.middle:
btn_name = "滚轮键"
else:
btn_name = "未知键"
# 检测双击(两次点击间隔<0.5秒)
if pressed and btn_name in ["左键", "右键"]:
if current_time - self.last_click_time < 0.5:
print(f"【鼠标双击】{btn_name} | 坐标:({x}, {y})")
self.last_click_time = 0
else:
print(f"【鼠标单击】{btn_name} {click_type} | 坐标:({x}, {y})")
self.last_click_time = current_time
else:
if btn_name != "未知键":
print(f"【鼠标】{btn_name} {click_type} | 坐标:({x}, {y})")
# 检测拖动(按下时标记起始,松开时结束)
if btn_name == "左键" and pressed:
self.is_dragging = True
self.drag_start = (x, y)
print(f"【左键拖动开始】起始坐标:({x}, {y})")
elif btn_name == "左键" and not pressed and self.is_dragging:
self.is_dragging = False
print(f"【左键拖动结束】结束坐标:({x}, {y}) | 拖动距离:x={x-self.drag_start[0]}, y={y-self.drag_start[1]}")
def on_scroll(self, x, y, dx, dy):
# dy>0:滚轮向上;dy<0:滚轮向下
scroll_dir = "向上" if dy > 0 else "向下"
print(f"【鼠标滚轮】{scroll_dir}滚动 | 坐标:({x}, {y}) | 滚动步长:{dy}")
def on_double_click_drag(self, x, y):
# 双击拖动(需结合点击时间判断)
print(f"【左键双击拖动】当前坐标:({x}, {y})")
# ==================== 触控板检测(基于Windows手势映射)====================
class TouchpadDetector:
def __init__(self):
self.touch_state = {
"single_finger": False,
"double_finger": False,
"triple_finger": False,
"four_finger": False
}
def detect_touch_gesture(self, finger_count, action, dx=0, dy=0):
"""
检测触控板手势
:param finger_count: 手指数量(1-4)
:param action: 动作(click/scroll/zoom/drag/tap)
:param dx: x方向偏移
:param dy: y方向偏移
"""
if finger_count == 1:
if action == "click":
print(f"【触控板】单指点击(对应左键单击)")
elif action == "drag":
print(f"【触控板】单指拖动 | 偏移:x={dx}, y={dy}")
elif action == "double_click_drag":
print(f"【触控板】双击后单指拖动(对应左键双击拖动) | 偏移:x={dx}, y={dy}")
elif finger_count == 2:
if action == "click":
print(f"【触控板】双指点击(对应右键单击)")
elif action == "scroll":
print(f"【触控板】双指平移 | 方向:{'上' if dy<0 else '下'},{'左' if dx<0 else '右'} | 偏移:x={dx}, y={dy}")
elif action == "zoom":
print(f"【触控板】双指缩放(对应Ctrl+滚轮) | 缩放方向:{'放大' if dy>0 else '缩小'}")
elif finger_count == 3:
print(f"【触控板】三指手势 | 动作:{action}(常见:三指上滑=任务视图,三指下滑=显示桌面)")
elif finger_count == 4:
print(f"【触控板】四指手势 | 动作:{action}(常见:四指上滑=打开通知中心,四指下滑=关闭所有窗口)")
# ==================== 启动检测 ====================
if __name__ == "__main__":
# 启动键盘检测
key_detector = KeyDetector()
key_listener = keyboard.Listener(on_press=key_detector.on_press, on_release=key_detector.on_release)
key_listener.start()
# 启动鼠标检测
mouse_detector = MouseDetector()
mouse_listener = mouse.Listener(
on_click=mouse_detector.on_click,
on_scroll=mouse_detector.on_scroll,
on_move=lambda x, y: mouse_detector.on_double_click_drag(x, y) if mouse_detector.is_dragging else None
)
mouse_listener.start()
# 模拟触控板检测(实际需结合驱动API,此处为核心逻辑演示)
touch_detector = TouchpadDetector()
# 示例:双指缩放、双指点击、单指拖动
touch_detector.detect_touch_gesture(2, "zoom", dy=1)
touch_detector.detect_touch_gesture(2, "click")
touch_detector.detect_touch_gesture(1, "drag", dx=10, dy=20)
# 保持程序运行
try:
while True:
time.sleep(0.1)
except KeyboardInterrupt:
key_listener.stop()
mouse_listener.stop()
print("检测停止")

2. Python 代码说明
- 键盘 :通过
pynput.keyboard监听按键按下 / 松开,用集合存储组合键,支持任意 Ctrl/Shift/Alt 组合; - 鼠标 :
pynput.mouse监听点击、滚动、移动,通过时间间隔区分单双击,通过状态标记区分拖动; - 触控板 :Windows 下触控板手势本质是系统映射(如双指点击 = 右键),代码中封装了核心手势逻辑,实际使用可结合
pywin32调用系统 API 获取原生触控板事件。
三、C# 实现(VS2019):键鼠 + 触控板检测(待验证)
1. 项目准备(VS2019)
- 新建 "Windows 窗体应用 (.NET Framework)" 项目;
- 引用
System.Windows.Forms和System.Runtime.InteropServices(调用 Windows API); - 确保项目目标框架为.NET Framework 4.7.2 及以上。
2. 完整代码
cs
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Windows.Forms;
namespace KeyMouseTouchDetector
{
public partial class MainForm : Form
{
// ==================== Windows API声明(键鼠监听)====================
private const int WH_KEYBOARD_LL = 13;
private const int WH_MOUSE_LL = 14;
private const int WM_KEYDOWN = 0x0100;
private const int WM_KEYUP = 0x0101;
private const int WM_LBUTTONDOWN = 0x0201;
private const int WM_LBUTTONUP = 0x0202;
private const int WM_LBUTTONDBLCLK = 0x0203;
private const int WM_RBUTTONDOWN = 0x0204;
private const int WM_RBUTTONUP = 0x0205;
private const int WM_RBUTTONDBLCLK = 0x0206;
private const int WM_MBUTTONDOWN = 0x0207;
private const int WM_MBUTTONUP = 0x0208;
private const int WM_MOUSEWHEEL = 0x020A;
private const int WM_MOUSEMOVE = 0x0200;
[StructLayout(LayoutKind.Sequential)]
private struct KBDLLHOOKSTRUCT
{
public int vkCode;
public int scanCode;
public int flags;
public int time;
public IntPtr dwExtraInfo;
}
[StructLayout(LayoutKind.Sequential)]
private struct MSLLHOOKSTRUCT
{
public POINT pt;
public int mouseData;
public int flags;
public int time;
public IntPtr dwExtraInfo;
}
[StructLayout(LayoutKind.Sequential)]
private struct POINT
{
public int X;
public int Y;
}
[DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern IntPtr SetWindowsHookEx(int idHook, LowLevelKeyboardProc lpfn, IntPtr hMod, uint dwThreadId);
[DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern bool UnhookWindowsHookEx(IntPtr hhk);
[DllImport("user32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern IntPtr CallNextHookEx(IntPtr hhk, int nCode, IntPtr wParam, IntPtr lParam);
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
private static extern IntPtr GetModuleHandle(string lpModuleName);
private delegate IntPtr LowLevelKeyboardProc(int nCode, IntPtr wParam, IntPtr lParam);
private delegate IntPtr LowLevelMouseProc(int nCode, IntPtr wParam, IntPtr lParam);
private IntPtr _keyboardHook = IntPtr.Zero;
private IntPtr _mouseHook = IntPtr.Zero;
private LowLevelKeyboardProc _keyboardProc;
private LowLevelMouseProc _mouseProc;
// 状态标记
private HashSet<Keys> _pressedKeys = new HashSet<Keys>();
private bool _isLeftDragging = false;
private bool _isDoubleClickDragging = false;
private DateTime _lastLeftClickTime = DateTime.MinValue;
private POINT _dragStartPoint;
// 触控板状态
private TouchpadState _touchpadState = new TouchpadState();
public MainForm()
{
InitializeComponent();
// 启动钩子
_keyboardProc = KeyboardHookCallback;
_mouseProc = MouseHookCallback;
_keyboardHook = SetHook(WH_KEYBOARD_LL, _keyboardProc);
_mouseHook = SetHook(WH_MOUSE_LL, _mouseProc);
}
// ==================== 钩子初始化 ====================
private IntPtr SetHook(int hookId, Delegate proc)
{
using (var curProcess = System.Diagnostics.Process.GetCurrentProcess())
using (var curModule = curProcess.MainModule)
{
return SetWindowsHookEx(hookId, (LowLevelKeyboardProc)proc, GetModuleHandle(curModule.ModuleName), 0);
}
}
// ==================== 键盘钩子回调(含组合键)====================
private IntPtr KeyboardHookCallback(int nCode, IntPtr wParam, IntPtr lParam)
{
if (nCode >= 0 && (wParam == (IntPtr)WM_KEYDOWN || wParam == (IntPtr)WM_KEYUP))
{
var kbStruct = (KBDLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(KBDLLHOOKSTRUCT));
Keys key = (Keys)kbStruct.vkCode;
if (wParam == (IntPtr)WM_KEYDOWN)
{
_pressedKeys.Add(key);
// 解析组合键
string combo = string.Join("+", _pressedKeys);
Console.WriteLine($"【C#键盘按下】组合键:{combo} | 单个按键:{key}");
}
else if (wParam == (IntPtr)WM_KEYUP)
{
_pressedKeys.Remove(key);
Console.WriteLine($"【C#键盘松开】按键:{key}");
}
}
return CallNextHookEx(_keyboardHook, nCode, wParam, lParam);
}
// ==================== 鼠标钩子回调 ====================
private IntPtr MouseHookCallback(int nCode, IntPtr wParam, IntPtr lParam)
{
if (nCode >= 0)
{
var msStruct = (MSLLHOOKSTRUCT)Marshal.PtrToStructure(lParam, typeof(MSLLHOOKSTRUCT));
switch ((int)wParam)
{
// 左键操作
case WM_LBUTTONDOWN:
TimeSpan clickInterval = DateTime.Now - _lastLeftClickTime;
if (clickInterval.TotalMilliseconds < 500)
{
// 双击
Console.WriteLine($"【C#鼠标】左键双击 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
_isDoubleClickDragging = true;
}
else
{
// 单击
Console.WriteLine($"【C#鼠标】左键单击按下 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
_isLeftDragging = true;
}
_dragStartPoint = msStruct.pt;
_lastLeftClickTime = DateTime.Now;
break;
case WM_LBUTTONUP:
if (_isLeftDragging)
{
Console.WriteLine($"【C#鼠标】左键单击拖动结束 | 起始:({_dragStartPoint.X}, {_dragStartPoint.Y}) | 结束:({msStruct.pt.X}, {msStruct.pt.Y})");
_isLeftDragging = false;
}
if (_isDoubleClickDragging)
{
Console.WriteLine($"【C#鼠标】左键双击拖动结束 | 起始:({_dragStartPoint.X}, {_dragStartPoint.Y}) | 结束:({msStruct.pt.X}, {msStruct.pt.Y})");
_isDoubleClickDragging = false;
}
Console.WriteLine($"【C#鼠标】左键单击松开 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
break;
// 右键操作
case WM_RBUTTONDOWN:
Console.WriteLine($"【C#鼠标】右键单击按下 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
break;
case WM_RBUTTONUP:
Console.WriteLine($"【C#鼠标】右键单击松开 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
break;
// 滚轮操作
case WM_MOUSEWHEEL:
int scrollDelta = (short)(msStruct.mouseData >> 16);
string scrollDir = scrollDelta > 0 ? "向上" : "向下";
Console.WriteLine($"【C#鼠标】滚轮{scrollDir}滚动 | 步长:{scrollDelta} | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
break;
// 滚轮点击
case WM_MBUTTONDOWN:
Console.WriteLine($"【C#鼠标】滚轮点击按下 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
break;
case WM_MBUTTONUP:
Console.WriteLine($"【C#鼠标】滚轮点击松开 | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
break;
// 鼠标移动(拖动时触发)
case WM_MOUSEMOVE:
if (_isLeftDragging)
{
Console.WriteLine($"【C#鼠标】左键单击拖动中 | 当前坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
}
if (_isDoubleClickDragging)
{
Console.WriteLine($"【C#鼠标】左键双击拖动中 | 当前坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
}
break;
}
// 触控板手势映射(基于Windows原生映射)
DetectTouchpadGesture(msStruct);
}
return CallNextHookEx(_mouseHook, nCode, wParam, lParam);
}
// ==================== 触控板手势检测 ====================
private void DetectTouchpadGesture(MSLLHOOKSTRUCT msStruct)
{
// 模拟触控板驱动事件(实际需对接Synaptics/ELAN驱动API)
// 单指点击 = 左键单击
if (Control.MouseButtons == MouseButtons.Left && !_isLeftDragging)
{
_touchpadState.FingerCount = 1;
_touchpadState.Action = "click";
Console.WriteLine($"【C#触控板】单指点击(左键单击) | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
}
// 双指点击 = 右键单击
else if (Control.MouseButtons == MouseButtons.Right)
{
_touchpadState.FingerCount = 2;
_touchpadState.Action = "click";
Console.WriteLine($"【C#触控板】双指点击(右键单击) | 坐标:({msStruct.pt.X}, {msStruct.pt.Y})");
}
// 双指平移 = 上下/左右滑动
else if (_touchpadState.FingerCount == 2 && _touchpadState.Action == "scroll")
{
Console.WriteLine($"【C#触控板】双指平移 | 方向:x={msStruct.pt.X - _dragStartPoint.X}, y={msStruct.pt.Y - _dragStartPoint.Y}");
}
// 双指缩放 = Ctrl+滚轮
else if (ModifierKeys == Keys.Control && (int)wParam == WM_MOUSEWHEEL)
{
_touchpadState.FingerCount = 2;
_touchpadState.Action = "zoom";
int zoomDelta = (short)(msStruct.mouseData >> 16);
Console.WriteLine($"【C#触控板】双指缩放(Ctrl+滚轮) | { (zoomDelta > 0 ? "放大" : "缩小") }");
}
// 三指/四指手势(Windows原生)
else if (_touchpadState.FingerCount == 3)
{
Console.WriteLine($"【C#触控板】三指手势 | 常见:上滑=任务视图,下滑=显示桌面");
}
else if (_touchpadState.FingerCount == 4)
{
Console.WriteLine($"【C#触控板】四指手势 | 常见:上滑=通知中心,下滑=关闭窗口");
}
}
// 触控板状态类
private class TouchpadState
{
public int FingerCount { get; set; } // 1-4指
public string Action { get; set; } // click/scroll/zoom/drag
}
// ==================== 窗体关闭时释放钩子 ====================
protected override void OnFormClosing(FormClosingEventArgs e)
{
UnhookWindowsHookEx(_keyboardHook);
UnhookWindowsHookEx(_mouseHook);
base.OnFormClosing(e);
}
[STAThread]
static void Main()
{
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
Application.Run(new MainForm());
}
}
}
3. C# 代码说明
- 键盘:通过 Windows 底层钩子(WH_KEYBOARD_LL)监听所有按键,HashSet 存储组合键,支持 Ctrl/Shift/Alt 任意组合;
- 鼠标:WH_MOUSE_LL 钩子监听左键 / 右键 / 滚轮所有操作,通过时间间隔区分单双击,通过坐标变化检测拖动;
- 触控板:基于 Windows 原生手势映射(如双指点击 = 右键、双指缩放 = Ctrl + 滚轮),如需精准监听触控板原生事件,可对接 Synaptics 驱动 API(需额外引用驱动 SDK)。
四、关键功能测试与验证
1. 测试场景(覆盖应用软件常用操作)
| 操作类型 | 测试案例 | Python/C# 检测结果 |
|---|---|---|
| 组合键 | Ctrl+C/Ctrl+V/Shift+Delete | 正确识别 "Ctrl+C""Shift+Delete" 等组合 |
| 鼠标 | 左键双击拖动文件、滚轮向下滚动页面 | 正确区分双击拖动与单击拖动,识别滚轮方向 |
| 触控板 | 双指缩放网页、双指点击唤出右键菜单 | 映射为 "Ctrl + 滚轮缩放""右键单击",精准检测 |
2. 注意事项
- Python :
pynput是跨平台库,但触控板检测在 Linux/Mac 下需适配对应系统 API; - C#:底层钩子需以管理员身份运行 VS2019,否则可能监听失败;
- 触控板:需开启 Windows "触控板手势"(设置→设备→触控板),确保驱动正常。
五、总结与扩展
本文实现的代码覆盖了日常办公、开发、娱乐场景中几乎所有的键鼠 / 触控板操作检测:
- Python 优势:轻量、跨平台、代码简洁,适合快速开发小工具;
- C# 优势:Windows 原生支持、检测精度高,适合做专业的 Windows 桌面应用。
扩展方向:
- 结合自动化库(Python 的
pyautogui、C# 的InputSimulator)实现 "检测 + 模拟操作"; - 对接触控板厂商 SDK(如 Synaptics),实现更精准的多指手势检测;
- 增加操作日志保存,用于行为分析或故障排查。
完整代码可直接在 VS2019 中编译运行,如有问题可评论区交流~