android远程控制

jsync2025-03-14 14:18

前言

本文章仅供学习交流，所涉及的技术要点都已标明了出处。

最近突然想使用下在android上远程控制另一个android设备，于是下载了一些软件试试，发现todesk和蒲公英app都是共享桌面免费，但是控制收费，于是想着要不自己写一个得了，正好最近研究了下systemui的源码，可以从这里面入手。

1.梳理技术要点

  1. 共享端需要模拟手势操作
  2. 控制端与共享端远程通信
  3. 共享端画面实时传输

2.方案

2.1共享端模拟手势操作

2.1.1.无障碍

AccessibilityService可以参考这个

优点： 基本手势操作都可以实现，，并且可以识别当前页面节点，做自动化脚本处理等。

缺点：需要用户手动赋予无障碍权限，且不稳定，退出应用后需要重新赋予无障碍权限。

2.1.2.adb命令行

优点：基本手势操作，页面节点获取，屏幕抓取等都非常方便。

缺点：普通应用没有权限使用。需要root权限才能执行。

java 复制代码 
private var cmdOs: OutputStream? = null
/**
 * 模拟按键 比如 电源键。。。
 */
fun pointKeyCode(code: Int) {
    val cmd = String.format("input keyevent %s \n", code)
    exec(cmd)
}

/**
 *  截屏
 *  adb shell screencap -p /sdcard/xxx.png
 *  图片将放在：sdcard/xxx.png
 *  @param path 完整路径
 */
fun screenCap(path: String) {
    exec("screencap -p $path \n")
}
/**
 * 滑动屏幕
 * @param x1    起点x轴坐标 单位px
 * @param y1    起点y轴坐标 单位px
 * @param x2    终点x轴坐标 单位px
 * @param y2    终点y轴坐标 单位px
 */
fun swipe(x1: String?, y1: String?, x2: String?, y2: String?) {
    val cmd = String.format("input swipe %s %s %s %s \n", x1, y1, x2, y2)
    exec(cmd)
}

/**
 * 点击屏幕坐标
 * @param x     横坐标 单位px
 * @param y     纵坐标 单位px
 */
fun pointXY(x: String?, y: String?) {
    val cmd = String.format("input tap %s %s \n", x, y)
    exec(cmd)
}

/**
 * 执行ADB命令： input tap 125 340
 */
fun exec(cmd: String) {
    Log.e("eee", cmd)
    try {
        if (cmdOs == null) {
            cmdOs = Runtime.getRuntime().exec("su").outputStream
## }
        cmdOs?.write(cmd.toByteArray())
        cmdOs?.flush()
        //cmdOs.close();
    } catch (e: Exception) {
        e.printStackTrace()
        Log.e("GK", e.message!!)
    }
}

2.1.3.基于adb调试

类似于scrcpy，原理是通过开启wifi无线调试，然后在另一个设备上启动一个scrcpy的服务链接到adb无线调试，参考这个scrcpy-android。 这种方案需要在局域网中互联，如果远程组局域网，表现就非常卡顿，可能需要相应优化，没有深入研究。

2.1.4.系统输入InputManager

在研究adb命令控制的源码后，发现adb命令控制是基于 frameworks/base/services/core/java/com/android/server/input/InputShellCommand.java

java 复制代码 
InputManagerGlobal.getInstance().injectInputEvent(event,
        InputManager.INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH);

但是需要系统应用，且需要权限

xml 复制代码 
<permission android:name="android.permission.INJECT_EVENTS" android:protectionLevel="signature" />

可以参考这篇文章adb shell

2.2控制端与共享端远程通信

这里需要长连接，使用mqtt进行消息传递，可以参考这篇文章mqtt

这里需要搭建mqtt服务，在阿里云申请免费服务器3个月后，使用centos搭建了mqtt，难度不大，可以参考搭建流程mqtt搭建流程

需要注意的一点是，搭建完成后，一定要配置好防火墙端口，不然外网访问不到。

2.3.共享端画面实时传输

自己写一套太夸张了，直接白嫖免费的吧，声网接入sdk，每月免费时长使用，使用屏幕共享功能就可以了。

3.最终方案

由于我下载编译了aosp源码，直接就想到用系统权限相关的方案。

  1. 在SystemUIService中注册一个广播接收者，监听我们应用发送过来的事件。（包括手势事件，打开app并自动开启无障碍权限，杀死应用等）
  2. 控制端监听View的触摸事件，收集事件序列，并使用mqtt发送给共享端。
  3. 共享端接收mqtt事件，或执行无障碍自动化操作，或者向SystemUIService发送触摸事件广播

4.附录

4.1.SystemUIService部分代码

java 复制代码 
private ExecutorService mExecutor;
private String dk_package = "com.jsync.capp";
private static final int DEFAULT_DEVICE_ID = 0;
private static final float DEFAULT_PRESSURE = 1.0f;
private static final float NO_PRESSURE = 0.0f;
private static final float DEFAULT_SIZE = 1.0f;
private static final int DEFAULT_META_STATE = 0;
private static final float DEFAULT_PRECISION_X = 1.0f;
private static final float DEFAULT_PRECISION_Y = 1.0f;
private static final int DEFAULT_EDGE_FLAGS = 0;
private static final int DEFAULT_BUTTON_STATE = 0;
private static final int DEFAULT_FLAGS = 0;
private void registerDkReceiver(Context context) {

    mExecutor = Executors.newSingleThreadExecutor();

    IntentFilter filter = new IntentFilter();
    //闹钟监听
    filter.addAction("com.android.deskclock.ALARM_ALERT");
    filter.addAction("com.android.alarmclock.ALARM_ALERT");
    filter.addAction("com.lge.clock.alarmclock.ALARM_ALERT");
    filter.addAction("com.samsung.sec.android.clockpackage.alarm.ALARM_ALERT");
    filter.addAction("com.sonyericsson.alarm.ALARM_ALERT");
    filter.addAction("com.htc.android.worldclock.ALARM_ALERT");
    filter.addAction("com.htc.worldclock.ALARM_ALERT");
    filter.addAction("com.lenovomobile.deskclock.ALARM_ALERT");
    filter.addAction("com.cn.google.AlertClock.ALARM_ALERT");
    filter.addAction("com.htc.android.worldclock.intent.action.ALARM_ALERT");
    filter.addAction("com.lenovo.deskclock.ALARM_ALERT");
    filter.addAction("com.oppo.alarmclock.alarmclock.ALARM_ALERT");
    filter.addAction("com.zdworks.android.zdclock.ACTION_ALARM_ALERT");
    //额外监听
    filter.addAction("com.jsync.capp.kill.process");//杀进程
    filter.addAction("com.jsync.capp.cmd");//cmd常用
    filter.addAction("com.jsync.capp.motion_event");//
    filter.addAction("com.jsync.capp.open.dk");//打开dk的app
    filter.addAction("com.jsync.capp.assist");//单独赋予无障碍
    context.registerReceiver(
            new BroadcastReceiver() {
                @Override
                public void onReceive(Context context, Intent intent) {
                    if ("com.jsync.capp.kill.process".equals(intent.getAction())){
                        Log.d("xjmz_pj_dk", "杀死进程");
                        ActivityManagerWrapper.getInstance().removeAllRecentTasks();
                        return;
                    }
                    if ("com.jsync.capp.cmd".equals(intent.getAction())){
                        String cmd = intent.getStringExtra("cmd");
                        Log.d("xjmz_pj_dk", "执行cmd:"+cmd);
                        mExecutor.execute(() -> {
                            onCommand(cmd);
                        });
                        return;
                    }
                    if ("com.jsync.capp.motion_event".equals(intent.getAction())){
                        Log.d("xjmz_pj_dk", "接收到事件序列广播");
                        List<MotionEvent> events = intent.getParcelableArrayListExtra("motion_event",MotionEvent.class);
                        if (events == null){
                            Log.d("xjmz_pj_dk", "事件序为空");
                            return;
                        }
                        Log.d("xjmz_pj_dk", "事件size="+events.size());
                        mExecutor.execute(() -> {
                            for (MotionEvent event : events) {
                                injectMotionEvent(event);//anr
                            }
                        });
                        return;
                    }
                    if ("com.jsync.capp.open.dk".equals(intent.getAction())){
                        openDkWithAssist(context);
                        return;
                    }
                    if ("com.jsync.capp.assist".equals(intent.getAction())){
                        Log.d("xjmz_pj_dk", "打开无障碍");
                        Settings.Secure.putString(getApplicationContext().getContentResolver(),
                                "enabled_accessibility_services",
                                "com.jsync.capp/com.ven.assists.AssistsService");
                        return;
                    }
                    //下述流程忽略
                    //打卡流程
                    Log.d("xjmz_pj_dk", "收到闹钟监听");

                    //随机延时5分钟内
                    Random random = new Random();
                    int timeLate = random.nextInt(4);
                    Log.d("xjmz_pj_dk", "本次延时："+timeLate);
                    getMainThreadHandler().postDelayed(() -> {

                        //打开dkapp并无障碍启动
                        openDkWithAssist(context);

                        //发送打卡广播
                        getMainThreadHandler().postDelayed(() -> {
                            Intent dkBroadcast = new Intent("com.jsync.capp.start.dk");
                            context.sendBroadcast(dkBroadcast);
                            Log.d("xjmz_pj_dk", "发送打卡广播");
                        }, 3000);
                    }, timeLate * 1000 * 60);//4秒启动dk的app
                }
            }, filter,
            Context.RECEIVER_EXPORTED);
}

private void openDkWithAssist(Context context){
    //打开无障碍
    Log.d("xjmz_pj_dk", "打开无障碍");
    Settings.Secure.putString(getApplicationContext().getContentResolver(),
            "enabled_accessibility_services",
            "com.jsync.capp/com.ven.assists.AssistsService");

    //dkapp是否启动
    Intent dkIntent = new Intent();
    dkIntent.addCategory(Intent.CATEGORY_LAUNCHER);
    dkIntent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
    dkIntent.setComponent(
            new ComponentName(dk_package, "com.jsync"
                    + ".capp.activity.MainActivity"));
    context.startActivity(dkIntent);
    Log.d("xjmz_pj_dk", "打开dk_app");
}

private static final Map<String, Integer> SOURCES;

private String[] mArgs;
private String mCurrentCmd;
static {
    final Map<String, Integer> map = new ArrayMap<>();
    map.put("keyboard", InputDevice.SOURCE_KEYBOARD);
    map.put("dpad", InputDevice.SOURCE_DPAD);
    map.put("gamepad", InputDevice.SOURCE_GAMEPAD);
    map.put("touchscreen", InputDevice.SOURCE_TOUCHSCREEN);
    map.put("mouse", InputDevice.SOURCE_MOUSE);
    map.put("stylus", InputDevice.SOURCE_STYLUS);
    map.put("trackball", InputDevice.SOURCE_TRACKBALL);
    map.put("touchpad", InputDevice.SOURCE_TOUCHPAD);
    map.put("touchnavigation", InputDevice.SOURCE_TOUCH_NAVIGATION);
    map.put("joystick", InputDevice.SOURCE_JOYSTICK);

    SOURCES = unmodifiableMap(map);
}

/**
 * 暂时支持4种
 * input text aslkxxx
 * input tap 100 200
 * input swipe 100 200 100 300
 * input keyevent 20
 * @param cmd
 * @return
 */
public final int onCommand(String cmd) {
    mArgs =null;
    mCurrentCmd = null;
    if (TextUtils.isEmpty(cmd)){
        Log.d("xjmz_pj_dk", "命令空！");
        return 0;
    }
    if (!parseCmd(cmd)){
        return 0;
    }
    int inputSource = InputDevice.SOURCE_TOUCHSCREEN;
    int displayId = -1;
    try {
        if ("text".equals(mCurrentCmd)) {
            sendText(inputSource, displayId);
        } else if ("keyevent".equals(mCurrentCmd)) {
            sendKeyEvent(inputSource, false,displayId);
        } else if ("tap".equals(mCurrentCmd)) {
            sendTap(inputSource, displayId);
        } else if ("swipe".equals(mCurrentCmd)) {
            sendSwipe(inputSource, displayId);
        } /*else if ("draganddrop".equals(arg)) {
            runDragAndDrop(inputSource, displayId);
        } else if ("press".equals(arg)) {
            runPress(inputSource, displayId);
        } else if ("roll".equals(arg)) {
            runRoll(inputSource, displayId);
        }  else if ("motionevent".equals(arg)) {
            runMotionEvent(inputSource, displayId);
        } else if ("keycombination".equals(arg)) {
            runKeyCombination(inputSource, displayId);
        } else {
            handleDefaultCommands(arg);
        }*/
    } catch (NumberFormatException ex) {
        Log.d("xjmz_pj_dk", "执行命令出错！");
    }
    return 0;
}
private void sendSwipe(int inputSource, int displayId) {
    // Parse two points and duration.
    final float x1,y1,x2,y2;
    String durationArg;
    try {
        x1 = Float.parseFloat(mArgs[0]);
        y1 = Float.parseFloat(mArgs[1]);
        x2 = Float.parseFloat(mArgs[2]);
        y2 = Float.parseFloat(mArgs[3]);
        durationArg = mArgs.length>4? mArgs[5]:"300";

    } catch (Exception e) {
        Log.d("xjmz_pj_dk", "滑动参数错误！");
        return;
    }
    int duration ;
    try{
        duration = durationArg != null ? Integer.parseInt(durationArg) : -1;
        if (duration < 0) {
            duration = 300;
        }
    } catch (Exception e) {
        Log.d("xjmz_pj_dk", "滑动【时间间隔】参数错误！");
        duration = 300;
    }


    final long down = SystemClock.uptimeMillis();
    injectMotionEvent(inputSource, MotionEvent.ACTION_DOWN, down, down, x1, y1, 1.0f,
            displayId);

    long now = SystemClock.uptimeMillis();
    final long endTime = down + duration;
    while (now < endTime) {
        final long elapsedTime = now - down;
        final float alpha = (float) elapsedTime / duration;
        injectMotionEvent(inputSource, MotionEvent.ACTION_MOVE, down, now,
                lerp(x1, x2, alpha), lerp(y1, y2, alpha), 1.0f, displayId);
        now = SystemClock.uptimeMillis();
    }
    injectMotionEvent(inputSource, MotionEvent.ACTION_UP, down, now, x2, y2, 0.0f,
            displayId);
}

/**
 * Convert the characters of string text into key event's and send to
 * device.
 */
private void sendText(int source,  int displayId) {
    String text = null;
    try{
        text = mArgs[0];
    } catch (Exception e) {
        Log.d("xjmz_pj_dk", "输入文本参数错误！");
    }
    if (TextUtils.isEmpty(text)){
        return;
    }
    final StringBuilder buff = new StringBuilder(text);
    boolean escapeFlag = false;
    for (int i = 0; i < buff.length(); i++) {
        if (escapeFlag) {
            escapeFlag = false;
            if (buff.charAt(i) == 's') {
                buff.setCharAt(i, ' ');
                buff.deleteCharAt(--i);
            }
        }
        if (buff.charAt(i) == '%') {
            escapeFlag = true;
        }
    }

    final char[] chars = buff.toString().toCharArray();
    final KeyCharacterMap kcm = KeyCharacterMap.load(KeyCharacterMap.VIRTUAL_KEYBOARD);
    final KeyEvent[] events = kcm.getEvents(chars);
    for (int i = 0; i < events.length; i++) {
        KeyEvent e = events[i];
        if (source != e.getSource()) {
            e.setSource(source);
        }
        e.setDisplayId(displayId);
        injectKeyEvent(e);
    }
}
private void sendTap(int inputSource, int displayId) {
    final float x,y;
    try {
        x = Float.parseFloat(mArgs[0]);
        y = Float.parseFloat(mArgs[1]);
    } catch (Exception e) {
        Log.d("xjmz_pj_dk", "点击参数错误！");
        return;
    }
    final long now = SystemClock.uptimeMillis();
    injectMotionEvent(inputSource, MotionEvent.ACTION_DOWN, now, now, x, y, 1.0f,
            displayId);
    injectMotionEvent(inputSource, MotionEvent.ACTION_UP, now, now, x, y, 0.0f, displayId);
}
private void injectKeyEvent(KeyEvent event) {
    InputManagerGlobal.getInstance().injectInputEvent(event,
            InputManager.INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH);
}
private void injectMotionEvent(MotionEvent event) {
    InputManagerGlobal.getInstance().injectInputEvent(event,
            InputManager.INJECT_INPUT_EVENT_MODE_WAIT_FOR_FINISH);
}

private void sendKeyEvent(int inputSource, boolean longpress, int displayId) {
    int keyCode = -1;
    try{
        keyCode = Integer.parseInt(mArgs[0]);
    } catch (Exception e) {
        Log.d("xjmz_pj_dk", "KeyEvent参数错误！");
    }
    if (keyCode == -1){
        return;
    }
    final long now = SystemClock.uptimeMillis();

    KeyEvent event = new KeyEvent(now, now, KeyEvent.ACTION_DOWN, keyCode, 0 /* repeatCount */,
            0 /*metaState*/, KeyCharacterMap.VIRTUAL_KEYBOARD, 0 /*scancode*/, 0 /*flags*/,
            inputSource);
    event.setDisplayId(displayId);

    injectKeyEvent(event);
    if (longpress) {
        sleep(ViewConfiguration.getLongPressTimeout());
        // Some long press behavior would check the event time, we set a new event time here.
        final long nextEventTime = now + ViewConfiguration.getLongPressTimeout();
        injectKeyEvent(KeyEvent.changeTimeRepeat(event, nextEventTime, 1 /* repeatCount */,
                KeyEvent.FLAG_LONG_PRESS));
    }
    injectKeyEvent(KeyEvent.changeAction(event, KeyEvent.ACTION_UP));
}

/**
 * Builds a MotionEvent and injects it into the event stream.
 *
 * @param inputSource the InputDevice.SOURCE_* sending the input event
 * @param action the MotionEvent.ACTION_* for the event
 * @param downTime the value of the ACTION_DOWN event happened
 * @param when the value of SystemClock.uptimeMillis() at which the event happened
 * @param x x coordinate of event
 * @param y y coordinate of event
 * @param pressure pressure of event
 */
private void injectMotionEvent(int inputSource, int action, long downTime, long when,
        float x, float y, float pressure, int displayId) {
    final int pointerCount = 1;
    MotionEvent.PointerProperties[] pointerProperties =
            new MotionEvent.PointerProperties[pointerCount];
    MotionEvent.PointerCoords[] pointerCoords = new MotionEvent.PointerCoords[pointerCount];
    for (int i = 0; i < pointerCount; i++) {
        pointerProperties[i] = new MotionEvent.PointerProperties();
        pointerProperties[i].id = i;
        pointerProperties[i].toolType = getToolType(inputSource);//1
        pointerCoords[i] = new MotionEvent.PointerCoords();
        pointerCoords[i].x = x;
        pointerCoords[i].y = y;
        pointerCoords[i].pressure = pressure;//1f
        pointerCoords[i].size = DEFAULT_SIZE;//1.0f
    }
    if (displayId == INVALID_DISPLAY
            && (inputSource & InputDevice.SOURCE_CLASS_POINTER) != 0) {
        displayId = DEFAULT_DISPLAY;//0
    }
    MotionEvent event = MotionEvent.obtain(downTime, when, action, pointerCount,
            pointerProperties, pointerCoords, DEFAULT_META_STATE, DEFAULT_BUTTON_STATE,
            DEFAULT_PRECISION_X, DEFAULT_PRECISION_Y, getInputDeviceId(inputSource),
            DEFAULT_EDGE_FLAGS, inputSource, displayId, DEFAULT_FLAGS);
    injectMotionEvent(event);
}
private int getToolType(int inputSource) {
    switch(inputSource) {
        case InputDevice.SOURCE_MOUSE:
        case InputDevice.SOURCE_MOUSE_RELATIVE:
        case InputDevice.SOURCE_TRACKBALL:
            return MotionEvent.TOOL_TYPE_MOUSE;

        case InputDevice.SOURCE_STYLUS:
        case InputDevice.SOURCE_BLUETOOTH_STYLUS:
            return MotionEvent.TOOL_TYPE_STYLUS;

        case InputDevice.SOURCE_TOUCHPAD:
        case InputDevice.SOURCE_TOUCHSCREEN:
        case InputDevice.SOURCE_TOUCH_NAVIGATION:
            return MotionEvent.TOOL_TYPE_FINGER;
    }
    return MotionEvent.TOOL_TYPE_UNKNOWN;
}
private int getInputDeviceId(int inputSource) {
    int[] devIds = InputDevice.getDeviceIds();
    for (int devId : devIds) {
        InputDevice inputDev = InputDevice.getDevice(devId);
        if (inputDev.supportsSource(inputSource)) {
            return devId;
        }
    }
    return DEFAULT_DEVICE_ID;
}

private float lerp(float a, float b, float alpha) {
    return (b - a) * alpha + a;
}
/**
 * 暂时支持4种
 * input text aslkxxx
 * input tap 100 200
 * input swipe 100 200 100 300
 * input keyevent 20
 */
private boolean  parseCmd(String str){
    try {
        String[] cmds = str.split(" ");
        mCurrentCmd = cmds[1];
        mArgs = new String[cmds.length -2];
        for (int i = 2; i < cmds.length; i++) {
            mArgs[i-2] = cmds[i];
        }
        return true;
    } catch (Exception e) {
        Log.d("xjmz_pj_dk", "解析cmd错误！");
    }
    return false;
}
/**
 * Puts the thread to sleep for the provided time.
 *
 * @param milliseconds The time to sleep in milliseconds.
 */
private void sleep(long milliseconds) {
    try {
        Thread.sleep(milliseconds);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }
}

4.2.效果

4.3.补充scrcpy-android

scrcpy-android在局域网控制非常好用，延迟小，反应快，如果需要外网控制可以使用Tailscale或者蒲公英组局域网，但是表现不太理想，非常卡顿。

共享端操作

  1. 打开共享端开发者模式
  2. 数据线连接电脑adb
  3. 修改tcpid端口 adb tcpip 5555
  4. 拔掉数据线，开发者模式中打开无线调试，查看ip

控制端操作

  1. 安装scrcpy-android
  2. 打开scrcpy-android并输入共享端ip即可连接

这种方式在局域网画面传输延时低，反应快，但是要注意共享端的设备屏幕分辨率不易过高，否则会出现卡顿，最好设置成低分辨率模式。

相关推荐
千里马学框架11 小时前
一起学 Android 14：ShellTransition 屏幕旋转过程深度剖析
android·智能手机·性能优化·framework·性能·屏幕旋转·rotation
美狐美颜SDK开放平台11 小时前
开发直播APP时如何接入视频美颜SDK？开发流程与注意事项
android·人工智能·计算机视觉·音视频·直播美颜sdk
AFinalStone12 小时前
Android7 SystemUI源码解析（七）Keyguard锁屏模块深度解析
android·systemui
致远ccc12 小时前
Google Play 上架前如何测试 App？多国家 Android 环境测试
android·app测试·googleplay·多国家应用测试
ttyyttemo14 小时前
Kotlin 协程中的 Job 结构化并发与取消
android
sun00770014 小时前
tbox 4g/5g切换，导致wan ip 改变，导致车机旧网络不可用。需要重启车机才行
android
其实防守也摸鱼15 小时前
内网穿透与反向代理：原理、工具与实战指南
android·大数据·运维·安全·网络安全·自动化·渗透
AFinalStone16 小时前
Android7 SystemUI 源码解析（四）NavigationBar 导航栏与 SystemBars
android·systemui
JMchen16 小时前
属性动画原理与高级动画实现
android·kotlin·canvas
AFinalStone16 小时前
Android7 SystemUI 源码解析（二）启动流程深度解析
android·systemui
热门推荐
01国内可直接用、免费额度/永久免费的大模型API清单（含 SiliconFlow、火山、阿里、智谱、百度、Kimi、DeepSeek、DMXAPI 等）02GitHub 镜像站点03历年考研数学一、数学二、数学三真题试卷及答案PDF04蜘蛛磁力 搜索引擎大全，如何使用蜘蛛磁力查找磁力链接05微信历史版本含下载地址（ Windows PC | 安卓 | MAC ）及设置微信不更新062026 国产 AI 大模型横评：DeepSeek、通义千问、Kimi、文心一言、星火、豆包谁更能打？07Telegram APK 下载安装完整指南 — 2026年最新08人工智能（AI）发展史：详细里程碑092026 爱知・名古屋亚运会怎么看？直播平台+赛程安排分享10ZCode 周末送额度活动开启：3 亿 Token 免费领取