Android性能优化之截屏时黑屏卡顿问题

Android性能优化之截屏时黑屏卡顿问题

1.前言：

之前在做云游戏项目时遇到了一个黑屏的问题，接触过云游戏的小伙伴肯定都知道，他的实现方式是以推拉流+websocket展现给用户的，推拉流油很多种协议RTSP、TCP、UDP、RTMP等，然后根据设备id、游玩时长、游戏名称、游戏id等拉起一个小游戏，然后以视频流的方式呈现给用户，这其他云游戏有几种方式可以进行操控，鼠标、遥控器、手柄、手机扫码等，当然手柄还分单手柄和双手柄，可以在手机、TV、盒子、投影、平板等各种设备上进行畅玩，今天要说的是场景是推拉流的同时手机进行扫码，h5通过websocket截屏的方式进行消息和数据发送传递，然而在4.4-6.0的TV和盒子上面长时间进行扫码操作的时候会导致用户游戏界面黑屏然后直接卡死，这是一个非常严重的问题.

2.截屏实现：

因为之前项目的版本很多所以录频权限不一样，现在的项目都需要适配Android15录屏权限

2.1 请求截取手机屏幕权限

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
private void createScreenCaptureIntent() {
    MediaProjectionManager mediaProjectionManager = (MediaProjectionManager)
            getSystemService(Context.MEDIA_PROJECTION_SERVICE);
    startActivityForResult(
            mediaProjectionManager.createScreenCaptureIntent(),
            1001);
}

2.2 创建截屏画面

• 这里由于是云游戏和在TV大屏上面运行，所以画面必现保持高清
• 图片的格式必现是PixelFormat.RGBA_8888，565会导致画面模糊
• 项目要求高清画质，游戏体验也得提升

@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
private void createVirtualDisplay(Intent data) {
    mImageReader = ImageReader.newInstance(mSWDisplay.getWidth(), mSWDisplay.getHeight(), PixelFormat.RGBA_8888, 2);
    MediaProjection mMediaProjection = ((MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE)).getMediaProjection(Activity.RESULT_OK, data);
    mMediaProjection.createVirtualDisplay("screen",
            mSWDisplay.getWidth(), mSWDisplay.getHeight(), 2, DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
            mImageReader.getSurface(), null, null);
}

3.截取游戏画面和图片：

private void acquireScreenImage() {
    Image image = mImageReader.acquireLatestImage();
    if (image == null) {
        return;
    }
    int width = image.getWidth();
​
    int height = image.getHeight();
​
    final Image.Plane[] planes = image.getPlanes();
​
    final ByteBuffer buffer = planes[0].getBuffer();
​
    int pixelStride = planes[0].getPixelStride();
​
    int rowStride = planes[0].getRowStride();
​
    int rowPadding = rowStride - pixelStride * width;
​
    Bitmap bitmap = Bitmap.createBitmap(width + rowPadding / pixelStride, height, Bitmap.Config.ARGB_8888);
​
    bitmap.copyPixelsFromBuffer(buffer);
    image.close();
}

4.初始化扫码：

private void initQRCode() {
    LogUtils.e("Account" + SPManager.getInstance().getAccount());
    mQRCodeUrl = "http://www.example.com/control.html" + "?userId=" + SPManager.getInstance().getAccount() + "&apkId=" + mAppId;
    mQRImageView.setImageBitmap(ZXingUtils.createQRImage(mQRCodeUrl, ConvertUtils.dp2px(100), ConvertUtils.dp2px(100)));
    mQRCodeLayout.setVisibility(_handlerGame.hasGameController() || mSupportRemoteControl || Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP ? View.GONE : View.VISIBLE);
}

5.websocket初始化：

//websocket地址
protected String getWSUrl() {
  String suffix = "/" + SPManager.getInstance().getAccount() + "/" + mAppId + "/" +                         Constants.channelId;
    return "ws://"+"xxxx:8180"+"/paas/android/webSocket/socketStatus" + suffix;
}
​
//链接websocket
private void connectTimeWebSocket() {
    OkHttpClient client = new OkHttpClient.Builder().build();
    Request mRequest = new Request.Builder().url(getWSUrl()).build();
    LogUtils.d("--timeWebSocket--", getWSUrl());
    mWebSocketTime = client.newWebSocket(mRequest, new WebSocketListener() {
        @Override
        public void onClosed(WebSocket webSocket, int code, String reason) {
            super.onClosed(webSocket, code, reason);
            LogUtils.e("WebSocket..", "onClosed " + reason);
        }
​
        @Override
        public void onClosing(WebSocket webSocket, int code, String reason) {
            super.onClosing(webSocket, code, reason);
            LogUtils.e("WebSocket..", "onClosing " + code + reason);
        }
​
        @Override
        public void onFailure(WebSocket webSocket, Throwable t, okhttp3.Response response) {
            super.onFailure(webSocket, t, response);
            LogUtils.e("WebSocket..", "onFailure" + t);
            LogUtils.e("WebSocket..", "onFailure" + response);
        }
​
        @Override
        public void onMessage(WebSocket webSocket, String text) {
            super.onMessage(webSocket, text);
            LogUtils.e("WebSocket..", "onMessage: " + text);
            Gson gson = new Gson();
            WSInfo wsInfo = gson.fromJson(text, WSInfo.class);
            if (wsInfo == null || isDestroyed()) {
                return;
            }
            if (WSInfo.TYPE_CONNECT.equals(wsInfo.getType())) {
                if (WSInfo.RESULT_TRUE.equals(wsInfo.getResult())) {
                    // 与服务端建立了连接
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {
                            connectTimeWebSocketSuccess();
                        }
                    });
                } else {
                    ToastUtils.showLong(Constants.ERROR_MSG_10009);
                    finish();
                }
            }
            // 与服务端心跳检测
            if (WSInfo.TYPE_PING.equals(wsInfo.getType())) {
                wsInfo.setType(WSInfo.TYPE_PONG);
                webSocket.send(gson.toJson(wsInfo));
            }
​
            runOnUiThread(new Runnable() {
                @Override
                public void run() {
                    onWSMessage(webSocket, wsInfo, gson);
                }
            });
        }
​
        @Override
        public void onMessage(WebSocket webSocket, ByteString bytes) {
            super.onMessage(webSocket, bytes);
            LogUtils.e("WebSocket..", "onMessage bytes: " + bytes);
        }
​
        @Override
        public void onOpen(WebSocket webSocket, okhttp3.Response response) {
            super.onOpen(webSocket, response);
            LogUtils.e("WebSocket..", "onOpen");
        }
    });
}

6.游戏sdk初始化：

private void startPlay(Bundle bundle) {
​
    if (bundle != null) {
        int groupId = Integer.valueOf(SPManager.getInstance().getBaseGroupId());
        LogUtils.e("groupId： " + SPManager.getInstance().getBaseGroupId());
        appBackground = bundle.getString("appBackground", "");
        mAppId = bundle.getInt("appId", 1);
        LogUtils.d("appId： " + mAppId);
        long onlineTime = bundle.getInt("onlineTime", 1);
        final String padCode = bundle.getString("padCode", null);
        mManual = bundle.getString(Constants.MANUAL, "");
        //操控方式 0手柄 1遥控器
        mOperationMode = bundle.getStringArrayList(Constants.OPERATION_MODE);
        LogUtils.d("OperationMode", mOperationMode);
        //是否显示虚拟手柄二维码，若操控方式不为空且操控方式为遥控器时不显示，否则显示
        mSupportRemoteControl = mOperationMode != null && mOperationMode.contains(Constants.OPERATION_MODE_REMOTE_CONTROL);
        final int apiLevel = 2;
        final int useSSL = 0;
        String ip = NetWorkUtils.getIPAddress(this);
        LogUtils.d("--ipAddress--",ip);
        PlaySdkManager.connectDevice(DataConstants.getSWSign(), padCode, onlineTime, groupId, packageName,
                null, 0, 15000, new PlaySdkManager.OnResponseListener() {
                    public void onResponse(int result, String content) {
                        LogUtils.d("connectRequest, result:" + result + ", content:" + content);
                        do {
                            if (result == 0) {
                                mPlaySdkManager = new PlaySdkManager(VideoPlayActivity.this, mIsUseSfDecode);
                                mPlaySdkManager.setSdkCallback(VideoPlayActivity.this);
                                if (_handlerGame != null) {
                                    _handlerGame.setPlaySdkManager(mPlaySdkManager);
                                }
                                if (mIsUseSfDecode) {
                                    if (mPlaySdkManager.setParams(content, packageName, apiLevel, useSSL,
                                            mSWViewDisplay, VideoPlayActivity.this) != 0) {
                                        break;
                                    }
                                } else {
                                    //5、set game parameters
                                    if (mPlaySdkManager.setParams(content, packageName, apiLevel, useSSL,
                                            mSWDisplay, VideoPlayActivity.this) != 0) {
                                        break;
                                    }
                                }
                                mPadCode = mPlaySdkManager.getPadCode();
                                mPadCodeView.setText("设备编号：" + mPadCode);
                                if (mPlaySdkManager.start() != 0) {
                                    break;
                                }
                                return;
                            }
                        }
                        while (false);
                    }
                });
    }
}

7.优化后的游戏截图：

• 使用bitmapsetScale对图片进行缩放
• 使用Luban压缩进行按质量和比例压缩
• 使用bitmap.recycle()回收图片资源

7.1 对图片进行缩放处理：

private void acquireScreenImage() {
    Image image = mImageReader.acquireLatestImage();
    if (image == null) {
        return;
    }
    int width = image.getWidth();
​
    int height = image.getHeight();
​
    final Image.Plane[] planes = image.getPlanes();
​
    final ByteBuffer buffer = planes[0].getBuffer();
​
    int pixelStride = planes[0].getPixelStride();
​
    int rowStride = planes[0].getRowStride();
​
    int rowPadding = rowStride - pixelStride * width;
​
    Bitmap bitmap = Bitmap.createBitmap(width + rowPadding / pixelStride, height, Bitmap.Config.ARGB_8888);
​
    bitmap.copyPixelsFromBuffer(buffer);
    if (Constants.PACKAGE_DY.equals(packageName)) {
        bitmap = Bitmap.createBitmap(bitmap, 0, 0, width, height);
    } else {
        Matrix matrix = new Matrix();
        matrix.setScale(0.35f, 0.35f);
        bitmap = Bitmap.createBitmap(bitmap, 0, 0, width, height, matrix, true);
    }
    if (bitmap != null) {
        compressionImage((BitmapUtils.bitmapToString(bitmap)), bitmap);
    }
    image.close();
}

7.2 使用luban压缩图片：

可以根据项目自行设置压缩的比例和质量，

/**
 * 压缩图片
 *
 * @param imageList
 * @param bitmap
 */
private void compressionImage(String imageList, Bitmap bitmap) {
    Luban.with(this)
            .load(imageList)
            .ignoreBy(100)
            .setCompressListener(new OnCompressListener() {
                @Override
                public void onStart() {
                    LogUtils.d("-----------开始压缩 == ");
                }
​
                @Override
                public void onSuccess(File file) {
                    try {
                        ByteArrayOutputStream baos = new ByteArrayOutputStream();
                        bitmap.compress(Bitmap.CompressFormat.JPEG, 100, baos);
                        byte[] data = baos.toByteArray();
                        mWebSocket.send(ByteString.of(data));
                        baos.flush();
                        baos.close();
                        if (bitmap != null && !bitmap.isRecycled()) {
                            bitmap.recycle();
                        }
                        LogUtils.d("-----------压缩成功 file.length()== " + file.length());
​
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                }
​
                @Override
                public void onError(Throwable e) {
                    LogUtils.d("-----------压缩失败 == ");
                }
            }).launch();
}

8.在onDestroy()回收资源：

• 游戏sdk断开连接，停止拉流
• 释放手柄
• handler回收
• 广播取消注册
• 计时器回收
• websocket端口连接

@Override
protected void onDestroy() {
    mSendScreenImage = false;
    if (mPlaySdkManager != null) {
        mPlaySdkManager.stop();
        mPlaySdkManager.release();
        PlaySdkManager.disconnectDevice(DataConstants.getSWSign(), mPadCode, mAppId, null);
    }
    if (_handlerGame != null) {
        _handlerGame.onRelease();
    }
    mHandler.removeMessages(mTipLoopWhat);
    unregisterReceiver(receiver);
    if (null != timer) {
        timer.cancel();
    }
    super.onDestroy();

    if (mWebSocket != null) {
        mWebSocket.cancel();
    }
    if (mWebSocketTime != null) {
        mWebSocketTime.cancel();
    }
    closeTipDialog();
}

9.总结：

通过以上的方式解决游戏黑屏卡顿问题，当然现在不能复现，也展示不了效果，因为项目接口都连接不上了，只能从以前的代码进行分析排查原因,然后去解决卡顿黑屏问题,实际项目比这里更复杂，由于不能抓取到trace信息，这里就不展示分析的过程了.本文做为性能优化的开篇，后面如果有项目可以g给出分析trace我会把整个完整的分析过程展示出来，要不然只讲问题和解决方式很枯燥，只有结合实际情况，然后分析原因，讲解过程，最后才是给出方案和验证.

• 保证质量的情况下处理黑屏问题
• 图片的格式不能使用RGB_565
• 产生问题的原因是由于长时间对TV盒子进行截屏、还有websocket心跳包、推拉流、扫码操控TV盒子的游戏界面，TV设备的内存和性能较低，各种线程比较耗资源，当然这种长时间操作不回收资源也是最主要的问题
• 对图片进行缩放处理
• 对图片进行压缩图片(按照比例和质量)
• 回收图片资源
• 退出游戏时也要注意回收资源

10.退出游戏效果截图：