【Android】VirtualDisplay创建流程及原理

Android VirtualDisplay创建流程及原理

  • Android DisplayManager提供了createVirtualDisplay接口,用于创建虚拟屏。虚拟屏可用于录屏(网上很多资料说这个功能),分屏幕(比如一块很长的屏幕,通过虚拟屏分出不同的区域)等等。

创建VirtualDisplay

  • DisplayManager中的函数原型如下。后两个Hide的API,只有平台的应用才可以使用。
java 复制代码
// frameworks/base/core/java/android/hardware/display/DisplayManager.java
public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags) {
}

public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
}
/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull String name, int width, int height, int densityDpi, @Nullable Surface surface,
		int flags, @Nullable VirtualDisplay.Callback callback, @Nullable Handler handler,
		@Nullable String uniqueId) {
}
/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull VirtualDisplayConfig virtualDisplayConfig,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
}
  • 补充一点,MediaProjection中也提供了 createVirtualDisplay这个接口,实际上也是通过调用DisplayManager实现的功能。
java 复制代码
// frameworks/base/media/java/android/media/projection/MediaProjection.java
    public VirtualDisplay createVirtualDisplay(@NonNull VirtualDisplayConfig virtualDisplayConfig,
            @Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
        DisplayManager dm = mContext.getSystemService(DisplayManager.class);
        // 调用DisplayManager的接口
        return dm.createVirtualDisplay(this, virtualDisplayConfig, callback, handler);
    }
  • 创建VirtualDisplay时,需要传入Surface。**VirtualDisplay上要绘制的内容,实际是通过传入的Surface显示出来的。**比如在主屏(根据物理屏,分配逻辑Display)上创建了一个SurfaceView,通过把这个SurfaceView传给VirtualDisplay。那么VirtualDisplay的 内容,实际上是在主屏的SurfaceView上显示的。下面是一段Android原生的例子。
java 复制代码
// frameworks/base/packages/SystemUI/tests/src/com/android/systemui/navigationbar/NavigationBarButtonTest.java
private Display createVirtualDisplay() {
	final String displayName = "NavVirtualDisplay";
	final DisplayInfo displayInfo = new DisplayInfo();
	mContext.getDisplay().getDisplayInfo(displayInfo);

	final DisplayManager displayManager = mContext.getSystemService(DisplayManager.class);

	// 创建ImageReader,通过它得到一张Surface
	mReader = ImageReader.newInstance(displayInfo.logicalWidth,
			displayInfo.logicalHeight, PixelFormat.RGBA_8888, 2);

	assertNotNull("ImageReader must not be null", mReader);

	// 创建虚拟屏,传入Surface。
	mVirtualDisplay = displayManager.createVirtualDisplay(displayName, displayInfo.logicalWidth,
			displayInfo.logicalHeight, displayInfo.logicalDensityDpi, mReader.getSurface(),
			0 /*flags*/);

	assertNotNull("virtual display must not be null", mVirtualDisplay);

	waitForDisplayReady(mVirtualDisplay.getDisplay().getDisplayId());

	return mVirtualDisplay.getDisplay();
}
  • 上面的例子中创建虚拟屏,返回Display(实际上是VirtualDislay)对象。有了Display对象,我们就可以将View绑定到这个虚拟的Display上了(绑定网上方法比较多可自行搜索)。关于Surface的创建,有很多种方法,比如通过SurfaceContron+Buffer这种方式也可以。
  • VituralDisplay创建时,需要提供flag。其值定义如下,可通过 "或"将flag组合。
java 复制代码
 
    public static final int VIRTUAL_DISPLAY_FLAG_PUBLIC = 1 << 0;

    public static final int VIRTUAL_DISPLAY_FLAG_PRESENTATION = 1 << 1;

    public static final int VIRTUAL_DISPLAY_FLAG_SECURE = 1 << 2;

    public static final int VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY = 1 << 3;

    public static final int VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR = 1 << 4;

    public static final int VIRTUAL_DISPLAY_FLAG_CAN_SHOW_WITH_INSECURE_KEYGUARD = 1 << 5;

    public static final int VIRTUAL_DISPLAY_FLAG_SUPPORTS_TOUCH = 1 << 6;

    public static final int VIRTUAL_DISPLAY_FLAG_ROTATES_WITH_CONTENT = 1 << 7;

    public static final int VIRTUAL_DISPLAY_FLAG_DESTROY_CONTENT_ON_REMOVAL = 1 << 8;

    public static final int VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS = 1 << 9;

    public static final int VIRTUAL_DISPLAY_FLAG_TRUSTED = 1 << 10;

    public static final int VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP = 1 << 11;
  • DisplayManager公开的接口中,有VirtualDisplay.Callback ,提供了其状态的回调。
java 复制代码
    public static abstract class Callback {
        /**
         * Called when the virtual display video projection has been
         * paused by the system or when the surface has been detached
         * by the application by calling setSurface(null).
         * The surface will not receive any more buffers while paused.
         */
         public void onPaused() { }

        /**
         * Called when the virtual display video projection has been
         * resumed after having been paused.
         */
         public void onResumed() { }

        /**
         * Called when the virtual display video projection has been
         * stopped by the system.  It will no longer receive frames
         * and it will never be resumed.  It is still the responsibility
         * of the application to release() the virtual display.
         */
        public void onStopped() { }
    }

VirtualDisplay原理

  • 关于VirtualDisplay的实现原理,主要从AndroidFramework角度进行分析。
java 复制代码
// /frameworks/base/core/java/android/hardware/display/DisplayManager.java
public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags) {
	return createVirtualDisplay(name, width, height, densityDpi, surface, flags, null, null);
}

public VirtualDisplay createVirtualDisplay(@NonNull String name,
		int width, int height, int densityDpi, @Nullable Surface surface, int flags,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
	final VirtualDisplayConfig.Builder builder = new VirtualDisplayConfig.Builder(name, width,
			height, densityDpi);
	builder.setFlags(flags);
	if (surface != null) {
		builder.setSurface(surface);
	}
	return createVirtualDisplay(null /* projection */, builder.build(), callback, handler);
}

// TODO : Remove this hidden API after remove all callers. (Refer to MultiDisplayService)
/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull String name, int width, int height, int densityDpi, @Nullable Surface surface,
		int flags, @Nullable VirtualDisplay.Callback callback, @Nullable Handler handler,
		@Nullable String uniqueId) {
	final VirtualDisplayConfig.Builder builder = new VirtualDisplayConfig.Builder(name, width,
			height, densityDpi);
	builder.setFlags(flags);
	if (uniqueId != null) {
		builder.setUniqueId(uniqueId);
	}
	if (surface != null) {
		builder.setSurface(surface);
	}
	return createVirtualDisplay(projection, builder.build(), callback, handler);
}

/** @hide */
public VirtualDisplay createVirtualDisplay(@Nullable MediaProjection projection,
		@NonNull VirtualDisplayConfig virtualDisplayConfig,
		@Nullable VirtualDisplay.Callback callback, @Nullable Handler handler) {
	// 走的这里,会调用到DisplayManagerGlobal中。
	return mGlobal.createVirtualDisplay(mContext, projection, virtualDisplayConfig, callback,
			handler);
}
  • DisplayManagerGlobal调用DMS(DisplayManagerService)服务创建虚拟屏,得到DMS返回的DisplayID后,通过DisplayID在Client端创建了VirtualDisplay对象。
java 复制代码
// /frameworks/base/core/java/android/hardware/display/DisplayManager.java
public VirtualDisplay createVirtualDisplay(@NonNull Context context, MediaProjection projection,
		@NonNull VirtualDisplayConfig virtualDisplayConfig, VirtualDisplay.Callback callback,
		Handler handler) {
	VirtualDisplayCallback callbackWrapper = new VirtualDisplayCallback(callback, handler);
	// 从MediaProjection过来的调用,这个地方非空。
	IMediaProjection projectionToken = projection != null ? projection.getProjection() : null;
	int displayId;
	try {
		// 告知DMS创建虚拟屏,并返回DisplayID
		displayId = mDm.createVirtualDisplay(virtualDisplayConfig, callbackWrapper,
				projectionToken, context.getPackageName());
	} catch (RemoteException ex) {
		throw ex.rethrowFromSystemServer();
	}
	if (displayId < 0) {
		Log.e(TAG, "Could not create virtual display: " + virtualDisplayConfig.getName());
		return null;
	}
	
	// 通过DisplayID,取得Display对象信息(也是调用DMS得到的)
	Display display = getRealDisplay(displayId);
	if (display == null) {
		Log.wtf(TAG, "Could not obtain display info for newly created "
				+ "virtual display: " + virtualDisplayConfig.getName());
		try {
			// 创建失败,需要释放
			mDm.releaseVirtualDisplay(callbackWrapper);
		} catch (RemoteException ex) {
			throw ex.rethrowFromSystemServer();
		}
		return null;
	}
	
	// 创建VirtualDisplay
	return new VirtualDisplay(this, display, callbackWrapper,
			virtualDisplayConfig.getSurface());
}
  • DisplayManagerService(DMS)中创建DisplayDevice并添加到Device列表中管理
java 复制代码
// /frameworks/base/services/core/java/com/android/server/display/DisplayManagerService.java
@Override // Binder call
public int createVirtualDisplay(VirtualDisplayConfig virtualDisplayConfig,
		IVirtualDisplayCallback callback, IMediaProjection projection, String packageName) {
		
	// 检查uid与包名,是否相符。
	final int callingUid = Binder.getCallingUid();
	if (!validatePackageName(callingUid, packageName)) {
		throw new SecurityException("packageName must match the calling uid");
	}
	if (callback == null) {
		throw new IllegalArgumentException("appToken must not be null");
	}
	if (virtualDisplayConfig == null) {
		throw new IllegalArgumentException("virtualDisplayConfig must not be null");
	}
	//  拿到client端传过来的surface对象
	final Surface surface = virtualDisplayConfig.getSurface();
	int flags = virtualDisplayConfig.getFlags();

	if (surface != null && surface.isSingleBuffered()) {
		throw new IllegalArgumentException("Surface can't be single-buffered");
	}

	// 下面开始针对Flag,做一些逻辑判断。
	if ((flags & VIRTUAL_DISPLAY_FLAG_PUBLIC) != 0) {
		flags |= VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR;

		// Public displays can't be allowed to show content when locked.
		if ((flags & VIRTUAL_DISPLAY_FLAG_CAN_SHOW_WITH_INSECURE_KEYGUARD) != 0) {
			throw new IllegalArgumentException(
					"Public display must not be marked as SHOW_WHEN_LOCKED_INSECURE");
		}
	}
	if ((flags & VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY) != 0) {
		flags &= ~VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR;
	}
	if ((flags & VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR) != 0) {
		flags &= ~VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP;
	}

	if (projection != null) {
		try {
			if (!getProjectionService().isValidMediaProjection(projection)) {
				throw new SecurityException("Invalid media projection");
			}
			flags = projection.applyVirtualDisplayFlags(flags);
		} catch (RemoteException e) {
			throw new SecurityException("unable to validate media projection or flags");
		}
	}

	if (callingUid != Process.SYSTEM_UID &&
			(flags & VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR) != 0) {
		if (!canProjectVideo(projection)) {
			throw new SecurityException("Requires CAPTURE_VIDEO_OUTPUT or "
					+ "CAPTURE_SECURE_VIDEO_OUTPUT permission, or an appropriate "
					+ "MediaProjection token in order to create a screen sharing virtual "
					+ "display.");
		}
	}
	if (callingUid != Process.SYSTEM_UID && (flags & VIRTUAL_DISPLAY_FLAG_SECURE) != 0) {
		if (!canProjectSecureVideo(projection)) {
			throw new SecurityException("Requires CAPTURE_SECURE_VIDEO_OUTPUT "
					+ "or an appropriate MediaProjection token to create a "
					+ "secure virtual display.");
		}
	}

	if (callingUid != Process.SYSTEM_UID && (flags & VIRTUAL_DISPLAY_FLAG_TRUSTED) != 0) {
		if (!checkCallingPermission(ADD_TRUSTED_DISPLAY, "createVirtualDisplay()")) {
			EventLog.writeEvent(0x534e4554, "162627132", callingUid,
					"Attempt to create a trusted display without holding permission!");
			throw new SecurityException("Requires ADD_TRUSTED_DISPLAY permission to "
					+ "create a trusted virtual display.");
		}
	}

	if (callingUid != Process.SYSTEM_UID
			&& (flags & VIRTUAL_DISPLAY_FLAG_OWN_DISPLAY_GROUP) != 0) {
		if (!checkCallingPermission(ADD_TRUSTED_DISPLAY, "createVirtualDisplay()")) {
			throw new SecurityException("Requires ADD_TRUSTED_DISPLAY permission to "
					+ "create a virtual display which is not in the default DisplayGroup.");
		}
	}

	if ((flags & VIRTUAL_DISPLAY_FLAG_TRUSTED) == 0) {
		flags &= ~VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS;
	}

	// Sometimes users can have sensitive information in system decoration windows. An app
	// could create a virtual display with system decorations support and read the user info
	// from the surface.
	// We should only allow adding flag VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
	// to trusted virtual displays.
	final int trustedDisplayWithSysDecorFlag =
			(VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
					| VIRTUAL_DISPLAY_FLAG_TRUSTED);
	if ((flags & trustedDisplayWithSysDecorFlag)
			== VIRTUAL_DISPLAY_FLAG_SHOULD_SHOW_SYSTEM_DECORATIONS
			&& !checkCallingPermission(INTERNAL_SYSTEM_WINDOW, "createVirtualDisplay()")) {
			throw new SecurityException("Requires INTERNAL_SYSTEM_WINDOW permission");
	}

	final long token = Binder.clearCallingIdentity();
	try {
		// 调用内部实现
		return createVirtualDisplayInternal(callback, projection, callingUid, packageName,
				surface, flags, virtualDisplayConfig);
	} finally {
		Binder.restoreCallingIdentity(token);
	}
}

// /frameworks/base/services/core/java/com/android/server/display/DisplayManagerService.java
private int createVirtualDisplayInternal(IVirtualDisplayCallback callback,
		IMediaProjection projection, int callingUid, String packageName, Surface surface,
		int flags, VirtualDisplayConfig virtualDisplayConfig) {
	synchronized (mSyncRoot) {
		if (mVirtualDisplayAdapter == null) {
			Slog.w(TAG, "Rejecting request to create private virtual display "
					+ "because the virtual display adapter is not available.");
			return -1;
		}
		// 为虚拟屏创建Device(告知surfaceflinger创建Display)
		DisplayDevice device = mVirtualDisplayAdapter.createVirtualDisplayLocked(
				callback, projection, callingUid, packageName, surface, flags,
				virtualDisplayConfig);
		if (device == null) {
			return -1;
		}

		// 发送添加Device通知,这里比较重要
		mDisplayDeviceRepo.onDisplayDeviceEvent(device,
				DisplayAdapter.DISPLAY_DEVICE_EVENT_ADDED);
		// 检查Display是否创建成功
		final LogicalDisplay display = mLogicalDisplayMapper.getDisplayLocked(device);
		if (display != null) {
			return display.getDisplayIdLocked();
		}

		// Something weird happened and the logical display was not created.
		Slog.w(TAG, "Rejecting request to create virtual display "
				+ "because the logical display was not created.");
		mVirtualDisplayAdapter.releaseVirtualDisplayLocked(callback.asBinder());
		mDisplayDeviceRepo.onDisplayDeviceEvent(device,
				DisplayAdapter.DISPLAY_DEVICE_EVENT_REMOVED);
	}
	return -1;
}
  • 接下来DMS开始调用SurfaceFlinger的接口,创建Display。并将Display放入自身的List中管理。
java 复制代码
// /frameworks/base/services/core/java/com/android/server/display/VirtualDisplayAdapter.java
public DisplayDevice createVirtualDisplayLocked(IVirtualDisplayCallback callback,
		IMediaProjection projection, int ownerUid, String ownerPackageName, Surface surface,
		int flags, VirtualDisplayConfig virtualDisplayConfig) {
	String name = virtualDisplayConfig.getName();
	// VIRTUAL_DISPLAY_FLAG_SECURE 的用途,是判断是否为安全的Display,这个参数会告知SurfaceFlinger
	boolean secure = (flags & VIRTUAL_DISPLAY_FLAG_SECURE) != 0;
	IBinder appToken = callback.asBinder();
	// 调用SurfaceFligner创建Display(Display的type是virtual)
	IBinder displayToken = mSurfaceControlDisplayFactory.createDisplay(name, secure);
	final String baseUniqueId =
			UNIQUE_ID_PREFIX + ownerPackageName + "," + ownerUid + "," + name + ",";
	final int uniqueIndex = getNextUniqueIndex(baseUniqueId);
	String uniqueId = virtualDisplayConfig.getUniqueId();
	if (uniqueId == null) {
		uniqueId = baseUniqueId + uniqueIndex;
	} else {
		uniqueId = UNIQUE_ID_PREFIX + ownerPackageName + ":" + uniqueId;
	}
	// 通过SurfaceFligner返回的displayToken,创建Device对象
	VirtualDisplayDevice device = new VirtualDisplayDevice(displayToken, appToken,
			ownerUid, ownerPackageName, surface, flags, new Callback(callback, mHandler),
			uniqueId, uniqueIndex, virtualDisplayConfig);
	//  放到虚拟屏的List中管理。
	mVirtualDisplayDevices.put(appToken, device);

	try {
		if (projection != null) {
			projection.registerCallback(new MediaProjectionCallback(appToken));
		}
		appToken.linkToDeath(device, 0);
	} catch (RemoteException ex) {
		mVirtualDisplayDevices.remove(appToken);
		device.destroyLocked(false);
		return null;
	}

	// Return the display device without actually sending the event indicating
	// that it was added.  The caller will handle it.
	return device;
}

// /frameworks/base/services/core/java/com/android/server/display/DisplayDeviceRepository.java
//  mDisplayDeviceRepo.onDisplayDeviceEvent(device,
//				DisplayAdapter.DISPLAY_DEVICE_EVENT_ADDED); 
// 这段代码,会调用到下面的函数中。
private void handleDisplayDeviceAdded(DisplayDevice device) {
	synchronized (mSyncRoot) {
		DisplayDeviceInfo info = device.getDisplayDeviceInfoLocked();
		if (mDisplayDevices.contains(device)) {
			Slog.w(TAG, "Attempted to add already added display device: " + info);
			return;
		}
		Slog.i(TAG, "Display device added: " + info);
		device.mDebugLastLoggedDeviceInfo = info;
		// 需要是将Device(就是上面创建的虚拟屏幕Device)放入到DMS的管理list
		mDisplayDevices.add(device);
		// 通知Device添加,会调用到LogicalDisplayMappe的handleDisplayDeviceAddedLocked中。
		sendEventLocked(device, DISPLAY_DEVICE_EVENT_ADDED);
	}
}
// /frameworks/base/services/core/java/com/android/server/display/LogicalDisplayMapper.java
private void handleDisplayDeviceAddedLocked(DisplayDevice device) {
	DisplayDeviceInfo deviceInfo = device.getDisplayDeviceInfoLocked();
	// Internal Displays need to have additional initialization.
	// This initializes a default dynamic display layout for INTERNAL
	// devices, which is used as a fallback in case no static layout definitions
	// exist or cannot be loaded.
	if (deviceInfo.type == Display.TYPE_INTERNAL) {
		initializeInternalDisplayDeviceLocked(device);
	}

	// Create a logical display for the new display device
	LogicalDisplay display = createNewLogicalDisplayLocked(
			device, Layout.assignDisplayIdLocked(false /*isDefault*/));

	// 刷新布局和display配置
	applyLayoutLocked();
	updateLogicalDisplaysLocked();
}
  • 虚拟屏幕的创建,Client端通过Surface告知的DisplayID,创建VirtualDisplay对象。通过DisplayID,与DMS打交道。DMS服务端,通过SurfaceFlinger创建虚拟屏,拿到SurfaceFligner的DisplayToken,然后通过它创建VirtualDisplayDevice + LogicalDisplay来管理虚拟屏幕。
如何上屏?
  • 创建虚拟屏幕的时候,会传入了一张Surface(比如绑定主屏的一张Buffer)。虚拟屏通过这张Surface拿到Surface对应的Buffer,将上屏内容绘制到这个Buffer上,然后提交到画面流水线上(SurfaceFlinger)。通过SurfaceFlinger将这个这个Buffer最终由SurfaceFlinger描画并显示到Surface所在那张Display上(根据VirtualDisplay位置去显示。)
相关推荐
CYRUS_STUDIO2 小时前
手把手教你改造 AAR:解包、注入逻辑、重打包,一条龙玩转第三方 SDK!
android·逆向
CYRUS_STUDIO3 小时前
Android 源码如何导入 Android Studio?踩坑与解决方案详解
android·android studio·源码阅读
前端赵哈哈4 小时前
初学者入门:Android 实现 Tab 点击切换(TabLayout + ViewPager2)
android·java·android studio
一条上岸小咸鱼7 小时前
Kotlin 控制流(二):返回和跳转
android·kotlin
Jasonakeke7 小时前
【重学 MySQL】九十二、 MySQL8 密码强度评估与配置指南
android·数据库·mysql
Mertrix_ITCH7 小时前
在 Android Studio 中修改 APK 启动图标(2025826)
android·ide·android studio
荏苒追寻7 小时前
Android OpenGL基础1——常用概念及方法解释
android
人生游戏牛马NPC1号7 小时前
学习 Android (十七) 学习 OpenCV (二)
android·opencv·学习
恋猫de小郭8 小时前
谷歌开启 Android 开发者身份验证,明年可能开始禁止“未经验证”应用的侧载,要求所有开发者向谷歌表明身份
android·前端·flutter
用户098 小时前
Gradle声明式构建总结
android