背景:
按照需求,需要支持APP在手机息屏时进行推流、录像。
技术要点:
1、手机在息屏时能够打开camera获取预览数据
2、获取预览数据时进行编码以及合成视频
一、息屏时获取camera预览数据:
①Camera.setPreviewDisplay(SurfaceHolder holder):
一般常规的打开camera后(Camera.open(int cameraId)),给相机设置预览setPreviewDisplay(SurfaceHolder holder),holder通过surfaceview获取。但是者在surfaceDestroyed(xxxxxx)后无法获取预览数据,所以setPreviewDisplay(SurfaceHolder holder)此方法无法满足息屏的需求。
②Camera.setPreviewTexture(SurfaceTexture surfaceTexture):
此方法通过创建一个new SurfaceTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES)传入就可以实现息屏获取相机的预览数据。这样就可以避免直接使用TextureView带来的onSurfaceTextureDestroyed(xxxx)导致息屏后无法获取预览数据。
二、预览camera预览数据:
①Camera.setPreviewTexture(SurfaceTexture surfaceTexture):
获取到yuv数据进行转换成bitmap,然后用Imageview或者Surfaceview直接显示。
此方法带来的弊端:
1、每一帧数据都要生成bitmap,短时间频繁的创建对象会导致STW,从而导致ANR
2、预览数据不流畅,是用Imageview或者Surfaceview手动方式展示的
②Camera.setPreviewDisplay(SurfaceHolder holder):
此方法是Android自带的,没有上述的弊端:ANR、画面卡顿,但是在息屏时无法获取预览数据
③Camera.setPreviewTexture(SurfaceTexture surfaceTexture)+Camera.setPreviewDisplay(SurfaceHolder holder):
此方法既解决了预览问题也解决了息屏获取预览数据问题,但是此方法在MediaMuxer两种模式转换合成音视频时无法合成连续的音视频,只能亮屏时合成一段,息屏时合成一段。不过也尝试在转换模式时,MediaMuxer继续写入数据,虽然视频可以播放但是会导致写入失败,视频画面卡顿在转换的那一帧画面。因为在转换模式时,编码的数据出问题了,大小比之前的要小很多,此问题待研究。
三、解决方案:
采用上述的第三种方法:
Camera.setPreviewTexture(SurfaceTexture surfaceTexture)+Camera.setPreviewDisplay(SurfaceHolder holder);
息屏、切换前后置摄像头时先释放相机releaseCamera(),代码如下:
override fun releaseCamera() {
try {
stopBackgroundThread()
mCamera?.stopPreview()
mCamera?.setPreviewCallbackWithBuffer(null)
mCamera?.release()
mCamera = null
} catch (runError: RuntimeException) {
KLog.e(TAG, "releaseCamera happened error: " + runError.message)
} catch (e: Exception) {
KLog.e(TAG, "releaseCamera error: $e")
}
}
然后再重新打开相机openCamera,代码如下:
override fun openCamera(
cameraId: Int,
imageFormat: Int,
holder: SurfaceHolder?
) {
mCameraId = cameraId
this.previewFormat = imageFormat
surfaceHolder = holder
mSurfaceTexture = SurfaceTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES)
openCamera(surfaceHolder, mSurfaceTexture!!, cameraId)
}
private fun openCamera(
surfaceHolder: SurfaceHolder?,
surfaceTexture: SurfaceTexture,
cameraId: Int
) {
if (cameraId < 0 /*|| cameraId > Camera.getNumberOfCameras() - 1*/) {
Log.w(
TAG,
"openCamera failed, cameraId=" + cameraId + ", Camera.getNumberOfCameras()=" + Camera.getNumberOfCameras()
)
return
}
startBackgroundThread()
try {
// Log.i(TAG,"surfaceCreated open camera cameraId=$cameraId start")
mCamera = Camera.open(cameraId)
mCamera?.setDisplayOrientation(90)
if (surfaceHolder == null) {
mCamera?.setPreviewTexture(surfaceTexture)
} else {
mCamera?.setPreviewDisplay(surfaceHolder)
}
// set preview format @{
this.previewFormat = setCameraPreviewFormat(mCamera!!, this.previewFormat)
// @}
// 设置fps@{
val minFps: Int = 30000
val maxFps: Int = 30000
setCameraPreviewFpsRange(mCamera!!, minFps, maxFps)
// @}
// 设置预览尺寸 @{
val hasSetPreviewSize = setCameraPreviewSize(mCamera!!)
if (hasSetPreviewSize.size > 1) {
/* previewWidth = hasSetPreviewSize[0]
previewHeight = hasSetPreviewSize[1]
GBApp.getInstance().previewWidth = hasSetPreviewSize[0]
GBApp.getInstance().previewHeight = hasSetPreviewSize[1]*/
previewWidth = 640
previewHeight = 480
GBApp.instance!!.previewWidth = 640
GBApp.instance!!.previewHeight = 480
}
// @}
// 设置照片尺寸 @{
setCameraPictureSize(mCamera!!)
// @}
// 设置预览回调函数@{
mCamera?.setPreviewCallbackWithBuffer(mCameraCallbacks)
Log.i(
TAG,
"ImageFormat: $previewFormat bits per pixel=" + ImageFormat.getBitsPerPixel(
previewFormat
)
)
// 初始化数组
for (index in 0 until previewDataSize) {
val previewData = if (previewFormat != ImageFormat.YV12) {
ByteArray(
previewWidth * previewHeight * ImageFormat.getBitsPerPixel(
previewFormat
) / 8
)
} else {
val size = ImageUtils.getYV12ImagePixelSize(previewWidth, previewHeight)
ByteArray(size)
}
previewDataArray.add(previewData)
}
//addAllPreviewCallbackData()
mCamera?.addCallbackBuffer(ByteArray(previewWidth * previewHeight * 3 / 2))
// @}
//autoRatioTextureView()
mCamera?.startPreview()
} catch (localIOException: IOException) {
Log.e(
TAG,
"surfaceCreated open camera localIOException cameraId=" + cameraId + ", error=" + localIOException.message,
localIOException
)
} catch (run: RuntimeException) {
Log.e(
TAG,
"open camera RuntimeException error=" + run.message
)
} catch (e: Exception) {
Log.e(
TAG,
"surfaceCreated open camera cameraId=" + cameraId + ", error=" + e.message,
e
)
}
}
此情况依旧会导致在切换相机时,出现录制的视频卡在某一帧,解决方案如下:
依旧使用SurfaceView预览相机
1、相机停止写入数据pauseRecord()
// 根据 status 状态是否写入数据
public void pauseRecord() {
if (status == Status.RECORDING) {
pauseMoment = System.nanoTime() / 1000;
status = Status.PAUSED;
if (listener != null) listener.onStatusChange(status);
}
}
2、释放相机
fun releaseCamera() {
try {
stopBackgroundThread()
mCamera?.stopPreview()
mCamera?.setPreviewCallbackWithBuffer(null)
mCamera?.release()
mCamera = null
} catch (runError: RuntimeException) {
KLog.e(TAG, "releaseCamera happened error: " + runError.message)
} catch (e: Exception) {
KLog.e(TAG, "releaseCamera error: $e")
}
}
3、继续录制视频
fun doResumeRecord(eventData: ResumeRecordEvent) {
// 打开相机
GBApp.instance?.service?.doOpenCamera(
OpenCameraEvent(
eventData.holder,
VideoTaskUtil.instance.mCameraId,
ImageFormat.NV21,
eventData.eventType
)
)
// 请求关键帧
camera2Base?.videoEncoder?.requestKeyframe()
// 继续写入音视频数据
camera2Base?.resumeRecord()
}
public void resumeRecord() {
if (status == Status.PAUSED) {
pauseTime += System.nanoTime() / 1000 - pauseMoment;
status = Status.RESUMED;
if (listener != null) listener.onStatusChange(status);
}
}
如果合成的视频在后续还会卡在某一帧,可以把之前的视频数据队列清空,这样避免因为切换相机之前的垃圾数据导致问题,然后执行上面的步骤