MM-Camera架构-ProcessCaptureRequest 流程分析

埋头干饭ing2023-10-09 14:03

文章目录

  • processCaptureRequest\_3\_4
    • [1.1 mDevice](#1.1 mDevice)
    • [1.2 mDevice->ops->process\_capture\_request](#1.2 mDevice->ops->process_capture_request)
    • [1.3 hardware to vendor](#1.3 hardware to vendor)
  • mct\_shimlayer\_process\_event
    • [2.1 mct\_shimlayer\_handle\_parm](#2.1 mct_shimlayer_handle_parm)
    • [2.2 mct\_shimlayer\_reg\_buffer](#2.2 mct_shimlayer_reg_buffer)

processCaptureRequest_3_4

sdm660的摄像头走camera hal 3.4接口，每个预览帧，都是由camera service发起，通过camera HAL的CameraDeviceSession::processCaptureRequest_3_4接口，通知hal，请求一个预览帧。

hardware/interfaces/camera/device/3.4/default/CameraDeviceSession.cpp

c 复制代码 
Return<void> CameraDeviceSession::processCaptureRequest_3_4(
        const hidl_vec<V3_4::CaptureRequest>& requests,
        const hidl_vec<V3_2::BufferCache>& cachesToRemove,
        ICameraDeviceSession::processCaptureRequest_3_4_cb _hidl_cb)  {
    for (size_t i = 0; i < requests.size(); i++, numRequestProcessed++) {
        s = processOneCaptureRequest_3_4(requests[i]);
        }
    ...
}

Status CameraDeviceSession::processOneCaptureRequest_3_4(const V3_4::CaptureRequest& request)  {
    ...
    //@ 1.1 - 1.2
    status_t ret = mDevice->ops->process_capture_request(mDevice, &halRequest);
    ...
}

1.1 mDevice

mDevice由camera_device_t或者camera3_device_t定义。camera1和camera3代表的是CAMERA DEVICE API VERSION,camera3和canera1完全不一样。这里mDevice是camera3_device_t定义。

camera3_device_t :是 camera HAL 核心结构体之一，是对 venor camera HAL 实现的抽象，APP 通过 Framework对底层Camera设备（物理设备和逻辑虚拟设备）的操作都是通过对 camera3_device_t 实例对象来进行的。
camera3_device_t:主要用来表示Camera设备，其中定义了Camera3_device_ops操作集合，用来实现正常获取图像数据以及控制Camera的功能。

cpp 复制代码 
typedef struct camera3_device {
    hw_device_t common;
    camera3_device_ops_t *ops;
    void *priv;
} camera3_device_t;

HAL3的核心接口都是在camera3_device_ops中被定义。
camera3_device_ops结构体定义了一系列的函数指针，用来指向平台厂商实际的实现方法。

cpp 复制代码 
typedef struct camera3_device_ops {
    /*
    initialize
    何时被调用：在camera_modul_t中的open方法之后，其他camera3_device_ops方法之前被调用。
    主要作用：将上层实现的回调方法注册到HAL中，并根据需要在该方法中加入自定义的一些初始化操作。
    返回时间：在5ms内返回，最长不能超过10ms。
    */
    int (*initialize)(const struct camera3_device *,
            const camera3_callback_ops_t *callback_ops);

    /*
    configure_streams
    何时被调用：在Initialize方法完成之后，在调用process_capture_request方法之前被调用
    主要作用：重设当前正在运行的Pipeline以及设执行的输入输出流，其中它回见stream_list中的新的数据流替换之前配置的数据流。
    返回时间：500ms内返回，最长不能超过1000ms
    */
    int (*configure_streams)(const struct camera3_device *,
            camera3_stream_configuration_t *stream_list);

    int (*register_stream_buffers)(const struct camera3_device *,
            const camera3_stream_buffer_set_t *buffer_set);

    const camera_metadata_t* (*construct_default_request_settings)(
            const struct camera3_device *,
            int type);

    /*
    process_capture_request
    主要作用：下发单次新的capture request到HAL中，上层必须保证该方法的调用都是在一个线程中完成，而且该方法是异步的，
    其结果是通过HAL调用另一个接口process_capture_result()来返回结果给上层，在使用过程中，通过in-flight机制，保证短时间内下发足够多的requst，从而满足帧率要求。
    */
    int (*process_capture_request)(const struct camera3_device *,
            camera3_capture_request_t *request);

    void (*get_metadata_vendor_tag_ops)(const struct camera3_device*,
            vendor_tag_query_ops_t* ops);

    void (*dump)(const struct camera3_device *, int fd);

    int (*flush)(const struct camera3_device *);

    void (*signal_stream_flush)(const struct camera3_device*,
            uint32_t num_streams,
            const camera3_stream_t* const* streams);

    int (*is_reconfiguration_required)(const struct camera3_device*,
            const camera_metadata_t* old_session_params,
            const camera_metadata_t* new_session_params);

    /* reserved for future use */
    void *reserved[6];
} camera3_device_ops_t;

1.2 mDevice->ops->process_capture_request

继续mDevice->ops->process_capture_request

在hardware/qcom/camera/QCamera2/HAL3/QCamera3HWI.h中camera3_device_ops_t定义了mCameraOps:

static camera3_device_ops_t mCameraOps;

cpp 复制代码 
camera3_device_ops_t QCamera3HardwareInterface::mCameraOps = {
    .initialize                         = QCamera3HardwareInterface::initialize,
    .configure_streams                  = QCamera3HardwareInterface::configure_streams,
    .register_stream_buffers            = NULL,
    .construct_default_request_settings = QCamera3HardwareInterface::construct_default_request_settings,
    .process_capture_request            = QCamera3HardwareInterface::process_capture_request,
    .get_metadata_vendor_tag_ops        = NULL,
    .dump                               = QCamera3HardwareInterface::dump,
    .flush                              = QCamera3HardwareInterface::flush,
    .reserved                           = {0},
};

hardware/qcom/camera/QCamera2/HAL3/QCamera3HWI.cpp

cpp 复制代码 
int QCamera3HardwareInterface::process_capture_request(
                    const struct camera3_device *device,
                    camera3_capture_request_t *request)
{
    LOGE("sundpmy_ process_capture_request E");
    ...
    QCamera3HardwareInterface *hw =
        reinterpret_cast<QCamera3HardwareInterface *>(device->priv);

    int rc = hw->orchestrateRequest(request);
    LOGE("sundpmy_ process_capture_request X");
    return rc;
}

int32_t QCamera3HardwareInterface::orchestrateRequest(
        camera3_capture_request_t *request)
{
    int32_t ret = NO_ERROR;
    ...
    ret = processCaptureRequest(request, internallyRequestedStreams);
    ...
}

1.3 hardware to vendor

QCamera2Factory构造的时候调用get_num_of_cameras_to_expose，一直到用到mm_camera_load_shim_lib。

cpp 复制代码 
 QCamera2Factory::QCamera2Factory()
 {
     mHalDescriptors = NULL;
     mCallbacks = NULL;
     mNumOfCameras = get_num_of_cameras();
     mNumOfCameras_expose = get_num_of_cameras_to_expose();
        get_num_of_cameras_to_expose()---> 
                      get_num_of_cameras()--->
                           mm_camera_load_shim_lib() 
}

mm_camera_load_shim_lib里面dlopen:libmmcamera2_mct_shimlayer.so。dlsym获取入口函数：mct_shimlayer_process_module_init

c 复制代码 
#define SHIMLAYER_LIB "/system/vendor/lib/libmmcamera2_mct_shimlayer.so"

int32_t mm_camera_load_shim_lib()
{
    const char* error = NULL;
    void *qdaemon_lib = NULL;

    LOGD("E");
    qdaemon_lib = dlopen(SHIMLAYER_LIB, RTLD_NOW);
    if (!qdaemon_lib) {
        error = dlerror();
        LOGE("dlopen failed with error %s", error ? error : "");
        return -1;
    }

    *(void **)&mm_camera_shim_module_init =
            dlsym(qdaemon_lib, "mct_shimlayer_process_module_init");
    if (!mm_camera_shim_module_init) {
        error = dlerror();
        LOGE("dlsym failed with error code %s", error ? error: "");
        dlclose(qdaemon_lib);
        return -1;
    }

    return mm_camera_shim_module_init(&g_cam_ctrl.cam_shim_ops);
}

mct_shimlayer_process_module_init给shim_ops_tbl赋值： @vendor/qcom/proprietary/mm-camera/mm-camera2/media-controller/mct_shim_layer/mct_shim_layer.c

c 复制代码 
int mct_shimlayer_process_module_init(mm_camera_shim_ops_t
  *shim_ops_tbl)
{
    ...
    shim_ops_tbl->mm_camera_shim_open_session = mct_shimlayer_start_session;
    shim_ops_tbl->mm_camera_shim_close_session = mct_shimlayer_stop_session;
    shim_ops_tbl->mm_camera_shim_send_cmd = mct_shimlayer_process_event;
    ...
}

mm_camera_shim_send_cmd的调用：

c 复制代码 
int32_t mm_camera_module_send_cmd(cam_shim_packet_t *event)
{
    int32_t rc = -1;
    if(g_cam_ctrl.cam_shim_ops.mm_camera_shim_send_cmd) {
        rc = g_cam_ctrl.cam_shim_ops.mm_camera_shim_send_cmd(event);
    }
    return rc;
}

mm_camera_module_send_cmd是个包装函数：

mm_camera_module_send_cmd被调用的地方比较多，打印log看一下：

c 复制代码 
D mm-camera: <CPP   >< HIGH> 1485: cpp_hardware_process_frame: [CPP_BUF:] stream_status:0xe7f1608c, iden:0x40002, cur_frame_id:17
E QCamera : <MCI><ERROR> mm_stream_unmap_buf: 2327: sundp_mc func:mm_stream_unmap_buf line:2327 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_stream_map_buf: 2168: sundp_mc func:mm_stream_map_buf line:2168 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_camera_util_s_ctrl: 2111: sundp_mc func:mm_camera_util_s_ctrl line:2111 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =0
D mm-camera: <CPP   >< HIGH> 1485: cpp_hardware_process_frame: [CPP_BUF:] stream_status:0xe7f1608c, iden:0x40002, cur_frame_id:18
E QCamera : <MCI><ERROR> mm_stream_unmap_buf: 2327: sundp_mc func:mm_stream_unmap_buf line:2327 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_stream_map_buf: 2168: sundp_mc func:mm_stream_map_buf line:2168 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_camera_util_s_ctrl: 2111: sundp_mc func:mm_camera_util_s_ctrl line:2111 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =0
D mm-camera: <CPP   >< HIGH> 1485: cpp_hardware_process_frame: [CPP_BUF:] stream_status:0xe7f1608c, iden:0x40002, cur_frame_id:19
E QCamera : <MCI><ERROR> mm_stream_unmap_buf: 2327: sundp_mc func:mm_stream_unmap_buf line:2327 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_stream_map_buf: 2168: sundp_mc func:mm_stream_map_buf line:2168 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_camera_util_s_ctrl: 2111: sundp_mc func:mm_camera_util_s_ctrl line:2111 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =0
D mm-camera: <CPP   >< HIGH> 1485: cpp_hardware_process_frame: [CPP_BUF:] stream_status:0xe7f1608c, iden:0x40002, cur_frame_id:20
E QCamera : <MCI><ERROR> mm_stream_unmap_buf: 2327: sundp_mc func:mm_stream_unmap_buf line:2327 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_stream_map_buf: 2168: sundp_mc func:mm_stream_map_buf line:2168 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =2
E QCamera : <MCI><ERROR> mm_camera_util_s_ctrl: 2111: sundp_mc func:mm_camera_util_s_ctrl line:2111 here
E mm-camera: <SHIM  ><ERROR> 481: mct_shimlayer_process_event: sundp_mct Received event from HAL type =0
D mm-camera: <CPP   >< HIGH> 1485: cpp_hardware_process_frame: [CPP_BUF:] stream_status:0xe7f1608c, iden:0x40002, cur_frame_id:21

以上log，可以看出每一帧，每次captureRequest，mm_camera_module_send_cmd会被调用三次。func:mm_stream_map_buf，func:mm_camera_util_s_ctrl，以及每一帧结束后的func:mm_stream_unmap_buf。

mct_shimlayer_process_event

从cameraservice层到qcom的hal层，到了shimlayer。 vendor/qcom/proprietary/mm-camera/mm-camera2/media-controller/mct_shim_layer/mct_shim_layer.c

从日志看，每次captureRequest，mct_shimlayer_process_event会被调用两次，cmd_type分别是CAM_SHIM_REG_BUF和CAM_SHIM_SET_PARM。

c 复制代码 
packet->cmd_type----->
typedef enum {
    CAM_SHIM_SET_PARM,   /*v4l2 set parameter*/
    CAM_SHIM_GET_PARM,   /*v4l2 get parameter*/
    CAM_SHIM_REG_BUF,    /*Reg/unreg buffers with back-end*/
    CAM_SHIM_BUNDLE_CMD, /*Bundled command for streams*/
} cam_shim_cmd_type;

int mct_shimlayer_process_event(cam_shim_packet_t *packet)
{
  switch (packet->cmd_type) {
    case CAM_SHIM_SET_PARM:
    case CAM_SHIM_GET_PARM: {
      session_id = packet->session_id;
      parm_event = &packet->cmd_data;
      @2.1
      rc = mct_shimlayer_handle_parm(packet->cmd_type, session_id, parm_event);  
    }
      break;
      
    case CAM_SHIM_REG_BUF: {
      session_id = packet->session_id;
      reg_buf = &packet->reg_buf;
      @2.2
      rc = mct_shimlayer_reg_buffer(session_id, reg_buf);  
    }
      break;
    ......
}

接下来看mct_shimlayer_process_event中的2.1 和 2.2。

2.1 mct_shimlayer_handle_parm

2.2 mct_shimlayer_reg_buffer

相关推荐
异次元的归来18 天前
UE5相机系统初探（一）
ue5·游戏引擎·camera
通俗_易懂23 天前
44-RK3588s调试 camera-engine-rkaiq(rkaiq_3A_server)
人工智能·计算机视觉·rk3588·驱动·camera·imx415
点PY24 天前
camera和lidar外参标定
camera·标定·lidar
如果可以0031 个月前
Android Camera系列（四）：TextureView+OpenGL ES+Camera
android·camera·opengl es·textureview
repinkply1 个月前
Android Camera性能优化概述
android·linux·性能优化·camera
Cmatrix2042 个月前
高通Camx-内存池架构/ImageBuffer
android·架构·camera
404detective3 个月前
Matlab R2022b使用Camera Calibrator工具箱张正友标定法进行相机标定附带标定前后对比代码
matlab·相机标定·camera·深度相机·张正友标定法·calibrator·棋盘法
JerryHe5 个月前
Android Camera API发展历程
android·数码相机·camera·camera api
plmm烟酒僧5 个月前
ffmpeg编码图象时报错Invalid buffer size, packet size * ＜ expected frame_size *
linux·ffmpeg·camera·yuv·jpg
daisy.skye5 个月前
isp效果库相关参数——镜像翻转
计算机视觉·camera·isp
热门推荐
01ICLR24大模型提示(1/11) | BadChain：大型语言模型的后门思维链提示02PyTorch机器学习实现液态神经网络03【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总04【音视频 | opus】opus编解码库(opus-1.4)详细介绍以及使用——附带解码示例代码05Coze扣子平台完整体验和实践（附国内和国际版对比）06(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明07IT专业入门，高考假期预习指南08玄机平台应急响应—webshell查杀09RAG 实践- Ollama+RagFlow 部署本地知识库10〔AI 绘画〕Stable Diffusion 之 解决绘制多人或面部很小的人物时面部崩坏问题 篇