Camera相机人脸识别系列专题分析之十九:MTK ISP6S平台FDNode传递三方FFD到APP流程解析

【关注我,后续持续新增专题博文,谢谢!!!】

上一篇我们讲了:

这一篇我们开始讲: Camera相机人脸识别系列专题分析之十九:MTK平台FDNode传递三方FFD到APP流程解析

目录

一、背景

二、:OcamMeta传递FFD到APP

2.1:OcamMeta

[2.2 :OcamMeta::process更新FFD](#2.2 :OcamMeta::process更新FFD)

[2.3 :setFfdData2App](#2.3 :setFfdData2App)


一、背景

我们知道app和hal属于两个不同进程,而进程间通信在camera领域一个重要的知识点就是camerameta,最典型的就是存放3A信息,我们也可以客制化存在FFD相关信息,来实现进程间通信。

二、:OcamMeta传递FFD到APP

2.1:OcamMeta

ocam_meta.cpp是对camera meta统一管理,之前的FFD数据就是在这里通过写入meta中,最后APP会读取meta来完成获取HAL的FFD数据。

而核心流程是CamThread::CamThreadHandle::threadLoop线程中,通过enqueNode最后调用到OcamMeta::process来完成meta的更新。

2.2 :OcamMeta::process更新FFD

OcamMeta::process函数中,会更新很多camera相关meta,FFD相关meta最终是通过setFfdData2App来完成

cpp 复制代码
MERROR OcamMeta::process(RequestPtr pRequest, RequestCallbackPtr pCallback)
{
    if( pRequest != NULL )
    {
        IMetadata *pIMetadataDynamic1  = acquire(pRequest->mIMetadataDynamic1);
        IMetadata *pOMetadataApp     = acquire(pRequest->mOMetadataApp);
        IMetadata *pOMetadataHal     = acquire(pRequest->mOMetadataHal);
        mDataMetaToAsync.mFaceNum = 0;
        thirdFaceDataAppJoint facedata;
        memset(&facedata, 0, sizeof(thirdFaceDataAppJoint));
        if (pIMetadataHal1 != NULL) {
            MINT64 p1timestamp = 0;
            IMetadata::IEntry entryP1Timestamp = pRequest->mIMetadataHal1->acquire()->entryFor(MTK_P1NODE_FRAME_START_TIMESTAMP);
            if (!entryP1Timestamp.isEmpty()) {
                p1timestamp = entryP1Timestamp.itemAt(0, Type2Type<MINT64>());
            } else {
                MY_LOGE("Cant get p1 timestamp meta.");
            }

            auto fdReader = IFDContainer::createInstance(LOG_TAG, IFDContainer::eFDContainer_Opt_Read);
            if (fdReader != NULL) {
                MBOOL fdResult = MFALSE;
                FD_DATATYPE fdData;
                fdResult = fdReader->cloneLatestFD(fdData, mOpenID);
                if (fdResult) {
                    mFDdata = std::make_shared<FD_DATATYPE>();
                    fdData.clone(*mFDdata);
                    mDataMetaToAsync.mFaceNum = mFDdata->facedata.number_of_faces;
                    MY_LOGD_IF(mLogDebug, "FDNode Result FaceNum(%d),openID(%d)", mFDdata->facedata.number_of_faces, mOpenID);
                } else {
                    MY_LOGI("fdData is not found");
                }
            } else {
                MY_LOGD_IF(mLogDebug, "Can not get FDContainer");
            }

            if (mDataMetaToAsync.mIsUseCusFd) {
                if (thirdIFDContainer::hasInstance(thirdIFDContainer::third_eFDContainer_Opt_Read)) {
                    auto thirdFdReader = thirdIFDContainer::createInstance(LOG_TAG, thirdIFDContainer::third_eFDContainer_Opt_Read);
                    auto fdData = thirdFdReader->queryLock(p1timestamp - FD_TOLERENCE, p1timestamp);
                    if (fdData.size() > 0) {
                        auto fdChunk = fdData.back();
                        facedata = fdChunk->thirdFaceData;
                        if (mDataMetaToAsync.mBeautyLevel > 0 && fdChunk->sensorId == mOpenID) {
                            setFfdData2App(facedata, pOMetadataApp);
                        }
                    }
                    thirdFdReader->queryUnlock(fdData);
                }
            }

2.3 :setFfdData2App

流程如下:

  1. 创建:IMetadata::IEntry faceInfoEntry(third_CONTROL_CAPTURE_FB_FACE_INFO);
  2. 写入各种FFD数据结构字段:faceInfoEntry.push_back
  3. 更新meta:pOMetadataApp->update(third_CONTROL_CAPTURE_FB_FACE_INFO, faceInfoEntry);
  4. APP会去读这个tag拿到meta,并解析各个FFD字段,拿到FFD各种变量。
cpp 复制代码
void OcamMeta::setFfdData2App(thirdFaceDataAppJoint facedata, IMetadata* pOMetadataApp)
{
    MSize fdImgSize(facedata.fdProcessInfo.ImgSize.w, facedata.fdProcessInfo.ImgSize.h);
    int face_count = 0;
    face_count = facedata.faceInfoOri.face_num;
    for (MINT32 i = 0; i < face_count; i++) {
        int32_t left = (int32_t)facedata.faceInfoOri.face_roi[i].faceRect.left;
        int32_t top = (int32_t)facedata.faceInfoOri.face_roi[i].faceRect.top;
        int32_t right = (int32_t)facedata.faceInfoOri.face_roi[i].faceRect.width + left;
        int32_t bottom = (int32_t)facedata.faceInfoOri.face_roi[i].faceRect.height + top;
        convertCoord(left, top, fdImgSize, facedata.fdProcessInfo);
        convertCoord(right, bottom, fdImgSize, facedata.fdProcessInfo);
        facedata.faceInfoOri.face_roi[i].faceRect.left = left;
        facedata.faceInfoOri.face_roi[i].faceRect.top = top;
        facedata.faceInfoOri.face_roi[i].faceRect.width = right - left;
        facedata.faceInfoOri.face_roi[i].faceRect.height = bottom - top;
    }
    for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
        for (MINT32 j = 0; j < MAX_FFD_NUM; j++) {
            convertCoord(facedata.faceInfoOri.ffd_data[i].x[j],
                facedata.faceInfoOri.ffd_data[i].y[j],
                fdImgSize, facedata.fdProcessInfo);
        }
    }
    for (MINT32 i = 0; i < MAX_FACE_ROIS; i++) {
        for (MINT32 j = 0; j < MAX_FFD_NUM_512; j++) {
            convertCoord(facedata.ffd296Data.ffdData[i].x[j],
                facedata.ffd296Data.ffdData[i].y[j],
                fdImgSize, facedata.fdProcessInfo);
        }
    }
    if (pOMetadataApp) {
        IMetadata::IEntry faceInfoEntry(third_CONTROL_CAPTURE_FB_FACE_INFO);
        int face_num = 0;
        faceInfoEntry.push_back(facedata.faceInfoOri.versionId,Type2Type<MINT32>());
        face_num = facedata.faceInfoOri.face_num ? facedata.faceInfoOri.face_num : mDataMetaToAsync.mFaceNum;
        faceInfoEntry.push_back(face_num,Type2Type<MINT32>());
        faceInfoEntry.push_back(facedata.faceInfoOri.master_index,Type2Type<MINT32>());
        faceInfoEntry.push_back(facedata.faceInfoOri.points_count,Type2Type<MINT32>());
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.age[i],Type2Type<MINT32>());
        }
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.gender[i],Type2Type<MINT32>());
        }
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            for (MINT32 j = 0; j < MAX_FFD_NUM; j++) {
                faceInfoEntry.push_back(facedata.faceInfoOri.ffd_data[i].x[j],Type2Type<MINT32>());
            }
            for (MINT32 j = 0; j < MAX_FFD_NUM; j++) {
                faceInfoEntry.push_back(facedata.faceInfoOri.ffd_data[i].y[j],Type2Type<MINT32>());
            }
            for (MINT32 j = 0; j < MAX_FFD_NUM; j++) {
                faceInfoEntry.push_back(facedata.faceInfoOri.ffd_data[i].occlusion[j],Type2Type<MINT32>());
            }
        }
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.yaw[i],Type2Type<MINT32>());
        }
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.pitch[i],Type2Type<MINT32>());
        }
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.roll[i],Type2Type<MINT32>());
        }
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.faceLuma[i],Type2Type<MINT32>());
        }
        // for decide the back lighting
        faceInfoEntry.push_back(facedata.faceInfoOri.faceLumaRatio,Type2Type<MINT32>());
        //colorTemperature
        faceInfoEntry.push_back(facedata.faceInfoOri.colorTemperature,Type2Type<MINT32>());
        //lip_HSV
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM * HSV_CHANNEL_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.lip_HSV[i],Type2Type<MINT32>());
        }
        //iso
        facedata.faceInfoOri.iso = mDataMetaToAsync.mIsoValue;
        faceInfoEntry.push_back(facedata.faceInfoOri.iso,Type2Type<MINT32>());
        //feature
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.feature[i],Type2Type<MINT32>());
        }
        //faceid
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.faceid[i],Type2Type<MINT32>());
        }
        //faceRect
        for (MINT32 i = 0; i < MAX_ATTRI_FACE_NUM; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.face_roi[i].faceRect.left,Type2Type<MINT32>());
            faceInfoEntry.push_back(facedata.faceInfoOri.face_roi[i].faceRect.top,Type2Type<MINT32>());
            faceInfoEntry.push_back(facedata.faceInfoOri.face_roi[i].faceRect.width,Type2Type<MINT32>());
            faceInfoEntry.push_back(facedata.faceInfoOri.face_roi[i].faceRect.height,Type2Type<MINT32>());
        }

        faceInfoEntry.push_back(facedata.faceInfoOri.fdDimensionW,Type2Type<MINT32>());
        faceInfoEntry.push_back(facedata.faceInfoOri.fdDimensionH,Type2Type<MINT32>());
        // reserve
        MINT32 RESERVE_ITEM1 = 4;
        // reserve
        for (MINT32 i = 0; i < RESERVE_ITEM1; i++) {
            faceInfoEntry.push_back(facedata.faceInfoOri.reserve[i],Type2Type<MINT32>());
        }
        pOMetadataApp->update(third_CONTROL_CAPTURE_FB_FACE_INFO, faceInfoEntry);
        // send ffd data to app
        IMetadata::IEntry ffdInfoEntry(third_CONTROL_CAPTURE_FFDPOINTS_INFO);
        for (MINT32 i = 0; i < MAX_FACE_ROIS; i++)  {
            for (MINT32 j = 0; j < MAX_FFD_NUM_512; j++) {
                ffdInfoEntry.push_back(facedata.ffd296Data.ffdData[i].x[j],Type2Type<MINT32>());
            }
            for (MINT32 j = 0; j < MAX_FFD_NUM_512; j++) {
                ffdInfoEntry.push_back(facedata.ffd296Data.ffdData[i].y[j],Type2Type<MINT32>());
            }
            for (MINT32 j = 0; j < MAX_FFD_NUM_512; j++) {
                ffdInfoEntry.push_back(facedata.ffd296Data.ffdData[i].occlusion[j],Type2Type<MINT32>());
            }
        }
        ffdInfoEntry.push_back(facedata.ffd296Data.pointsCount,Type2Type<MINT32>());
        pOMetadataApp->update(third_CONTROL_CAPTURE_FFDPOINTS_INFO, ffdInfoEntry);
    }
}

【关注我,后续持续新增专题博文,谢谢!!!】

下一篇讲解:

相关推荐
张较瘦_41 分钟前
[论文阅读] 人工智能 + 软件工程 | 增强RESTful API测试:针对MongoDB的搜索式模糊测试新方法
论文阅读·人工智能·软件工程
Wendy14412 小时前
【边缘填充】——图像预处理(OpenCV)
人工智能·opencv·计算机视觉
钱彬 (Qian Bin)2 小时前
《使用Qt Quick从零构建AI螺丝瑕疵检测系统》——8. AI赋能(下):在Qt中部署YOLOv8模型
人工智能·qt·yolo·qml·qt quick·工业质检·螺丝瑕疵检测
giaoho2 小时前
Android 系统架构
android·系统架构
星月昭铭3 小时前
Spring AI调用Embedding模型返回HTTP 400:Invalid HTTP request received分析处理
人工智能·spring boot·python·spring·ai·embedding
大千AI助手4 小时前
直接偏好优化(DPO):原理、演进与大模型对齐新范式
人工智能·神经网络·算法·机器学习·dpo·大模型对齐·直接偏好优化
ReinaXue4 小时前
大模型【进阶】(四)QWen模型架构的解读
人工智能·神经网络·语言模型·transformer·语音识别·迁移学习·audiolm
静心问道4 小时前
Deja Vu: 利用上下文稀疏性提升大语言模型推理效率
人工智能·模型加速·ai技术应用
小妖同学学AI4 小时前
deepseek+飞书多维表格 打造小红书矩阵
人工智能·矩阵·飞书
阿明观察4 小时前
再谈亚马逊云科技(AWS)上海AI研究院7月22日关闭事件
人工智能