[Android] Graphic Buffer 的申请

前言:

MediaCodec 支持 texture mode,即MediaCodec解码video完毕后把 yuv 数据填入 GPU 共享出来的 graphic buffer 里面,app 会把 video 的 yuv数据 和 ui 的数据通过通过软件渲染组件(opengl等)发送给GPU 进行一并渲染。这样做的效率较低,但是稳定性较好。且性能取决于GPU的性能。

另外一种是 surface mode,就是常用的模式,这种模式下 app 不再获取 yuv 数据,而是只负责输送 es 数据并决定 render 还是 discard 解码后的数据,且是根据 bufferid 来判断的,根本没有机会接触解码后的 yuv 数据,所有的一切都在平台完成。这种办法效率搞且功耗也很低,因为不需要GPU的参与。大部分情况下都是这种模式。

本文简单理清一下 Graphic Buffer 的框架,以对 texture mode 中的 Graphic Buffer 申请流程有一个大致的概念。




Graphic Buffer 的申请

始于 ANativeWindow 类

frameworks/native/libs/nativewindow/include/system/window.h

cpp 复制代码
struct ANativeWindow
{
#ifdef __cplusplus
    ANativeWindow()
        : flags(0), minSwapInterval(0), maxSwapInterval(0), xdpi(0), ydpi(0)
    {
        common.magic = ANDROID_NATIVE_WINDOW_MAGIC;
        common.version = sizeof(ANativeWindow);
        memset(common.reserved, 0, sizeof(common.reserved));
    }

    /* Implement the methods that sp<ANativeWindow> expects so that it
       can be used to automatically refcount ANativeWindow's. */
    void incStrong(const void* /*id*/) const {
        common.incRef(const_cast<android_native_base_t*>(&common));
    }
    void decStrong(const void* /*id*/) const {
        common.decRef(const_cast<android_native_base_t*>(&common));
    }
#endif

    struct android_native_base_t common;

    /* flags describing some attributes of this surface or its updater */
    const uint32_t flags;

    /* min swap interval supported by this updated */
    const int   minSwapInterval;

    /* max swap interval supported by this updated */
    const int   maxSwapInterval;

    /* horizontal and vertical resolution in DPI */
    const float xdpi;
    const float ydpi;

    /* Some storage reserved for the OEM's driver. */
    intptr_t    oem[4];

    /*
     * Set the swap interval for this surface.
     *
     * Returns 0 on success or -errno on error.
     */
    int     (*setSwapInterval)(struct ANativeWindow* window,
                int interval);

    /*
     * Hook called by EGL to acquire a buffer. After this call, the buffer
     * is not locked, so its content cannot be modified. This call may block if
     * no buffers are available.
     *
     * The window holds a reference to the buffer between dequeueBuffer and
     * either queueBuffer or cancelBuffer, so clients only need their own
     * reference if they might use the buffer after queueing or canceling it.
     * Holding a reference to a buffer after queueing or canceling it is only
     * allowed if a specific buffer count has been set.
     *
     * Returns 0 on success or -errno on error.
     *
     * XXX: This function is deprecated.  It will continue to work for some
     * time for binary compatibility, but the new dequeueBuffer function that
     * outputs a fence file descriptor should be used in its place.
     */
    int     (*dequeueBuffer_DEPRECATED)(struct ANativeWindow* window,
                struct ANativeWindowBuffer** buffer);

    /*
     * hook called by EGL to lock a buffer. This MUST be called before modifying
     * the content of a buffer. The buffer must have been acquired with
     * dequeueBuffer first.
     *
     * Returns 0 on success or -errno on error.
     *
     * XXX: This function is deprecated.  It will continue to work for some
     * time for binary compatibility, but it is essentially a no-op, and calls
     * to it should be removed.
     */
    int     (*lockBuffer_DEPRECATED)(struct ANativeWindow* window,
                struct ANativeWindowBuffer* buffer);

    /*
     * Hook called by EGL when modifications to the render buffer are done.
     * This unlocks and post the buffer.
     *
     * The window holds a reference to the buffer between dequeueBuffer and
     * either queueBuffer or cancelBuffer, so clients only need their own
     * reference if they might use the buffer after queueing or canceling it.
     * Holding a reference to a buffer after queueing or canceling it is only
     * allowed if a specific buffer count has been set.
     *
     * Buffers MUST be queued in the same order than they were dequeued.
     *
     * Returns 0 on success or -errno on error.
     *
     * XXX: This function is deprecated.  It will continue to work for some
     * time for binary compatibility, but the new queueBuffer function that
     * takes a fence file descriptor should be used in its place (pass a value
     * of -1 for the fence file descriptor if there is no valid one to pass).
     */
    int     (*queueBuffer_DEPRECATED)(struct ANativeWindow* window,
                struct ANativeWindowBuffer* buffer);

    /*
     * hook used to retrieve information about the native window.
     *
     * Returns 0 on success or -errno on error.
     */
    int     (*query)(const struct ANativeWindow* window,
                int what, int* value);

    /*
     * hook used to perform various operations on the surface.
     * (*perform)() is a generic mechanism to add functionality to
     * ANativeWindow while keeping backward binary compatibility.
     *
     * DO NOT CALL THIS HOOK DIRECTLY.  Instead, use the helper functions
     * defined below.
     *
     * (*perform)() returns -ENOENT if the 'what' parameter is not supported
     * by the surface's implementation.
     *
     * See above for a list of valid operations, such as
     * NATIVE_WINDOW_SET_USAGE or NATIVE_WINDOW_CONNECT
     */
    int     (*perform)(struct ANativeWindow* window,
                int operation, ... );

    /*
     * Hook used to cancel a buffer that has been dequeued.
     * No synchronization is performed between dequeue() and cancel(), so
     * either external synchronization is needed, or these functions must be
     * called from the same thread.
     *
     * The window holds a reference to the buffer between dequeueBuffer and
     * either queueBuffer or cancelBuffer, so clients only need their own
     * reference if they might use the buffer after queueing or canceling it.
     * Holding a reference to a buffer after queueing or canceling it is only
     * allowed if a specific buffer count has been set.
     *
     * XXX: This function is deprecated.  It will continue to work for some
     * time for binary compatibility, but the new cancelBuffer function that
     * takes a fence file descriptor should be used in its place (pass a value
     * of -1 for the fence file descriptor if there is no valid one to pass).
     */
    int     (*cancelBuffer_DEPRECATED)(struct ANativeWindow* window,
                struct ANativeWindowBuffer* buffer);

    /*
     * Hook called by EGL to acquire a buffer. This call may block if no
     * buffers are available.
     *
     * The window holds a reference to the buffer between dequeueBuffer and
     * either queueBuffer or cancelBuffer, so clients only need their own
     * reference if they might use the buffer after queueing or canceling it.
     * Holding a reference to a buffer after queueing or canceling it is only
     * allowed if a specific buffer count has been set.
     *
     * The libsync fence file descriptor returned in the int pointed to by the
     * fenceFd argument will refer to the fence that must signal before the
     * dequeued buffer may be written to.  A value of -1 indicates that the
     * caller may access the buffer immediately without waiting on a fence.  If
     * a valid file descriptor is returned (i.e. any value except -1) then the
     * caller is responsible for closing the file descriptor.
     *
     * Returns 0 on success or -errno on error.
     */
    int     (*dequeueBuffer)(struct ANativeWindow* window,
                struct ANativeWindowBuffer** buffer, int* fenceFd);

    /*
     * Hook called by EGL when modifications to the render buffer are done.
     * This unlocks and post the buffer.
     *
     * The window holds a reference to the buffer between dequeueBuffer and
     * either queueBuffer or cancelBuffer, so clients only need their own
     * reference if they might use the buffer after queueing or canceling it.
     * Holding a reference to a buffer after queueing or canceling it is only
     * allowed if a specific buffer count has been set.
     *
     * The fenceFd argument specifies a libsync fence file descriptor for a
     * fence that must signal before the buffer can be accessed.  If the buffer
     * can be accessed immediately then a value of -1 should be used.  The
     * caller must not use the file descriptor after it is passed to
     * queueBuffer, and the ANativeWindow implementation is responsible for
     * closing it.
     *
     * Returns 0 on success or -errno on error.
     */
    int     (*queueBuffer)(struct ANativeWindow* window,
                struct ANativeWindowBuffer* buffer, int fenceFd);

    /*
     * Hook used to cancel a buffer that has been dequeued.
     * No synchronization is performed between dequeue() and cancel(), so
     * either external synchronization is needed, or these functions must be
     * called from the same thread.
     *
     * The window holds a reference to the buffer between dequeueBuffer and
     * either queueBuffer or cancelBuffer, so clients only need their own
     * reference if they might use the buffer after queueing or canceling it.
     * Holding a reference to a buffer after queueing or canceling it is only
     * allowed if a specific buffer count has been set.
     *
     * The fenceFd argument specifies a libsync fence file decsriptor for a
     * fence that must signal before the buffer can be accessed.  If the buffer
     * can be accessed immediately then a value of -1 should be used.
     *
     * Note that if the client has not waited on the fence that was returned
     * from dequeueBuffer, that same fence should be passed to cancelBuffer to
     * ensure that future uses of the buffer are preceded by a wait on that
     * fence.  The caller must not use the file descriptor after it is passed
     * to cancelBuffer, and the ANativeWindow implementation is responsible for
     * closing it.
     *
     * Returns 0 on success or -errno on error.
     */
    int     (*cancelBuffer)(struct ANativeWindow* window,
                struct ANativeWindowBuffer* buffer, int fenceFd);
};

上述代码中有很多函数指针,比如

cpp 复制代码
    int     (*dequeueBuffer)(struct ANativeWindow* window,
                struct ANativeWindowBuffer** buffer, int* fenceFd);

这些指针的具体实现在其子类中有实现和设置,这里是一种 C 的 hook 写法,而不是 c++ 的虚函数写法,可能出于兼容性考虑。

\frameworks\native\libs\nativewindow\ANativeWindow.cpp 里面并不是 ANativeWindow的实现,而是对其封装,比如:

cpp 复制代码
int ANativeWindow_dequeueBuffer(ANativeWindow* window, ANativeWindowBuffer** buffer, int* fenceFd) {
    return window->dequeueBuffer(window, buffer, fenceFd);
}

可以看到,这里调用了 ANativeWindow的dequeueBuffer函数,而这个函数在 ANativeWindow 结构里是一个函数指针,而这个函数指针又指向哪里呢?


Hook的实现者Surface

\frameworks\native\libs\gui\include\gui\Surface.h

\frameworks\native\libs\gui\Surface.cpp

cpp 复制代码
class Surface
    : public ANativeObjectBase<ANativeWindow, Surface, RefBase>
{

}

Surface 是 ANativeWindow的子类,但是考虑到 ANativeWindow通过 函数指针的方式实现多态,所以Surface里面一定有设置Hook的地方,那就是在构造函数里。

cpp 复制代码
Surface::Surface(const sp<IGraphicBufferProducer>& bufferProducer, bool controlledByApp)
      : mGraphicBufferProducer(bufferProducer),
        mCrop(Rect::EMPTY_RECT),
        mBufferAge(0),
        mGenerationNumber(0),
        mSharedBufferMode(false),
        mAutoRefresh(false),
        mSharedBufferSlot(BufferItem::INVALID_BUFFER_SLOT),
        mSharedBufferHasBeenQueued(false),
        mQueriedSupportedTimestamps(false),
        mFrameTimestampsSupportsPresent(false),
        mEnableFrameTimestamps(false),
        mFrameEventHistory(std::make_unique<ProducerFrameEventHistory>()) {
    // Initialize the ANativeWindow function pointers.
    ANativeWindow::setSwapInterval  = hook_setSwapInterval;
    ANativeWindow::dequeueBuffer    = hook_dequeueBuffer;
    ANativeWindow::cancelBuffer     = hook_cancelBuffer;
    ANativeWindow::queueBuffer      = hook_queueBuffer;
    ANativeWindow::query            = hook_query;
    ANativeWindow::perform          = hook_perform;

    ANativeWindow::dequeueBuffer_DEPRECATED = hook_dequeueBuffer_DEPRECATED;
    ANativeWindow::cancelBuffer_DEPRECATED  = hook_cancelBuffer_DEPRECATED;
    ANativeWindow::lockBuffer_DEPRECATED    = hook_lockBuffer_DEPRECATED;
    ANativeWindow::queueBuffer_DEPRECATED   = hook_queueBuffer_DEPRECATED;

至此,所有调用 ANativeWindow_xxx 的入口都会被导入 Surface 里的相应接口中,进入 hook_xxx。

cpp 复制代码
int Surface::hook_dequeueBuffer(ANativeWindow* window,
        ANativeWindowBuffer** buffer, int* fenceFd) {
    Surface* c = getSelf(window);
    return c->dequeueBuffer(buffer, fenceFd);
}

进而在进入 Surface 的 xxx 函数。从 xxx 函数开始,和 graphic service 打交道就正式开始了。

cpp 复制代码
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
    ATRACE_CALL();

    status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence, reqWidth, reqHeight,
                                                            reqFormat, reqUsage, &mBufferAge,
                                                            enableFrameTimestamps ? &frameTimestamps
                                                                                  : nullptr);


}

Graphic Buffer 服务

Surface 的 所有和 Graphic Buffer 相关的接口都会使用到成员 mGraphicBufferProducer

cpp 复制代码
sp<IGraphicBufferProducer> mGraphicBufferProducer;

这个成员的类型一看就是一个 binder Interface,继续看 IGraphicBufferProducer。

\frameworks\native\libs\gui\include\gui\IGraphicBufferProducer.h

cpp 复制代码
class IGraphicBufferProducer : public IInterface
{
public:
    using HGraphicBufferProducerV1_0 =
            ::android::hardware::graphics::bufferqueue::V1_0::
            IGraphicBufferProducer;
    using HGraphicBufferProducerV2_0 =
            ::android::hardware::graphics::bufferqueue::V2_0::
            IGraphicBufferProducer;

。。。

    virtual status_t dequeueBuffer(int* slot, sp<Fence>* fence, uint32_t w, uint32_t h,
                                   PixelFormat format, uint64_t usage, uint64_t* outBufferAge,
                                   FrameEventHistoryDelta* outTimestamps) = 0;

。。。

}

至此,一个binder 模型已经形成。

Surface::mGraphicBufferProducer as Bp <--IGraphicBufferProducer BINDER--> ? as Bn

那么这里的Bn是什么呢?只需要在源码里搜一下哪些类继承IGraphicBufferProducer 即可。其中一个常用的类就是 BufferQueueProducer。

\frameworks\native\libs\gui\include\gui\BufferQueueProducer.h

cpp 复制代码
class BufferQueueProducer : public BnGraphicBufferProducer,
                            private IBinder::DeathRecipient {

。。。
    virtual status_t queueBuffer(int slot,
            const QueueBufferInput& input, QueueBufferOutput* output);
。。。

}

继承自BnGraphicBufferProducer,Bn开头,一看就知道是server端。

cpp 复制代码
status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
                                            uint32_t width, uint32_t height, PixelFormat format,
                                            uint64_t usage, uint64_t* outBufferAge,
                                            FrameEventHistoryDelta* outTimestamps) {

        sp<GraphicBuffer> graphicBuffer = new GraphicBuffer(
                width, height, format, BQ_LAYER_COUNT, usage,
                {mConsumerName.string(), mConsumerName.size()});

        status_t error = graphicBuffer->initCheck();


}

在上面的代码里有 new GraphicBuffer 和 initCheck() 这两个动作,这里就是在创建管理对象并进行初始化检查。


内存管理

\frameworks\native\libs\ui\include\ui\GraphicBuffer.h

cpp 复制代码
class GraphicBuffer
    : public ANativeObjectBase<ANativeWindowBuffer, GraphicBuffer, RefBase>,
      public Flattenable<GraphicBuffer>
{

    GraphicBuffer(uint32_t inWidth, uint32_t inHeight, PixelFormat inFormat,
            uint32_t inUsage, std::string requestorName = "<Unknown>");

}

\frameworks\native\libs\ui\GraphicBuffer.cpp

cpp 复制代码
GraphicBuffer::GraphicBuffer(uint32_t inWidth, uint32_t inHeight, PixelFormat inFormat,
                             uint32_t inLayerCount, uint64_t inUsage, std::string requestorName)
      : GraphicBuffer() {
    mInitCheck = initWithSize(inWidth, inHeight, inFormat, inLayerCount, inUsage,
                              std::move(requestorName));
}
cpp 复制代码
status_t GraphicBuffer::initWithSize(uint32_t inWidth, uint32_t inHeight,
        PixelFormat inFormat, uint32_t inLayerCount, uint64_t inUsage,
        std::string requestorName)
{
    GraphicBufferAllocator& allocator = GraphicBufferAllocator::get();
    uint32_t outStride = 0;
    status_t err = allocator.allocate(inWidth, inHeight, inFormat, inLayerCount,
            inUsage, &handle, &outStride, mId,
            std::move(requestorName));
    if (err == NO_ERROR) {
        mBufferMapper.getTransportSize(handle, &mTransportNumFds, &mTransportNumInts);

        width = static_cast<int>(inWidth);
        height = static_cast<int>(inHeight);
        format = inFormat;
        layerCount = inLayerCount;
        usage = inUsage;
        usage_deprecated = int(usage);
        stride = static_cast<int>(outStride);
    }
    return err;
}

上面创建 Graphic Buffer 的过程中会有一个 initWithSize , 这是初始化的过程,在初始化过程里又又 allocator.allocate,可见是一个内存分配的过程。


内存分配 GraphicBufferAllocator

\frameworks\native\libs\ui\include\ui\GraphicBufferAllocator.h

\frameworks\native\libs\ui\GraphicBufferAllocator.cpp

cpp 复制代码
class GraphicBufferAllocator : public Singleton<GraphicBufferAllocator>
{

}

allocate 方法如下

cpp 复制代码
status_t GraphicBufferAllocator::allocate(uint32_t width, uint32_t height,
        PixelFormat format, uint32_t layerCount, uint64_t usage,
        buffer_handle_t* handle, uint32_t* stride,
        uint64_t /*graphicBufferId*/, std::string requestorName)
{

。。。
    status_t error =
            mAllocator->allocate(width, height, format, layerCount, usage, 1, stride, handle);
。。。

}

mAllocator 是什么 ?

cpp 复制代码
sp<hardware::graphics::allocator::V2_0::IAllocator> mAllocator;

又是一个 Interface ,这次是 hardware 的,因此其实现是由硬件完成的。具体细节参考 /hardware/interfaces/graphics/allocator/2.0/

相关推荐
七月.末35 分钟前
安卓aab包的安装教程,附带adb环境的配置
android·adb
SRC_BLUE_176 小时前
SQLI LABS | Less-39 GET-Stacked Query Injection-Intiger Based
android·网络安全·adb·less
无尽的大道9 小时前
Android打包流程图
android
镭封10 小时前
android studio 配置过程
android·ide·android studio
夜雨星辰48711 小时前
Android Studio 学习——整体框架和概念
android·学习·android studio
邹阿涛涛涛涛涛涛11 小时前
月之暗面招 Android 开发,大家快来投简历呀
android·人工智能·aigc
IAM四十二11 小时前
Jetpack Compose State 你用对了吗?
android·android jetpack·composer
奶茶喵喵叫11 小时前
Android开发中的隐藏控件技巧
android
Winston Wood13 小时前
Android中Activity启动的模式
android
众乐认证13 小时前
Android Auto 不再用于旧手机
android·google·智能手机·android auto